Multithreaded Programming

Overview

Programming in a multithreaded environment can be difficult. Sometimes programming errors are timing related as multiple threads interact. Multithread locks can become tangled and bugs can be difficult to reproduce. To alleviate these problems and enable the benefits of a multi-threaded core, Ejscript provides a suite of facilities to make multithreaded programming easier, more reliable, and more efficient.

The Problem of Multithreaded I/O Events

A particular thorny issue in multithreaded servers, is how to handle I/O without consuming a thread for each request. It is not practical to dedicate a thread to each HTTP request. A single browser will often send 6-10 simultaneous requests. If each request consumed a thread for the duration, the server would quickly consume too many threads and system performance would greatly suffer.

One solution is to use a thread pool. This allows a request to borrow a thread from a thread pool while the request is active and return the thread when it cannot immediately continue with servicing the request. A thread may not be able to continue because it is waiting for I/O from the client, or waiting for I/O to the client to drain over the network. Returning the thread to the pool allows other requests to use the thread while the first request is waiting for network I/O. When the network is ready, a thread can be obtained from the pool and the original request can continue. For this to work, an efficient network event service is essential. Ejscript uses such a thread pool and event service to efficiently manage thread resources.

However, this use of such an event service and thread pool raises another problem: races between the foreground request thread and the background async I/O event thread. It is easy for these two threads to simultaneously interact and corrupt critical data structures. A typical solution is to use multithread locks to serialize access to such data, but this is a crude solution and often leads to brittle applications. Ejscript has a better solution that effectively serializes all activity for an interpreter: per-interpreter event dispatchers.

Event Dispatchers

The Multithreaded Portable Runtime (MPR) used by Ejscript has a facility called Event Dispatchers. These are event queues on which all I/O and other event activity for an interpreter can be queued and serviced. Each interpreter has its own dispatcher and so events for the interpreter are serialized. When a network I/O event is received by Ejscript for a request, an event is queued on the interpreter's dispatcher. If the interpreter is currently active (using a thread from the thread pool), the event is queued and no further action is taken. When the request has finished its current activity, it will service events on its dispatcher queue and eventually service the I/O event. If the interpreter is currently idle, a thread is assigned from the thread pool for the request, and the thread is resumed to service the interpreter's dispatcher queue. This greatly simplifies Ejscript as all activity for an interpreter is thus serialized via the dispatcher queue.

Multithreaded Ejscript

By using event dispatchers, Ejscript serializes all activity, yet it can support many simultaneous requests due to its multithreaded core. Ejscript efficiently utilizes thread resources by using a thread pool and not dedicating threads permanently to requests. Threads are temporarily assigned only as required by active requests.

Modules

Ejscript can be extended by writing loadable modules. All module code can rely on the fact that event activity will be serialized and no locking is required to serialize I/O activity. Consequently, most module code can be effectively single-threaded.

Locking

If you have a requirement for a data structure that will be accessed and manipulated simultaneously by multiple threads, the MPR provides a suite of locking primitives. See MprSynch for the MPR Multithreaded Synchronization Services.