Multiple people will use your app at the same time, and you want to deliver your app as fast as possible. So you'll need some way to handle concurrency. Fear not! Most web servers already do this by default. But when you need to scale, you want to use concurrency in the most efficient way possible.
Different types of concurrency
There are multiple ways to handle concurrency: multi-process, multi-threading and event-driven. Each of these have their uses, pros and cons. In this article, you'll learn how they differ and when to use which.
Each worker process has the full codebase in memory. This makes this method pretty memory-intensive, and makes it hard to scale to larger infrastructures.
|Use case||One non-ruby example you probably know is the Chrome browser. It uses multi-process concurrency to give each tab their own process. It allows a single tab to crash without taking the full application down. In their case, it also helps to isolate exploits to a single tab.|
|Pros||Most simple to implement.|
Ignores difficulties with thread safety.
Each worker can crash without damaging the rest of the system.
|Cons||Each process loads the full codebase in memory. This makes it memory-intensive.|
Hence, it does not scale to large amounts of concurrent connections.
As opposed to the multi-process approach, all threads run within the same process. This means they share data such as global variables. Therefore, only small chunks of extra memory are used per thread.
Global Interpreter Lock
This brings us to the global interpreter lock (GIL) in MRI. The GIL is a lock around the execution of all Ruby code. Even though our threads appear to run in parallel, only one thread is active at a time.
IO operates outside of the GIL. When you execute a database query waiting for the result to come back, it won't lock. Another thread will have a chance to do some work in the meantime. If you do a lot of math and operations on hashes or arrays in threads, you will only utilize a single core if you use MRI. In most cases you still need multiple processes to fully utilize your machine. Or you could use Rubinius or jRuby, which don't have a GIL.
If you use multiple threads you have to be careful to write all code that manipulates shared data in a thread safe way. You can do this for example by using a Mutex to lock shared data structures before you manipulate them. This will ensure that other threads are not basing their work on stale data while you're changing the data.
|Use case||This is the "middle of the road" option. Used for a lot of standard web applications which should handle loads of short requests (such as a busy web application).|
|Pros||Uses less memory than multi-process.|
|Cons||You have to make sure your code is thread safe.|
If a thread causes a a crash, it can potentially take down your process.
The GIL locks all operations except I/O.
Below you'll see a very simple event loop written in Ruby. The loop will take the event from the
event_queue and handle it. If there is no event, it will sleep and repeat to see if there are new events in the queue.
loop do if event_queue.any? handle_event(event_queue.pop) else sleep 0.1 end end
In this illustration, we're taking it a step further. The event loop now does a beautiful dance with the OS, queue and some memory.
Step by step
- The OS keeps track of network and disk availability.
- When the OS sees the I/O is ready, it sends an event to the queue.
- The queue is a list of events from which the event loop takes the top one.
- The event loop handles the event.
- It uses some memory to store meta data about the connections.
- It can send a new event directly into the event queue again. For example, a message to shut down the queue based on the contents of an event.
- If it wants to do an I/O operation, it tells the OS that it's interested in a specific I/O operation. The OS keeps track of the network and disk (see ) and adds an event again when I/O is ready.
|Use case||When using a lot of concurrent connections to your users. Think of services like Slack. Chrome notifications.|
|Pros||Almost no memory overhead per connection.|
Scales to a huge number of parallel connections.
|Cons||It's a difficult mental model to understand.|
Batch sizes must be small and predictable to avoid queues building up.
- For most apps threading makes sense, Ruby/Rails ecosystem seems to (slowly) be moving this way.
- If you run highly concurrent apps with long-running streams, event-loop allows you to scale.
- If you don't have a high traffic site, or you expect your workers to break go for good old multi-process.
And, it is possible to run an event loop, inside a thread, inside a multi-process setup. So yes, you can have your stroopwafel and eat it too!