Magicians never share their secrets. But we do. Sign up for our Ruby Magic email series and receive deep insights about garbage collection, memory allocation, concurrency and much more.
Multiple people will use your app at the same time, and you want to deliver your app as fast as possible. So you'll need some way to handle concurrency. Fear not! Most web servers already do this by default. But when you need to scale, you want to use concurrency in the most efficient way possible.
There are multiple ways to handle concurrency: multi-process, multi-threading and event-driven. Each of these have their uses, pros and cons. In this article, you'll learn how they differ and when to use which.
Each worker process has the full codebase in memory. This makes this method pretty memory-intensive, and makes it hard to scale to larger infrastructures.
|Use case||One non-ruby example you probably know is the Chrome browser. It uses multi-process concurrency to give each tab their own process. It allows a single tab to crash without taking the full application down. In their case, it also helps to isolate exploits to a single tab.|
Most simple to implement.
Ignores difficulties with thread safety.
Each worker can crash without damaging the rest of the system.
Each process loads the full codebase in memory. This makes it memory-intensive.
Hence, it does not scale to large amounts of concurrent connections.
As opposed to the multi-process approach, all threads run within the same process. This means they share data such as global variables. Therefore, only small chunks of extra memory are used per thread.
This brings us to the global interpreter lock (GIL) in MRI. The GIL is a lock around the execution of all Ruby code. Even though our threads appear to run in parallel, only one thread is active at a time.
IO operates outside of the GIL. When you execute a database query waiting for the result to come back, it won't lock. Another thread will have a chance to do some work in the meantime. If you do a lot of math and operations on hashes or arrays in threads, you will only utilize a single core if you use MRI. In most cases you still need multiple processes to fully utilize your machine. Or you could use Rubinius or jRuby, which don't have a GIL.
If you use multiple threads you have to be careful to write all code that manipulates shared data in a thread safe way. You can do this for example by using a Mutex to lock shared data structures before you manipulate them. This will ensure that other threads are not basing their work on stale data while you're changing the data.
|Use case||This is the "middle of the road" option. Used for a lot of standard web applications which should handle loads of short requests (such as a busy web application).|
|Pros||Uses less memory than multi-process.|
You have to make sure your code is thread safe.
If a thread causes a a crash, it can potentially take down your process.
The GIL locks all operations except I/O.
Below you'll see a very simple event loop written in Ruby. The loop will take the event from the
event_queue and handle it. If there is no event, it will sleep and repeat to see if there are new events in the queue.
1 2 3 4 5 6 7
loop do if event_queue.any? handle_event(event_queue.pop) else sleep 0.1 end end
In this illustration, we're taking it a step further. The event loop now does a beautiful dance with the OS, queue and some memory.
|Use case||When using a lot of concurrent connections to your users. Think of services like Slack. Chrome notifications.|
Almost no memory overhead per connection.
Scales to a huge number of parallel connections.
It's a difficult mental model to understand.
Batch sizes must be small and predictable to avoid queues building up.
And, it is possible to run an event loop, inside a thread, inside a multi-process setup. So yes, you can have your stroopwafel and eat it too!