Erlang is renowned for its remarkable fault tolerance and high concurrency. Erlang's scheduler efficiently handles many lightweight processes. The scheduler plays a crucial role in managing processes, concurrency, and system resources, efficiently coordinating these elements to help Erlang maintain fault tolerance and support high levels of concurrency in its applications.

This post will dissect some of the scheduler's key components and shed light on how it works internally.

Let's get started!

Processes in Erlang

Erlang processes are lightweight, independent units of execution managed by the Erlang runtime system. They are created and scheduled by the Erlang virtual machine (BEAM), and each Erlang process has its own memory space. These should not be confused with operating system processes or threads.

OS processes are entities managed by the OS and typically have more overhead in terms of memory and resources, as well as inter-process communication. While threads are lighter than OS processes, they are still heavier than a typical Erlang process and share a common memory space within a process. Communication is usually easier between threads of the same process but still requires synchronization mechanisms.

On the other hand, the Erlang VM (BEAM) is capable of spawning several lightweight processes with independent memory space and can communicate easily using message passing. The scheduler inside the VM manages processes, allocates resources to processes, and context-switches between them.

Erlang Scheduler Architecture — Preemptive Scheduling

In a single-core setup, only one process can occupy the CPU core at any given time. To emulate a concurrent system, the scheduler employs a preemptive, priority-based scheduling mechanism (don't worry, we will explore what this means soon) that rapidly switches between available processes to create the illusion that all processes are executing simultaneously. The Erlang Virtual Machine (BEAM) schedules processes to run sequentially, with one process running for a duration, being suspended, and then allowing another process to take its turn. This process management strategy is characteristic of concurrency and does not entail true parallelism (which is not possible on a single-core system). Tasks appear to run concurrently due to fast context switching, although they actually execute sequentially on a single core.

To identify processes for potential swapping, Erlang introduces the concept of "reductions". In Erlang, a reduction represents a unit of work performed by BEAM, encompassing fundamental operations such as function application, arithmetic calculations, or message passing. The scheduler keeps track of the reductions executed by each process, preempting a process when it reaches a certain reduction count, thereby allowing another process to run.

Reductions

The idea of "reduction" in Erlang is inherited from its Prolog ancestry. In Prolog, every execution step is termed a goal-reduction, involving breaking down a logic problem into individual components and solving each part accordingly.

To promote fairness among processes, Erlang's preemptive scheduling relies on reductions rather than time slices. If a process exhausts its allocated reductions, it can be preempted, even if its execution isn't complete. This approach prevents a single process from monopolizing the CPU for an extended period, fostering fairness among concurrent processes. By using reductions as the foundation for preemption, Erlang mitigates the risk of processes starving for CPU time. This design ensures that every process, irrespective of its workload, is periodically allowed to execute.

Moreover, reductions are flexibly applied based on the type of operation a process is performing. For example, certain operations, such as I/O operations, may consume various reductions. This adaptability allows the scheduler to effectively handle different types of operations.

In other languages, traditional blocking I/O operations can lead to inefficient resource utilization, as threads might be blocked while waiting for I/O to complete. Erlang's asynchronous and non-blocking I/O model allows processes to continue executing other tasks while waiting for I/O operations to complete. This minimizes the impact of blocking operations on overall system performance.

Priority

Each process in Erlang can have a priority value that decides how often the scheduler will execute that process. A process priority can be set using the Process.flag/2 (or process_flag/2 on Erlang) function call:

iex> Process.spawn(
  fn ->
    Process.flag(:priority, :high)
    # ...
  end,
  [:link]
)

There are 4 priority levels: low, normal, high, and max. max is reserved for internal use in Erlang and should not be used in application code. Processes on each priority level get a separate run queue and are scheduled in the normal round-robin fashion as described above.

Except low and normal, Erlang executes the processes of each priority queue exclusively. This means that if there is a process in the max priority queue, only max priority processes will be executed, and all other processes will be blocked. Similarly, if there is a process in the high priority queue, low and normal processes will not be executed until all processes from the high priority queue are executed (or are non-runnable). low and normal queues behave slightly differently — the processes inside both queues are interleaved such that normal priority processes are executed more often than low priority ones, but normal processes do not block the execution of low priority processes.

Due to the blocking behavior of high priority processes, they should be used very rarely and only for short-lived tasks. Overusing high priority processes is bound to lead to outages and affect the responsiveness of an application, as all other regular OTP processes run on normal priority.

Another point that's important here is that Erlang places no restrictions on communication between different priority levels of processes. So a high-priority process can wait for a message from a lower-priority process. This is allowed, but will effectively lower the priority of the high-priority process.

Running on Multiple Cores

Up to this point, we've explored how the scheduler orchestrates processes within a single-core system. However, in a multi-core environment, where additional resources are available for parallel processing, the Erlang VM creates a dedicated "run queue" for each available core. This enables true parallelism, as multiple processes can run simultaneously (one on each available core). Within each run queue, all processes adhere to the preemptive scheduling mechanism we discussed earlier.

Typically, Erlang processes share the same run queue as their parent process, and a work-stealing algorithm may come into play to ensure load balancing. For instance, if there are two cores in the system and one consistently handles busy processes while the other remains relatively idle, the Erlang schedulers on both cores engage in periodic communication. This communication facilitates the movement of processes from the heavily loaded core to the less busy one, ensuring a more equitable distribution of workloads across both cores.

In the broader context, beyond Erlang's internal run queues, the operating system plays a role in managing the scheduling of threads onto OS-level cores. This implies that processes not only experience swapping within the Erlang run queue but also may undergo a complete context switch or be moved to a different core at the OS level.

Note that, because of the concept of work-stealing inside the Erlang VM, it is usually beneficial to run a single Erlang application instance on multiple cores rather than running separate instances of the same application on different cores of the same machine. In a single instance, the schedulers dedicated to each core can better communicate between them to share the load equitably compared to a multi-node cluster where schedulers cannot share process load (even if all of that cluster's nodes are on the same physical machine).

Performance and Optimization

Erlang's scheduler takes out most of the complexities involved in building a concurrent system. It automatically frees the developer from having to think about things like lock contention, thread overhead, and load balancing by handling these issues out of the box with its preemptive scheduling algorithm.

Erlang provides various scheduler-related parameters that can be tuned for optimal performance. Parameters like +S (scheduler count) and +P (maximum number of processes) allow you to configure the number of scheduler threads and processes.

For example, you can start Erlang with erl +S Schedulers:SchedulerOnline to control the number of scheduler threads. By default, Erlang uses the number of CPU cores to identify these values automatically. Note that while both Scheduler and SchedulerOnline accept values up to 1024, starting more schedulers than the number of CPU cores does not have any positive benefits for an app.

Another possible step to fine-tune performance is to control the priorities of processes, as we've discussed. It is indeed possible to execute certain high-priority tasks in Erlang.

Nevertheless, this comes with an inherent risk of potentially rendering a system unresponsive and increasing latency, as processes with a high priority may block all other normal/low-priority processes. Conversely, marking tasks identified as intentionally low priority can be advantageous to prioritize other processes above them.

So be careful and use your judgment.

Wrapping Up

In this post, we've seen that the Erlang scheduler stands as a cornerstone in Erlang's architecture, fostering fault tolerance, concurrency, and adaptability. Its preemptive and dynamic nature equips developers to build resilient, highly concurrent systems capable of handling failures and utilizing resources optimally. Understanding the intricacies of the Erlang scheduler can empower you to craft scalable and robust distributed applications.

If you are interested in learning more about the scheduler, I recommend checking out the Scheduling chapter in The BEAM Book and Processes from the Erlang Efficiency Guide.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

P.P.S. AppSignal has an integration for Erlang — check it out.