Node.js Resiliency Concepts: The Circuit Breaker

Andrei Andrei Gaspar on

In 2009 Node.js opened up a door for front-end developers to dip their toes into the world of servers without having to leave the comfort of their language.

It’s almost effortless to get started with Node. You can basically copy-paste an entire HTTP server into existence and then install an ODM and you’ve got your CRUD app ready to roll!

However, if we’ve learned anything from the amazing Spider-Man, it’s that with great power, comes great responsibility.

So, in this article, we’re going to discuss how you can wield your Node-given powers responsibly, and design servers that don’t just work, but are also resilient and adaptive to failures.

πŸ‘‹ As you’re exploring Node.js resiliency concepts, you might want to explore AppSignal for Node.js as well. We provide you with out-of-the-box support for Node.js Core, Express, Next.js, Apollo Server, node-postgres and node-redis.

Resiliency and Chill

One of the biggest names in the industry when it comes to server resiliency design is Netflix. They are extremely dedicated to designing robust systems that will serve us all seasons of Grey’s Anatomy any minute of the day!

But what is this “resiliency” anyway?

Well, resiliency is just a fancy word for the ability of your system to recover from failures and continue operating.

If the power goes out and it continues to work, your system is resilient. If there is an equipment failure and the system keeps on going, it is even more resilient. If you hit it with a baseball bat and the system is still up… you get the idea.

However, in our case, we’re more interested in providing API resiliency. So, let’s see how we would identify a resilient API. What are some of the core principles of a resilient API?

Well, let’s learn from the pros. Let’s see what Netflix has to say about it.

Netflix defines the principles of resiliency as follows:

They are also responsible for fault tolerance libraries and sophisticated tools for dealing with latency and fault tolerance in distributed systems.

To deal with the problem of fault tolerance, most of these solutions use a popular software design pattern called circuit-breaker, which is the exact pattern that we’re going to be discussing in detail in the upcoming sections.

The Circuit Breaker Pattern

The Circuit Breaker in software design is named after it’s equivalent in electrical engineering, where it serves as a switch designed to stop the flow of the current in an electric circuit. It is used as a safety measure to protect the circuit from overload or short circuit.

Circuit breakers come in all shapes and sizes, there are some that reset automatically, some that need to be reset manually, but they all essentially do the same thing β€” open the circuit if there’s trouble.

The Circuit Breaker was popularized by Miachel Nygard with his book Release It!, where he describes this pattern along with other useful information about architecting resilient and performant software.

So if the electrical circuit breaker manages the flow of current, what does it’s software equivalent do?

The circuit breaker manages the flow of requests to an upstream resource.

Let’s think of the upstream resource as a remote server for the time being, but it is certainly not limited to being that. Circuit breakers can also be used locally to protect one part of your system from failure from another part.

The circuit breaker monitors for failures, and when the failures reach a certain threshold, it trips and none of the successive calls will be forwarded to the upstream resource.

Why Would We Bother Using a Circuit Breaker?

With the rising popularity of microservices, it is common for apps to make remote calls to other apps running on different processes across a network. It is often the case that the system is spread out across multiple machines as well.

Some of these services act as dependencies for others, and it is not unusual to have multiple dependencies upstream.

Even if we forget about microservices altogether, think about how common it is for applications to make remote calls. It is almost unavoidable that it will have integrations and will rely on upstream resources.

Another popular case is an API gateway, where a service’s primary purpose is to proxy requests upstream. In this case, the health of the application is very closely tied to the health of the upstream resource.

So, we have all these cases where requests are being passed upstream, but why use a circuit breaker? And why don’t we just let the request fail at its own pace?

Preserve Resources

Wasteful calls pile up on the upstream resource which might be already struggling with serving previous requests, further escalating the problem.

Wasteful calls can also be a big problem for the service making those calls.

Resources such as threads might be consumed while waiting for the upstream resource to respond, which can lead to resource exhaustion.

This can in turn lead to the service being unable to handle other requests.

So, wasteful calls can bring down services, and the failure can cascade to other services throughout the application.

Fail Fast

Imagine you’re throwing a party on a Saturday evening. You’re making preparations, sending invitations to all your friends.

Would you prefer them to respond instantly, or would you prefer them to respond the day after the party?

I know, I’d go with option one.

We want responses fast so that we can adapt to them even if it means not getting what we asked for.

This concept in systems design is called failing fast.

Fail Proactively

When upstream resources give us lemons, we make lemonade.

You might not be able to prevent upstream failures, but you can always manage them proactively, and make the most out of what you got.

Here are some common solutions to improve the failure:

Avoid Polluting the Logs

Your monitoring solution is one of the most important components of your system. Without it, you’re completely blind to what happens inside the dark realm of containers and Linux servers.

Metrics and logs are your eyes and ears. And the better the quality of logs, the better you’re able to understand what happens with your system.

If requests keep failing and you don’t have a system in place that handles the situation gracefully, it will end up pumping ungodly amounts of pollution into your logs.

Circuit Breaker States

The circuit breaker has 3 main states which give us a clue about the health of the upstream resource or endpoint that we’re targeting.

Even though these are the conventional names of circuit breaker states, I prefer not to use them because I find them deceptive and can be misleading for developers.

When people see Open they’re intuitively associating it with OK, and Closed sounds a lot like something went wrong.

What I prefer to use instead are colors e.g. Red, Yellow, Green or descriptive names like Failing, Stabilizing, OK.

So, for this demonstration, we’re going to use colors to describe states, but remember, this is just personal preference!

Creating Your Own Circuit Breaker

There are plenty of libraries out there that we could use to implement our circuit breaker, but that would beat the purpose of the article since our goal is to understand how the circuit breaker pattern is implemented.

So let’s reinvent the wheel to learn how the wheel works.

What we are going to code:

We are going to use TypeScript to implement these features.

So, let’s dive in!

The first thing we want to do is to navigate to an empty directory of our choice, which will be our work directory, and execute the npm init command.

1
npm init -y

Once we have the package.json file, it’s time to install our main dependencies.

1
npm install --save express axios

Since we’re using TypeScript, we’ll also need some dev dependencies, so let’s install those as well.

1
npm install --save-dev typescript @types/express @types/axios

Next, we’re going to need a tsconfig.json file to hold our TypeScript configuration. You can use the one below.

1
2
3
4
5
6
7
8
9
10
11
12
13
{
  "compilerOptions": {
    "outDir": "./build",
    "lib": [ "es5", "es6" ],
    "module": "commonjs",
    "target": "es6",
    "sourceMap": true
  },
  "exclude": [
    "node_modules"
  ]
}

Great, now our work directory should contain a node_modules directory and three files: package.json, package-lock.json, and tsconfig.json.

It is time to copy-paste a basic Express server into existence.

Create a file called index.ts and paste the following lines of code into it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
// index.ts

import {Request, Response} from "express";

const express = require("express");
const app = express();

const port = 3000;


app.get( '/', (req: Request, res: Response) => {

    if ( Math.random() > 0.5 ) {
        res.status( 200 ).send( "Success!" );
    } else {
        res.status( 400 ).send( "Failed!" );
    }

});

app.listen( port, () => console.log( `Listening at http://localhost:${ port }` ) );

The above code snippet summons a simple express server that will be listening to GET requests on localhost:3000, and randomly failing with status 400 or responding with status 200. We’ll be able to use this endpoint to test our Circuit Breaker.

Before we go further with the implementation, let’s add a couple of convenience scripts to our package.json file so that we can build and start the server using npm commands.

In the scripts section of your package.json, copy and paste the following:

1
2
3
4
5
6
...
  "scripts": {
    "build": "tsc",
    "start-server": "npm run build && node build/index.js"
  },
 ...

This will allow you to start your server with a simple npm command.

1
npm run start-server

Once the command is executed, the server should print “Listening at.. localhost:3000” to the console.

So far so good! Let’s move on to the meat of the article, which is the Circuit Breaker itself!

Let’s create a circuit-breaker directory, which will contain all the assets related to the Circuit Breaker.

1
mkdir circuit-breaker

Now, let’s navigate into this directory and start thinking about the components that we’ll need to make the circuit breaker a reality.

First, we talked about states, so let’s create a file called BreakerStates.ts to define our states.

We’re going to use an enum and color codes for the states, to make it a bit more developer-friendly.

In the BreakerStates.ts file let’s declare an enum like so:

1
2
3
4
5
6
7
8
// circuit-breaker/BreakerStates.ts

export enum BreakerState {
    GREEN = "GREEN",
    RED = "RED",
    YELLOW = "YELLOW"
}

Great, now that we have the states, what else do we need?

We’ll need some configuration options for our Circuit Breaker that will answer the following questions for us:

So, immediately, we can see that we’ll need a public class named BreakerOptions that can hold these properties. We could also opt for an interface trick here, but let’s stick to the conventional class-based approach.

Let’s create a file called BreakerOptions.ts and define our public class.

1
2
3
4
5
6
7
8
// circuit-breaker/BreakerOptions.ts

export class BreakerOptions { constructor(
    public failureThreshold: number,
    public successThreshold: number,
    public timeout: number
){}}

Once we have the States and Options defined, we can start planning the CircuitBreaker class implementation. Since the circuit breaker will be making requests, and we’re using Axios as our HTTP library, we’ll have Axios as our dependency for this class.

Let’s think about the properties that we’ll have in the class.

Let’s not forget about the BreakerOptions we defined! We’ll need to store those inside the class as well. It would also be smart to make them optional and have default values defined for them within the class.

That is a handful of properties to be defined. So let’s set all this up before we move to the logic implementation.

Let’s create a file called CircuitBreaker.ts where we’ll define our CircuitBreaker class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
// circuit-breaker/CircuitBreaker.ts

import { BreakerOptions } from "./BreakerOptions";
import { BreakerState } from "./BreakerStates";
import { AxiosRequestConfig } from "axios";

const axios = require("axios");


class CircuitBreaker {
    private request: AxiosRequestConfig;
    private state: BreakerState;

    private failureCount: number;
    private successCount: number;

    private nextAttempt: number;

    // Options
    private failureThreshold: number;
    private successThreshold: number;
    private timeout: number;


    constructor(request: AxiosRequestConfig, options?: BreakerOptions) {

        this.request        = request;
        this.state          = BreakerState.GREEN;

        this.failureCount   = 0;
        this.successCount   = 0;
        this.nextAttempt    = Date.now();

        if ( options ) {
            this.failureThreshold   = options.failureThreshold;
            this.successThreshold   = options.successThreshold;
            this.timeout            = options.timeout;
        } else {
            // Define defaults
            this.failureThreshold   = 3;
            this.successThreshold   = 2;
            this.timeout            = 3500;
        }
    }

}

Now it’s time to think about the methods that we’ll need. Let’s plan them out and then we can start implementing them one by one.

So let’s start at the beginning and define our log method as such:

1
2
3
4
5
6
7
8
9
10
11
12
// circuit-breaker/CircuitBreaker.ts

   private log(result: string): void {

        console.table({
            Result: result,
            Timestamp: Date.now(),
            Successes: this.successCount,
            Failures: this.failureCount,
            State: this.state
        });
    }

All it’s responsible for is taking the result and displaying it in a nice tabular format, including other details about the current state of our Circuit Breaker.

Let’s move on to the success method and define some logic. Here’s what it should do for us.

Sounds easy enough, let’s write the code!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// circuit-breaker/CircuitBreaker.ts

    private success(res: any): any {

        this.failureCount = 0;

        if ( this.state === BreakerState.YELLOW ) {
            this.successCount++;

            if ( this.successCount > this.successThreshold ) {
                this.successCount = 0;
                this.state = BreakerState.GREEN;
            }
        }

        this.log( "Success" );

        return res;

    }

Great, we have success down β€” we’ll do the same for failure. Here’s the gist of it.

Here’s the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// circuit-breaker/CircuitBreaker.ts

    private failure(res: any): any {

        this.failureCount++;

        if ( this.failureCount >= this.failureThreshold ) {
            this.state = BreakerState.RED;

            this.nextAttempt = Date.now() + this.timeout;
        }

        this.log( "Failure" );

        return res;
    }

And finally, the most important method to define, the exec method! This stands at the core of our mechanism. Let’s see what it should do for us.

Simple enough, right? Let’s see how the implementation looks.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// circuit-breaker/CircuitBreaker.ts

    public async exec(): Promise<void> {

        if ( this.state === BreakerState.RED ) {

            if ( this.nextAttempt <= Date.now() ) {
                this.state = BreakerState.YELLOW;
            } else {
                throw new Error( "Circuit suspended. You shall not pass." );
            }
        }

        try {
            const response = await axios( this.request );

            if ( response.status === 200 ) {
                return this.success( response.data );
            } else {
                return this.failure( response.data );
            }
        } catch ( err ) {
            return this.failure( err.message );
        }
    }

So, now that we have our CircuitBreaker class all set up, it’s time to see how we can use it to perform requests.

Before anything else though, here’s the complete implementation of the class, you can review it to see if it matches yours!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
// circuit-breaker/CircuitBreaker.ts

import { BreakerOptions } from "./BreakerOptions";
import { BreakerState } from "./BreakerStates";
import { AxiosRequestConfig } from "axios";

const axios = require("axios");



export class CircuitBreaker {
    private request: AxiosRequestConfig;
    private state: BreakerState;

    private failureCount: number;
    private successCount: number;

    private nextAttempt: number;

    // Options
    private failureThreshold: number;
    private successThreshold: number;
    private timeout: number;



    constructor(request: AxiosRequestConfig, options?: BreakerOptions) {

        this.request        = request;
        this.state          = BreakerState.GREEN;

        this.failureCount   = 0;
        this.successCount   = 0;
        this.nextAttempt    = Date.now();

        if ( options ) {
            this.failureThreshold   = options.failureThreshold;
            this.successThreshold   = options.successThreshold;
            this.timeout            = options.timeout;
        } else {
            // Define defaults
            this.failureThreshold   = 3;
            this.successThreshold   = 2;
            this.timeout            = 3500;
        }
    }



    private log(result: string): void {

        console.table({
            Result: result,
            Timestamp: Date.now(),
            Successes: this.successCount,
            Failures: this.failureCount,
            State: this.state
        });
    }



    public async exec(): Promise<void> {

        if ( this.state === BreakerState.RED ) {

            if ( this.nextAttempt <= Date.now() ) {
                this.state = BreakerState.YELLOW;
            } else {
                throw new Error( "Circuit suspended. You shall not pass." );
            }
        }

        try {
            const response = await axios( this.request );

            if ( response.status === 200 ) {
                return this.success( response.data );
            } else {
                return this.failure( response.data );
            }
        } catch ( err ) {
            return this.failure( err.message );
        }
    }



    private success(res: any): any {

        this.failureCount = 0;

        if ( this.state === BreakerState.YELLOW ) {
            this.successCount++;

            if ( this.successCount > this.successThreshold ) {
                this.successCount = 0;
                this.state = BreakerState.GREEN;
            }
        }

        this.log( "Success" );

        return res;

    }



    private failure(res: any): any {

        this.failureCount++;

        if ( this.failureCount >= this.failureThreshold ) {
            this.state = BreakerState.RED;

            this.nextAttempt = Date.now() + this.timeout;
        }

        this.log( "Failure" );

        return res;
    }

}

Looking good? Great!

Alongside our index.ts file, we can create a test.ts file as well, that will contain a couple of lines of code for testing our masterpiece.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// test.ts
import { CircuitBreaker } from "./circuit-breaker/CircuitBreaker";



const circuitBreaker = new CircuitBreaker({
    method: "get",
    url: "http://localhost:3000"
});


setInterval(() => {
    circuitBreaker
        .exec()
        .then( console.log )
        .catch( console.error )
}, 1000 );

In the code above, we imported the CircuitBreaker, created an instance of it and started calling the exec() method at an interval of 1 second.

Let’s add one more script to our package.json file to be able to run this test conveniently.

The scripts section should look like this, updated with the test-breaker script:

1
2
3
4
5
6
7
...
  "scripts": {
    "build": "tsc",
    "start-server": "npm run build && node build/index.js",
    "test-breaker": "npm run build && node build/test.js"
  },
  ...

Now, let’s make sure the server is running!

1
npm run start-server

And in a separate terminal window, let’s run the circuit breaker test as well.

1
npm run test-breaker

Once executed, here’s an example of the log stream that you should be seeing in your terminal.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
Success!
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  (index)  β”‚    Values     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  Result   β”‚   'Failure'   β”‚
β”‚ Timestamp β”‚ 1592222319902 β”‚
β”‚ Successes β”‚       0       β”‚
β”‚ Failures  β”‚       1       β”‚
β”‚   State   β”‚    'GREEN'    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
Request failed with status code 400
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  (index)  β”‚    Values     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  Result   β”‚   'Failure'   β”‚
β”‚ Timestamp β”‚ 1592222320906 β”‚
β”‚ Successes β”‚       0       β”‚
β”‚ Failures  β”‚       2       β”‚
β”‚   State   β”‚    'GREEN'    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
..............
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  (index)  β”‚    Values     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  Result   β”‚   'Failure'   β”‚
β”‚ Timestamp β”‚ 1592222321904 β”‚
β”‚ Successes β”‚       0       β”‚
β”‚ Failures  β”‚       3       β”‚
β”‚   State   β”‚     'RED'     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
...............
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  (index)  β”‚    Values     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  Result   β”‚   'Failure'   β”‚
β”‚ Timestamp β”‚ 1592222331941 β”‚
β”‚ Successes β”‚       2       β”‚
β”‚ Failures  β”‚       1       β”‚
β”‚   State   β”‚   'YELLOW'    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
...............

From this point forward, you can have as much fun with it as you like.

You can start and stop the server while the circuit breaker is running to notice what happens, and you can also create different breakers with different BreakerOptions like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
// test.ts

import { CircuitBreaker } from "./circuit-breaker/CircuitBreaker";
import { BreakerOptions } from "./circuit-breaker/BreakerOptions";



const breaker1 = new CircuitBreaker({
    method: "get",
    url: "http://localhost:3000"
}, new BreakerOptions( 3, 5, 5000 ) );


const breaker2 = new CircuitBreaker({
    method: "get",
    url: "http://localhost:3000"
}, new BreakerOptions( 6, 7, 1000 ) );


setInterval(() => {
    breaker1
        .exec()
        .then( console.log )
        .catch( console.error )
}, 500 );

setInterval(() => {
    breaker2
        .exec()
        .then( console.log )
        .catch( console.error )
}, 1500 );

Implementation Granularity

Once you have it up and running, the design choices are in your hands. You can choose to make a circuit breaker responsible for an entire upstream service or just target individual endpoints depending on your needs.

Feel free to use different HTTP integrations, experiment with extending the breaker options and define multiple endpoints in your server to test with.

Here are additional feature ideas to consider:

Parting Words

This sums up our overview of the Circuit Breaker pattern! I hope this article helped you grasp a few resiliency principles and it sparked your imagination to try extending this boilerplate with some creative solutions.

We reinvented the wheel to understand how it works, but custom solutions are not always the best choice. You have to analyze complexity and keep maintenance overhead in sight.

Once you’re comfortable with the basics, I would suggest you check out a few npm packages which are designed specifically for this purpose. There are a couple of candidates out there like opossum, hystrixJS, and brakes.

It all depends on your requirements and I trust you to make the right decisions in your journey to improve system resiliency!

P.S. If you liked this post, subscribe to our new JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you’d love an all-in-one APM for Node or you’re already familiar with AppSignal, go and check out AppSignal for Node.js.

Daydreaming about APIs and imagining web services β€” our guest author Andrei is a solutions architect by day and the co-founder of Boardme by night. When he’s not typing frantically in a terminal, he’s exploring nature, pretends to draw, and supplies bystanders with unsolicited gym advice.

10 latest articles

Go back
Javascript sorcery icon

Subscribe to

JavaScript Sorcery

A true sorcerer combines ancient wisdom and new discoveries. We'll provide you with both. Sign up for our JavaScript Sorcery email series and receive deep insights about JavaScript, error tracking and other developments.

We'd like to set cookies, read why.