Build Serverless APIs with Node.js and AWS Lambda

AWS Lambda has been around for a few years now, and it remains the most popular way to experiment with serverless technology. If you're not familiar with serverless, it's a model of development in which managing, provisioning, and scaling servers is abstracted away from application development. Servers do exist in a serverless world, but they are completely managed by the cloud provider, allowing developers to focus on packaging their code for deployment.

AWS Lambda is a type of Function-as-a-Service (FaaS) offering that allows code execution on demand in response to preconfigured events or requests. This post will introduce you to AWS Lambda and guide you on creating and deploying Lambda functions with Node.js and AWS SAM.

Let's get started!

Prerequisites

Before you proceed with this tutorial, ensure that you have Node.js 14.x LTS installed on your computer, as this is the latest release that AWS Lambda supports at the time of writing. However, the contents of this article should stay relevant even when newer releases are supported. You can use Volta to install and manage multiple versions of Node.js on your computer. Also, ensure you sign up for a free AWS account if you don't have one already.

If you intend to run your AWS Lambda function locally (which I'll be demonstrating in this article), you'll need Docker installed on your computer. Follow these instructions to set up Docker for your operating system before proceeding with the rest of this tutorial.

Install the AWS CLI and AWS SAM CLI

In this guide, we'll be using both the AWS CLI and the AWS Serverless Application Model (SAM) CLI to develop our serverless functions and deploy them to AWS.

The former interacts with AWS services on the command line, while the latter helps with building, debugging, deploying, and invoking Lambda functions. Read more about AWS SAM in the docs.

The exact way to install both CLI tools will differ depending on your operating system. You can install or upgrade to the latest version of AWS CLI for Linux, macOS, and Windows by following the instructions on this page. To install the AWS SAM CLI, follow the relevant guide for your operating system:

Here are the versions of AWS CLI and AWS SAM CLI that I installed while writing this guide:

$ aws --version
aws-cli/2.4.1 Python/3.8.8 Linux/4.19.128-microsoft-standard exe/x86_64.ubuntu.20 prompt/off
 
$ sam --version
SAM CLI, version 1.36.0

After you've installed both CLI tools, follow this guide to set up your AWS credentials so that you can interact successfully with your AWS account via the CLIs.

$ aws configure list
      Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key     ****************ZFEF shared-credentials-file
secret_key     ****************BnOU shared-credentials-file
    region                us-east-1      config-file    ~/.aws/config

Create Your First AWS Lambda Function with Node.js

Let's start by writing a simple hello world function to demonstrate how AWS Lambda works. Run the command below to initialize a new project:

$ sam init --runtime nodejs15.x --name aws-lambda-nodejs-example

When prompted, choose AWS Quick Start Templates under template source, Zip under package type, and Hello World Example under application templates.

Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1
 
What package type would you like to use?
        1 - Zip (artifact is a zip uploaded to S3)
        2 - Image (artifact is an image uploaded to an ECR image repository)
Package type: 1
 
Cloning from https://github.com/aws/aws-sam-cli-app-templates
 
AWS quick start application templates:
        1 - Hello World Example
        2 - Step Functions Sample App (Stock Trader)
        3 - Quick Start: From Scratch
        4 - Quick Start: Scheduled Events
        5 - Quick Start: S3
        6 - Quick Start: SNS
        7 - Quick Start: SQS
        8 - Quick Start: Web Backend
Template selection: 1

Once the command exits, change into the freshly minted aws-lambda-nodejs-example folder. It should have the following folder structure:

.
├── events
│   └── event.json
├── hello-world
│   ├── app.js
│   ├── package.json
│   └── tests
│       └── unit
│           └── test-handler.js
├── README.md
└── template.yaml

Here's a short description of the important files and directories in the project:

template.yaml: Defines the AWS resources for your application.
hello-world/app.js: Contains the Lambda function logic.
hello-world/package.json: Contains any Node.js dependencies required by the application.
hello-world/tests/: Contains unit tests for your Lambda functions.

Open up the template.yaml file in your text editor and take note of the following lines:

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello-world/
      Handler: app.lambdaHandler
      Runtime: nodejs14.x
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get

These lines describe the name of your Lambda function (HelloWorldFunction), the runtime used to execute it (nodejs14.x), and the type of trigger for the function (Api). This indicates that your Lambda function will execute when a GET request is sent to the /hello route via API Gateway. Note that there are several other ways to invoke Lambda functions.

The CodeUri line indicates that the code for the HelloWorldFunction is in the hello-world directory. The Handler property specifies app.js as the file with the function code, which should have a named export called lambdaHandler.

Open up the hello-world/app.js file and examine its contents:

let response;
 
exports.lambdaHandler = async (event, context) => {
  try {
    response = {
      statusCode: 200,
      body: JSON.stringify({
        message: "hello world",
      }),
    };
  } catch (err) {
    console.log(err);
    return err;
  }
 
  return response;
};

This simple function takes two parameters and returns a response object containing a 'hello world' message. The first parameter is the JSON payload sent by the invoker of the function, while the second is the context object which contains information about the function invocation and execution environment. This handler is async, so you can use return or throw to return a response or an error, respectively. Non-async handlers must use a third callback parameter (not shown here).

Go ahead and modify the function to return the reserved environmental variables defined in the runtime environment instead of the hello world message.

exports.lambdaHandler = async (event, context) => {
  let response;
 
  try {
    const environmentalVariables = {
      handler: process.env._HANDLER,
      aws_region: process.env.AWS_REGION,
      aws_execution_env: process.env.AWS_EXECUTION_ENV,
      aws_lambda_function_name: process.env.AWS_LAMBDA_FUNCTION_NAME,
      aws_lambda_function_name: process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE,
      aws_lambda_function_version: process.env.AWS_LAMBDA_FUNCTION_VERSION,
      aws_lambda_log_group_name: process.env.AWS_LAMBDA_LOG_GROUP_NAME,
      aws_lambda_log_stream_name: process.env.AWS_LAMBDA_LOG_STREAM_NAME,
      aws_lambda_runtime_api: process.env.AWS_LAMBDA_RUNTIME_API,
      lang: process.env.LANG,
      tz: process.env.TZ,
      lambda_task_root: process.env.LAMBDA_TASK_ROOT,
      lambda_runtime_dir: process.env.LAMBDA_RUNTIME_DIR,
      path: process.env.PATH,
      ld_library_path: process.env.LD_LIBRARY_PATH,
    };
 
    response = {
      statusCode: 200,
      body: JSON.stringify(environmentalVariables),
    };
  } catch (err) {
    response = {
      statusCode: 500,
      error: err,
    };
 
    console.log(err);
  }
 
  return response;
};

We can access the environmental variables through process.env and, after aggregating them in an object, return them in the response object. API Gateway uses the statusCode property to add the right HTTP status code to the generated response.

Testing Your AWS Lambda Function Locally

Before deploying your function, you'll want to test it locally to confirm it works as expected. To do so, run the following SAM command at the root of your project directory:

$ sam local start-api
Mounting HelloWorldFunction at http://127.0.0.1:3000/hello [GET]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2021-11-25 20:56:52  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)

This command requires Docker, so make sure it's installed and running on your computer. Otherwise, you might get an error message similar to the one shown below:

Error: Running AWS SAM projects locally requires Docker. Have you got it installed and running?

Once the application is running, make a GET request to http://localhost:3000/hello. This will cause AWS SAM to start a Docker container to run the function. Once the container is up and running, the function will execute and the following result will be returned:

$ curl http://localhost:3000/hello
{"handler":"app.lambdaHandler","aws_region":"us-east-1","aws_execution_env":"AWS_Lambda_nodejs14.x","aws_lambda_function_name":"128","aws_lambda_function_version":"$LATEST","aws_lambda_log_group_name":"aws/lambda/HelloWorldFunction","aws_lambda_log_stream_name":"$LATEST","aws_lambda_runtime_api":"127.0.0.1:9001","lang":"en_US.UTF-8","tz":":/etc/localtime","lambda_task_root":"/var/task","lambda_runtime_dir":"/var/runtime","path":"/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin","ld_library_path":"/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib"}

You can use jq to prettify the output if you have it installed:

$ curl http://localhost:3000/hello | jq
{
  "handler": "app.lambdaHandler",
  "aws_region": "us-east-1",
  "aws_execution_env": "AWS_Lambda_nodejs14.x",
  "aws_lambda_function_name": "128",
  "aws_lambda_function_version": "$LATEST",
  "aws_lambda_log_group_name": "aws/lambda/HelloWorldFunction",
  "aws_lambda_log_stream_name": "$LATEST",
  "aws_lambda_runtime_api": "127.0.0.1:9001",
  "lang": "en_US.UTF-8",
  "tz": ":/etc/localtime",
  "lambda_task_root": "/var/task",
  "lambda_runtime_dir": "/var/runtime",
  "path": "/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin",
  "ld_library_path": "/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib"
}

You can also test your Lambda function without making an HTTP request to trigger it. The SAM CLI provides a way to invoke the function using a predefined JSON file. Run the following command in your project root to try it out:

$ sam local invoke "HelloWorldFunction" --event events/event.json
Invoking app.lambdaHandler (nodejs14.x)
Skip pulling image and use local one: amazon/aws-sam-cli-emulation-image-nodejs14.x:rapid-1.21.1.
 
Mounting /home/ayo/aws-lambda-nodejs-example/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container
START RequestId: 5ee01a30-d649-484d-9b9c-124ba3669525 Version: $LATEST
END RequestId: 5ee01a30-d649-484d-9b9c-124ba3669525
REPORT RequestId: 5ee01a30-d649-484d-9b9c-124ba3669525  Init Duration: 2.44 ms  Duration: 584.60 ms     Billed Duration: 600 ms   Memory Size: 128 MB     Max Memory Used: 128 MB
{"statusCode":200,"body":"{\"handler\":\"app.lambdaHandler\",\"aws_region\":\"us-east-1\",\"aws_execution_env\":\"AWS_Lambda_nodejs14.x\",\"aws_lambda_function_name\":\"128\",\"aws_lambda_function_version\":\"$LATEST\",\"aws_lambda_log_group_name\":\"aws/lambda/HelloWorldFunction\",\"aws_lambda_log_stream_name\":\"$LATEST\",\"aws_lambda_runtime_api\":\"127.0.0.1:9001\",\"lang\":\"en_US.UTF-8\",\"tz\":\":/etc/localtime\",\"lambda_task_root\":\"/var/task\",\"lambda_runtime_dir\":\"/var/runtime\",\"path\":\"/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin\",\"ld_library_path\":\"/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib\"}

The event that triggers a Lambda function usually comes with a JSON payload. You should provide this payload using the --event option (as demonstrated above) to invoke the function locally. This payload passes as the first argument to the Lambda function. The event.json file is created by the SAM CLI when initializing the project, so it may be used for this purpose. Learn more about events.

When you invoke a function locally, you'll get some information on:

the Docker container image used to execute the function
how long it ran for
how much memory was used

You'll also get the actual return value of your function below the runtime information.

Deploying the Lambda Function to the AWS Cloud

Once you are happy with how your function runs locally, you can deploy it to the AWS Cloud through the SAM CLI. First, run sam build to generate artifacts that target AWS Lambda's execution environment:

$ sam build
Building codeuri: /home/ayo/dev/demo/aws-lambda-nodejs-example/hello-world runtime: nodejs14.x metadata: {} architecture: x86_64 functions: ['HelloWorldFunction']
Running NodejsNpmBuilder:NpmPack
Running NodejsNpmBuilder:CopyNpmrc
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:NpmInstall
Running NodejsNpmBuilder:CleanUpNpmrc
 
Build Succeeded

Next, run sam deploy --guided to deploy the function, and answer the prompts as shown below:

$ sam deploy --guided
 
Configuring SAM deploy
======================
 
Looking for config file [samconfig.toml] :  Not found
 
Setting default arguments for 'sam deploy'
=========================================
Stack Name [sam-app]:
AWS Region [us-east-1]:
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]: y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: y
#Preserves the state of previously provisioned resources when an operation fails
Disable rollback [y/N]: n
HelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y
Save arguments to configuration file [Y/n]: y
SAM configuration file [samconfig.toml]: y
SAM configuration environment [default]:
 
. . .

Once deployment is successful, you will see the API Gateway URL in the output. It should have the following structure:

https://<API_ID>.execute-api.<AWS_REGION>.amazonaws.com/Prod/hello/

Once you make a GET request to that URL, your function will execute, and you'll get a similar output as before:

$ curl https://s032akg5bh.execute-api.us-east-1.amazonaws.com/Prod/hello/ | jq
{
  "handler": "app.lambdaHandler",
  "aws_region": "us-east-1",
  "aws_execution_env": "AWS_Lambda_nodejs14.x",
  "aws_lambda_function_name": "128",
  "aws_lambda_function_version": "$LATEST",
  "aws_lambda_log_group_name": "/aws/lambda/sam-app-HelloWorldFunction-QK428gXjzGYj",
  "aws_lambda_log_stream_name": "2021/11/26/[$LATEST]46ab610defd746fcae6474da20515190",
  "aws_lambda_runtime_api": "127.0.0.1:9001",
  "lang": "en_US.UTF-8",
  "tz": ":UTC",
  "lambda_task_root": "/var/task",
  "lambda_runtime_dir": "/var/runtime",
  "path": "/var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin",
  "ld_library_path": "/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib"
}

Congratulations, you've successfully deployed your first Lambda function to production!

Using NPM Packages in AWS Lambda Functions

Let's go ahead and create a new Lambda function that uses some NPM packages to perform a web scraping task. Create a new folder called quotes-scraper in your project directory and initialize it with a package.json file:

$ mkdir quotes-scraper
$ cd quotes-scraper
$ npm init -y

Afterward, create an app.js file in the root of the quotes-scraper directory and populate it with the following content:

const axios = require("axios");
const cheerio = require("cheerio");
 
const url = "http://quotes.toscrape.com/";
 
exports.lambdaHandler = async (_event, _context) => {
  try {
    const response = await axios(url);
    const $ = cheerio.load(response.data);
    const container = $(".container .quote");
    const quotes = [];
 
    container.each(function () {
      const text = $(this).find(".text").text();
      const author = $(this).find(".author").text();
      const tags = $(this).find(".tag");
 
      const tagArray = [];
 
      tags.each(function () {
        const tagText = $(this).text();
        tagArray.push(tagText);
      });
 
      quotes.push({
        text,
        author,
        tag: tagArray,
      });
    });
 
    return {
      statusCode: 200,
      body: JSON.stringify(quotes),
    };
  } catch (err) {
    console.log(err);
    throw err;
  }
};

This code scrapes the quotes on this website and returns them as a JSON object. It uses axios to fetch the HTML and cheerio to extract the relevant parts. Ensure you install both dependencies in the quotes-scraper directory:

$ npm install axios cheerio

Afterward, open the template.yml file in your project root and add the following code to the Resources section:

Resources:
  . . .
 
  QuotesScraperFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: quotes-scraper/
      Handler: app.lambdaHandler
      Runtime: nodejs14.x
      Events:
        QuotesScraper:
          Type: Api
          Properties:
            Path: /quotes
            Method: get

Next, add the following snippet to the Output section:

Output:
  . . .
 
  QuotesScraperApi:
    Description: "API Gateway endpoint URL for Prod stage for Quotes Scraper function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/quotes/"
  QuotesScraperFunction:
    Description: "Quotes Scraper Function ARN"
    Value: !GetAtt QuotesScraperFunction.Arn
  QuotesScraperFunctionIamRole:
    Description: "Implicit IAM Role created for Quotes Scraper function"
    Value: !GetAtt QuotesScraperFunctionRole.Arn

Save and close the file, then invoke your new function through the SAM CLI:

$ sam build
$ sam local invoke "QuotesScraperFunction" | jq
[
  {
    text: '“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”',
    author: 'Albert Einstein',
    tag: [ 'change', 'deep-thoughts', 'thinking', 'world' ]
  },
  {
    text: '“It is our choices, Harry, that show what we truly are, far more than our abilities.”',
    author: 'J.K. Rowling',
    tag: [ 'abilities', 'choices' ]
  },
  . . .
]

Go ahead and deploy your function with the sam deploy command. Afterward, you will be able to invoke the function through the API Gateway endpoint:

https://<API_ID>.execute-api.<AWS_REGION>.amazonaws.com/Prod/quotes/

Creating Serverless APIs with AWS Lambda and Node.js: Wrap Up and Next Steps

I hope this article has helped you learn the basics of building serverless APIs with the help of AWS Lambda and Node.js.

There are so many other topics concerning Lambda functions that we've not covered here, including authentication, logging and monitoring, caching, persisting to a database, and more.

To build on the knowledge you've gained through this tutorial, read up on those topics and check out some of the recommended practices for working effectively with AWS Lambda functions.

Thanks for reading, and happy coding!

P.S. If you liked this post, subscribe to our JavaScript Sorcery list for a monthly deep dive into more magical JavaScript tips and tricks.

P.P.S. If you need an APM for your Node.js app, go and check out the AppSignal APM for Node.js.