AWS Lambda with Node.js: Getting Started

AWS Lambda service is used for building event-driven applications that are highly scalable, but many people are unclear about using it. You may have heard about terms such as serverless, function-as-a-service or AWS Lambda. If you want to learn more about these terms, then you are in luck. This article shares details about AWS Lambda serverless and to build a scalable image processing app with AWS Lambda and Node.js .

Defining Serverless: An Overview

There was a time when everything online was hosted on physical machines known as servers. The servers were kept in server rooms and companies mostly built and looked after their own data centers. However, this required a lot of resources, time and cost. 

In recent years, a new technology known as cloud computing has emerged in the market.  Today, all types of applications can be hosted on it easily. This means that you don’t need a data center of your own.

You can easily deploy your applications on a cloud server in minutes in any part of the world. Yet, scaling, server provisioning and maintenance was a hectic task. Fortunately, Serverless, a new shift in cloud computing technology has emerged. This has resolved the issue of server provisioning, logging, monitoring and maintenance of the entire infrastructure. This helps you to break your business logic into small single-purpose functions and work on it:

Serverless takes away the responsibility of managing servers from you, but it still requires servers and is not completely Serverless. Amazon Web Services is one such entity that falls under this category. It takes care of servers.

What are Amazon Web Services?

Amazon Web Service or commonly referred as the AWS is a renowned name in the cloud computing industry. According to their statistics, AWS offers an extremely dependable, scalable and economical infrastructure. It also hosts hundreds of thousands of businesses around 190 countries in the world.  As per Canalys’ 2018 report, AWS owns a 32.6% market share which is greater than any other company.

With this fact established, let’s move on to teaching you something that will completely blow you away.

AWS Lambda Functions

Computing service provided by AWS is called Lambda. This helps you run your code without having to deal with the cloud servers. With the help of an event, a lambda function will be triggered and die down after execution. The Lambda function only performs one thing such as fetching or creating a blog post or sending an email.

3 Ways to create a Lambda Function on AWS:

  1. You can use AWS console, a web interface offered by AWS for accessing and managing their services. However, it takes a lot of time and effort to write an application from the console and hence it’s not a recommended option.
  2. AWS also provides  Cloud based IDE. you can write, run and debug your code from the browser through it.
  3. Lastly, you can always use your local development environment with any text editor. Deploy the code with a single command. This article explores this option.

Creating an AWS Account

You must have a AWS account to trigger a lambda function.The account requirements include an email address, phone number and a legitimate credit card. You can always opt for a free tier account by AWS which allows you to use almost all the AWS services without paying anything for a year.

Here are the steps for Account Setup:

  1. Visit the AWS console.
  2. Choose “Create a Free Account.”
  3. Enter your email address, choose a strong password, contact and credit card details. Make sure all the details entered are correct.
  4. Complete identity verification process via Amazon’s phone call.
  5. You will receive a 4-digit number on your computer screen. Enter it on your phone’s keypad.
  6. Choose the free plan.
  7. Welldone! You have signed up for a brand new AWS account.

Local Development Environment Set Up

This article tutorial uses serverless framework, a CLI tool written inNode.js to write and deploy Lambda functions. It is compatible with AWS, Microsoft Azure, Google Cloud Platform, Spotinst, Kubeless, IBM OpenWhisk and more.

It is also easy to install the Serverless framework. First, you require a Node.js runtime. Install Node.js 8.10 runtime version which is compatible with AWS Lambda. Also, make sure your local development environment is close to the production environment including the runtime.

If you already have other Node.js versions installed, make use of NVM to install Node.js 8.10 runtime. NVM also helps to switch between Node.js versions.

$ nvm install v8.10

For switching between Node.js versions, do this:

$ nvm use 8.10

After Node.js runtime is ready, you need to to install the Serverless framework:

$ npm install -g serverless

For checking the Serverless framework installation.

$ serverless --version
1.40.0

How to Create a Programmatic User on AWS

The Lambda function doesn’t live in your local environment permanently. It must be transferred into the AWS environment. This procedure is called deployment. Serverless framework requires a way to access AWS resources and deploy your Lambda functions. 

This requires a programmatic user account. This account does not log into AWS console. It provides access to AWS resources through API calls with the help of access keys that you will create next.

Steps to Create a Programmatic User Account

  1. Sign in to AWS console and choose the IAM user.

2. Select Add user to start the account creation process.

3. Type lambda-example-cli as username. Enable programmatic access by checking the checkbox and click on Next: permissions to proceed.

4. Select Attach existing policy directly and search Administrator access. Check the AdinistratorAccess box. The policy is an object that defines the permissions of a user, group or role.

5. Click on the Create user button to view this screen.

6. Download or copy a CSV file that has your access Key ID and access key secret. Keep this file safe. These access keys help  to make API calls. Anyone who gets it can make API calls and can control your AWS account.

7. Configure serverless CLI through AWS credentials in order to deploy the code.

serverless config credentials --provider aws --key <your_access_key_id&gt; --secret <your_access_key_secret&gt;

Let’s first create a simple hello world app with Lambda and Node.js to get started. After that we will create an advanced app that downloads an image from a URL, rescale it and upload it to AWS S3, a scalable object storage service.

Start by using the Serverless CLI tool: 

$ serverless create --template hello-world

If the above command runs successfully, you will be able to have two files with you.

.
├── handler.js
└── serverless.yml

We supplied the –template argument to let Serverless CLI know our choice of templates. There are dozens of templates the Serverless CLI tool supports. You can find them in this repository.

This command supplies the -template argument to indicate Serverless CLI about our templates choice. Serverless CLI supports a variety of templates available in a repository.

Handler.js

Handler.js is a Lambda function where you will make your logic:

'use strict';
module.exports.helloWorld = (event, context, callback) =&gt; {
...

It accepts three arguments: event, context, and a callback.

Event

The event argument contains event data. There are different event types, and each often contains different attributes. Understanding how Lambda functions work can be a bit hard to grasp at first.

Event argument has event data. There are different event types and each type has attributes. The way Lambda functions work are a bit hard to understand at first. First thing you must know is that a Lambda function is triggered by a service and doesn’t run on its own.  Here is a list of services to invoke Lambda functions.

Context

The context argument is used to pass the runtime parameter to Lambda function.

Callback

Callback argument is used to return responses to the caller.

Serverless.yml

Serverless.yml  has API definition and other resources. These are required by your application to work properly. The article covers S3 for storing images.

Make some changes to serverless.yml. Change the runtime property to nodejs8.10. Add a new property region to the provider object. This will deploy the app to the specified region (we will do it). However, specifying it is optional and AWS will use us-east-1 by default unless specified by us. However, always choose regions close to users in production due to latency.

service: serverless-hello-world
# The `provider` block defines where your service will be deployed
provider:
  name: aws
  runtime: nodejs8.10
  region: eu-west-1
....

Deploying the App

Deploy the app with a deploy argument. Enter the  following command from the console:

$ serverless deploy

You will see the result in your console on completion. Note the endpoint here, as it’s quite important.

...
api keys:
  None
endpoints:
  GET - https://ss7n639ye3.execute-api.eu-west-1.amazonaws.com/dev/hello-world
functions:
  helloWorld: serverless-hello-world-dev-helloWorld
...

When you can access the endpoint in your browser, you will also see a request printed back to you. Pat yourself on the back. You have done your first Lambda app.

Going Advanced

Hello world app built previously was quite simple. Lets go a bit advance and build the image processing app discussed above.

You can start a new project or modify Hello World app.

Edit serverless.yml as follows:

# filename: serverless.yml
service: ImageUploaderService

# The `provider` block defines where your service will be deployed
custom:
  bucket: getting-started-lambda-example
provider:
  name: aws
  runtime: nodejs8.10
  region: eu-west-1
  stackName: imageUploader
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "s3:PutObject" 
      Resource: 
        - "arn:aws:s3:::${self:custom.bucket}/*" 

# The `functions` block defines what code to deploy
functions:
  UploadImage:
    handler: uploadImage.handler
    # The `events` block defines how to trigger the uploadImage.handler code
    events:
      - http:
        path: upload
        method: post
        cors: true
    environment:
      Bucket: ${self:custom.bucket} 
resources:
  Resources:
    StorageBucket:
      Type: "AWS::S3::Bucket"
      Properties:
        BucketName: ${self:custom.bucket}

The YAML file has a custom object and the bucket’s name of the bucket is defined here. You can choose a different bucket name, as you won’t be able to choose the same name I have used unless i delete it. According to the AWS documentation, “Amazon S3 bucket name is globally unique and the namespace is shared by all AWS accounts.” This means that you can not use the same bucks name after it’s created by a user through another AWS account in any AWS region until the bucket is deleted.

You will also see that we have renamed stackName as ImageUploader. A stack is a collection of AWS resources which one manages as a single unit. IamRoleStatement is also defined as global. Lambda function needs permission for accessing these AWS resources. In our case, we require permission for writing to S3 bucket. This permission is given in the IAM role statements.

Below Lambda function Upload Image, a new object named environment is added. This helps to set environment variables. We can get these via process.env object during execution. Note the handler’s name here.

We concluded it by defining the S3 bucket resource for storing images.

Adding npm packages

Dont start from scratch. Use your favorite npm packages in Lambda apps. They will be packaged with your functions on deployment.

Use uuid package to generate unique names for images and jimp for manipulating the uploaded images.Create a package.json file.

npm init

Answer the questions to get started. 

npm install jimp uuid

Update the handler’s function. Rename the function to UploadImage.js. It’s a good convention to name your function after its functionality.

// filename: uploadImage.js

"use strict";

const AWS = require("aws-sdk");
const uuid = require("uuid/v4");
const Jimp = require("jimp");
const s3 = new AWS.S3();
const width = 200;
const height = 200;
const imageType = "image/png";
const bucket = process.env.Bucket;

module.exports.handler = (event, context, callback) =&gt; {
    let requestBody = JSON.parse(event.body);
    let photoUrl = requestBody.photoUrl;
    let objectId = uuid();
    let objectKey = `resize-${width}x${height}-${objectId}.png`;

    fetchImage(photoUrl)
        .then(image =&gt; image.resize(width, height)
            .getBufferAsync(imageType))
        .then(resizedBuffer =&gt; uploadToS3(resizedBuffer, objectKey))
        .then(function(response) {
            console.log(`Image ${objectKey} was uploaed and resized`);
            callback(null, {
                statusCode: 200, 
                body: JSON.stringify(response)
            });
        })
        .catch(error =&gt; console.log(error));
};

/**
* @param {*} data
* @param {string} key
*/
function uploadToS3(data, key) {
    return s3
        .putObject({
            Bucket: bucket,
            Key: key,
            Body: data,
            ContentType: imageType
        })
        .promise();
}

/**
* @param {url}
* @returns {Promise}
*/
function fetchImage(url) {
    return Jimp.read(url);
)

In the uploadImage.js we have used fetchimage method for getting the image from the URL. Read more about jimp package’s working in the readme file.

After you have rescaled the image, it’s time to store it in the S3 bucket with the help of putObject method in the AWS SDK.

How to log in AWS Lambda functions

Logging gives clarity about how the applications run in production. This saves time when troubleshooting a problem. There are different log aggregating services such as Retrace, AWS cloudwatch and Lambda that work well together.

AWS Lambda monitors functions on your behalf and shares metrics in a report through Amazon CloudWatch. The metrics include total requests, duration and error rates. In addition to logging and monitoring, you can also log an event with console.log from your code.

console.log('An error occurred')

The handler function (uploadImage.js) we log into AWS CloudWatch when an image is successfully processed and when an errors occurs.

Deploying and testing

Deploy the existing or a new app with this Serverless deploy command:

serverless deploy

This is the output which you will also get. Note the endpoint again.

.....
  None
endpoints:
  POST - https://0sdampzeaj.execute-api.eu-west-1.amazonaws.com/dev/upload
functions:
  UploadImage: ImageUploaderService-dev-UploadImage
layers:

Make a curl request to the endpoint, so that the image is downloaded from the URL, rescaled and stored to S3 bucket. Don’t forget to change the post endpoint to the one in your console.

curl -H "Content-type: application/json" -d '{"photoUrl":"https://www.petmd.com/sites/default/files/what-does-it-mean-when-cat-wags-tail.jpg"}' 'https://0sdampzeaj.execute-api.eu-west-1.amazonaws.com/dev/upload'

Check the logs in CloudWatch and images in S3 bucket.

Summary

You learned what AWS is and how to setup an AWS account with access keys. You also learned to build the hello world app using Lambda and Node.js running in the cloud. Lastly, you also learned to create a photo processing app using the Serverless framework! That’s a lot to cover for a beginner!

Move on and build your knowledge from there about the Serverless framework and how to test the Lambda functions. Check out “Serverless Local Development” by Gareth McCumskey, a serverless and web developer.

These resources provide a great learning path to understanding AWS Lambda with Node.js.

Leave a comment

Your email address will not be published. Required fields are marked *