May 5, 2020

Serving your first Express js App in production

Prerequisites:

Install NodeJs on the server and the dev machine. See for instructions: https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs/development_environment

Create a new Express App

(Do this on your dev machine first and then transfer the app, once ready, to the server using a git push-pull or any other method you like)

For a full fledged WebApp we can use "expressgenerator" to quickly setup a project directory and install all the dependencies (including express). However to keep things simple for a basic express app, we will not be using the "expressgenerator". Instructions for using expressgenerator are still appended below for reference.

Basic App without expressgenerator

Make a project directory and initialize a project

mkdir MyApp && cd MyApp
npm init

Install Express

npm install express --save

Create an "app.js" file

touch app.js

That's it. With the basic app we only intent to serve some functions that accept data from a http request and return some data as a response (no html templating, rendering, serving webpages, etc.), and thus avoid adding more dependencies and bloated folder structures. For a more full-fledged webapp or website, starting with expressgenerator might be the way to go.

Using ExpressGenerator

Install the generator:

npm install express-generator -g

Use it to generate the project

express MyApp

Install all dependencies

cd MyApp
npm install

This would install "express" and a bunch of other dependencies usually used in a full-fledged WebApp serving requests for webpages, handling html templates etc.

Your first http-request triggered function

Irrespective of the method you used above to create your project structure. You now have an "app.js" file in your folder.

The minimal amount of code you need in your app.js file to create an express app serving responses to http requests is

// import any requirements
const express = require('express');

// initialize the express app
const app = express()

// write the function called when a specific path is requested
app.get('/function_path', (request,response) => {
    response.send('Hi, here\'s my first function') 
	})

// set up the app to listen to a port on the local network
app.listen(8000,() => console.log('MyApp listening on port 8000!'))

That's it !

We can now see how to serve this app locally on the dev machine (to take this to production, there are a few more changes that will be recommended, so don't send this off to the production server just yet)

Serving your app locally

Just run the app from the terminal using node

node app.js

and you should see "MyApp listening on port 8000!" in your terminal. Now you can navigate your browser to the address "127.0.0.1:8000/function_path" and you should see our message "Hi, here's my first function", appear in the browser.

That's it ! You can use this method now to add more functions to the app and use node to locally test the functionality of your app.

However, to take the app into production, we must take care of a few security issues and have the app served through a production server instead of using the local port 8000.

Preparing the app for a production server

There are several excellent in-depth articles highlighting the best practices when deploying your nodejs applications on a production server. (See, for example, https://expressjs.com/en/advanced/best-practice-performance.html)

Here's a short gist of what you need to do.

Introduce middleware to enhance security and performance

The one security middleware that is commonly suggested for nodejs applications in production is called "helmet"

Install helmet using:

npm install helmet --save

Add "helmet" to your app by adding the following line to app.js

...
const express = require('express')
// add this requirement
const helmet = require('helmet')
...
// after app initialization
// add the following app.use line
app.use(helmet())
...

Helmet (https://www.npmjs.com/package/helmet) sets a bunch of http header options for node app that are aimed to protect the apps against most known methods of attacks.

Another commonly recommended middleware is "compression". You can similarly add "compression" to your app using "npm install compression --save" and add "const compression = require('compression'); app.use(compression())".

'compression' uses gzip compression to compress the size of the response generated by your app, thus reducing the outgoing traffic from your server and getting the response faster to the client (due to its smaller size, and thus faster loading at the client end). This can have a significant impact on the performance for high volume client-server interaction.

However there are two arguments against using 'compression' in your NodeJs app. The first is that the compression of responses doesn't need to happen within your app in a synchronous manner (synchronous execution would be mean that while the response is being compressed, the app isn't serving other requests). This task of gzip compression can easily be handled by another process sitting between the app and the client (this process is usually the reverse proxy server that handles the passing of messages between the app and the client and most servers like Apache or Nginx have an in-built module for this purpose).

The second reason for not using 'compression' at all (either in your app or in the reverse proxy server), is that gzip compression of responses opens up your app to a known method of attack on encrypted HTTPS connections between the server and client. The attack method is called 'BREACH' and you can read more about it on https://en.wikipedia.org/wiki/BREACH.

Thus in low volume or for security critical connections, I would recommend not using 'compression' at all.

In summary, the only additional middleware, I can safely recommend to add to your app is 'helmet' as we added above.

Set the NODE_ENV variable to "production"

Setting the NODE_ENV to "production" removes all the excess functionality in Node that is meant for a debugging and development environment. It's estimated to make your app in production run about 3 times faster.

The NODE_ENV variable can easily be set using a "export NODE_ENV='production' " statement in a bash environment.  This is however only appropriate on a development computer. On the server we want the NODE_ENV variable to be set for the process in which the automatic launch of the node app occurs (see the next point for clarification). We will be running the node app as a daemon process which automatically restarts after a crash or system reboot. Thus we need to inject the code to set the node environment into this daemon launch process and for this simply setting the NODE_ENV in a bash shell won't do. Instead, we will set the NODE_ENV value within the "daemonization" setup we will use for our app (see next point)

Ensure your app restarts automatically after a crash

We will use a process manager called "pm2" to "daemonize" the app, i.e., the app restarts automatically after any crash or system reboot, to ensure that the app is always running and available on our server.

To install pm2, use

sudo npm install -g pm2

Within our project folder run the command

pm2 ecosystem

This will generate a sample config file (named "ecosystem.config.js"). We can set our NODE_ENV variable within this file as

module.exports = {
  apps : [{
    name: "MyApp",
    script: "./app.js",
    env: {
      NODE_ENV: "production",
    }
  }]
}

Then we can start the app as a daemon within using the following shell command

pm2 start ecosystem.config.js

This will launch an instance of your app within the pm2 process. Whenever the app crashes, pm2 will relaunch the app automatically with the config defined above. However, we need to also ensure that pm2 itself is a daemon and ensure that pm2 restarts after any crashes or after system reboot. This can be executed by issuing the shell command and following the prompted instructions

pm2 startup systemd

That's it! Our app is now ready to run on a production server. There are a few more config changes we can make to run multiple parallel instances of our app within pm2, thus improving our response time from the server to the client.

For this we can modify our config file as,

module.exports = {
  apps : [{
    name: "MyApp",
    script: "./app.js",
    instances: "max",
    exec_mode: "cluster",
    env: {
      NODE_ENV: "production",
    }
  }]
}

In the above modification we added an "instances: 'max', exec_mode: 'cluster', " statement. This tells pm2 to run as many instances of the MyApp as there are compute cores in the processor (i.e. on a 4-core processor, this will launch 4 instances of our app, each app having its own memory space and running in parallel so that we can serve multiple requests in parallel). You could also specify an exact number for the instances instead of using max (e.g. "instances: 3, ..." would launch 3 instances irrespective of the number of cores.)

You can check out a more detailed note on managing the pm2 ecosystem here:

PM2 - Ecosystem File
Advanced process manager for production Node.js applications. Load balancer, logs facility, startup script, micro service management, at a glance.

Setup a production server (reverse proxy)

Finally, we setup the reverse proxy server that passes the messages back and forth between a client and our app (or rather the cluster of app instances we launched in the previous step). We will use Nginx in this case.

To configure an Nginx server, we simply create a server configuration file with the site-available folder and create a symbolic link to it from within the sites-enable folder . If you aren't familiar with the process of setting up Nginx, the following post describes the process in detail

Configuring Nginx to serve multiple domains
Prerequisites We assume that the server is running on Ubuntu 18.04; for other operating systems, kindly look for the equivalent commands for OS-specific features and paths. The configuration of Nginx will remain the same independent of the host OS.

However, here is the short gist of it. Install Nginx using "sudo apt-get install nginx"

Then, create a text file at the location "/etc/nginx/sites-available/". Let's call the file "MyNodeApp" (no .txt extension). On linux you can do this as

sudo nano /etc/nginx/sites-available/MyNodeApp

Then enter the following content as the configuration for our Nginx server. Here we assume that you already have a domain name or static IP pointing to your production server. You can get a domain name for free from FreeNom.com or one is often offered along with server hosting services.

server{
    listen 80
    server_name YOUR_DOMAIN_NAME
	location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host;

        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_pass http://127.0.0.1:8000;
    }
}

This passes the messages back and forth between port '80' that connects your computer/server to the internet and port 8000, that connects the Nginx server to your app.

Then you can activate this server configuration by creating the symbolic link of this file into the site-enabled folder using

sudo ln -s /etc/nginx/sites-available/MyNodeApp /etc/nginx/sites-enabled

That's it ! Now your you can start Nginx

sudo systemctl start nginx

or if already running, reload Nginx

sudo systemctl reload nginx

Your function is now callable from the internet at "YOUR_DOMAIN_NAME/function_path"