Express

HTTP

  • In order to support the full spectrum of possible HTTP applications, the Node.js HTTP API is very low-level.
  • It deals with stream handling and message parsing only.
  • It parses a message into headers and body but it does not parse the actual headers or the body.

HTTP Classes

  • Agent
  • ClientRequest extends Stream
  • Server extends extends net.Server extends EventEmitter
  • ServerResponse extends Stream
  • IncomingMessage extends Stream.Readable

Making a fetch request

  • http.request(options[, callback])
  • http.request(url[, options][, callback])
  • http.get(options[, callback])
  • http.get(url[, options][, callback])
const http = require('http');
const options = {
  hostname: 'jsonplaceholder.typicode.com',
  port: 80,
  path: '/todos/1',
};

http.get(options, (res) => {
  res.pipe(process.stdout);
});

const http = require('http');
const url = 'http://jsonplaceholder.typicode.com/todos/2';

http.get(url, {}, (res) => {
  res.pipe(process.stdout);
});

Creating a server

  • http.createServer([options][, requestListener])
const http = require('http');

const server = http.createServer((req, res) => {
  console.log(req.url);
  res.end('done\n');
});

server.listen(8080);

What's Express?

Fast, unopinionated, minimalist web framework for Node.js

  • NodeJS based web framework.
  • Inspired by Ruby's Sinatra
const express = require('express');
const app = express();

app.get('/', (req, res) => res.send('Hello World!'));

app.listen(3000,
    () => console.log('Example app listening on port 3000!'));

Why Express.js

  • Configurable
  • Middle-wares
  • Session Management
  • Routing
  • Views and Templates
  • Robust and Secure

 

Express Basic Architecture

  • Model View Controller (MVC) is a software architecture pattern, commonly used to implement user interfaces: it is therefore a popular choice for architecting web apps.
  • it separates out the application logic into three separate parts, promoting modularity and ease of collaboration and reuse.

MVC Theory

The Model

  • The model defines what data the app should contain.
  • If the state of this data changes, then the model will usually notify the view (so the display can change as needed) and sometimes the controller

The View

The view defines how the app's data should be displayed.

The Controller

The controller contains logic that updates the model and/or view in response to input from the users of the app.

MVC Architecture

Advantages & disadvantages

  • Simultaneous development – Multiple developers can work simultaneously on the model, controller and views.
  • High cohesion – MVC enables logical grouping of related actions on a controller together. The views for a specific model are also grouped together.
  • Low coupling – The very nature of the MVC framework is such that there is low coupling among models, views or controllers
  • Ease of modification – Because of the separation of responsibilities, future development or modification is easier
  • Multiple views for a model – Models can have multiple views
  • Code navigability – The framework navigation can be complex because it introduces new layers of abstraction and requires users to adapt to the decomposition criteria of MVC.
  • Multi-artifact consistency – Decomposing a feature into three artifacts causes scattering. Thus, requiring developers to maintain the consistency of multiple representations at once.
  • Pronounced learning curve – Knowledge on multiple technologies becomes the norm. Developers using MVC need to be skilled in multiple technologies.

Express Generator

Express' application generator

quickly create an application skeleton

$ npm install -g express-generator
$ express --view=ejs /tmp/foo
$ cd /tmp/foo
$ npm install
$ npm start

.
├── app.js
├── bin
│   └── www
├── package.json
├── public
│   ├── images
│   ├── javascripts
│   └── stylesheets
│       └── style.css
├── routes
│   ├── index.js
│   └── users.js
└── views
    ├── error.pug
    ├── index.pug
    └── layout.pug

Basic Routing

  • Routing refers to determining how an application responds to a client request to a particular endpoint, which is a URI (or path) and a specific HTTP request method (GET, POST, and so on).
  • Each route can have one or more handler functions, which are executed when the route is matched.
app.METHOD(PATH, HANDLER);

Where:

  • app is an instance of express.
  • METHOD is an HTTP request method, in lowercase.
  • PATH is a path on the server.
  • HANDLER is the function executed when the route is matched.
app.get('/', function (req, res) {
  res.send('Hello World!')
});

app.post('/', function (req, res) {
  res.send('Got a POST request')
});
app.put('/user', function (req, res) {
  res.send('PUT request at /user')
});

app.delete('/user', function (req, res) {
  res.send('DELETE request at /user')
});

Body Parser

Node.js body parsing middleware.

Parse incoming request bodies in a middleware before your handlers, available under the req.body property.

Route paths

app.get('/ab?cd', function (req, res) {
  res.send('ab?cd')
})

app.get('/ab+cd', function (req, res) {
  res.send('ab+cd')
})

app.get(/.*fly$/, function (req, res) {
  res.send('/.*fly$/')
})

matches acd and abcd

matches abcd, abbcd, abbbcd, and so on.

match butterfly and dragonfly, but not butterflyman, dragonflyman, and so on.

Route parameters

app.get('/users/:userId/books/:bookId', function (req, res) {
  res.send(req.params)
})

Route path: /users/:userId/books/:bookId
Request URL: http://localhost:3000/users/34/books/8989
req.params: { "userId": "34", "bookId": "8989" }

Route path: /user/:userId(\d+)
Request URL: http://localhost:3000/user/42
req.params: {"userId": "42"}

To have more control over the exact string that can be matched by a route parameter, you can append a regular expression in parentheses (()):

express.Router

  • Use the express.Router class to create modular, mountable route handlers.
  • A Router instance is a complete middleware and routing system; for this reason, it is often referred to as a “mini-app”.
var express = require('express')
var router = express.Router()

// middleware that is specific to this router
router.use(function timeLog (req, res, next) {
  console.log('Time: ', Date.now())
  next()
})
// define the home page route
router.get('/', function (req, res) {
  res.send('Birds home page')
})
// define the about route
router.get('/about', function (req, res) {
  res.send('About birds')
})
module.exports = router
var birds = require('./birds');

app.use('/birds', birds)

The app will now be able to handle requests to /birds and /birds/about, as well as call the timeLog middleware function that is specific to the route.

What is Middleware?

  • Middleware functions are functions that have access to the request object (req), the response object (res), and the next middleware function in the application’s request-response cycle.
  • The next middleware function is commonly denoted by a variable named next.

An Express application is essentially a series of middleware function calls.

  • Execute any code.
  • Make changes to the request and the response objects.
  • End the request-response cycle.
  • Call the next middleware function in the stack.

Middleware functions can perform the following tasks:

  • If the current middleware function does not end the request-response cycle, it must call next() to pass control to the next middleware function.
  • Otherwise, the request will be left hanging.
  • Middlewares are executed in the order that they are specified.

 

  • Application-level middleware
  • Router-level middleware
  • Error-handling middleware
  • Built-in middleware
  • Third-party middleware

An Express application can use the following types of middleware:

app.use(function (req, res, next) {
  console.log('Time:', Date.now())
  next()
});

app.get('/user/:id', function (req, res, next) {
  res.send('USER')
});

app.use(function (err, req, res, next) {
  console.error(err.stack)
  res.status(500).send('Something broke!')
});

app.use(cookieParser());

Application-level

Application-level

Error-Handling

Third-Party middleware

// a middleware sub-stack that handles GET requests to the /user/:id path
router.get('/user/:id', function (req, res, next) {
  // if the user ID is 0, skip to the next router
  if (req.params.id === '0') next('route')
  // otherwise pass control to the next middleware function in this stack
  else next()
}, function (req, res, next) {
  // render a regular page
  res.render('regular')
});

Router-level

Built-in

Express has the following built-in middleware functions:

  • express.static serves static assets such as HTML files, images, and so on.
  • express.json parses incoming requests with JSON payloads.
  • express.urlencoded parses incoming requests with URL-encoded payloads.

View Template Engines

  • Express supports many template engines - 

  • A template engine enables you to use static template files in your application.

  • At runtime, the template engine replaces variables in a template file with actual values, and transforms the template into an HTML file sent to the client.

  • This approach makes it easier to design an HTML page.

 

  • Pug -- Haml inspired template engine (formerly Jade)

  • Haml.js -- Haml implementation

  • EJS -- Embedded JavaScript template engine

  • hbs -- adapter for Handlebars.js, an extension of Mustache.js template engine

  • React -- renders React components on server.

and many more ...

app.set('view engine', 'pug');

app.get('/', function (req, res) {
  res.render('index', { 
    title: 'Hey',
    message: 'Hello there!'
    });
});

An Example

  • To render template files, set the following application setting properties:

  • views, the directory where the template files are located. Eg: app.set('views', './views'). This defaults to the views directory in the application root directory.

  • view engine, the template engine to use. For example, to use the Pug template engine: app.set('view engine', 'pug').

html
  head
    title= title
  body
    h1= message

index.pug

View Layout and Partials

Passwords

  • The most important aspect of a user account system is how user passwords are protected.
  • User account databases are hacked frequently, so you absolutely must do something to protect your users' passwords if your website is ever breached.

A cryptosystem should be secure even if everything about the system, except the key, is public knowledge. - Kerckhoffs's principle

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

Terms (Hash Functions)

  • A hash function is any function that can be used to map data of arbitrary size to data of a fixed size.
  • it is deterministic so the same message always results in the same hash
  • it is quick to compute the hash value for any given message
  • it is infeasible to generate a message from its hash value except by trying all possible messages
  • a small change to a message should change the hash value so extensively that the new hash value appears uncorrelated with the old hash value
  • it is infeasible to find two different messages with the same hash value
  • Encryption is the process of encoding a message or information in such a way that only authorized parties can access it and those who are not authorized cannot.
  • They provide a 1:1 mapping between an arbitrary length input and output.
  • The important thing to note is that it's reversible using some method.

Bad Solution #1: plain text password

  • This is insecure because if a hacker gains access to your database, they'll be able to use that password to login as that user on your system.
  • Or even worse, if that user uses the same password for other sites on the internet, the hacker can now login there as well. Your users will be very unhappy.
  • Sony: https://arstechnica.com/tech-policy/2011/06/sony-hacked-yet-again-plaintext-passwords-posted/

Solution #2: sha1(password)

  • A better solution is to store a "one-way hash" of the password, typically using a function like md5() or sha1()
  • This solution is more secure than storing the plain text password, because in theory it should be impossible to "undo" a one-way hash function and find an input string that outputs the same hash value.
  • Hackers don't need to "undo" the hash function at all; they can just keep guessing input passwords until they find a match.
  • You might think that there are too many possible passwords for this technique to be feasible. 
  • But there are far fewer common passwords than you'd think. Most people use passwords that are based on dictionary words (possibly with a few extra numbers or letters thrown in).
  • And most hash functions like sha1() can be executed very quickly -- one computer can literally try billions of combinations each second.
  • A modern server can calculate the MD5 hash of about 330MB every second. If your users have passwords which are lowercase, alphanumeric, and 6 characters long, you can try every single possible password of that size in around 40 seconds.
  • sha1(password) is what LinkedIn used to store its passwords. And in 2012, a large set of those password hashes were leaked.

Solution #3: sha1(FIXED_SALT + password)

  • One attempt to make things more secure is to "salt" the password before hashing it
  • The salt is supposed to be a long random string of bytes. If the hacker gains access to these new password hashes (but not the salt), it will make it much more difficult for the hacker to guess the passwords because they would also need to know the salt.
  • However, if the hacker has broken into your server, they probably also have access to your source code as well, so they'll learn the salt too.

Solution #4: sha1(PER_USER_SALT + password)

  • The next step up in security is to create a new column in the database and store a different salt for each user.
  • The salt is randomly created when the user account is first created (or when the user changes their password).
  • By having a per-user-salt, we get one huge benefit: the hacker can't attack all of your user's passwords at the same time. 
for user in users:
    PER_USER_SALT = user["salt"]

    for password in LIST_OF_COMMON_PASSWORDS:
        if sha1(PER_USER_SALT + password) in database_table:
            print "Hacker wins! I guessed a password!", password

Salts (alone) Will Not Help You

  • You can use huge salts or many salts or hand-harvested, shade-grown, organic Himalayan pink salt.
  • It doesn’t affect how fast an attacker can try a candidate password, given the hash and the salt from your database.
  • Don't use MD5, SHA1, SHA2, SHA3, etc?

Double Hashing & Wacky Hash Functions

  •  It's easy to get carried away and try to combine different hash functions, hoping that the result will be more secure. In practice, though, there is very little benefit to doing it.
  • All it does is create interoperability problems, and can sometimes even make the hashes less secure.
  • An attacker cannot attack a hash when he doesn't know the algorithm, but note Kerckhoffs's principle, that the attacker will usually have access to the source code (especially if it's free or open source software), and that given a few password-hash pairs from the target system, it is not difficult to reverse engineer the algorithm. 
md5(sha1(password))
md5(md5(salt) + md5(password))
sha1(sha1(password))
sha1(str_rot13(password + salt))
md5(sha1(md5(md5(password) + sha1(password)) + md5(password)))

Solution: bcrypt/pbkdf2 (password)

  • bcrypt() takes about 100ms to compute, which is about 10,000x slower than sha1().
  • 100ms is fast enough that the user won't notice when they log in, but slow enough that it becomes less feasible to execute against a long list of likely passwords.
  • For instance, if a hacker wants to compute bcrypt() against a list of a billion likely passwords, it will take about 30,000 cpu-hours (about $1200) -- and that's for a single password. Certainly not impossible, but way more work than most hackers are willing to do.
  • Because bcrypt() is so slow, it makes the idea of rainbow tables attractive again, so a per-user-salt is built into the Bcrypt system.
  • In fact, bcrypt stores the salt in the same string as the password hash, so you won't even have to create a separate database column for the salt.

Security Best Practices

  • Don’t use deprecated or vulnerable versions of Express
  • Use TLS
  • Use Helmet library
  • Use cookies securely
  • Ensure your dependencies are secure
  • Avoid other known vulnerabilities
  • Additional considerations

Security best practices for Express applications in production include:

Node.js Security Checklist

Use TLS

  • If your app deals with or transmits sensitive data, use Transport Layer Security (TLS) to secure the connection and the data.
  • This technology encrypts data before it is sent from the client to the server, thus preventing some common (and easy) hacks.
  • Although Ajax and POST requests might not be visibly obvious and seem “hidden” in browsers, their network traffic is vulnerable to packet sniffing and man-in-the-middle attacks.

Use Helmet

  • csp sets the Content-Security-Policy header to help prevent cross-site scripting attacks and other cross-site injections.
  • hidePoweredBy removes the X-Powered-By header.
  • hpkp Adds Public Key Pinning headers to prevent man-in-the-middle attacks with forged certificates.
  • hsts sets Strict-Transport-Security header that enforces secure (HTTP over SSL/TLS) connections to the server.
  • ieNoOpen sets X-Download-Options for IE8+.
  • noCache sets Cache-Control and Pragma headers to disable client-side caching.

Helmet is actually just a collection of smaller middleware functions that set security-related HTTP response headers:

  • noSniff sets X-Content-Type-Options to prevent browsers from MIME-sniffing a response away from the declared content-type.
  • frameguard sets the X-Frame-Options header to provide clickjacking protection.
  • xssFilter sets X-XSS-Protection to enable the Cross-site scripting (XSS) filter in most recent web browsers.

Performance Best Practices

  • Use gzip compression
  • Don’t use synchronous functions
  • Do logging correctly
  • Handle exceptions properly
  • Here are some things you can do in your code to improve your application’s performance:
  • Use gzip compression
  • Don’t use synchronous functions
  • Do logging correctly
  • Handle exceptions properly
  • Set NODE_ENV to “production”
  • Ensure your app automatically restarts
  • Run your app in a cluster
  • Cache request results
  • Use a load balancer
  • Use a reverse proxy

Things to do in your environment / setup (the ops part):

Things to do in your code

Things to do in your code (the dev part):

Don’t use synchronous functions

  • Synchronous functions and methods tie up the executing process until they return.
  • A single call to a synchronous function might return in a few microseconds or milliseconds, however in high-traffic websites, these calls add up and reduce the performance of the app.
  • Avoid their use in production.
  • If you are using Node.js 4.0+ or io.js 2.1.0+, you can use the --trace-sync-io command-line flag to print a warning and a stack trace whenever your application uses a synchronous API.

Do logging correctly

  • Using console.log() or console.error() to print log messages to the terminal is common practice in development.
  • But these functions are synchronous when the destination is a terminal or a file, so they are not suitable for production.

Handle exceptions properly

  • Not handling exceptions and taking appropriate actions will make your Express app crash and go offline.
  • Ensure your app automatically restarts
  • One thing you should not do is to listen for the uncaughtException event, emitted when an exception bubbles all the way back to the event loop.
  • In Express, the best practice is to use the next() function to propagate errors through the middleware chain.

Use try-catch

app.get('/search', function (req, res) {
  // Simulating async operation
  setImmediate(function () {
    var jsonStr = req.query.params
    try {
      var jsonObj = JSON.parse(jsonStr)
      res.send('Success')
    } catch (e) {
      res.status(400).send('Invalid JSON string')
    }
  })
})

Use promises

app.get('/', function (req, res, next) {
  // do some sync stuff
  queryDb()
    .then(function (data) {
      // handle data
      return makeCsv(data)
    })
    .then(function (csv) {
      // handle csv
    })
    .catch(next)
})

app.use(function (err, req, res, next) {
  // handle error
})

However, there are two caveats:

  • All your asynchronous code must return promises (except emitters). If a particular library does not return promises, convert the base object by using a helper function like Bluebird.promisifyAll().
  • Event emitters (like streams) can still cause uncaught exceptions. So make sure you are handling the error event properly; for example:
const wrap = fn => (...args) => fn(...args).catch(args[2])

app.get('/', wrap(async (req, res, next) => {
  let company = await getCompanyById(req.query.id)
  let stream = getLogoStreamById(company.id)
  stream.on('error', next).pipe(res)
}))

Use a load balancer

A load balancer is usually a reverse proxy that orchestrates traffic to and from multiple application instances and servers.

  • An ordinary forward proxy is an intermediate server that sits between the client and the origin server.
  • In order to get content from the origin server, the client sends a request to the proxy naming the origin server as the target.
  • The proxy then requests the content from the origin server and returns it to the client. The client must be specially configured to use the forward proxy to access other sites.
  • A reverse proxy , by contrast, appears to the client just like an ordinary web server. No special configuration on the client is necessary.
  • The client makes ordinary requests for content in the namespace of the reverse proxy.
  • The reverse proxy then decides where to send those requests and returns the content as if it were itself the origin.
  • A typical usage of a reverse proxy is to provide Internet users access to a server that is behind a firewall.
  • Reverse proxies can also be used to balance load among several back-end servers or to provide caching for a slower back-end server.
  • In addition, reverse proxies can be used simply to bring several servers into the same URL space.
Made with Slides.com