Node.js Scalability Tips

Luciano Mammino (@loige)

2020-09-15

👋 Hello, I am Luciano!

Principal Software Engineer at FabFitFun

Blog: loige.co

Twitter: @loige

GitHub: @lmammino

nodejsdp.link/buy

CITYJS SPECIAL DISCOUNT: 20% OFF on Amazon.com - www.amazon.com/gp/mpc/A2L4T1QGT3DXYX

@loige

Get the slides!

loige.link/node-scale-city

Node.js + Scalability? 🤔

@loige

🤣

@loige

"Scalability is the property of a system to handle a growing amount of work by adding resources to the system"

— Wikipedia

@loige

"A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added"

— Werner Vogels

@loige

🛫
Tip 1.
Establish a baseline

@loige

/?data=hello%20cityjs

https://repl.it/@lmammino/QRGen

@loige

const { createServer } = require('http')
const { URL } = require('url')
const QRCode = require('qrcode')

createServer(function handler (req, res) {
  const url = new URL(req.url, 'http://localhost:8080')
  const data = url.searchParams.get('data')
  if (!data) {
    res.writeHead(400) // bad request
    return res.end()
  }
  res.writeHead(200, { 'Content-Type': 'image/png' })
  QRCode.toFileStream(res, data, { width: 300 })
})
  .listen(8080)

@loige

autocannon -c 200 --on-port / -- node server.js

wrk

node server.js&
wrk -t8 -c200 -d10s http://localhost:8080/

@loige

autocannon -c 200 --on-port /?data=hello%20cityjs -- node server.js
Running 10s test @ http://localhost:8080/?data=hello%20cityjs
200 connections

┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐
│ Stat    │ 2.5%    │ 50%     │ 97.5%   │ 99%     │ Avg        │ Stdev   │ Max        │
├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤
│ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │
└─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘
┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐
│ Stat      │ 1%  │ 2.5% │ 50%     │ 97.5%  │ Avg    │ Stdev  │ Min     │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Req/Sec   │ 0   │ 0    │ 30      │ 199    │ 99.5   │ 94.27  │ 30      │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Bytes/Sec │ 0 B │ 0 B  │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │
└───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘

Req/Bytes counts sampled once per second.

995 requests in 10.08s, 1.68 MB read

@loige

⛅️
Tip 1-bis
Also, find out your ceiling

@loige

const { createServer } = require('http')

createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/') {
    res.writeHead(200, { 'Content-Type': 'text/plain' })
    res.end('Hello World\n')
  } else {
    res.statusCode = 404
    res.end()
  }
})
  .listen(8080)

@loige

autocannon -c 200 --on-port / -- node server.js
Running 10s test @ http://localhost:8080/
200 connections

┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%   │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤
│ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%    │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 21087   │ 21087   │ 34623  │ 35487   │ 33258.4 │ 4107.01 │ 21077   │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB  │ 3.29 MB │
└───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

333k requests in 10.1s, 51.9 MB read

@loige

🍾
Tip 2.
Find your bottleneck

@loige

Clinic.js

clinicjs.org

@loige

clinic doctor --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node server.js

@loige

clinic flame --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node server.js

@loige

clinic bubble --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node server.js

@loige

🎳
Tip 3.
Understand your goals

@loige

What do we optimize for?

Throughput?

Memory?

Latency?

@loige

👁
Tip 4.
Always "observe"

@loige

I mean, in production!

Logs - Metrics - Traces

@loige

🚀
Tip 5.
Scale your architecture

@loige

Performance != Scalability

@loige

How can we scale a system
by adding resources?

@loige

The "Scale Cube"

x-axis

cloning

z-axis

partitioning

y-axis

functional decomposition

@loige

Cloning

Inside the same server

Reverse proxy

Load Balancer

Using multiple server

@loige

The cluster module

Master process

Worker process

@loige

const cluster = require('cluster')
const numCPUs = require('os').cpus().length

if (cluster.isMaster) {
  // Fork workers
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork()
  }
} else {
  // Worker code
  require('./server.js')
}

3-4x req/sec

(8 core)

@loige