Luciano Mammino (@loige)

Node.js Scalability Tips

Vijayawada - 2021-12-13

@loige

Get the slides! 👇

loige.link/node-adc

HEllo!

👋 I'm Luciano (🇮🇹🍕🍝🤌)

👨‍💻 Senior Architect @ fourTheorem (Dublin 🇮🇪)

nodejsdp.link

📔 Co-Author of Node.js Design Patterns 👉

Let's connect!

loige.co (blog)

@loige (twitter)

loige (twitch)

lmammino (github)

We are business focused technologists that deliver.

Accelerated Serverless | AI as a Service | Platform Modernisation

We are hiring: do you want to work with us?

loige

"A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added"

— Werner Vogels

@loige

🛫
Tip 1.
Establish a baseline

@loige

/?data=ciao👋

Site: qrgen.lmammino.repl.co - Code: replit.com/@lmammino/QRGen

@loige

import { createServer } from 'http'
import QRCode from 'qrcode'

createServer(function handler (req, res) {
  const url = new URL(req.url, 'http://localhost:8080')
  const data = url.searchParams.get('data')
  if (!data) {
    res.writeHead(200, {'Content-Type': 'text/html'})
    return res.end(`<html>
      <body>
        <p>To use this app you need to set the <code>data</code> querystring parameter.</p>
        <p>Try for example <a href="/?data=hello">/?data=hello</a></p>
      </body>
    </html>`)
  }
  res.writeHead(200, { 'Content-Type': 'image/png' })
  QRCode.toFileStream(res, data, { width: 300 })
}).listen(8080)

@loige

autocannon -c 200 --on-port / -- node server.js

wrk

node server.js&
wrk -t8 -c200 -d10s http://localhost:8080/

@loige

autocannon -c 200 --on-port /?data=ciao👋 -- node server.js
Running 10s test @ http://localhost:8080/?data=ciao👋
200 connections

┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐
│ Stat    │ 2.5%    │ 50%     │ 97.5%   │ 99%     │ Avg        │ Stdev   │ Max        │
├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤
│ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │
└─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘
┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐
│ Stat      │ 1%  │ 2.5% │ 50%     │ 97.5%  │ Avg    │ Stdev  │ Min     │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Req/Sec   │ 0   │ 0    │ 30      │ 199    │ 99.5   │ 94.27  │ 30      │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Bytes/Sec │ 0 B │ 0 B  │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │
└───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘

Req/Bytes counts sampled once per second.

995 requests in 10.08s, 1.68 MB read

@loige

⛅️
Tip 1-bis
Also, find out your ceiling

@loige

import { createServer } from 'http'

createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/') {
    res.writeHead(200, { 'Content-Type': 'text/plain' })
    res.end('Hello World\n')
  } else {
    res.statusCode = 404
    res.end()
  }
}).listen(8080)

@loige

autocannon -c 200 --on-port / -- node server.js
Running 10s test @ http://localhost:8080/
200 connections

┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%   │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤
│ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%    │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 21087   │ 21087   │ 34623  │ 35487   │ 33258.4 │ 4107.01 │ 21077   │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB  │ 3.29 MB │
└───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

333k requests in 10.1s, 51.9 MB read

@loige

🍾
Tip 2.
Find your bottleneck

@loige

Clinic.js

clinicjs.org

@loige

clinic doctor --autocannon [ -c 200 '/?data=ciao👋' ] -- node server.js

@loige

clinic flame --autocannon [ -c 200 '/?data=ciao👋' ] -- node server.js

@loige

clinic bubble --autocannon [ -c 200 '/?data=ciao👋' ] -- node server.js

@loige

🎳
Tip 3.
Understand your goals

@loige

What do we optimize for?

Throughput?

Memory?

Latency?

@loige

👁
Tip 4.
Always "observe"

@loige

I mean, in production!

Logs - Metrics - Traces

@loige

🚀
Tip 5.
Scale your architecture

@loige

Performance != Scalability

@loige

How can we scale a system
by adding resources?

@loige

The "Scale Cube"

x-axis

cloning

z-axis

partitioning

y-axis

functional decomposition

@loige

Cloning

Inside the same server

Load Balancer

Using multiple server

@loige

Reverse proxy

The cluster module

Master process

Worker process

@loige

import cluster from 'cluster'
import { cpus } from 'os'

const numCPUs = cpus().length

if (cluster.isMaster) {
  // Fork workers
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork()
  }
} else {
  // Worker code here...
}

@loige

3-4x req/sec

(8 core)