Luciano Mammino PRO
Cloud developer, entrepreneur, fighter, butterfly maker! #nodejs #javascript - Author of https://www.nodejsdesignpatterns.com , Founder of https://fullstackbulletin.com
Node.js Scalability Tips
Luciano Mammino (@loige)
2020-09-15
👋 Hello, I am Luciano!
Principal Software Engineer at FabFitFun
Get the slides!
Node.js + Scalability? 🤔
🤣
"Scalability is the property of a system to handle a growing amount of work by adding resources to the system"
"A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added"
🛫
Tip 1.
Establish a baseline
/?data=hello%20cityjs
const { createServer } = require('http')
const { URL } = require('url')
const QRCode = require('qrcode')
createServer(function handler (req, res) {
const url = new URL(req.url, 'http://localhost:8080')
const data = url.searchParams.get('data')
if (!data) {
res.writeHead(400) // bad request
return res.end()
}
res.writeHead(200, { 'Content-Type': 'image/png' })
QRCode.toFileStream(res, data, { width: 300 })
})
.listen(8080)
autocannon -c 200 --on-port / -- node server.jsnode server.js&
wrk -t8 -c200 -d10s http://localhost:8080/autocannon -c 200 --on-port /?data=hello%20cityjs -- node server.js
Running 10s test @ http://localhost:8080/?data=hello%20cityjs
200 connections
┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤
│ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │
└─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘
┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │
├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤
│ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │
└───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘
Req/Bytes counts sampled once per second.
995 requests in 10.08s, 1.68 MB read⛅️
Tip 1-bis
Also, find out your ceiling
const { createServer } = require('http')
createServer((req, res) => {
if (req.method === 'GET' && req.url === '/') {
res.writeHead(200, { 'Content-Type': 'text/plain' })
res.end('Hello World\n')
} else {
res.statusCode = 404
res.end()
}
})
.listen(8080)autocannon -c 200 --on-port / -- node server.js
Running 10s test @ http://localhost:8080/
200 connections
┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐
│ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │
├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤
│ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │
└─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │
├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │
└───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘
Req/Bytes counts sampled once per second.
333k requests in 10.1s, 51.9 MB read🍾
Tip 2.
Find your bottleneck
clinic doctor --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node server.js
clinic flame --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node server.js
clinic bubble --autocannon [ -c 200 '/?data=hello%20cityjs' ] -- node server.js
🎳
Tip 3.
Understand your goals
What do we optimize for?
Throughput?
Memory?
Latency?
👁
Tip 4.
Always "observe"
I mean, in production!
Logs - Metrics - Traces
🚀
Tip 5.
Scale your architecture
Performance != Scalability
How can we scale a system
by adding resources?
The "Scale Cube"
x-axis
cloning
z-axis
partitioning
y-axis
functional decomposition
Cloning
Inside the same server
Reverse proxy
Load Balancer
Using multiple server
The cluster module
Master process
Worker process
Worker process
Worker process
const cluster = require('cluster')
const numCPUs = require('os').cpus().length
if (cluster.isMaster) {
// Fork workers
for (let i = 0; i < numCPUs; i++) {
cluster.fork()
}
} else {
// Worker code
require('./server.js')
}
3-4x req/sec
(8 core)
You could also use Worker Threads
Check out piscina!
Cloning is the easiest strategy to scale a service...
... as long as your application is "Stateless"
Functional decomposition
a.k.a. "Micro-services"
API Gateway
/products
/cart
products DB
cart DB
Functional decomposition
a.k.a. "Micro-services"
API Gateway
/products
/cart
Functional decomposition can also be combined with cloning!
products DB
cart DB
Node.js is great for microservices
Microservices can also help with
scaling the organisation!
Microservices add complexity
- Observability
- Deployments
- Versioning
- Integration
Partitioning
Service and Data Partitioning along Customer Boundaries
Shard partitioning
/products/[A-L]/
/products/[M-Z]/
DB 1
DB 2
Partitioning is generally used
to scale databases
and
SaaS software geographically
Summary
🛫 Establish a baseline
🍾 Find your bottleneck
🎳 Understand your goals
👁 Always "observe"
🚀 Scale your architecture
(cloning, decomposition & partitioning)
Thank you!
Special thanks to @StefanoAbalsamo, @matteocollina, @dagonzago, @NullishCoalesce, @DublinSvelte, @KViglucci, @gjohnson391, @lucamaraschi, @laurekamalandua, @giltayar, @mrm8488, @adrirai, @harafise, @EugeneWare, @Jauny, @tlivings, @michaelcfine, @leojino, @shahidontech, @Lordoomer, @zsadigov, @dottorblaster
Icons and SVGs by freepik.com
Node.js: scalability tips - CityJS 2020
By Luciano Mammino
Node.js: scalability tips - CityJS 2020
You finally built that amazing start-up idea you had in mind for years and you did it using Node.js! That's Great! You just launched it on Hacker News and you are very happy and proud... but now more and more people are using it and you start to have a nasty fear that Node.js won't scale because you now... it's single-threaded! Is your project doomed now? Do you have to invest your time on rewriting it in something like C++ or maybe Rust or even Go? You'd rather invest your time on adding valuable features for your users rather than learning a new language and rewriting everything from scratch, but what if nothing works anymore? And... by the way, what the heck "single-threaded" really means?! Fear no more, dear fellow developer! In this talk, we will discuss the architecture of Node.js going through its strengths and its weaknesses. We will then talk about scalability and I will share some valuable tips and tricks to make your Node.js app scale! Spoiler alert: you probably won't need Go or Rust :)
