Scaling with node
Thierry Nilles
streamroot.io
@thierrynilles
Our experience on growth
What we Do
Classic CDN
with Streamroot
DEmo
demo.streamroot.io
A server? For P2P?
We need to keep track of viewers
So that we can connect them together
First Attempt
The big picture
Tracker
Redis
socket.io
Issues
Tracker going down!
~1000
viewers
So we scaled
Horizontally
Tracker
Tracker
Tracker
HAProxy
Redis
Wait... Trackers should
Talk to each other
Tracker
Tracker
Tracker
Redis
No problem
Use redis built-in pubSUB!
"This guy wants to talk"
got it
got it
Autoscaling
Home-made
Didn't use amazon
Autoscaling
— Why?
— Scales only on simple metrics (cpu, memory)
— What to use instead?
— Custom autoscaling which is based on # of connections per sec
+ # of messages per sec
How we did it
Tracker
Tracker
Tracker
Monitoring agent
Periodically checks trackers' health
Tracker
Runs on node too
Launch new instances
... or terminate them if needed
The only downsiDe
IT
COSTS
MONEY!
$
$
$
CPU Profiling
Fine optimizations
Saving money + time
Setting up the profiler
var agent = require('strong-agent');
agent.metrics.startCpuProfiling();
// Run some code here
setTimeout(function() {
var data = agent.metrics.stopCpuProfiling();
fs.writeFileSync('./CPU-' + Date.now() + '.cpuprofile', data);
}, 1000 * 30);Analyze over 30 seconds
JSON File
How to read
this generated data with chrome
Chrome profiles
(program) = native code + Idle
Flame chart
Testing Methodology
Wrote a client simulator
1. Configured it with the following parameters:
Number of clients to emulate: 500
Connections per sec: 3
2. Run the test with strong agent
3. Run another test increasing the # of connections per sec
4. Repeat until it starts to slow down
What takes time?
What takes time overall:
Our messages size
Is quite big
v=0
o=- 8762946660211567753 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio data
a=msid-semantic: WMS
m=audio 1 RTP/SAVPF 111 103 104 0 8 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:PMG42memfIKiWttR
a=ice-pwd:GhKvhH7GxwXCISLVpcFQL4/Z
a=ice-options:google-ice
a=fingerprint:sha-256 57:6D:FB:99:A0:20:74:50:AB:56:00:90:75:0B:07:53:4E:47:C7:A5:72:6A:7B:8B:2B:32:87:E9:6D:14:F4:06
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=recvonly
a=rtcp-mux
a=rtpmap:111 opus/48000/2
...*for each viewer, several times
Reducing
The messages size
Group the messages together
Send less data
&
Group the messages
Chrome does ICE trickling — around 12 messages
Can be grouped into one bigger message
Results
Without optimization
With optimization
31.87% Idle + Native
63.55% Idle + Native
Group the messages
Messages/Sec
84
168
Nice!
*On my computer
Send less data
We optimized message length
Results
Without optimization
With optimization
31.87% Idle + Native
49.96% Idle + Native
Send less DATA
Messages/Sec
84
112
Total
Messages/Sec
84
196
*On my computer
We tried
To send binary
{
meta: 'foo',
data: 89,
heavy: myBigObject
}VS
3,308 ms
241 ms
{
meta: 'foo',
data: 89,
heavy: [ArrayBuffer]
}on ucs2encode() + ucs2decode() (used by socket.io)
Beware of local optimizations!
Results
Without optimization
With "optimization"
31.87% Idle + Native
1.99% Idle + Native
Binary messages
Messages/Sec
84
84
huh?
*On my computer
Next steps
Find the next bottleneck
Clusterize by client/video/region
Optimize slow node modules
Thank you!
How to scale in node
By Thierry Nilles
How to scale in node
Slides from the December 3rd nodejs meetup in Boston