meets


Your host today




Software developer with 15+ years experience
Currently at MediaSmart Mobile
Shameless tinkerer since forever
@pinchito, alexfernandez, alejandrofer

Node.js



A Gentle Introduction

JavaScript on the Server

Chrome JS engine: V8 (en cómic)

Adapted for the server by Ryan Dahl

Free software (know your software)

Supported by Joyent

Running Model

Asynchronous

Non-blocking

Event-driven


Similar servers:

Python: Tornado, Twisted

Ruby: EventMachine

Java: Akka


Source: nodejs.org/about, Wikipedia

Events? Why Not Threads?


A programmer had a problem.

He thought to himself,

"I know, I'll solve it with threads!".

has Now problems. two he

Source: Davidlohr Bueso

Three Generations


Processes: CGI, PHP core


Threads: Apache MTM, Java Servlets


Events: nginx, node.js


Node.js is fast

But is it really fast?


Not when running sequential code

... but it is in concurrent execution


Only one running thread (essentially threadless)

Multi-process by using the cluster module


Node.js is highly scalable

Node.js is very linear

Success Story: PayPal

Account Overview page


2 engineers for 4 months

Developed twice as fast as the Java version

33% less code, 40% less files


Supports twice as many requests per second

Response times 35% faster


Source: Node.js at PayPal

Success Story: MediaSmart Mobile


Initial platform: one engineer for a year

Current platform: three engineers for another year

Serves mobile ads for performance campaigns


Supports 20K requests/second (at least)

Scales to 30 servers (that we know of)

Latency < 80 ms


Source: internal data

Practical Session 1


Create node.js project



Add to GitHub



'Ello world!

Technical Specification

Socket server, port 1702


When a connection is opened, asks: "Hello?"

After receiving "hello" it answers "world" and ends

Any other message should result in "ERROR"


Technical note: careful with your carriage returns!

Always "\r\n".


Doc: node.js net module

Sample Code

var net = require('net');

var server = net.createServer(function(connection) {
console.log('Connection open');
connection.write('Hello?');
connection.on('data', function(data) {
if (String(data).trim() != 'hello') {
connection.write('ERROR');
} else {
connection.write('world');
console.log('Connection closed');
}
});
});
server.listen(port);

It contains three errors

Source: hello-world.js

Success!



Memcached



The key-value in-memory database

What is Memcached?


Created in 2003 by Brad Fitzpatrick for LiveJournal


NoSQL before NoSQL was cool


Key-value


In-memory only storage


Free software of course!

Layered caching

Storage for precomputed values

Almost instantaneous responses (~10 µs)


One of two great inventions in horizontal architectures
(the other one is queue managers)

Basic Memcached API

set [key] [flags] [exptime] [bytes]\r\n
[data block]\r\n
\=> STORED
\=> NOT_STORED

store a value in memory

get [key]* \r\n
\=> VALUE [key] [flags] [bytes]\r\n
[data block]\r\n
END\r\n
extract one (or more) values by key
delete [key]
\=> DELETED
\=> NOT_FOUND
delete a value from memory


Source: protocol.txt

Complete API

get, gets

set, add, replace, append, prepend, cas

delete

incr, decr,

touch

slabs

stats

flush_all, version, quit


Source: protocol.txt

Practical Session 2: Simplecached


Simplified memcached server


get, set, delete


Simplified protocol


Only accepts strings (without line feeds)

Technical Specification

Memcached-type server on port 11311

set [key] [value]\r\n
\=> STORED
\=> NOT_STORED

store a value in memory

get [key]\r\n
\=> VALUE [key] [value]\r\n
\=> END\r\n
extract a value by key
delete [key]\r\n
\=> DELETED
\=> NOT_FOUND
delete a value from memory

Any non-complying message must result in ERROR.

Hints

string.trim(): remove whitespace at the ends

string.split(separator): split a string into an array of pieces

array.slice(begin): return a substring from begin

array.join(separator): join an array into a string


A JavaScript object is an associative map by key


The value can contain spaces


Careful with error cases

Extra Round: Quit Command


Add a quit command:


quit\r\n

closes the client


Hint: connection.end(): closes a connection


Verify how easy it is to modify your software!

Success!


Probably not

NPM


Node Package Manager

Anatomy of a Package (or module)


Read me file: README.md


Definition: package.json


Entry point: index.js


Code: lib/

Documentation: doc/

Binaries: bin/

NPM Commands


  • npm install: install a package or more
  • npm install -g: global installation
  • npm update: update one or more packages
  • npm remove: remove a package
  • npm test: run package tests


Source: npm-index

Directory Resolution

Local:

            node_modules
../node_modules
../../node_modules
...
$HOME/node_modules
./node_modules

Global:

{prefix}/lib/node_modules

where {prefix} is usually /usr/local


Source: npm-folders

aNATOMy of a package.json

{
"name": "simplecached",
"version": "0.0.1",
"description": "Simplified memcached server.",
"contributors": ["Alex Fernández <alexfernandeznpm@gmail.com>"],
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/alexfernandez/simplecached"
},
"dependencies": {
"testing": "0.1.x"
},
"keywords" : ["simplecached", "didactic", "memcached", "caching"],
"bin": {
"simplecached": "bin/server.js"
},
"scripts": {
"test": "node test.js"
},
}

Source: package.json

Practical Session 3: Packages

We will use the simplecached package later on


Install it:

npm install simplecached

Imported with require:

var Client = require('simplecached').Client;

var client = new Client(port);

client.set(key, value, function(error) {});

client.get(key, function(error, value) {}); 

Source: simplecached


Create a package.json


Name and description


Depends on simplecached


Move everything to the correct location:
/lib, /bin, index.js

Use npm


Run npm install without arguments

Run npm list to see what you have installed

Run npm remove simplecached
and then npm list

Publish with npm publish


Take it back! with npm unpublish --force


Success!

Test Your Code

Just Do it!

The value of tests


Customers are happy when your code works


Untested code does not work!

Trust me on this


Do not ever sell, announce or deploy untested code

Do not talk about tests separately from code

Asynchronous Tests


Use testing


Build your async tests


Create a test function for every unit test


Run all tests together

Testing API


testing.success(message, callback); 

consider that the test succeeded

testing.failure(message, callback); 

consider that the test failed

testing.assert(condition, message, callback); 

verify a condition, fail if not true

testing.run(tests, testing.show) 

run a set of tests and show the results

Automate!


Let the machine work for you


Create a control API

Use it in your tests


Three basic rules:

  • One single button
  • Fail fast and loudly
  • No human intervention


Practical Session 4: Tests

The simplecached package contains a client. Use it!


Installation:

npm install simplecached

Imported with require:

var Client = require('simplecached').Client;

var client = new Client(port);

client.set(key, value, function(error) {});

client.get(key, function(error, value) {}); 

Source: simplecached

Technical Specification


  • Open a socket to port 11311
  • Pick a random key and a value containing a space
  • Send the command 'get [key]'
  • Check that it returns nothing ('END')
  • Send 'set [key] [value]', returns 'STORED'
  • Send 'get [key]'; returns the [value] and 'END'
  • Send 'delete [key]', returns 'DELETED'
  • Send 'get [key]', returns 'END'
  • Close the socket


Technical note: commands must be run in sequence

Success!


Bonus Round




Continuations

Continuations


Functional programming concept


CPS: Continuation-passing Style


State is passed around as parameters


Ideal for asynchronous calls

Callback Hell


Pyramid of callbacks


Infinite nested continuations


Abuse of lambdas (unnamed functions)


Hard to follow, debug, refactor


Modern GOTOs?

Simple Solutions


Use named functions


Create an object with named functions

Closures become attributes


Use async.waterfall()


Use EventEmitter

Other Models


Promises


Reactive programming


Yield: almost there


Coroutines (and not goroutines): not yet there


Practical Session 5


Refactor the pyramid of callbacks


Do not keep more than two nested callbacks


Choose your poisonmethod

Success!



Thanks!