The Art  of Node

An Introduction to Node.js





James M. Greene

http://jamesgreene.net/


**  Title used with permission from Max Ogden

About Me



Why I Love Node



  • JavaScript... without browser compatibility concerns!

  • Works cross-platform. Period.

  • A wonderful, active, and fierce community.

  • It enabled us to bring web development to
    the next level in many aspects: real-time,
    tooling, automation, etc.

What is Node?



"Node.js is a platform built on Chrome's JavaScript runtime
for easily building fast, scalable network applications.

Node.js uses an event-driven, non-blocking I/O model
that makes it lightweight and efficient, perfect for
data-intensive real-time applications that run
across distributed devices."

What is Node, really?



Node is...


  • a "server-side JavaScript engine"

  • that runs on all major operating systems

  • and excels at asynchronous I/O

What is Node, really?



Node is not...

  • A  web browser

  • A  panacea  (well, almost...)

  • A "toy" that isn't production ready

  • A  fad

What is Node good for?






Here's a better question....

What is Node bad for?




  • Long-running synchronous processing

  • CPU-intensive operations

...And Everything Else?




Yes, Node is — or can be — good
for  [almost?]  everything else!

What is Node

exceptionally great  for?


  • Data-intensive real-time applications

  • Applications that need to run cross-platform

  • Asynchronous processing

  • The UI Layer of your backend (a.k.a. "the
    middle end"), even/especially if your backend
    business logic services are in another language

Why should I care about Node?




Isn't it only used by "hipster" startups?

I work in the enterprise!

Noteworthy Non-Tech Companies

Using Node in Production


  • Walmart
  • Netflix
  • PayPal
  • eBay
  • Dow Jones
  • LinkedIn
  • New York Times
  • Uber
  • airbnb

But Can it scale?





Oh yes, it can scale.


Where's the Proof?


My favorite example:

"Walmart's Node.js Memory Leak",
a blog post by Node's current
project lead, 
TJ Fontaine.

Walmart's Memory Leak


The Facts:

  • 8 MB memory leak per day

  • 1 Node.js process

  • 4 byte memory leak per session disconnect

Walmart's Memory Leak


How many sessions did this 1 process handle in 1 day?

You do the math:


8 MB (8,388,608  bytes) leak per day
÷
4 bytes leaked per session disconnect
=
______________ session disconnects?

Walmart's Memory Leak



Doing the math with the data provided
shows us that Walmart's

SINGLE (1) Node.js process

handled a total of

2.1 MILLION sessions per day!

So... Can it scale?




How do I use Node?




# Starts a Node REPLnode
# Runs a single command with Nodenode -e "console.log('Node Process ID: ' + process.pid);"
# Run a particular script with Nodenode my_app.js

How should I structure

my Node application?




Modules.

...but what if I am using
framework ABC?




Modules.

...but I AM USING FRAMEWORK XYZ!
It's ""different""!




Modules.

Concept #1: ModuleS



  • CommonJS-esque module system

  • Intentionally small set of core modules

  • Empowers "userland" modules

  • Also allows for native "userland" modules

  • Encourages application composition

The Module System


Importing Modules

// Import the "fs" (file system) module from Node.js core
var fs = require('fs');


// Import the "async" module from Node.js userland // (after running `npm install async`)var async = require('async');

// Import a module relative to the app/module you're working onvar awesomeUtil = require('./util/awesome');

The Module System


Exporting Modules

module.exports = function() {  // A utility method or constructor
};

module.exports = {  // An "API" object};

The Module SyStem


Putting It All Together

// Contents of "util/awesome.js"module.exports = {
  awesome: function() {
    console.log("AWESOME!");
  }
};

// Contents of "app.js"var awesomeUtil = require("./util/awesome");awesomeUtil.awesome();

Don't Reinvent the Wheel




  • Get familiar with the Node core modules

  • Search through the "userland" modules

Where can I find TheSE
""userland"" modules?




They are published to

What is NPM?



Node Package Manager


or, some believe:

Node Packaged Modules


Anyway, just refer to it "NPM".

How do I use NPM?




NPM has many commands available.

Read the docs.

How do I use NPM?


The Basics

# Install a dependency globally
npm install -g grunt-cli            

# Scaffold your new Node module/app (mainly "package.json") npm init
# Install a "dev" dependency locally and update "package.json"npm install --save-dev grunt
# Publish a new module of your own to NPM npm publish

Is there a Node module for ___?




Probably!

Search NPM.

There are currently
~65,000 modules
available!

I Couldn't Find Any!



OK, so make one!  It's easy.

# Scaffold your new Node module (mainly "package.json")npm init

# Code your module up! Tweak "package.json" as you go.

# Share it with the worldnpm publish

Existing Modules For...

Web apps



EXISTING MODULES FOR...

command Line Apps



EXISTING MODULES FOR...

Desktop APPS



EXISTING MODULES FOR...

Build Tools



EXISTING MODULES FOR...

Real-Time Communication



CONCEPT #2: AsynchronousITY



  • Programming in Node is event-driven

  • Event-driven programming is asynchronous

  • The Event Loop manages execution flow

The Event Loop



Node's Event Loop is a construct within the
runtime 
itself, not a library.

This trait is unique to Node.

Other popular event-driven systems in other
language stacks are provided as libraries, e.g.:

  • Ruby’s Event Machine
  • Python’s Twisted

The Event Loop



  • Does not wait for anything to "finish"

  • Waits for everything to "finish"

  • Umm... WAT?

The Event Loop


Does not wait for anything to "finish".


In other words, it spends no time waiting.

It moves on to any other tasks that are
ready to be queued and/or processed.


This is the main key to Node's speed!

The Event Loop


Waits for everything to "finish".


In other words, it is aware of every unresolved
callback (via the V8 garbage collector).

The process will not exit until all callbacks are resolved.

The Event Loop In Practice


// app.js

process.on("exit", function() {
  console.log("Exiting... asynchronous flow");
});

process.nextTick(function() {
  console.log("Next tick, asynchronous flow");
});

console.log("Synchronous flow");

# Results
$> node app.js
Synchronous flow
Next tick, asynchronous flow
Exiting... asynchronous flow

This SoundS Familiar...



  • So now we've seen Node's Event Loop.

  • We know it's part of the runtime, not a library,
    which makes it different than Ruby's Event
    Machine and Python's Twisted.

  • But is it truly unique, as claimed earlier?
    Or does this concept sound vaguely familiar
    to anyone?

Perhaps... Our Old Friends?


Web Browsers



Browsers' Main Threads
are Event Loops!

The Event Loop





For more info on the Event Loop,
check out this StrongLoop blog post:

Beware...


Node's asynchronous nature can lead your

code into what is commonly referred to as


Callback Hell!!!

The Pyramid of DooM


step1(function (value1) {
  step2(value1, function(value2) {
    step3(value2, function(value3) {
      step4(value3, function(value4) {
        // Do something with value4
      });
    });
  });
});

Strategies to Avoid Callback hell



There are a few strategies that you can
employ to successfully avoid callback hell:

  • Don't do anything complex  :)

  • Promises (e.g. Q)

  • Asynchronous workflow libraries (e.g. async)

  • Event emitters  (part of Node core)

Promises

Flatten the pyramid!
Q.fcall(promisedStep1)
  .then(promisedStep2)
  .then(promisedStep3)
  .then(promisedStep4)
  .then(function (value4) {
    // Do something with value4
  })
  .catch(function (error) {
    // Handle any error from all above steps
  })
  .done();

For more on promises, check out this great presentation:

JavaScript Promises by Adam Terlson


Async Workflow Libraries


Like async and its many useful workflow patterns:

async.waterfall([
  function (callback) {
    callback(null, 'one', 'two');
  },
  function (arg1, arg2, callback) {
    callback(null, 'three');
  },
  function (arg1, callback) {
    // arg1 now equals 'three'
    callback(null, 'done');
  }
], function (err, result) {
  // result now equals 'done'    
}); 

Concept #3: Events



  • When discussing the Event Loop, we learned
    that Node follows an event-driven paradigm.

  • As such, it only makes sense that events and
    EventEmitters are offered within Node's
    core modules.

Events & Emitters in the browser


// EventTarget interfaces
document.addEventListener("DOMContentReady", handlerFn, false);

// jQuery events$(".priamry-btn").on("click", handlerFn);

// Unhandled exception handlers (sort of)window.onerror = function(msg, file, line, col, err) { return true;};

Events & Emitters in Node


// Global events
process.on("exit", function() { console.log("Exiting...");});

// Individual objects' eventssocketClient.on("connect", handlerFn);

Custom Emitters in Node

// Create your own Emitters
function MyClass() {}
var util = require('util');var EventEmitter = require('events').EventEmitter;util.inherits(MyClass, EventEmitter);

var instance = new MyClass();instance.on("ping", function(e) { instance.emit("pong");});instance.on("pong", function(e) { console.log("Ping! Pong!");});instance.emit("ping");

Callback Hell is still possible


Depending on the event model, it is still
100% possible for event emitters to
land you in Callback Hell:

range.on("preheat", function() {
  pot.on("boil", function() {
    rice.on("cooked", function() {
      dinner.serve(rice);
    });
  });
}); 

Concept #4: Streams


  • Relying on just the asynchronous APIs for I/O will
    quickly highlight a potential problem: buffering

  • To counter this potential problem, Node provides
    a battle-tested API for "Streams" to allow for
    streaming partial data

  • The Stream interface provides a few basic methods
    and events (it's an EventEmitter) to use/extend

A Common Async Problem


Let's serve a file over HTTP!

var http = require('http');
var fs = require('fs');
var path = require('path');

var server = http.createServer(
  function (req, res) {
    var file = path.join(__dirname, 'data.txt');

    // BUFFERING BUFFERING BUFFERING...
    fs.readFile(file, function (err, data) {
      res.end(data);
    });
  }
);
server.listen(8000);

The Streams Solution


Let's serve a file over HTTP!

var http = require('http');
var fs = require('fs');
var path = require('path');

var server = http.createServer(
  function (req, res) {
    var file = path.join(__dirname, 'data.txt');

    // Open a readable stream
    var stream = fs.createReadStream(file);

    // Stream chunks of bytes as they come
    stream.pipe(res);
  }
);
server.listen(8000);

More On Streams



@substack already did it better:

So... server-side Only, right?




Not exactly. 


There are a few exceptions....

Isomorphic Web Frameworks



Front-end Bundlers



Bonus Topic: Debugging




If time allows....

Debugging Node


Use node-inspector.  Period.

Awesome and easy.

# Installnpm install -g node-inspector

# Runnode-debug -p 8181 app.js

Debugging Talk VIdeo



Chris Rueber gave a talk on "Debugging Node.js"
at the February 2014 meetup for NodeMN. 

Video:

Learning Resources


Questions?




Slides:

The Art of Node

By James M. Greene

The Art of Node

By now, most developers have heard of Node.js — the dominant "server-side JavaScript engine" — but many still haven't worked with it. Come listen, learn, and play along as we explore the world of Node.

  • 4,142