What is Node.js?

Node.js® is a platform built on  Chrome's JavaScript runtime  for easily building fast, scalable network applications. Node.js uses an event-driven , non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

Source: Node.js

History

• 2009 Created by Ryan Dahl of Joyent 
• 2011 Node.js package manager - npm released
• 2011 Windows version is released
• 2012 npm creator Issac Schlueter became manager
• 2014 io.js is forked from Node.js with open governance
• 2015 Node.js Foundation is founded and manage Node.js and io.js development

Ryan Dahl

Creator of Node.js

Source: Node.js - Wikipedia

Why Node.js?

Node.js is *FAST*

• V8 based JS runtime
• Event driven
• Non-blocking I/O

Source: What Makes Node.js Faster Than Java?

Who is using Node.js?

Source: Node.js Wiki & Node.js Pass Hosting Providers

The Basics

Install Node.js

Download Link: https://nodejs.org/en/download/

Write Your First Node.js App

• Type in following code with any editor you prefer and save as hello.js

• Run the app with node

• Get

console.log('Hello world!');

$ node hello.js
Hello world!

Modulization

• Node.js follows Common Module Definition (CMD)
  • Synchronous loading
  • Export module properties or methods through module.exports
  • Load module through require()
• Support C/C++ addons

• Share and use 3rd-party modules via NPM

Source: Node.js Modules

// FILENAME: mymodule.js
// following assignment can be simplified as
// `exports.sum = ...`
module.exports.sum = function(a, b) {
    return a + b;
};

// FILENAME: test.js
// load mymodule.js
var mymod = require('./mymodule');

// call sum method
console.log(mymod.sum(1, 2));

Node.js API

• Node.js APIs are provided through built-in modules or global objects
• Most APIs provide two forms: sync and async

• See https://nodejs.org/api/ for full Node.js API references

Source: Node.js API

// load file system module
var fs = require('fs');

// read file through sync API
var buf = fs.readFileSync('test.txt');

// read file through async API
fs.readFile('test.txt', function(err, buf) {
// when failed, err will be assigned to Error object.
});

Node.js API - Events

Many objects in Node emit events. For example,

• net.Server emits an event each time a peer connects to it
• fs.readStream emits an event when the file is opened
• ...

EventEmitter API

• emit: emit an event
• on: attach event handler
• once: attach event handler which will be removed after first fired

Source: Node.js Events API

// FILENAME: events-demo.js
var util = require("util");
var events = require('events');

// write your own class
var MyClass = function() {
    // invoke super class constructor
    events.EventEmitter.call(this);
};

// inherit from `events.EventEmitter`
util.inherits(MyClass, events.EventEmitter);

var myobj = new MyClass();

// attach event handler with `EventEmitter.on()`
myobj.on('myEvent', function(msg1, msg2, msg3) {
    console.log(msg1, msg2, msg3);
});

// emit "myEvent" with message
myobj.emit('myEvent', '1st', '2nd', '3rd');

Node.js API - Timers

• Node.js provides all standard JS timer functions: setTimeout / clearTimeout / setInterval / clearInterval.

• process.nextTick - Once the current event loop turn runs to completion, call the callback function. This is NOT a simple alias to setTimeout(fn, 0), it's much more efficient. It runs before any additional I/O events (including timers) fire in subsequent ticks of the event loop.
• Node.js will NOT exit if there are timers in event loop

Source: process.nextTick

// FILENAME: timers.js

setTimeout(function() {
    console.log('setTimeout(0)');
}, 0);

setInterval(function() {
    console.log('setInterval(1s)');
}, 1000);

process.nextTick(function() {
    console.log('nextTick');
});

$ node timers.js
nextTick
setTimeout(0)
setInterval(1s)
setInterval(1s)
setInterval(1s)
setInterval(1s)
setInterval(1s)

Node.js API - Buffer

Pure JavaScript is Unicode friendly but not nice to binary data. When dealing with TCP streams or the file system, it's necessary to handle octet streams. Node has several strategies for manipulating, creating, and consuming octet streams.

Raw data is stored in instances of the Buffer class. A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. A Buffer cannot be resized.

Converting between Buffers and JavaScript string objects requires an explicit encoding method. 'utf8' is an encoding for multibyte encoded Unicode characters. Many web pages and other document formats use UTF-8.

Source: Node.js Buffer

Node.js API - Stream

A stream is an abstract interface implemented by various objects in Node.

• Readable stream is a source of data that you are reading from. e.g. http request on server
• Writable stream is a destination that you are writing data to. e.g. http response on server
• Duplex stream is bidirectional stream implementing both Readable and Writable interfaces. e.g. tcp sockets
• Transform stream is a Duplex stream where the output is in some way computed from the input. e.g. crypto stream

Source: Node.js Streams

Example - HTTP Server

Source: Nodejs.org

// FILENAME: http-server.js
// load http module
var http = require('http');
// create an HTTP server
var server = http.createServer();
// attach the handler to 'request' event of server
server.on('request', function (req, res) {
  // write Content-Type header to the response with status code 200 - OK
  res.writeHead(200, {'Content-Type': 'text/plain'});
  // `res.end` write to the response and close the response stream
  res.end('Hello World\n');
});
// bind the server and it will block the exiting
server.listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');

$ node ./http-server.js
Server running at http://127.0.0.1:1337/

Launch server with Node.js

Access the server by browser with http://localhost:1337/

Node Package Manager - NPM

Source: about npm

• npm is the package manager for Node.js. It was created in 2009 as an open source project to help JavaScript developers easily share packaged modules of code.
• The npm Registry is a public collection of packages of open-source code for Node.js, front-end web apps, mobile apps, robots, routers, and countless other needs of the JavaScript community.
• npm is the command line client that allows developers to install and publish those packages.
• npm, Inc. is the company that hosts and maintains all of the above.

What is NPM?

package.json

Source: npm package.json

{
    "name": "my-pkg",
    "versoin": "1.0.1",
    "main": "main.js",
    "description": "This is my first npm package :)",
    "dependencies": {
        "ws": "0.7.2"
    }
}

• package.json - metadata of npm package in JSON format
  • name (required) - name of the package
  • version (required) - version
  • main (optional) - entrance file of the package used by require(module_name)
  • dependencies (optional) - list of 3rd-party npm packages and versions that your package depends on.
  • description (optional) - description of the package
  • ...​​

Usage of NPM Modules

• Installation: you don't need to install npm beause npm is installed as part of Node.js
• Use npm CLI
  • npm init <folder> - initialize a local npm module. It can help to create a package.json from step-by-step guide.
  • npm install [[module_name] [--save]] - install module_name. If module_name is not provided, 3rd-party modules listed in `dependencies` field of package.json in current folder will be installed recursively. They will be installed in "node_modules" folder relative to `package.json`. Once `--save` is given, the installed module with corresponding version will be updated to `package.
  • npm rm <module_name> - remove installed modules
  • npm search <module_name> - search for 3rd-party modules from npm registry
• ​Use npm modules in your code
  • `require(module_name)` will search for module_name in `node_modules` in current and all parent folders

Source: npm package.json

Publish You Module 

Creating a user

• If you don't have one, create it with npm adduser.
• If you created one on the site, use npm login to store the credentials on the client.
• Test: Use npm config ls to ensure that the credentials are stored on your client. Check that it has been added to the registry by going to http://npmjs.com/~.

Publishing the package

• Use npm publish to publish the package.
• Test: Go to http://npmjs.com/package/<package>. You should see the information for your new package.

Source: Publishing NPM Modules

Update You Module 

Updating the package

• When you make changes, you can update the package using npm version <update_type>, where update_type is one of the semantic versioning release types, patch, minor, or major. This command will change the version number in package.json. Note that this will also add a tag with this release number to your git repository if you have one.
• After updating the version number, you can npm publish again.
• Test: Go to http://npmjs.com/package/<package>. The package number should be updated.

Source: Publishing NPM Modules

Async Programming

Differences of Sync and Async API

Text

When Use Node.js API

• Always use async API on server side scripting

• Always check errors first in async callback

Note

// FILENAME: async-api.js
var fs = require('fs');

// 1. use async API
fs.raedFile('some/file', function(err, buf) {
    // 2. process error
    if (processError(err)) return;

    // put your logic here
});

function processError(err) {
    if (err) {
        console.error(err);
        return true;
    } else {
        return false;
    }
}

Nested Callback is Harmful

// FILENAME: nested-callback.js
var fs = require('fs');

fs.readdir('some/dir', function(err, files) {
    // LEVEL - 1
    // process err
    if (processError(err)) {
        return;
    }

    files.forEach(function(file) {
        fs.stat('some/dir/' + file, function(err, stats) {
            // LEVEL - 2
            // process err
            if (processError(err)) {
                return;
            }

            if (stats.isFile()) {
                fs.readFile('some/dir/' + file, function(err, buf) {
                    // LEVEL - 3
                    if (processError(err)) {
                        return;
                    }

                    // dare you put more nested callback here?

                }); // fs.readFile
            }
        }); // fs.stat
    }); // files.forEach
    
}); // fs.readdir

Avoid Nested Callback

Best Known Methods

// FILENAME: async-bkm1.js
var fs = require('fs');

// 1. use async API
fs.readdir('some/dir', function(err, files) {
    // 2. process error
    if (processError(err)) return;

    // 3. avoid too much nested callbacks ...
    files.forEach(function(file) {
        statFile(file);
    });
});

// 3. ... and do another async call in a separate function
function statFile(file) {
    fs.statFile('some/dir/' + file, function(err, stats) {
        // 2. process error
        if (processError(err)) return;

        if (stats.isFile()) readFile(file);
    });
}

// 3. ... and do another async call in a separate function
function readFile(file) {
    fs.readFile('some/dir/' + file, function(err, buf) {
        // process error
        // and put your logic below
    });
}

by refactoring into separate functions

Best Known Methods

Avoid Nested Callback

by using Promise

Promise

Source: Promise

What is Promise?

Concept

A Promise represents a proxy for a value not necessarily known when the promise is created. It allows you to associate handlers to an asynchronous action's eventual success value or failure reason.

 

This lets asynchronous methods return values like synchronous methods: instead of the final value, the asynchronous method returns a promise of having a value at some point in the future.

 

Promise is a standard object since of ES6.

Source: Promise

The States of a Promise

Concept

A Promise is in one of these states:

• pending: initial state, not fulfilled or rejected.
• fulfilled: meaning that the operation completed successfully.
• rejected: meaning that the operation failed.

Source:

Promise API

• Constructor
  • new Promise(function(resolve, reject) {  }) - Create a promise with an executor, which is a function object with two arguments resolve and reject. The first argument fulfills the promise, the second argument rejects it. We can call these functions, once our operation is completed.
• Methods
  • promise.then(onFulfilled, onRejected) - Appends fulfillment and rejection handlers to the promise, and returns a new promise resolving to the return value of the called handler
  • promise.catch(onRejected) - Appends a rejection handler callback to the promise, and returns a new promise resolving to the return value of the callback if it is called, or to its original fulfillment value if the promise is instead fulfilled.

Source: Promise

Promise API (cont.)

• Static Methods
  • Promise.all(iterable) - Returns a promise that resolves when all of the promises in the iterable argument have resolved. This is useful for aggregating results of multiple promises together.
  • Promise.race(iterable) - Returns a promise that resolves or rejects as soon as one of the promises in the iterable resolves or rejects, with the value or reason from that promise.
  • Promise.reject(reason) - Returns a Promise object that is rejected with the given reason.
  • Promise.resolve(value) - Returns a Promise object that is resolved with the given value.
    • If the value is a thenable (i.e. has a then method), the returned promise will "follow" that thenable, adopting its eventual state;
    • otherwise the returned promise will be fulfilled with the value. Generally, if you want to know if a value is a promise or not - Promise.resolve(value) it instead and work with the return value as a promise.

Source: Promise

Write and Use Promise

// FILENAME: promise-simple.js
console.log('before promise');

new Promise(function(resolve, reject) {
    resolve(1234);
}) // p1
.then(function(val) { // onResolve
    console.log('p1 resolved with ' + val);
}, function(err) {  // onReject
    console.error('you should not see this because ' +
                  'no ops rejected previously');
}); // p2

console.log('after promise');

$ node promise.js
before promise
after promise
p1 resolved with 1234

• "after promise" is printed before promise output because promise is async

Write and Use Promise

// FILENAME: promise.js
new Promise(function(resolve, reject) {
    // delay 2 seconds
    setTimeout(function() {
        // ... then resolve with 1234
        resolve(1234);
    }, 2000);
}) // p1
.then(function(val) {
    console.log('p1 resolved with ' + val);
    // return a bare value "abcd"
    return 'abcd';
}) // p2
.then(function(val) {
    console.log('p2 resolved with ' + val);
    // return a thenable in rejected state
   return Promise.reject('bad things happens in p3');
}) // p3
.then(function(val) {
    console.log('you should not see this because ' +
                'previous ops rejected');
}) // p4
.catch(function(reason) { // onReject
    // error happens
    console.error('rejected with reason: '
                + reason);
    // explicitly return a thenable in resolved state
    return Promise.resolve(Math.random());
}) // p5
.then(function(val) {
    console.log('p5 resolved with ' + val);
}, function(err) {  // onReject
    console.error('you should not see this because ' +
                  'no ops rejected previously');
}); // p6

• p1 - is resolved after 2s delay
• p2 - returns bare value which is treated as resolved and then passed to p3
• p4 - is skipped because p3 returns a rejected thenable
• p5 - any rejected values will fall to next nearest onReject callback of .then() or .catch()
• p5 - returning of .catch() is a resolved value passed to p6

Rewrite Async Functions with Promise

// FILENAME: async-callback.js
function async(value, callback) {
    // do some stuff

    // if fail, invoke callback with
    // the error
    var err = hasError();
    if (err) {
        callback(err);
    }

    // and if success, invoke callback
    // with result
    callback(null, result);
}

// FILENAME: async-promise.js
// 1. No need to pass callback as argument
function async(value) {
    // 2. return the promise object
    return new Promise(function(resolve,
                                reject) {
        // do some stuff

        // 3. if fail, invoke reject with
        // the error
        var err = hasError();
        if (err) {
            reject(err);
        }

        // 4. and if success, invoke resolve
        // with the result
        resolve(result);
    });
}

1. No need to pass callback as argument
2. Return a Promise object
3. Invoke reject with error reasons when fail
4. Invoke resolve with result when succeed

Reduce Nested Callbacks

// FILENAME: async-nested-callback.js
// async1, async2, async3 invoke callback
// when it's done
async1(function(err, value1) {
    if (processError(err)) return;

    async2(value1, function(err, value2) {
        if (processError(err)) return;

        async3(value2, function(err, value3) {
            if (processError(err)) return;
            // ...
        });
    });
});

// FILENAME: async-with-promise.js
// async1, async2, async3 returns a resolved
// Promise on success or a rejected Promise
// on failure
async1()
.then(function(value1){
    return async2(value1);
})
.then(function(value2){
    return async3(value2);
})
.then(function() {
    // ...
})
.catch(processError);

Nested callbacks

Chainable callbacks

Reduce Nested Callbacks (cont.)

Final Version

// FILENAME: async-with-promise-final.js
// async1, async2, async3 returns a resolved
// Promise on success or a rejected Promise
// on failure
async1()
.then(async2)
.then(async3)
.then(function() {
    // ...
})
.catch(processError);

Checklist for Using Promise

• A promise should eventually be fulfilled or rejected - do NOT leave a promise pending forever

• A promise should be returned or handled with onReject of .then() or .catch() - make sure rejected promises are handled somewhere

Best Known Methods

Q

Source: Q on Github

A tool for creating and composing asynchronous promises in JavaScript. -- by Kris Kowal

The Q module can be loaded as:

• A <script> tag (creating a Q global variable) in browser
• A Node.js and CommonJS module, available in npm as the q package
• An AMD module by requirejs
• A component as microjs/q
• Using bower as q#1.0.1
• Using NuGet as Q

Q as Helpers of Node.js API

Q provides helper functions to wrap async Node.js APIs into promise-styled API

• Q.nfbind(nodeFunc, ...args)
• Q.nbind(nodeMethod, thisArg, ...args)
• Q.nfapply(nodeFunc, args)
• Q.nfcall(func, ...args)
• Q.npost(object, methodName, args)
• Q.ninvoke(object, methodName, ...args)

Source: Q API Reference

var fs = require('fs');
var Q = require('q');

// wrap and invoke `fs.readFile` to make it
// return a promise
Q.nfapply(fs.readFile, ['some.txt', 'utf-8'])
.then(function(txt) {
    console.log(txt);
}, function(err) {
    console.error(err);
});

Questions?

Node.js

• Node.js is a server side scripting language

• Use and share 3rd-party modules with NPM

• Asynchronous I/O makes Node.js fast

• Use Promise to avoid dirty nested callbacks