10 Hours of NodeJs

 

Introducing NodeJS in 10 hours

What are we covering?

  • HOUR 01 - Introduction
  • HOUR 02 - Understanding Modules
  • HOUR 03 - Event Loop & Event Emitter
  • HOUR 04 - File System and Buffers
  • HOUR 05 - Networking
  • HOUR 06 - Streams
  • HOUR 07 - Unit Testing !
  • HOUR 08 - Writing a web App - Part 1
  • HOUR 09 - Writing a web App - Part 2
  • HOUR 10 - Wrap Up

Introduction

Hour 01

What is NodeJS?

  • NodeJS is an open-source, cross-platform JavaScript runtime environment based on Chrome's V8 JavaScript Engine
  • NodeJS applications are written in JavaScript Language and can run on many different environments supported by the NodeJS runtime (OS X, Windows,  Linux, Free BSD & many more)

 

Why NodeJS

  • Speaks the lingua franca of the web!
  • Single language for the server and front-end
  • Dynamically Typed
  • Speed & non blocking I/O
  • Massive Ecosystem (npm modules ~198k packages)
  • Vibrant Community
  • Cross Platform (FreeBSD, Linux, OS X, Windows)
  • Seriously Productive
  • Builtin support for realtime and streaming
  • Fun, Easy and Enjoyable.

http://www.toptal.com/nodejs/why-the-hell-would-i-use-node-js

Where is node used today?

  • Netflix
  • eBay
  • New York Times
  • PayPal
  • Medium
  • Linkedin
  • Uber
NodeJS in Industries - From RisingStack

NodeJs Use Cases

  • Rest API's
  • Streaming & Realtime Applications
  • Command Line Tools

 

When to avoid?

  • CPU Intensive Applications
  • RDBMS Data Stores with lots of de-normalized data
  • Web applications with lots of server side rendering

 

Node Bindings (Socket Library, HTTP etc) 

Node Standard Library

Event Loop

(libev)

V8

Thread Pool

(libeio)

DNS

(c-ares)

Crypto

(OpenSSL)

JavaScript

C++

  • Single Threaded
  • Event Loop
  • Non Blocking IO
  • Excellent Support for HTTP

 

Installing NodeJS & NPM - USING NVM

# https://github.com/creationix/nvm

# For Linux & Mac OS
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.29.0/install.sh | bash
or Wget:
wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.29.0/install.sh | bash




# For Windows Machine
nvmw - https://github.com/hakobera/nvmw
nvm-windows - https://github.com/coreybutler/nvm-windows (has a windows installer)

NodeJS Development Environment

  • Get a good shell (if windows!)
  • Install Git (Git Bash)
  • A decent text editor (Atom, Sublime Text, Notepad++)
  • Install jshint - {npm install -g jshint}
  • Master the shell!

All About npm

  • NPM stands for Node Package Manager
  • Written by Isaac Schlueter
  • Manages 3rd party package installation
  • From 0.6.0 version of nodejs comes preinstalled
  • Used for initializing the nodejs applicaitons
  • Keeping track of out dated dependencies

NPM Commands

# npm commands

npm --version                            # displays the version of npm
npm init                                 # inits a new module with a package.json file
npm install <module>                     # installs the module and all its dependency
npm install -g                           # globally installs the module
npm install [-g] <module>  --save        # saves the module to the package.json dependencies section
npm install [-g] <module>  --save-dev    # saves the module to the package.json dev-dependencies section
npm ls -g                                # list global modules
npm ls <module>                          # list by a module bane
npm rm <module> # remove (alias)         # remove an installed module
npm outdated                             # list outdated modules
npm view                                 # view details of an installed module
npm prune                                # removes extraneous (unwanted) packages
npm search                               # search public packages from npm registry
npm update                               # updates all dependent modules based changes to package.json

npm help                                 # help on npm

Package.json

# Package.json, is present in the root directory of the nodejs applicaiton. It defines the property of the application / package.

# An npm init command generates a package.json file by interactively asking a set of questions

# Attributes of package.json

name             - name of the package
version          - version of the package
description      - description of the package
homepage         - homepage of the package
author           - author of the package
contributors     - name of the contributors to the package
dependencies     - list of dependencies. dependencies mentioned here in are installed to node_module folder of the package.
devDependencies  - 
repository       - repository type and url of the package
main             - entry point of the package
keywords         - keywords
scripts          - scripts to be run during various times of the project's lifecycle (example, start, test, pack etc)
config           - set configuration parameters used in package scripts

jshint

# JSHint is a static code analysis tool for JavaScript


# installation

    npm install -g jshint || npm i -g jshint

# or install it as your dev dependency

    npm install jshint --save-dev


# configuration (customize jshint)

    1)  --config <file>
    2)  .jshintrc
    3)  jshintConfig 
 

Example - Basic

mkdir hello-node && cd $_
npm init

# answer the questions asked by npm init
# touch index.js

    "use strict";

    console.log("Hello, world from nodejs");

# run the program
    node index.js

# Add the start up script
"scripts": {

    "start": "node index.js"
}

# Run using scripts:start attribute

npm start

Example - Advanced

# touch server.js

"use strict";

var http = require("http");
var port = process.env.PORT || 8888;

http.createServer( function (req, res) {
	res.writeHead(200, {"Content-Type": "text/plain"});
	res.write("Hello, World from nodejs");
	res.end();
}).listen(port);

console.log("HTTP Server listening to port :" + port);

Example - Non Blocking I/O

# touch nonBlockingDemo.js

"use strict";

var fs = require('fs');

fs.readFile('read-nb.txt', 'utf8', function(err, contents) {

	console.log(contents);
});

console.log('Hello there, the file is being read asynchronously');


Extras - Closures

# touch no-closure.js

"use strict";


function sayHello (message) {
    console.log(message);
}


function createMessage(name) {
    return "Hi " 
    + name 
    + " welcome to Node in 10 hours training!";
}

var message = createMessage('Jack');

sayHello(message);
# touch closure.js

"use strict";

// Define the closure
function HelloMaker (name) {

    var message = 'Hi ' 
    + name 
    + ' welcome to Node in 10 hours training!';

    return function sayHello() {
        console.log(message);
    }
}

// Create the closure
var helloJack = HelloMaker('Jack');

// Use the closure
helloJack();

Understanding Modules

Hour 02

What are NodeJS Modules?

  • Brings the concept of a 'Standard Library' to NodeJS
  • Allows developers to extend NodeJS through custom modules
  • Implements the CommonJS modules standard
  • A module is defined in a single file (one to one correspondence with file)
  • A module can export public functions which can be used by other modules
  • Variables local to the module are private to the module

CommonJS

  • Is a standard specification for modualrizing JavaScript
  • CommonJS does not provide any libraries of its own
  • Specifies require() and an exports object which work in conjunction to provide the module 
  • NodeJS standard library and all 3rd Party libraries use CommonJS standard to modularize the code 

Consuming Modules

To consume an existing module use the require function

// Require a code module or a module installed by NPM
var module = require('module-name');


// Require a module relative to current path
var my-module = require('./my-module');

Defining Modules

To define a module, use exports global variable

 

exports

vs

module.exports

/**
* File: my-math.js
* Style 1 - Export single function
*/

var square = function (n) { return n*n;};

exports.square = square

///////////////////////////////////////////////////////

/**
* File: my-math-lib.js
* Style 2 - Set of functions
*/

var square = function (n) { return n*n;};
var cube = function(n) {return square(n) * n;};

exports.square = square;
exports.cube = cube;

//////////////////////////////////////////////////////

/**
* File: my-math-lib-module.js
* Style 3 - Set of functions in module style
*/

var square = function (n) { return n*n;};
var cube = function(n) {return square(n) * n;};

module.exports = {
    square:square;
    cube: cube;
};

///////////////////////////////////////////////////////

/**
* File employee.js
* Style 4 : Object Constructor
*/

module.exports = function (firstname, lastname, department) {
    this.firstname = firstname;
    this.lastname = lastname;
    this.department = department;

    this.print = function() {
        return firstname + ' ' + lastname + ' - ' + department;
    }
};

/**
* file employee-service.js
*/
var Employee = require ('../models/employee');

var employee1 = new Employee('Jack', 'Peters', 'Mechanical');
console.log(employee1.print());

Resolving Modules

Core Modules
    binary modules - from lib folder
    Preferentially loaded over user defined names

File Modules
    Prefixed with /, ./ or ../
    If exact name is not found appends .js, .node or .json and does the search. 
    Throws exception if module is not found

node_modules folder
    If not available in native modules and no prefix such as /, ./, ../ etc

Folders
    Loads from main entry in package.json
    If package.json is not available in the folder, 
    tries with /index appended to the end of the require string [.js,  .node, .json]

node_modules folder

  • automatically created by npm when a module is installed
  • node_modules are not checked into version control system
  • They maintain deeply nested folder structure to manage each package dependency 
  • Due to deep nesting, it does not work very well Windows

Creating and Publishing npm modules

Modules Best Practices

Event Loop & Event Emitter

Hour 03

What is an Event Loop?

  • A software pattern
  • Facilitates async I/O
  • In NodeJS Event Loop is core concept for handling high throughput
  • Though event loop helps in asynchronous programming the loop itself is synchronous
  • An event loop in NodeJS continues to run until the last callback is completed

Event Loop in NodeJS

  • Depends on multiple libraries (libev)
  • Used OS efficiently to hold connections & resources
  •  

Traditional Vs Event Driven I/O

// Traditional Model
posts = db.query("SELECT * from posts WHERE author = '%john%'");

formattedPosts = formatPosts(post);

sendPosts(formattedPosts);
// create a callback
var postHandler = function(err, posts) {
    formatPosts(posts, function (err, formattedPosts) {
        sendPosts(formattedPosts, function(err) {
            //Handle Error
        };
    });
};

// var formatPosts = function (posts, callback) {};
// var sendPosts = function (formattedPosts, callback) {};

// Query the database with a callback
db.query("SELECT * FROM posts WHERE author ='%jack%'", postHandler);

Blocking the Event Loop


// Guess what happens here?

var opened = false;

setTimeout(function() {
    opened = true;
}, 1000);

while (!opened) {
    // wait till it is opened
}

console.log('Opened now!');

Event Emitter

  • Many NodeJS objects emit event
  • All objects that emit events are instances of events.EventEmitter
  • Examples
    • TCP Server emit 'connect' every time a client connects
    • ReadStream can emit 'data' each time data is available for reading

Event Emitter - Methods

Method Description
.addListener
.on
Adds an event listener
.once Listens exactly once
.removeListener Removes an event listener for an event
.removeAllListeners Removes all listeners for an event
.emit Emits an event

Example - Add Listener

"use strict";

var rl = require('readline').createInterface({
  input: require('fs').createReadStream('readme.txt')
});

rl.on('line', function (line) {
    console.log('Line from file:', line);
});

rl.on('close', function () {
    console.log('Closed::The end');
});

Example - Once

// Manual Method

var events = require('events'),
                emitter = new events.EventEmitter();
function listener() {
        console.log('one Timer');
        emitter.removeListener('oneTimer', listener);
}
emitter.on('oneTimer', listener);
emitter.emit('oneTimer');
emitter.emit('oneTimer');


// Using Once

var events = require('events'),
                emitter = new events.EventEmitter();
/* Use Once */
emitter.once('onceOnly', function() {
        console.log('one Only');
});
emitter.emit('onceOnly');
emitter.emit('onceOnly');

Example - Create EventEmitter

var EventEmitter = require('events').EventEmitter;
var util = require('util');

var Person = function (firstname, lastname) {
    this.firstname = firstname;
    this.lastname = lastname;
};

util.inherits(Person, EventEmitter);

// Create an method that emits an event
Person.prototype.sayHello = function () {
    this.emit('greet', {firstname: this.firstname, lastname: this.lastname});
};


// Create an instance of person and say hello

var p = new Person('Peter', 'Parker');

p.on('greet', function (data){
    console.log('Got a greet event from ' + data.firstname); 
});

p.sayHello();

Example - Emit Frequently

"use strict";

var util = require('util'),

EventEmitter = require('events').EventEmitter;

var Ticker = function(limit){ 
    var self = this; 
    
    this.limit = limit || 10;
    this.counter = 0;

    setInterval(function() {
        self.emit('tick');
        self.counter += 1;

        if(self.counter === self.limit) {
            console.log('limit reached -- removing event listener');
            self.emit('limit');
        }

    }, 1000);
};

util.inherits(Ticker,EventEmitter)

var ticker = new Ticker(); 

var tickListener = function(){
  console.log('TICK');
};

ticker.on('tick',tickListener);

ticker.on('limit', function(){
    ticker.removeListener('tick', tickListener);
    ticker.removeAllListeners('limit');
})

// As an exercise, remove the event listener after 100 ticks

Filesystem & Buffers

Hour 04

Buffers

  • JavaScript cannot natively handle binary data
  • JavaScript is good at handling unicode encoded strings
  • Server side javascript needs to handle various binary formats 
  • Initially NodeJS used strings, but this lead to problems, hence Buffer was introduced
  • Buffer is like an integer array, with its memory allocated outside V8 engine 
  • Buffers cannot be resized.
  • To convert a buffer to JavaScript object and vice versa, we must provide an encoding
  • Supported encodings are 'ascii', 'utf-8', 'utf16le', 'ucs2', 'base64' and 'binary'

Creating Buffers


// Couple of ways to create buffers

var buffer1 = new Buffer(1024);

// Defaults to utf-8
var buffer2 = new Buffer ('Hello, Friends!');

var buffer3 = new Buffer ([8, 6, 7, 5, 3, 0, 9]);

console.log(buffer1.toString('utf-8'));
console.log('\n');
console.log(buffer2.toString('utf-8'));
console.log('\n');
console.log(buffer3.toString('utf-8'));

Buffer Operations

// Write to a buffer

var buf = new Buffer(100);

buf.write('Hello, World!');
buf.write(' John!', 13);


// Reading from buffer
console.log(buf.toString());

// Reading from buffer like an array
console.log(buf[10].toString());


// Check if the buffer is actually a buffer
console.log("Buffer.isBuffer  " + Buffer.isBuffer(buf));

// Check bytelength required to encode
snowman = "☃";
console.log('Snowman length: ' + snowman.length);
console.log('Buffer Required for snowman: ' + Buffer.byteLength(snowman));

// Check the length of the buffer

console.log('Buffer length is ' + buf.length);

Slice & Copy Buffer

// Slice a buffer
// Slice does not create new memory, it uses the original memory underneath

var buffer = new Buffer ('this is the string in my buffer');
var slice = buffer.slice(10,20);

console.log(slice.toString());


// Copy a buffer
var copy = new Buffer(10);
var targetStart = 0,
    sourceStart = 10,
    sourceEnd = 20;

buffer.copy(copy, targetStart, sourceStart, sourceEnd);

console.log(copy.toString());

// manupulate the slice
slice.write('_-_-_-_-_-');

console.log(buffer.toString());

Buffer Exercise

// Slice a buffer
// Slice does not create new memory, it uses the original memory underneath

var buffer = new Buffer ('this is the string in my buffer');
var slice = buffer.slice(10,20);

console.log(slice.toString());


// Copy a buffer
var copy = new Buffer(10);
var targetStart = 0,
    sourceStart = 10,
    sourceEnd = 20;

buffer.copy(copy, targetStart, sourceStart, sourceEnd);

console.log(copy.toString());

// manupulate the slice
slice.write('_-_-_-_-_-');

console.log(buffer.toString());

File System

  • NodeJS has streaming api's for file system access
  • NodeJS provides fs module that abstracts the file system access
  • Most methods in file system module works asynchronously, but they also have their synchronous counterparts
  • fs module provides a lot of methods for file and directory operations
  • Ensure all opened files are closed in your node applications.

Synchronous & Asynchronous operations

"use strict";

var fs = require('fs');

// Asynchronous
fs.stat('./', function (err, stats) {

	if(err) {
		console.log(err.message);
		return;
	}
	
	console.log(stats);
});


// Synchronous
try {
	var myStat = fs.statSync('./notAvailable.txt');
	console.log(myStat);
}
catch (e) {
	console.log(e); 
}

File Size

"use strict";

var fs = require('fs');

fs.stat(__dirname+'/readme.txt',function(err,stats){ if (err) { throw err; }
    console.log(stats.size);
});

Read a File - Buffer

var fs = require('fs'); 

fs.open('./readme.txt','r',function(err,fd){
    if (err) throw err;
    var readBuffer = new Buffer(1024),
    bufferOffset = 0,
    bufferLength = readBuffer.length,
    filePosition = 100;

    fs.read(fd, readBuffer, bufferOffset, bufferLength, filePosition, function(err, readBytes) {
        if (err) throw err;
        console.log('just read ' + readBytes + ' bytes'); if (readBytes > 0) {
        console.log(readBuffer.slice(0, readBytes));
      }
    }); 
});

Write to a File

var fs=require('fs');

fs.open('./readme.txt','a',function(err,fd){ 
    var writeBuffer = new Buffer('writing this string'),
    bufferOffset = 0,
    bufferLength = writeBuffer.length, 
    filePosition = null;
    fs.write(fd,writeBuffer, bufferOffset, bufferLength, filePosition, function(err, written) {
        if (err) { throw err; }
          console.log('wrote ' + written + ' bytes');
        }
    ); 
});

Listing Directory Structure

"use strict";

var fs = require('fs');

fs.readdir('.', function (err, files) {

  if (err) throw err;

  files.forEach( function (file) {
      fs.stat('./' + file, function (err, stats) {
      
          if (err) throw err;
      
          if (stats.isFile()) {
            console.log("%s is file", file);
          }
          else if (stats.isDirectory ()) {
              console.log("%s is a directory", file);
          }
          console.log('stats:  %s',JSON.stringify(stats));
        }); 
    });
});

Watching Files

"use strict";

var fs = require('fs');

fs.watchFile('./watch.txt', function (curr, prev) {
  console.log('the current mtime is: ' + curr.mtime);
  console.log('the previous mtime was: ' + prev.mtime);
});

fs.writeFile('./watch.txt', "changed", function (err) {
  if (err) throw err;
  console.log("file changed");
});

Networking

Hour 05

Networking in NodeJS

  • TCP
  • UDP
  • HTTP
  • TLS
  • WebSockets

TCP Server & Client

// TCP Server 
var net = require('net');

net.createServer(function(socket){

    // new connection4
    socket.on('data', function(data) {
        // got data
    });

    socket.on('end', function(data) {
        // connection closed
    });

    socket.write('Some string');

}).listen(4001);

// TCP Client
var net=require('net');
var port=80;
var host='www.google.com'; 
// host is optional, if ommitted defaults to localhost
var conn = net.createConnection(port,host);

conn.on('data',function(data){ 
    console.log('some data has arrived')
});

conn.on('close',function(){ 
    console.log('connection closed');
});

conn.write("Hello, World!");
conn.close();

UDP Server & Client

// UDP Server 
var dgram = require('dgram');

var server=dgram.createSocket('udp4'); // can be 'udp6' for ipv6

server.on('message',function(message,rinfo){
  console.log('server got message: ' + message + ' from ' + rinfo.address + ':' + rinfo.port);
});

server.on('listening',function(){
    var address = server.address();
    console.log('server listening on ' + address.address + ':' + address.port);
});

server.bind(4000);

// UDP Client

var dgram = require('dgram');
var client = dgram.createSocket('udp4');
var message = new Buffer('A Datagram Message!!!');
client.send(message,0,message.length,4000,'localhost');
client.close();

HTTP Server

// HTTP Server 
var util = require('util');
var http = require('http');
var url = require('url');

http.createServer(function(req,res){ 
    res.writeHead(200, {'Content-Type':'application/json'});
    var response = url.parse(req.url, true);
    res.end(util.inspect(response));
}).listen(4000);


// HTTP Client
var http=require('http');
var options={
    host: 'localhost', port: 4000,
    path: '/'
};

http.get(options,function(res)
{ 
    console.log('got response: ' + res.statusCode);
}).on('error',function(err){ 
    console.log('got error: ' + err.message)
});

TLS / SSL

// Works based on public / private key combination
// based on openssl library

// Generate Private Key
// openssl genrsa -out my_private_key.pem 1024

// Public Key (CSR)
// openssl req -new -key my_private_key.pem -out my_public_key.pem

// Self Signed Certificate with CSR
// openssl x509 -req -in my_public_key.pem -signkey my_private_key.pem -out my_cert.pem

SSL Client

var tls = require('tls'), 
    fs = require('fs'), 
    port = 3000,
    host = 'localhost', 
    options = {
      key : fs.readFileSync('./my_private_key.pem'),
      cert : fs.readFileSync('./my_cert.pem')
    };

// To avoid errors due to self signed certificate
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"

var client=tls.connect(port,host,options,function(){ 
        console.log('connected');
        console.log('authorized: ' + client.authorized); 
        
        client.on('data', function(data) {
            client.write(data); 
            // just send data back to server });
        });
});

SSL Server

var tls = require('tls'), 
    fs = require('fs'), 
    options = {
      key : fs.readFileSync('./my_private_key.pem'),
      cert : fs.readFileSync('./my_cert.pem')
    };

tls.createServer(options,function(s){ 
    s.pipe(s);
}).listen(4040);

HTTPS Server


var https = require('https');
var fs = require('fs');

var options = {
  key : fs.readFileSync('./my_private_key.pem'),
  cert : fs.readFileSync('./my_cert.pem')
};

var a = https.createServer(options, function (req, res) {
  res.writeHead(200);
  res.end("hello world\n");
}).listen(4041);

Websocket Server

// npm install websocket
// Please note that there are many other websocket implementations example socket.io

var server = require('websocket').server, 
    http = require('http');

var socket = new server({
    httpServer: http.createServer().listen(1337)
});

socket.on('request', function(request) {
    var connection = request.accept(null, request.origin);

    connection.on('message', function(message) {
        console.log(message.utf8Data);
        connection.sendUTF('Welcome');
        setTimeout(function() {
            connection.sendUTF('Sending message back to the websocket client');
        }, 1000);
    });

    connection.on('close', function(connection) {
        console.log('connection closed');
    });
}); 

Websocket Client

<!DOCTYPE html>

<head>
    <meta charset="utf-8">
    <title>WEBSOCKET CLIENT</title>
    <meta name="description" content="">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootswatch/3.3.2/paper/bootstrap.min.css">
</head>
<body>
	<div id="content"></div>

	<script type="text/javascript">
    		var content = document.getElementById('content');
    		var socket = new WebSocket('ws://localhost:1337');
    		socket.onopen = function () {
        		socket.send('hello from the client');
   		 };

    		socket.onmessage = function (message) {
        		content.innerHTML += message.data +'<br />';
    		};

    		socket.onerror = function (error) {
        		console.log('WebSocket error: ' + error);
    		};
	</script>
</body>
</html>

STREAMS

Hour 06

Streams

  • Abstract interface implemented by various objects in nodejs
  • They are readable, writable or both  (Duplex)
  • All streams are instances of EventEmitter
  • A Duplex stream implements the methods of both readable and writable stream
  • We can implement our own stream by implementing appropriate interface method
    • Read only - _read
    • write only - _write, _writev
    • Read & Write - _read, _write, _writev
    • Transform - _transform, _flush [*Duplex]

Reading

"use strict";

var fs = require('fs');

try {
    var rstream = fs.createReadStream('./readme.txt');

    rstream.on('readable', function() {
        console.log('readable ', rstream.read());
    });

    rstream.on('end', function(){
        console.log('end');
    });

    // Uncomment and see what happens

    // rstream.on('data', function(data){
    //    console.log("data ::" + data);
    // })

}
catch (e) {
    console.log('Error occured ' + e);
}

Readstreams

can be paused and resumed

Writing

"use strict";

var fs = require('fs');

var wstream = fs.createWriteStream ('readme.txt');

var count = 1000;

while (count > 0) {
    wstream.write(count + ' \n');
    count -= 1;
}

wstream.end('');

wstream.on('end', function(){
    wstream.close();
})

TESTING

Hour 07

Automated Testing

  • Unit Testing
  • API Testing
  • Acceptance Testing
  • Frameworks and Libraries

NodeJS_10_H

By Hari Narasimhan

NodeJS_10_H

10 Hours of nodejs

  • 2,319