10 Hours of NodeJs
Introducing NodeJS in 10 hours
What are we covering?
- HOUR 01 - Introduction
- HOUR 02 - Understanding Modules
- HOUR 03 - Event Loop & Event Emitter
- HOUR 04 - File System and Buffers
- HOUR 05 - Networking
- HOUR 06 - Streams
- HOUR 07 - Unit Testing !
- HOUR 08 - Writing a web App - Part 1
- HOUR 09 - Writing a web App - Part 2
- HOUR 10 - Wrap Up
Introduction
Hour 01
What is NodeJS?
- NodeJS is an open-source, cross-platform JavaScript runtime environment based on Chrome's V8 JavaScript Engine
- NodeJS applications are written in JavaScript Language and can run on many different environments supported by the NodeJS runtime (OS X, Windows, Linux, Free BSD & many more)
Why NodeJS
- Speaks the lingua franca of the web!
- Single language for the server and front-end
- Dynamically Typed
- Speed & non blocking I/O
- Massive Ecosystem (npm modules ~198k packages)
- Vibrant Community
- Cross Platform (FreeBSD, Linux, OS X, Windows)
- Seriously Productive
- Builtin support for realtime and streaming
- Fun, Easy and Enjoyable.
http://www.toptal.com/nodejs/why-the-hell-would-i-use-node-js
Where is node used today?
- Netflix
- eBay
- New York Times
- PayPal
- Medium
- Uber
NodeJs Use Cases
- Rest API's
- Streaming & Realtime Applications
- Command Line Tools
When to avoid?
- CPU Intensive Applications
- RDBMS Data Stores with lots of de-normalized data
- Web applications with lots of server side rendering
Node Bindings (Socket Library, HTTP etc)
Node Standard Library
Event Loop
(libev)
V8
Thread Pool
(libeio)
DNS
(c-ares)
Crypto
(OpenSSL)
JavaScript
C++
- Single Threaded
- Event Loop
- Non Blocking IO
- Excellent Support for HTTP
Installing NodeJS & NPM - USING NVM
# https://github.com/creationix/nvm
# For Linux & Mac OS
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.29.0/install.sh | bash
or Wget:
wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.29.0/install.sh | bash
# For Windows Machine
nvmw - https://github.com/hakobera/nvmw
nvm-windows - https://github.com/coreybutler/nvm-windows (has a windows installer)
NodeJS Development Environment
- Get a good shell (if windows!)
- Install Git (Git Bash)
- A decent text editor (Sublime Text, Notepad++)
- Install jshint - {npm install -g jshint}
- Master the shell!
All About npm
- NPM stands for Node Package Manager
- Written by Isaac Schlueter
- Manages 3rd party package installation
- From 0.6.0 version of nodejs comes preinstalled
- Used for initializing the nodejs applicaitons
- Keeping track of out dated dependencies
NPM Commands
# npm commands
npm --version # displays the version of npm
npm init # inits a new module with a package.json file
npm install <module> # installs the module and all its dependency
npm install -g # globally installs the module
npm install [-g] <module> --save # saves the module to the package.json dependencies section
npm install [-g] <module> --save-dev # saves the module to the package.json dev-dependencies section
npm ls -g # list global modules
npm ls <module> # list by a module bane
npm rm <module> # remove (alias) # remove an installed module
npm outdated # list outdated modules
npm view # view details of an installed module
npm prune # removes extraneous (unwanted) packages
npm search # search public packages from npm registry
npm update # updates all dependent modules based changes to package.json
npm help # help on npm
Package.json
# Package.json, is present in the root directory of the nodejs applicaiton. It defines the property of the application / package.
# An npm init command generates a package.json file by interactively asking a set of questions
# Attributes of package.json
name - name of the package
version - version of the package
description - description of the package
homepage - homepage of the package
author - author of the package
contributors - name of the contributors to the package
dependencies - list of dependencies. dependencies mentioned here in are installed to node_module folder of the package.
devDependencies -
repository - repository type and url of the package
main - entry point of the package
keywords - keywords
scripts - scripts to be run during various times of the project's lifecycle (example, start, test, pack etc)
config - set configuration parameters used in package scripts
jshint
# JSHint is a static code analysis tool for JavaScript
# installation
npm install -g jshint || npm i -g jshint
# or install it as your dev dependency
npm install jshint --save-dev
# configuration (customize jshint)
1) --config <file>
2) .jshintrc
3) jshintConfig
Example - Basic
mkdir hello-node && cd $_
npm init
# answer the questions asked by npm init
# touch index.js
"use strict";
console.log("Hello, world from nodejs");
# run the program
node index.js
# Add the start up script
"scripts": {
start: "node index.js"
}
# Run using scripts:start attribute
npm start
Example - Advanced
# touch server.js
"use strict";
var http = require("http");
var port = process.env.PORT || 8888;
http.createServer( function (req, res) {
res.writeHead(200, {"Content-Type": "text/plain"});
res.write("Hello, World from nodejs");
res.end();
}).listen(port);
console.log("HTTP Server listening to port :" + port);
Example - Non Blocking I/O
# touch nonBlockingDemo.js
"use strict";
var fs = require('fs');
fs.readFile('read-nb.txt', 'utf8', function(err, contents) {
console.log(contents);
});
console.log('Hello there, the file is being read asynchronously');
Extras - Closures
# touch no-closure.js
"use strict";
function sayHello (message) {
console.log(message);
}
function createMessage(name) {
return "Hi "
+ name
+ " welcome to Node in 10 hours training!";
}
var message = createMessage('Jack');
sayHello(message);
# touch closure.js
"use strict";
// Define the closure
function HelloMaker (name) {
var message = 'Hi '
+ name
+ ' welcome to Node in 10 hours training!';
return function sayHello() {
console.log(message);
}
}
// Create the closure
var helloJack = HelloMaker('Jack');
// Use the closure
helloJack();
Understanding Modules
Hour 02
What are NodeJS Modules?
- Brings the concept of a 'Standard Library' to NodeJS
- Allows developers to extend NodeJS through custom modules
- Implements the CommonJS modules standard
- A module is defined in a single file (one to one correspondence with file)
- A module can export public functions which can be used by other modules
- Variables local to the module are private to the module
CommonJS
- Is a standard specification for modualrizing JavaScript
- CommonJS does not provide any libraries of its own
- Specifies require() and an exports object which work in conjunction to provide the module
- NodeJS standard library and all 3rd Party libraries use CommonJS standard to modularize the code
Consuming Modules
To consume an existing module use the require function
// Require a code module or a module installed by NPM
var module = require('module-name');
// Require a module relative to current path
var my-module = require('./my-module');
Defining Modules
To define a module, use exports global variable
exports
vs
module.exports
/**
* File: my-math.js
* Style 1 - Export single function
*/
var square = function (n) { return n*n;};
exports.square = square
///////////////////////////////////////////////////////
/**
* File: my-math-lib.js
* Style 2 - Set of functions
*/
var square = function (n) { return n*n;};
var cube = function(n) {return square(n) * n;};
exports.square = square;
exports.cube = cube;
//////////////////////////////////////////////////////
/**
* File: my-math-lib-module.js
* Style 3 - Set of functions in module style
*/
var square = function (n) { return n*n;};
var cube = function(n) {return square(n) * n;};
module.exports = {
square:square;
cube: cube;
};
///////////////////////////////////////////////////////
/**
* File employee.js
* Style 4 : Object Constructor
*/
module.exports = function (firstname, lastname, department) {
this.firstname = firstname;
this.lastname = lastname;
this.department = department;
this.print = function() {
return firstname + ' ' + lastname + ' - ' + department;
}
};
/**
* file employee-service.js
*/
var Employee = require ('../models/employee');
var employee1 = new Employee('Jack', 'Peters', 'Mechanical');
console.log(employee1.print());
Resolving Modules
Core Modules
binary modules - from lib folder
Preferentially loaded over user defined names
File Modules
Prefixed with /, ./ or ../
If exact name is not found appends .js, .node or .json and does the search.
Throws exception if module is not found
node_modules folder
If not available in native modules and no prefix such as /, ./, ../ etc
Folders
Loads from main entry in package.json
If package.json is not available in the folder,
tries with /index appended to the end of the require string [.js, .node, .json]
node_modules folder
- automatically created by npm when a module is installed
- node_modules are not checked into version control system
- They maintain deeply nested folder structure to manage each package dependency
- Due to deep nesting, it does not work very well Windows
Creating and Publishing npm modules
Modules Best Practices
Event Loop & Event Emitter
Hour 03
What is an Event Loop?
- A software pattern
- Facilitates async I/O
- In NodeJS Event Loop is core concept for handling high throughput
- Though event loop helps in asynchronous programming the loop itself is synchronous
- An event loop in NodeJS continues to run until the last callback is completed
Event Loop in NodeJS
- Depends on multiple libraries (libev)
- Used OS efficiently to hold connections & resources
Traditional Vs Event Driven I/O
// Traditional Model
posts = db.query("SELECT * from posts WHERE author = '%john%'");
formattedPosts = formatPosts(post);
sendPosts(formattedPosts);
// create a callback
var postHandler = function(err, posts) {
formatPosts(posts, function (err, formattedPosts) {
sendPosts(formattedPosts, function(err) {
//Handle Error
};
});
};
// var formatPosts = function (posts, callback) {};
// var sendPosts = function (formattedPosts, callback) {};
// Query the database with a callback
db.query("SELECT * FROM posts WHERE author ='%jack%'", postHandler);
Blocking the Event Loop
// Guess what happens here?
var opened = false;
setTimeout(function() {
opened = true;
}, 1000);
while (!opened) {
// wait till it is opened
}
console.log('Opened now!');
Event Emitter
- Many NodeJS objects emit event
- All objects that emit events are instances of events.EventEmitter
- Examples
- TCP Server emit 'connect' every time a client connects
- ReadStream can emit 'data' each time data is available for reading
Event Emitter - Methods
Method | Description |
---|---|
.addListener .on |
Adds an event listener |
.once | Listens exactly once |
.removeListener | Removes an event listener for an event |
.removeAllListeners | Removes all listeners for an event |
.emit | Emits an event |
There are few more
https://nodejs.org/api/events.html#events_class_events_eventemitter
Example - Add Listener
"use strict";
var rl = require('readline').createInterface({
input: require('fs').createReadStream('readme.txt')
});
rl.on('line', function (line) {
console.log('Line from file:', line);
});
rl.on('close', function () {
console.log('Closed::The end');
});
Example - Once
// Manual Method
var events = require('events'),
emitter = new events.EventEmitter();
function listener() {
console.log('one Timer');
emitter.removeListener('oneTimer', listener);
}
emitter.on('oneTimer', listener);
emitter.emit('oneTimer');
emitter.emit('oneTimer');
// Using Once
var events = require('events'),
emitter = new events.EventEmitter();
/* Use Once */
emitter.once('onceOnly', function() {
console.log('one Only');
});
emitter.emit('onceOnly');
emitter.emit('onceOnly');
Example - Create EventEmitter
var EventEmitter = require('events').EventEmitter;
var util = require('util');
var Person = function (firstname, lastname) {
this.firstname = firstname;
this.lastname = lastname;
};
util.inherits(Person, EventEmitter);
// Create an method that emits an event
Person.prototype.sayHello = function () {
this.emit('greet', {firstname: this.firstname, lastname: this.lastname});
};
// Create an instance of person and say hello
var p = new Person('Peter', 'Parker');
p.on('greet', function (data){
console.log('Got a greet event from ' + data.firstname);
});
Example - Emit Frequently
"use strict";
var util = require('util'),
EventEmitter = require('events').EventEmitter;
var Ticker = function(limit){
var self = this;
this.limit = limit || 10;
this.counter = 0;
setInterval(function() {
self.emit('tick');
self.counter += 1;
if(self.counter === self.limit) {
console.log('limit reached -- removing event listener');
self.emit('limit');
}
}, 1000);
};
util.inherits(Ticker,EventEmitter)
var ticker = new Ticker();
var tickListener = function(){
console.log('TICK');
};
ticker.on('tick',tickListener);
ticker.on('limit', function(){
ticker.removeListener('tick', tickListener);
ticker.removeAllListeners('limit');
})
// As an exercise, remove the event listener after 100 ticks
Filesystem & Buffers
Hour 04
Buffers
- JavaScript cannot natively handle binary data
- JavaScript is good at handling unicode encoded strings
- Server side javascript needs to handle various binary formats
- Initially NodeJS used strings, but this lead to problems, hence Buffer was introduced
- Buffer is like an integer array, with its memory allocated outside V8 engine
- Buffers cannot be resized.
- To convert a buffer to JavaScript object and vice versa, we must provide an encoding
- Supported encodings are 'ascii', 'utf-8', 'utf16le', 'ucs2', 'base64' and 'binary'
Creating Buffers
var Buffer = require('buffer');
// Couple of ways to create buffers
var buffer1 = new Buffer(1024);
// Defaults to utf-8
var buffer2 = new Buffer ('Hello, Friends!');
var buffer3 = new Buffer ([8, 6, 7, 5, 3, 0, 9]);
console.log(buffer1.toString('utf-8'));
console.log('\n');
console.log(buffer2.toString('utf-8'));
console.log('\n');
console.log(buffer3.toString('utf-8'));
Buffer Operations
// Write to a buffer
var buf = new Buffer(100);
buf.write('Hello, World!');
buf.write(' John!', 13);
// Reading from buffer
console.log(buf.toString());
// Reading from buffer like an array
console.log(buf[10].toString());
// Check if the buffer is actually a buffer
console.log("Buffer.isBuffer " + Buffer.isBuffer(buf));
// Check bytelength required to encode
snowman = "☃";
console.log('Snowman length: ' + snowman.length);
console.log('Buffer Required for snowman: ' + Buffer.byteLength(snowman));
// Check the length of the buffer
console.log('Buffer length is ' + buf.length);
Slice & Copy Buffer
// Slice a buffer
// Slice does not create new memory, it uses the original memory underneath
var buffer = new Buffer ('this is the string in my buffer');
var slice = buffer.slice(10,20);
console.log(slice.toString());
// Copy a buffer
var copy = new Buffer(10);
var targetStart = 0,
sourceStart = 10,
sourceEnd = 20;
buffer.copy(copy, targetStart, sourceStart, sourceEnd);
console.log(copy.toString());
// manupulate the slice
slice.write('_-_-_-_-_-');
console.log(buffer.toString());
Buffer Exercise
// Slice a buffer
// Slice does not create new memory, it uses the original memory underneath
var buffer = new Buffer ('this is the string in my buffer');
var slice = buffer.slice(10,20);
console.log(slice.toString());
// Copy a buffer
var copy = new Buffer(10);
var targetStart = 0,
sourceStart = 10,
sourceEnd = 20;
buffer.copy(copy, targetStart, sourceStart, sourceEnd);
console.log(copy.toString());
// manupulate the slice
slice.write('_-_-_-_-_-');
console.log(buffer.toString());
File System
- NodeJS has streaming api's for file system access
- NodeJS provides fs module that abstracts the file system access
- Most methods in file system module works asynchronously, but they also have their synchronous counterparts
- fs module provides a lot of methods for file and directory operations
- Ensure all opened files are closed in your node applications.
Synchronous & Asynchronous operations
"use strict";
var fs = require('fs');
// Asynchronous
fs.stat('./', function (err, stats) {
if(err) {
console.log(err.message);
return;
}
console.log(stats);
});
// Synchronous
try {
var myStat = fs.statSync('./notAvailable.txt');
console.log(myStat);
}
catch (e) {
console.log(e);
}
File Size
"use strict";
var fs = require('fs');
fs.stat(__dirname+'/readme.txt',function(err,stats){ if (err) { throw err; }
console.log(stats.size);
});
Read a File - Buffer
var fs = require('fs');
fs.open('./readme.txt','r',function(err,fd){
if (err) throw err;
var readBuffer = new Buffer(1024),
bufferOffset = 0,
bufferLength = readBuffer.length,
filePosition = 100;
fs.read(fd, readBuffer, bufferOffset, bufferLength, filePosition, function(err, readBytes) {
if (err) throw err;
console.log('just read ' + readBytes + ' bytes'); if (readBytes > 0) {
console.log(readBuffer.slice(0, readBytes));
}
});
});
Write to a File
var fs=require('fs');
fs.open('./readme.txt','a',function(err,fd){
var writeBuffer = new Buffer('writing this string'),
bufferOffset = 0,
bufferLength = writeBuffer.length,
filePosition = null;
fs.write(fd,writeBuffer, bufferOffset, bufferLength, filePosition, function(err, written) {
if (err) { throw err; }
console.log('wrote ' + written + ' bytes');
}
);
});
Listing Directory Structure
"use strict";
var fs = require('fs');
fs.readdir('.', function (err, files) {
if (err) throw err;
files.forEach( function (file) {
fs.stat('./' + file, function (err, stats) {
if (err) throw err;
if (stats.isFile()) {
console.log("%s is file", file);
}
else if (stats.isDirectory ()) {
console.log("%s is a directory", file);
}
console.log('stats: %s',JSON.stringify(stats));
});
});
});
Watching Files
"use strict";
var fs = require('fs');
fs.watchFile('./watch.txt', function (curr, prev) {
console.log('the current mtime is: ' + curr.mtime);
console.log('the previous mtime was: ' + prev.mtime);
});
fs.writeFile('./watch.txt', "changed", function (err) {
if (err) throw err;
console.log("file changed");
});
Networking
Hour 05
Networking in NodeJS
- TCP
- UDP
- HTTP
- TLS
- WebSockets
TCP Server & Client
// TCP Server
var net = require('net');
net.createServer(function(socket){
// new connection4
socket.on('data', function(data) {
// got data
});
socket.on('end', function(data) {
// connection closed
});
socket.write('Some string');
}).listen(4001);
// TCP Client
var net=require('net');
var port=80;
var host='www.google.com';
// host is optional, if ommitted defaults to localhost
var conn = net.createConnection(port,host);
conn.on('data',function(data){
console.log('some data has arrived')
});
conn.on('close',function(){
console.log('connection closed');
});
conn.write("Hello, World!");
conn.close();
UDP Server & Client
// UDP Server
var dgram = require('dgram');
var server=dgram.createSocket('udp4'); // can be 'udp6' for ipv6
server.on('message',function(message,rinfo){
console.log('server got message: ' + message + ' from ' + rinfo.address + ':' + rinfo.port);
});
server.on('listening',function(){
var address = server.address();
console.log('server listening on ' + address.address + ':' + address.port);
});
server.bind(4000);
// UDP Client
var dgram = require('dgram');
var client = dgram.createSocket('udp4');
var message = new Buffer('A Datagram Message!!!');
client.send(message,0,message.length,4000,'localhost');
client.close();
HTTP Server
// HTTP Server
var util = require('util');
var http = require('http');
var url = require('url');
http.createServer(function(req,res){
res.writeHead(200, {'Content-Type':'application/json'});
var response = url.parse(req.url, true);
res.end(util.inspect(response));
}).listen(4000);
// HTTP Client
var http=require('http');
var options={
host: 'localhost', port: 4000,
path: '/'
};
http.get(options,function(res)
{
console.log('got response: ' + res.statusCode);
}).on('error',function(err){
console.log('got error: ' + err.message)
});
TLS / SSL
// Works based on public / private key combination
// based on openssl library
// Generate Private Key
// openssl genrsa -out my_private_key.pem 1024
// Public Key (CSR)
// openssl req -new -key my_private_key.pem -out my_public_key.pem
// Self Signed Certificate with CSR
// openssl x509 -req -in my_public_key.pem -signkey my_private_key.pem -out my_cert.pem
SSL Client
var tls = require('tls'),
fs = require('fs'),
port = 3000,
host = 'localhost',
options = {
key : fs.readFileSync('./my_private_key.pem'),
cert : fs.readFileSync('./my_cert.pem')
};
// To avoid errors due to self signed certificate
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"
var client=tls.connect(port,host,options,function(){
console.log('connected');
console.log('authorized: ' + client.authorized);
client.on('data', function(data) {
client.write(data);
// just send data back to server });
});
});
SSL Server
var tls = require('tls'),
fs = require('fs'),
options = {
key : fs.readFileSync('./my_private_key.pem'),
cert : fs.readFileSync('./my_cert.pem')
};
tls.createServer(options,function(s){
s.pipe(s);
}).listen(4040);
Websocket Server
// npm install websocket
// Please note that there are many other websocket implementations example socket.io
var server = require('websocket').server,
http = require('http');
var socket = new server({
httpServer: http.createServer().listen(1337)
});
socket.on('request', function(request) {
var connection = request.accept(null, request.origin);
connection.on('message', function(message) {
console.log(message.utf8Data);
connection.sendUTF('Welcome');
setTimeout(function() {
connection.sendUTF('Sending message back to the websocket client');
}, 1000);
});
connection.on('close', function(connection) {
console.log('connection closed');
});
});
Websocket Client
<!DOCTYPE html>
<head>
<meta charset="utf-8">
<title>WEBSOCKET CLIENT</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootswatch/3.3.2/paper/bootstrap.min.css">
</head>
<body>
<div id="content"></div>
<script type="text/javascript">
var content = document.getElementById('content');
var socket = new WebSocket('ws://localhost:1337');
socket.onopen = function () {
socket.send('hello from the client');
};
socket.onmessage = function (message) {
content.innerHTML += message.data +'<br />';
};
socket.onerror = function (error) {
console.log('WebSocket error: ' + error);
};
</script>
</body>
</html>
STREAMS
Hour 06
Streams
- Abstract interface implemented by various objects in nodejs
- They are readable, writable or both (Duplex)
- All streams are instances of EventEmitter
- A Duplex stream implements the methods of both readable and writable stream
- We can implement our own stream by implementing appropriate interface method
- Read only - _read
- write only - _write, _writev
- Read & Write - _read, _write, _writev
- Transform - _transform, _flush [*Duplex]
Reading
"use strict";
var fs = require('fs');
try {
var rstream = fs.createReadStream('./readme.txt');
rstream.on('readable', function() {
console.log('readable ', rstream.read());
});
rstream.on('end', function(){
console.log('end');
});
// Uncomment and see what happens
// rstream.on('data', function(data){
// console.log("data ::" + data);
// })
}
catch (e) {
console.log('Error occured ' + e);
}
Readstreams
can be paused and resumed
Writing
"use strict";
var fs = require('fs');
var wstream = fs.createWriteStream ('readme.txt');
var count = 1000;
while (count > 0) {
wstream.write(count + ' \n');
count -= 1;
}
wstream.end('');
wstream.on('end', function(){
wstream.close();
})
TESTING
Hour 07
Automated Testing
- Unit Testing
- API Testing
- Acceptance Testing
NodeJS_10_H
By Michael PAMBO OGNANA
NodeJS_10_H
10 Hours of nodejs
- 1,068