Develop, Monitor and Scale REST APIs in Node.js

 

Prerequisites

  • Node( version 0.10.33) - http://nodejs.org/download/
  • Python( version 2.7.8​)
  • Mongodb
    • Start mongo
    • Connect to mongo with mongo shell 
      • $ mongo
  • Git (Full install)

  • For Windows - Microsoft Visual Studio C++ 2012 for Windows Desktop (Express version works well)

  • For Mac - XCode 

  • strongloop cli -> $ npm install -g strongloop

     

The Frontend is changing fast (Mobile First !)

And it isn’t done yet !

APIs are powering this change

OK... But how do I cross the legacy bridge?

Distinctly unique approaches to solve the same problem

What backend is being used for APIs ?

Node is FAST

 

.. and highly concurrent

 

Node is perfect for APIs

Node powers full-stack JS

What is Node.js?

Asynchronous, event-driven, non-blocking I/O platform that is perfect for real-time, data-intensive applications.

Single thread of execution

Built on Chrome’s JavaScript runtime engine, V8

Uses Libuv, a high performance evented I/O library that works on Linux and Windows (faster than Libev)

Threads Don’t Wait !

Behind the curtain

 

Yes, it's C back there

 

But what exactly is full stack

Full Stack JavaScript is powering mobility

A Compelling Option for Enterprise Mobility

  • Faster time to market vs. 100% native development
  • Reuse skills of existing web developer staff
  • Open Source stack / no vendor lock-in 
  • For Enterprise apps, performance is comparable to native apps
  • Near “write once, run anywhere” UI: tablets, phones, smart TVs, smart watches, cars, etc.

Gartner predicts by 2016 more than 50% of the        apps deployed will be Hybrid apps.

 

But is the backend really being written in JS ?

  • Node.js active contributors (>65K)
  • Very large and active developer community
  • Growing/ most modules (100K > Java, Ruby, PHP... )

Popular Frameworks

 

Express

var express = require('express');
var Item = require('models').Item;
var app = express();
var itemRoute = express.Router();
 
itemRoute.param('itemId', function(req, res, next, id) {
  Item.findById(req.params.itemId, function(err, item) {
    req.item = item;
    next();
  });
});
 
itemRoute.route('/:itemId')
  .get(function(req, res, next) {
    res.json(req.item);
  })
  .put(function(req, res, next) {
    req.item.set(req.body);
    req.item.save(function(err, item) {
      res.json(item);
    });
  })
  .post(function(req, res, next) {
    var item = new Item(req.body);
    item.save(function(err, item) {
      res.json(item);
    });
  })
  .delete(function(req, res, next) {
    req.item.remove(function(err) {
      res.json({});
    });
  })
  ;
 
app.use('/api/items', itemRoute);
app.listen(8080);

Sample code (Router using express 4.x)

 

  • Pros

    1. Little learning curve, Express is nearly a standard for Node.js web application

    2.  Fully customizable

  • Cons

    1. All end points need to be created manually, you end up doing a lot of the same code (or worse, start rolling your own libraries after a while)

    2. Every end point needs to be tested (or at the very least I recommend that you hit the end points with HTTP consumer to make sure they are actually there and don’t throw 500s)

    3. Refactoring becomes painful because everything needs to be updated everywhere

    4. Doesn’t come with anything “standard”, have to figure out your own approach

RESTful APIs with Restify

 

var restify = require('restify');
var Item = require('models').Item;
var app = restify.createServer()
 
app.use(function(req, res, next) {
  if (req.params.itemId) {
    Item.findById(req.params.itemId, function(err, item) {
      req.item = item;
      next();
    });
  }
  else {
    next();
  }
});
 
app.get('/api/items/:itemId', function(req, res, next) {
  res.send(200, req.item);
});
 
app.put('/api/items/:itemId', function(req, res, next) {
  req.item.set(req.body);
  req.item.save(function(err, item) {
    res.send(204, item);
  });
});
 
app.post('/api/items/:itemId', function(req, res, next) {
  var item = new Item(req.body);
  item.save(function(err, item) {
    res.send(201, item);
  });
});
 
app.delete('/api/items/:itemId', function(req, res, next) {
  req.item.remove(function(err) {
    res.send(204, {});
  });
});
 
app.listen(8080);

Sample Code (Router using restify)

 

  • Pros

    1. Automatic DTrace support for all your handlers (if you’re running on a platform that supports DTrace)

    2. Doesn’t have unnecessary functionality like templating and rendering

    3. Built in throttling

    4. Built in SPDY support

  • Cons

    1. The cons are all the same with Restify as they are with Express, lots of manual labor.

var Hapi = require('hapi');
var Item = require('models').Item;
var server = Hapi.createServer('0.0.0.0', 8080);
 
server.ext('onPreHandler', function(req, next) {
  if (req.params.itemId) {
    Item.findById(req.params.itemId, function(err, item) {
      req.item = item;
      next();
    });
  }
  else {
    next();
  }
});
 
server.route([
  {
    path: '/api/items/{itemId}',
    method: 'GET',
    config: {
      handler: function(req, reply) {
        reply(req.item);
      }
    }
  },
  {
    path: '/api/items',
    method: 'PUT',
    config: {
      handler: function(req, reply) {
        req.item.set(req.body);
        req.item.save(function(err, item) {
          reply(item).code(204);
        });
      }
    }
  },
  {
    path: '/api/items',
    method: 'POST',
    config: {
      handler: function(req, reply) {
        var item = new Item(req.body);
        item.save(function(err, item) {
          reply(item).code(201);
        });
      }
    }
  },
  {
    path: '/api/items/{itemId}',
    method: 'DELETE',
    config: {
      handler: function(req, reply) {
        req.item.remove(function(err) {
          reply({}).code(204);
        });
      }
    }
  }
]);
 
server.start();

Sample Code (Router using hapi)

 

  • Pros

    1. Very granular control over request handling

    2. Detailed API reference with support for documentation generation

  • Cons

    1. As with Express and Restifyhapi gives you great construction blocks, but you are left to your own devices figuring out how to use them.

Sample Code (Router using Loopback)

Sample Code (Router using Loopback)

Code needed

var loopback = require('loopback');
var Item = require('./models').Item;
var app = module.exports = loopback();
 
app.model(Item);
app.use('/api', loopback.rest());
app.listen(8080);

Endpoint automatically needed

DELETE /items/{id}
GET /items
GET /items/count
GET /items/findOne
GET /items/{id}
GET /items/{id}/exists
POST /items
PUT /items
PUT /items/{id}

Sample Code (API explorer and test)

Code needed

var explorer = require('loopback-explorer');
app.use('/explorer', explorer(app, {basePath: '/api'}));

API explorer automatically created

Code needed

var loopback = require('loopback');
var explorer = require('loopback-explorer');
var remoting = require('strong-remoting');
var Item = require('./models').Item;
var app = module.exports = loopback();
var rpc = remoting.create();
 
function echo(ping, callback) {
  callback(null, ping);
}
 
echo.shared = true;
echo.accepts = {arg: 'ping'};
echo.returns = {arg: 'echo'};
 
rpc.exports.system = {
  echo: echo
};
 
app.model(Item);
app.use('/api', loopback.rest());
app.use('/explorer', explorer(app, {basePath: '/api'}));
app.use('/rpc', rpc.handler('rest'));
app.listen(8080);

Remotable calls

$ curl "http://localhost:8080/rpc/system/echo?ping=hello"
{
  "echo": "hello"
}
  • Pros

    1. Very quick RESTful API development

    2. Convention over configuration

    3. Built in models ready to use

    4. RPC support

    5. Fully configurable when needed

    6. Extensive documentation

    7. Fulltime team working on the project

    8. Available commercial support

  • Cons

    1. Learning curve can be pretty steep because there are so many moving parts

LoopBack Studio

  • Visual ORM

  • Discovery

  • Migration

  • DEV - API Design & Composition

  • OPS – Provisioning, Scaling & Monitoring

 

A visual tool for creating and managing Node.js powered REST APIs

API Design & Composition

Migration

mysql> use coffee;
Database changed
mysql> show tables;
+------------------+
| Tables_in_coffee |
+------------------+
| Coffee           |
+------------------+
1 row in set (0.15 sec)

mysql> describe Coffee;
+-----------+--------------+------+-----+---------+----------------+
| Field     | Type         | Null | Key | Default | Extra          |
+-----------+--------------+------+-----+---------+----------------+
| id        | int(11)      | NO   | PRI | NULL    | auto_increment |
| Name      | varchar(512) | YES  |     | NULL    |                |
| RoastType | varchar(512) | YES  |     | NULL    |                |
+-----------+--------------+------+-----+---------+----------------+
3 rows in set (0.07 sec)

Discovery

Profiling using studio

**Use the loadtest.js script to generate load

Monitoring - On-premises and Hosted

Register and get license key at http://strongloop.com

Supported metrics backends are:

- `statsd://[<host>][:<port>][/<scope>]`
- `graphite://[<host>][:<port>]`
- `syslog:[?[application=<application>][,priority=<priority>]` (syslog is the
  Unix logging framework, it doesn't exist on Windows)
- `splunk://[<host>]:<port>`
- `log:[<file>]`
- `debug:[?pretty[=<true|false>]]`

Object Tracking

$ slc run --cluster cpus -d
$ slc runctl objects-start <pid>

Start a cluster

Start object tracking

Start CPU profiling

$ slc runctl cpu-start <pid>

Stop CPU profiling

$ slc runctl cpu-start <pid>
  1. Open .cpuprofile files with Chrome Developer Tools for viewing and analysis.

$ slc runctl heap-snapshot <pid>

Take a heap snapshot

  1. Open .cpuprofile files with Chrome Developer Tools for viewing and analysis.

Hosted monitoring

Title Text

Loopback - An Open-source Node.js Framework

Develop APIs from scratch with the open source Loopback.io framework.

  • Middleware API Engine

  • ORM

  • Aggregation & Mashups

  • API Explorer (Swagger)

  • Fine Grained Access Control (ACLs)

  • Data Replication

  • IsoMorphic JS and Client SDKs. 

 

Create new app

$ slc loopback

     _-----_
    |       |    .--------------------------.
    |--(o)--|    |  Let's create a LoopBack |
   `---------´   |       application!       |
    ( _´U`_ )    '--------------------------'
    /___A___\    
     |  ~  |     
   __'.___.'__   
 ´   `  |° ´ Y ` 

[?] Enter a directory name where to create the project: (.) 

Connect an API to a datasource

$ cd  myproject 
$ slc loopback:datasource

Install the connector

$ npm install loopback-connector-mongodb --save

Create models

 

$ slc loopback:model

API Explorer and FrontEnd App Integration

$ slc run
Browse your REST API at http://localhost:3000/explorer
Web server listening at: http://localhost:3000/

API Explorer 

Query your data using the mongo CLI

> use test
switched to db test
> show collections;
CoffeeShop
system.indexes
> db.CoffeeShop.find().limit(4)
{ "Name" : "Aesop's Tables", "pos" : { "lat" : 37.5332, "lng" : -85.7302 }, "_id" : ObjectId("543f0787f9fed2bb759fc146") }
{ "Name" : "Award Winners Cafe", "pos" : { "lat" : 41.3896, "lng" : -88.12595 }, "_id" : ObjectId("543f0787f9fed2bb759fc147") }
{ "Name" : "Acadia Cafe", "pos" : { "lat" : 44.454, "lng" : -68.04902 }, "_id" : ObjectId("543f0787f9fed2bb759fc148") }
{ "Name" : "Adams Coffee Shop", "pos" : { "lat" : 42.25639, "lng" : -71.01119 }, "_id" : ObjectId("543f0787f9fed2bb759fc149") }
> 

Model Relations

$ slc loopback:relation

Customer - hasMany Review

  • property name for the relation: reviews
  • custom foreign key: authorId

CoffeeShop - hasMany Review

  • property name for the relation: reviews
  • custom foreign key: coffeeshop

Review - belongsTo Customer

  • property name for the relation: author
  • custom foreign key: authorId

Review - belongsTo CoffeeShop

  • property name for the relation: coffeeshop
  • custom foreign key: coffeeshop

Fine Grained Access Control (ACLs) 

$ slc loopback:acl
[?] Select the model to apply the ACL entry to: CoffeeShop
[?] Select the ACL scope: All methods and properties
[?] Select the access type: All (match all types)
[?] Select the role: Any unauthenticated user
[?] Select the permission to apply: Explicitly deny access
getIndexes()

StrongLoop Controller

  • OSS Controller

  • Cluster Mgmt.

  • Hot Deploy

  • Rolling Restart

  • Cluster State Mgmt.

  • Process Mgmt.

  • Deploy

 

Debugging 

 

Run your app in the debugger

$ slc debug

Build and package the application

 

$ rm -rf node_modules // Shouldnt commit node_modules into master branch  
$ git init .
$ git commit -a -m "Initial commit" 
$ slc build --onto deploy --install --commit

Run StrongLoop Process Manager server


$ slc pm -l 7777

Deploy to Process Manager Server

$ slc deploy http://localhost:7777 deploy  

Run an application cluster

$ slc run --cluster=cpus

Control the cluster using slc runctl

  • View cluster status

  • Change cluster size

  • Restart worker processes

  • ​Stop c

Manage application logs

$ slc run --cluster cpus --log getstarted.%w.log

Open Source mBaaS (mobile Backend as a Service)

Push Notification

GeoLocation

CoffeeShop.attachTo(oracle);
var here = new GeoPoint({lat: 10.32424, lng: 5.84978});
CoffeeShop.find( {where: {location: {near: here}}, limit:3}, function(err, nearbyShops) {
console.info(nearbyShops); // [CoffeeShop, ...]
});

Storage Service

Offline Sync

Thank you!

Made with Slides.com