Transport and Servers

Transport

How do we talk to servers?

  • 7-Layer OSI Model
  • We make requests
  • which are addressed to a domain (e.g. www.google.com)
  • which is mapped to an external IP (e.g. 8.8.8.8)
  • the request goes to our ISP and then...
  • out through a sea of routers (all with routing tables)
  • until a server attached to that router is confirmed
  • the server responds and the route is re-traced back to the client

How is this possible??

 

Protocols

  • HTTP(S) - HyperText Transfer Protocol (Secure)

  • FTP - File Transfer Protocol

  • SMTP - Simple Mail Transfer Protocol (href="mailto: ...")

  • GIT

  • WS - Web Sockets

  • SSH - Secure Shell

  • TEL - Telephone Service (href="tel: ...")

  • SMS - Short Messaging Service (href="sms: ...")

  • skype, telnet, IRC, etc. etc.

Agreed Ways of Doing Things

HTTP(S)

What can we send/receive?

  • Static Files
    • inc. files templated by the server
    • files like .js, .css,
    • .doc, .pdf
  • Data
    • xml
    • JSON <-- this is the preferred way we send data

 

List of common types

JSON

  • JavaScript Object Notation
  • Turns js arrays and objects into transportable string
  • Natural methods for conversion:  
    • JSON.stringify(obj) - prepare for transport
    • JSON.parse(str) - decode post-transport
[
  {
    name: 'James',
    age: 40,
    isDeveloper: true  
  }
]
[
  {
    "name": "James",
    "age": 40,
    "isDeveloper": true  
  }
]

stringify

parse

Notes on stringify

Ways to use it

Methods (HTTP VERBS)

  • GET
  • POST
  • PUT
  • PATCH
  • DELETE
  • HEAD
  • OPTIONS

Full List: https://www.iana.org/assignments/http-methods/http-methods.xhtml

GET

  • GET is when you just ask for a resource
  • Your browser makes GET requests when you enter a URL in the address bar and hit return
  • GET is the default action on an HTML Form
  • GET requests have no body, so any information you need to pass with the request to enable the server to make choices is put in the parameters: /users?p=1
  • If you're sending private information don't use GET

POST

  • POST is when you want to send information to a server privately
  • POST requests have a request body in which the information is stored (basically it's like an object)
  • POST often sends NEW information to a server

PUT

  • PUT works like POST except that it is meant to send information to update information the server currently holds
  • It's supposed to be for upsert-ing (add or replace in collection)
  • PUT also has a request body
  • body usually contains the changes to a record

DELETE

  • DELETE does what it says on the tin
  • It does have a request body but it didn't used to and its use is sketchy at best...

HEAD, PATCH & OPTIONS

  • HEAD gets the headers you'd get if you made a GET request to that URL
    • Headers are a list of key-pair values you can send with a request to help the two machines understand how to deal with the request.
  • PATCH is meant to be for partial updating of a resource
  • OPTIONS shows what methods are available (so it might return [GET, POST, OPTIONS]

HTTP HEADERS

See here for standard headers

Headers are information the transportation layer needs to know about, not your application.

 

Headers contain 'meta' information about the call itself:

  • How is it encoded?
  • What response type will it accept in return?
  • Does it have a security token?

 

Headers can be the standard ones or you can make your own custom ones.

How do we know if it is successful or not?

HTTP Status Codes

  • 100s - information (getting there...)
    • 100 CONTINUE
  • 200s - success (Here you go!)
    • 200 OK
    • 201 CREATED
  • 300s - redirection (resource location shennanigans)
    • 302 FOUND
  • 400s - client errors (You f*cked up)
    • 400 - BAD REQUEST
    • 401 - UNAUTHORISED
    • 404 - NOT FOUND
  • 500s - server errors (I f*cked up)
    • 500 - SERVER ERROR

Full List:

https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

 

As cats: https://http.cat/

As dogs: https://httpstatusdogs.com/

THE POSTMAN CLIENT

Services

  • A service DOES SOMETHING for you - it RETURNS something, like a function does...
  • An example might be an image service that gives you resized images: https://unsplash.it/
  • Or processes text: https://www.foaas.com/
  • It does not hold state
  • You are not able to modify anything within it

APIs

  • Stands for: Application Programming Interface
  • Allow users to interact with and use your services
  • APIs are for intelligent interaction
  • Many Types
  • You CAN modify records that it holds
  • You CAN send commands that will enact events (possibly real life ones)
  • Examples: Booking a delivery slot with tesco or manipulating tweets from a twitter account
  • We will look at REST APIs

What is REST?

  • REpresentational State Transfer
  • Sharing the state of your app between the client and server to keep them in sync
  • e.g. Shopping basket:
    • See what's in basket
    • Add to basket
    • Modify basket items (quantity, colour options, etc.)
    • Remove from Basket
  • CRUD (Create, Read, Update & Delete)
  • Synonymous with POST, GET, PUT, DELETE
  • The system runs off intelligently structured URLs:
    • GET /cars <-- Gets all the cars
    • GET /cars/ferarri <-- Gets all the Ferraris
    • POST /cars <-- Adds a car to the list

Anatomy of a URL

https://jmsherry.github.io/ajax-templating/#APIS

Try it out

http://restler3.luracast.com/examples/_007_crud/index.php/authors

GET    authors      ⇠ Authors::index()
POST   authors      ⇠ Authors::post()
DELETE authors/{id} ⇠ Authors::delete()
GET    authors/{id} ⇠ Authors::get()
PUT    authors/{id} ⇠ Authors::put()

Server-side

We're going to build a REST server

  • We'll use Node
  • with Express.js - a server framework
  • and MongoDB - a NOSQL database package
  • With Mongoose.js (for ORM - Object Relational Mapping)

Procedure for starting a project

  • Use your command line
  • Create a new folder in a relevant place and move into it
  • git init to turn it into a git repo
  • npm init to get package management
  • git add -A && git commit -m "npm initialised"
  • npm i express mongoose body-parser
  • Create a public folder, which contains HTML, CSS & JS
  • Create a server.js file which we'll run to start the server

Express JS

https://expressjs.com/en/api.html

Creating a basic server

var express = require('express');
var app = express();

app.listen(3333, function(){
    console.log('Server is listening');
});
  • Get the express software
  • execute it to return a working server
  • set the server to listen on a port (3333)
  • and log a message when it is listening

To run this file, go to your command line and do:

npm start

(in your server.js)

Creating a file server

var express = require('express');
var app = express();

app.use(express.static('public'));

app.listen(3333, function(){
    console.log('Server is listening');
});
  • app.use();
    • loads middleware. Middlewares are functions which are run when a request is made of the server
  • express.static is a middleware function that allows static file serving
  • Create some html files in the public directory and then request them from localhost:3333/<myFile>.html

Route Handlers

var express = require('express');
var app = express();

app.use(express.static('public'));

app.get('/cars', function(request, response){
    var cars = [{name: 'ferarri'}, {name: 'lamborghini'}];
    return res.json(cars);
});

app.listen(3333, function(){
    console.log('Server is listening');
});
  • Route handlers act like user interaction events (click, mouseover, etc.) in the browser
  • a GET request aimed at a certain URL (e.g. /cars) will run the handler function
  • You get two 3 arguments passed to a hander

Getting the request body

var express = require('express');
var bodyParser = require('body-parser');
var app = express();

app.use(express.static('public'));

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: false }));

// parse application/json
app.use(bodyParser.json());

app.get('/cars', function(request, response){
    var cars = [{name: 'ferarri'}, {name: 'lamborghini'}];
    return res.json(cars);
});

app.post('/cars', function(request, response){
    return res.json(req.body);
});

app.listen(3333, function(){
    console.log('Server is listening');
});
  • Route handlers act like user interaction events (click, mouseover, etc.) in the browser

Routing

  • Determines what functions run and what data is passed to them
  • Full guide here
  • Route handlers are loaded sequentially, so if 2 routes match the request the one which is written EARLIER will be triggered
  • Route Params:
app.get('/users/:userId/books/:bookId?', function (req, res) { // ? = optional
  res.send(req.params)
});

/*
*   Route path: /users/:userId/books/:bookId
*   Request URL: http://localhost:3000/users/34/books/8989
*   req.params: { "userId": "34", "bookId": "8989" }
*/

Query Params:

// GET /search?q=tobi+ferret
req.query.q
// => "tobi ferret"

Databases

Things that allow us to persist data

List: https://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis?

Installing MongoDB

https://docs.mongodb.com/manual/administration/install-community/

Connect to DB

var express = require('express');
var bodyParser = require('body-parser');
var mongoose = require('mongoose'); // DB control program

var app = express();

app.use(express.static('public'));

mongoose.Promise = global.Promise;
var promise = mongoose.connect('mongodb://localhost/first_servers');

promise.then(function(db) {
  console.log('DATABASE CONNECTED!!');
}).catch(function(err){
  console.log('CONNECTION ERROR', err);
});



app.listen(3333, function(){
    console.log('Server is listening');
});

Creating Records

  • MongoDB is a noSQL non-relational database
  • Each record is a text file
  • records don't have to bear any relationship to each other
  • If we want consistency then we should use ORM
  • Mongoose helps us create database record
  • It ensures that the information is properly formatted
  • Mongoose works by creating models
  • then storing information according rules (known as schemas) for creating that type of model
  • Each CRUD operation on a model has a callback with access to req, res and next()

Schemas

var mongoose = require('mongoose');
var Schema = mongoose.Schema;

var blogSchema = new Schema({
  title:  String,
  author: String,
  body:   String,
  comments: [{ body: String, date: Date }],
  date: { type: Date, default: Date.now },
  hidden: Boolean,
  meta: {
    votes: Number,
    favs:  Number
  }
});

var Blog = mongoose.model('Blog', blogSchema);

CRUD: READ

api.get('/people', function(req, res, next){

    // Person is a model, like Blog
    Person.find({ 
        occupation: /host/,
        'name.last': 'Ghost',
        age: { $gt: 17, $lt: 66 },
        likes: { $in: ['vaporizing', 'talking'] }
    })
    .limit(10)
    .sort({ occupation: -1 })
    .select({ name: 1, occupation: 1 })
    .exec(function(err, people){
        return res.json(people);
    });

});

http://mongoosejs.com/docs/queries.html

CRUD: CREATE

api.post('/tutors', function(req, res){

  var tutorData = req.body.tutor;
  console.log('tutorData', tutorData);

  var tutor = new Tutor(tutorData);
  console.log('tutor', tutor);

  tutor.save(function(err, model){
    if (err) {
      return res.status(500).send(err);
    }
    return res.sendStatus(201);
  });

});

http://mongoosejs.com/docs/models.html

CRUD: DELETE

api.delete('/tutors/:tutorid', function(req, res){

  var tutorIdForDeletion = req.params.tutorId;

  Tutor.remove({ _id: tutorIdForDeletion }, function (err) {
    if (err) return handleError(err);
    // removed!
    res.sendStatus(204);
  });

});

http://mongoosejs.com/docs/models.html

CRUD: UPDATE

api.put('/tutors/:tutorid', function(req, res){

    var tutorId = req.params.tutorId;

    Tutor.update({ _id: tutorId }, { age: 40 }, function (err, raw) {
        if (err) return handleError(err);
        console.log('The raw response from Mongo was ', raw);
        return res.sendStatus(200);
    });

});

http://mongoosejs.com/docs/api.html#model_Model.update

Task

  • Create a REST API
  • It can be about anything you want
  • You need 4(5) route handlers
    • GET /{things} - get all records
    • GET /{things}/:id? - get one record
    • POST: /{things} - add new
    • PUT: /{things}/:id - update a thing
    • DELETE /{things}/:id - delete a thing

JS: Transport & Servers

By James Sherry

JS: Transport & Servers

Focussing on HTTP, Servers and REST

  • 1,421

More from James Sherry