The Importance of GOOD Plumbing!

how do we manage the code/data?  

OVERVIEW TONIGHT!  

hello GitHub,

a little JSON gymnastics,

examples on BlueMix,

using Node.js and Express

http://nodey.mybluemix.net/

• Pitches and followups
  • Heather says GO FOR IT!!
• Hackvictoria?
• Bluemix day!
  • will get to the "labs"
  • SOSExample set off the fire alarms!
• Ain't no mountain high enough!
  • what is next in the "experiment"?

Web
Server

RECAP!

development repo, webserver, client, google machines!!!

Web
Server

GOOGLE

you &
me!

GitHub
git

Landscape

MAIN BRANCH of repo set up on Web Server...

Web
Server

GOOGLE

you &
me!

GitHub
git

clients query search engine for content...

Web
Server

GOOGLE

you &
me!

GitHub
git

many options returned (and rendered)!

Web
Server

GOOGLE

you &
me!

GitHub
git

was there EVER a time before GOOGLE?

client accesses web server

Web
Server

GOOGLE

you &
me!

GitHub
git

web server returns JS, CSS, HTML to client browser

Web
Server

GOOGLE

you &
me!

GitHub
git

interactions with content generate data
for analytics!

Web
Server

GOOGLE

you &
me!

GitHub
git

GitHub is free (typically open source...)

Hosting for Web Servers is *not* typically free!

What do you use?

GitHub
git

Web
Server

Open source projects?  Let's all update the same (shared) file at the same time!!!!!

https://guides.github.com/activities/hello-world/

GitHub

• code hosting platform

• version control and collaboration

• repositories, branches, commits, and pull requests

• "repos" are shared in "branches"

• shared changes are "commits" to a "branch"

• "pull requests" are a workflow mechanisms to review changes before they are "committed"  

GitHub?

• A repo organizes files in a project

  • files (data and code!), images, folders... 

  • README.md

    • markdown :)

    • we have a readme.md in our repo 

      • want to you add a link to YOUR repo there!!!

• LET THE GAMES BEGIN! :) 

Step 1: Create a Repo

• Branching allows us to have different versions of a file
• master is the definitive branch
  • other branches experiment with edits before committing them to master
• a branch off the master branch is a copy at that point in time.
  • if changes are made to the master you could pull in those updates
  • ready to share?  "merge" with master!
• GO FOR IT!!!!!

Step 2: Branch!!!

• your readme-edits branch, is a copy of master

  • make some edits!

  • saved changes are called commits 

  • each commit has an associated commit message,

    • a description explaining why a particular change was made

  • messages capture the history of your changes

• GO FOR IT!!!

Step 3: Make & Commit Changes!!!

• you have changes in a branch off of master

• you can open a pull request

  • ​you’re proposing your changes and requesting review

    • pull in your contribution and merge with the branch

  • pull requests show diffs, or differences, of the content from both branches

• @mention system 
• open pull requests in your own repository and merge them yourself!!!
• GO FOR IT!!!

Step 4: Open a Pull Request!!!

• After review

  • add the changes to the master branch

  • DELETE the temporary branch!

• GO FOR IT!!!

NOOOOOOWWWWWWWWWW  do the same for our shared README at:

              https://github.com/ycoady/FortUVic2016

Step 5: MERGE!!!

DATA

Full stack development:  Front end and Back end

JavaScript all the way is very popular

MEAN Stack!

MongoDB is a document database each document is a data structure like this... similar to JSON! :) 

{
   "_id" : ObjectId("54c955492b7c8eb21818bd09"),
   "address" : {
      "street" : "2 Avenue",
      "zipcode" : "10075",
      "building" : "1480",
      "coord" : [ -73.9557413, 40.7720266 ],
   },
   "borough" : "Manhattan",
   "cuisine" : "Italian",
   "grades" : [
      {
         "date" : ISODate("2014-10-01T00:00:00Z"),
         "grade" : "A",
         "score" : 11
      },
      {
         "date" : ISODate("2014-01-16T00:00:00Z"),
         "grade" : "B",
         "score" : 17
      }
   ],
   "name" : "Vella",
   "restaurant_id" : "41704620"
}

MongoDB is a document database

each document is a data structure like this...

similar to JSON! :) 

{
   "_id" : ObjectId("54c955492b7c8eb21818bd09"),
   "address" : {
      "street" : "2 Avenue",
      "zipcode" : "10075",
      "building" : "1480",
      "coord" : [ -73.9557413, 40.7720266 ],
   },
   "borough" : "Manhattan",
   "cuisine" : "Italian",
   "grades" : [
      {
         "date" : ISODate("2014-10-01T00:00:00Z"),
         "grade" : "A",
         "score" : 11
      },
      {
         "date" : ISODate("2014-01-16T00:00:00Z"),
         "grade" : "B",
         "score" : 17
      }
   ],
   "name" : "Vella",
   "restaurant_id" : "41704620"
}

MongoDB is a document database

each document is a data structure like this...

similar to JSON! :) 

_id is the primary key!

  ~
➜  ~  mongoimport --db test --collection restaurants --drop --file primer-dataset.json
connected to: 127.0.0.1
Mon Nov  2 00:49:01.901 dropping: test.restaurants
Mon Nov  2 00:49:03.078 check 9 25359
Mon Nov  2 00:49:03.084 imported 25359 objects
➜  ~

➜  ~  ps -aux | grep mongod
mongodb     53  0.2  4.1 553084 41804 ?        Ssl  Oct13  64:00 /usr/bin/mongod --config /etc/mongodb.conf
➜  ~

Getting Started with MongoDB (Node.js Edition)

mongodb has to be running...

can import a data set (drop means the old one is lost!)...

➜  ~  npm install mongodb
> kerberos@0.0.17 install /home/nitrous/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos
> (node-gyp rebuild) || (exit 0)
make: Entering directory `/home/nitrous/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos/build'
  CXX(target) Release/obj.target/kerberos/lib/kerberos.o
  CXX(target) Release/obj.target/kerberos/lib/worker.o
  CC(target) Release/obj.target/kerberos/lib/kerberosgss.o
../lib/kerberosgss.c:36:0: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
 #pragma clang diagnostic push
 ^
../lib/kerberosgss.c:37:0: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
 #pragma clang diagnostic ignored "-Wdeprecated-declarations"
 ^
../lib/kerberosgss.c: In function ‘authenticate_gss_client_wrap’:
../lib/kerberosgss.c:362:19: warning: variable ‘server_conf_flags’ set but not used [-Wunused-but-set-variable]
   char buf[4096], server_conf_flags;
                   ^
../lib/kerberosgss.c: At top level:
../lib/kerberosgss.c:930:0: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
 #pragma clang diagnostic pop
 ^
  CC(target) Release/obj.target/kerberos/lib/base64.o
  CXX(target) Release/obj.target/kerberos/lib/kerberos_context.o
  SOLINK_MODULE(target) Release/obj.target/kerberos.node
  COPY Release/kerberos.node
make: Leaving directory `/home/nitrous/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos/build'
mongodb@2.0.47 node_modules/mongodb
├── es6-promise@2.1.1
├── readable-stream@1.0.31 (isarray@0.0.1, inherits@2.0.1, string_decoder@0.10.31, core-util-is@1.0.1)
└── mongodb-core@1.2.20 (bson@0.4.19, kerberos@0.0.17)
➜  ~

Getting Started with Node.js 

//this is the way we will communicate with the server
var http = require('http');
var dispatcher = require('httpdispatcher');

var MongoClient = require('mongodb').MongoClient;
var assert = require('assert');
var ObjectId = require('mongodb').ObjectID;
var url = 'mongodb://0.0.0.0:27017/test';

var myText = "";  //building my own text here to ship to the client!


var findRestaurants = function(db, callback) {
   var cursor = db.collection('restaurants').find( { "address.zipcode": "10075" } );
   cursor.each(function(err, doc) {
      assert.equal(err, null);
      if (doc != null) {
         myText = myText + "{\"name\": \"" + doc.name + "\" , \"cuisine\": \"" + doc.cuisine + "\", \"location\" : {\"lng\":" + doc.address.coord[0] + ", \"lat\":" + doc.address.coord[1] + "} },";
      } else {
         callback();
      }
   });
};

MongoClient.connect(url, function(err, db) {
  assert.equal(null, err);
  findRestaurants(db, function() {
      db.close();
  });
});

//A sample GET request
dispatcher.onGet("/page1", function(req, res) {
      res.setHeader('Access-Control-Allow-Origin', '*');
      res.setHeader("Access-Control-Allow-Headers", 'Content-Type, X-Requested-With');
      res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
      res.setHeader('Access-Control-Allow-Credentials', 'true');
      res.setHeader('Content-Type', 'application/json');
      res.write("{\"venues\" : [" + myText.substring(0, myText.length - 1) + "]}");
      res.end();
});

//A sample POST request
dispatcher.onPost("/post1", function(req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Got Post Data');
});

//We need a function which handles requests and send response
function handleRequest(request, response){
    try {
        console.log(request.url);
        dispatcher.dispatch(request, response);
    } catch(err) {
        console.log(err);
    }
}

//Create a server
var server = http.createServer(handleRequest).listen(3000, '0.0.0.0');

//Writes to the console that we are in business!
console.log('Server running at http://0.0.0.0:3000/');

We can start node and declare a MongoClient and connect to it!

BACK TO AN EXAMPLE!

• HTTP has two operations we will use

  • Post

    • sending data to the server

  • Get

    • receiving data from the server

• Take a quick look at our example code

• https://github.com/ycoady/FortUVic2016/blob/master/app.js

plumbing simply does NOT get better than this!!!

next for the "experiment"?

• what do you need next?