DATA!!!

the importance of GOOD plumbing!

how do we manage it?

what's the big deal about visualization?!

OVERVIEW TONIGHT!

a little JSON gymnastics

examples on Nitrous

using Node.js and MongoDB

(https://docs.mongodb.org/getting-started/node/introduction/)

Google Analytics...

A/B testing, online campaigns
digital marketing
location of visitors
what people click on
performance of web pages
mobile access

how does it all work?

landscape! webserver, client, google machines!!!

landscape! webserver, client, ~~google~~ NOSEY machines!!!

landscape! repo, web-based front end, clients!!!

GitHub

Firechat is NOT using the internet, it turns phones into communication channels, using bluetooth!

We will see others!

Understanding it!!!

"Visual Data Analytics"

Understanding it????????

"Visual Data Analytics"

Our examples...

My Victoria
- filter social media, post pictures, dynamic!
CityVibes
- get events from server, filter and display with crowd sourced data
Taps and Pours
- get the restaurants, get the breweries, ...
"Binder"
- post and get issues, dynamic!

plumbing simply does NOT get better than this!!!

<script>

function myFunction(selection) {  
   if (selection === 1) 
      alert("yes");  
   else
      alert("no");
}
</script>

<p onclick="myFunction(1)"><a href="#!">First Option</a></p>
<p onclick="myFunction(2)"><a href="#!">Second Option</a></p>

IT IS ALL ABOUT THE PLUMBING!!!

(this IS plumbing TOO!!!)

{"restaurants": [
       {
           "name":"Drake",
           "breweries":["Hoyne","Phillips", "Moon Under Water"],
           "coord" : { "lat": 48.4280838370059, "long": -123.368818023075}
       },
       {
           "name":"Churchill",
           "breweries":["Spinnakers","Phillips"],
           "coord" : { "lat": 48.4255563713968, "long": -123.368125887101}

       }
]}

Mocking up the data with JSON... really makes you think!

{"restaurants": [
       {
           "name":"Drake",
           "coord" : { "lat": 48.4280838370059, "long": -123.368818023075}
       },
       {
           "name":"Churchill",
           "coord" : { "lat": 48.4255563713968, "long": -123.368125887101}
       }
]}

Or maybe there are just restaurants...

{"breweries": [
       {
           "name":"Hoyne",
           "rests":["Drake","Churchill"]
       },
       {
           "name":"Phillips",
           "rests":["Churchill"]
       }
]}

and could have a 2nd collection....


<script>
var brewdata = 
'{"breweries": [ {"name":"Hoyne","rests":["Drake","Churchill"]},{"name":"Phillips", "rests":["Churchill"]} ]}';

var restaurantdata =
'{"restaurants": [{"name":"Drake", "coord" : { "lat": 48.4280838370059, "long": -123.368818023075}}, {"name":"Churchill", "coord" : { "lat": 48.4255563713968, "long": -123.368125887101}} ]}';


/// you can play around by doing your link step in something that looks like this

var myBrews = JSON.parse(brewdata);
var myRests = JSON.parse(restaurantdata);


function myFunction() {  
  alert("Watch this! " + myBrews.breweries[0].name + "is at " + myBrews.breweries[0].rests[0] 
   + " located " + myRests.restaurants[0].coord.lat);
}
</script>

<p onclick="myFunction()"><a href="#!">Try it!</a></p>

starting to put it all together...

{"breweries": [
{
"name":"Hoyne",
"rests":["Drake","Churchill"]
},
{
"name":"Phillips",
"rests":["Churchill"]
}
]}

{ key : value } eg., {"name":"Phillips"}, or

{ key : [list of values] } eg., "rests":["Drake","Churchill"]

DATA

Full stack development: Front end and Back end

JavaScript all the way is very popular

MEAN Stack!

MongoDB is a document database each document is a data structure like this... similar to JSON! :)

{
   "_id" : ObjectId("54c955492b7c8eb21818bd09"),
   "address" : {
      "street" : "2 Avenue",
      "zipcode" : "10075",
      "building" : "1480",
      "coord" : [ -73.9557413, 40.7720266 ],
   },
   "borough" : "Manhattan",
   "cuisine" : "Italian",
   "grades" : [
      {
         "date" : ISODate("2014-10-01T00:00:00Z"),
         "grade" : "A",
         "score" : 11
      },
      {
         "date" : ISODate("2014-01-16T00:00:00Z"),
         "grade" : "B",
         "score" : 17
      }
   ],
   "name" : "Vella",
   "restaurant_id" : "41704620"
}

MongoDB is a document database

each document is a data structure like this...

similar to JSON! :)

{
   "_id" : ObjectId("54c955492b7c8eb21818bd09"),
   "address" : {
      "street" : "2 Avenue",
      "zipcode" : "10075",
      "building" : "1480",
      "coord" : [ -73.9557413, 40.7720266 ],
   },
   "borough" : "Manhattan",
   "cuisine" : "Italian",
   "grades" : [
      {
         "date" : ISODate("2014-10-01T00:00:00Z"),
         "grade" : "A",
         "score" : 11
      },
      {
         "date" : ISODate("2014-01-16T00:00:00Z"),
         "grade" : "B",
         "score" : 17
      }
   ],
   "name" : "Vella",
   "restaurant_id" : "41704620"
}

MongoDB is a document database

each document is a data structure like this...

similar to JSON! :)

_id is the primary key!

  ~
➜  ~  mongoimport --db test --collection restaurants --drop --file primer-dataset.json
connected to: 127.0.0.1
Mon Nov  2 00:49:01.901 dropping: test.restaurants
Mon Nov  2 00:49:03.078 check 9 25359
Mon Nov  2 00:49:03.084 imported 25359 objects
➜  ~

➜  ~  ps -aux | grep mongod
mongodb     53  0.2  4.1 553084 41804 ?        Ssl  Oct13  64:00 /usr/bin/mongod --config /etc/mongodb.conf
➜  ~

Getting Started with MongoDB (Node.js Edition)

mongodb has to be running...

can import a data set (drop means the old one is lost!)...

➜  ~  npm install mongodb
> kerberos@0.0.17 install /home/nitrous/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos
> (node-gyp rebuild) || (exit 0)
make: Entering directory `/home/nitrous/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos/build'
  CXX(target) Release/obj.target/kerberos/lib/kerberos.o
  CXX(target) Release/obj.target/kerberos/lib/worker.o
  CC(target) Release/obj.target/kerberos/lib/kerberosgss.o
../lib/kerberosgss.c:36:0: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
 #pragma clang diagnostic push
 ^
../lib/kerberosgss.c:37:0: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
 #pragma clang diagnostic ignored "-Wdeprecated-declarations"
 ^
../lib/kerberosgss.c: In function ‘authenticate_gss_client_wrap’:
../lib/kerberosgss.c:362:19: warning: variable ‘server_conf_flags’ set but not used [-Wunused-but-set-variable]
   char buf[4096], server_conf_flags;
                   ^
../lib/kerberosgss.c: At top level:
../lib/kerberosgss.c:930:0: warning: ignoring #pragma clang diagnostic [-Wunknown-pragmas]
 #pragma clang diagnostic pop
 ^
  CC(target) Release/obj.target/kerberos/lib/base64.o
  CXX(target) Release/obj.target/kerberos/lib/kerberos_context.o
  SOLINK_MODULE(target) Release/obj.target/kerberos.node
  COPY Release/kerberos.node
make: Leaving directory `/home/nitrous/node_modules/mongodb/node_modules/mongodb-core/node_modules/kerberos/build'
mongodb@2.0.47 node_modules/mongodb
├── es6-promise@2.1.1
├── readable-stream@1.0.31 (isarray@0.0.1, inherits@2.0.1, string_decoder@0.10.31, core-util-is@1.0.1)
└── mongodb-core@1.2.20 (bson@0.4.19, kerberos@0.0.17)
➜  ~

Getting Started with Node.js



//this is the way we will communicate with the server
var http = require('http');
var dispatcher = require('httpdispatcher');

var MongoClient = require('mongodb').MongoClient;
var assert = require('assert');
var ObjectId = require('mongodb').ObjectID;
var url = 'mongodb://0.0.0.0:27017/test';

var myText = "";  //building my own text here to ship to the client!


var findRestaurants = function(db, callback) {
   var cursor = db.collection('restaurants').find( { "address.zipcode": "10075" } );
   cursor.each(function(err, doc) {
      assert.equal(err, null);
      if (doc != null) {
         myText = myText + "{\"name\": \"" + doc.name + "\" , \"cuisine\": \"" + doc.cuisine + "\", \"location\" : {\"lng\":" + doc.address.coord[0] + ", \"lat\":" + doc.address.coord[1] + "} },";
      } else {
         callback();
      }
   });
};

MongoClient.connect(url, function(err, db) {
  assert.equal(null, err);
  findRestaurants(db, function() {
      db.close();
  });
});

//A sample GET request
dispatcher.onGet("/page1", function(req, res) {
      res.setHeader('Access-Control-Allow-Origin', '*');
      res.setHeader("Access-Control-Allow-Headers", 'Content-Type, X-Requested-With');
      res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
      res.setHeader('Access-Control-Allow-Credentials', 'true');
      res.setHeader('Content-Type', 'application/json');
      res.write("{\"venues\" : [" + myText.substring(0, myText.length - 1) + "]}");
      res.end();
});

//A sample POST request
dispatcher.onPost("/post1", function(req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Got Post Data');
});

//We need a function which handles requests and send response
function handleRequest(request, response){
    try {
        console.log(request.url);
        dispatcher.dispatch(request, response);
    } catch(err) {
        console.log(err);
    }
}

//Create a server
var server = http.createServer(handleRequest).listen(3000, '0.0.0.0');

//Writes to the console that we are in business!
console.log('Server running at http://0.0.0.0:3000/');

We can start node and declare a MongoClient and connect to it!

BACK TO AN EXAMPLE!

Let's walk through the example in our code repo:

sampleGmapsAndJSON.html

Let's REALLY pull apart the plumbing!!!!

IN THEORY, how hard is it to change the pipes around?

IN PRACTICE, how hard is it to change the pipes around?

DATA!!!

Google Analytics...

GitHub

We will see others!

Understanding it!!!

Understanding it????????

Our examples...

My Victoria

CityVibes

Taps and Pours

"Binder"

IT IS ALL ABOUT THE PLUMBING!!!

Mocking up the data with JSON... really makes you think!

Or maybe there are just restaurants...

and could have a 2nd collection....

starting to put it all together...

starting to put it all together...

starting to put it all together...

DATA

Getting Started with MongoDB (Node.js Edition)

Getting Started with Node.js

We can start node and declare a MongoClient and connect to it!

BACK TO AN EXAMPLE!