Events Over Status Fields

Before we proceed

- Not married to any technology

- Everything is awesome as a POC first

- Everyone should read Shapeup: https://basecamp.com/shapeup/shape-up.pdf

{
    "tags": {
        "album": "5403b0c2f5f5680200b45d89",
        "song": "5403b0c2f5f5680200b45d91",
        "locatorname": null,
        "eventtype": "startTrackPlayback",
        "artist": "5403b0c2f5f5680200b45d87",
        "locatortype": "buttonClick",
        "user": null,
        "device": "windows-unknown",
        "browser": "chrome"
    },
    "timestamp": 1427223676.554661,
    "host": "localhost",
    "message": null,
    "time_taken": null,
    "client": "UNKNOWN",
    "exc_info": null,
    "logger": "analytic",
    "type": "realtime_api",
    "levelname": "INFO"
}

Data now

  • Having a file means that any tool can use it
  • Conformed JSON standard is http://jsonlines.org/
  • This serves as an interface.
  • My process has all it takes to be repeatable.
  • Not a slave to technology

Protocols

  • Binary Protocols: MySQL Client/Server Protocol
  • Binary Protocols(Cont): JNI, RMI
  • Binary Protocols(Cont): Basic Socket connections, Thrift, Google Protocol Buffers,
  • HTTP Based: Rest, Soap
  • Websockets
  • File IO
  • Inspect websocket on patari.pk
  • They're seriously fast
  • Best Designed as a contract and implemented by the front end

https://cloud.google.com/blog/products/gcp/how-to-build-a-bi-dashboard-using-google-data-studio-and-bigquery

The ONION (Looks like a lot, just a weekend)

- Shareable embedded dashboards for analytics and other fun

 

Owning the stack means easy to consume data back in the APP via Pandas or BQ

(because backend dev's architects love building UI's)

Events Over Status Fields

By Iqbal Talaat Bhatti

Events Over Status Fields

Talk first given on the Sydney Mongodb Users Group on 25th August 2015.

  • 94