Fun with Streams and LevelDB

node.js intro


  • node.js is Google's V8 javascript VM + libuv
  • libuv provides I/O functionality not normally available to javascript developers
  • event-driven functional programming with asynchronous I/O
  • single threaded (in developer space, libuv takes care of threading I/O)
  • obey the law of the event loop...don't block, finish fast
  • currently in the .10 version
  • rabid community
  • https://github.com/joyent/node/wiki/Projects,-Applications,-and-Companies-Using-Node

Streams?

The way to push data through a functional pipeline


node core:
var s = new Stream();

WritableStream
ReadableStream

Sugar

  • Separates IO concerns from logic concerns

  • Allows modules to control "backpressure".
    • e.g. return false on write() and then emit 'drain'

  • use pipe() to send data to the next function in the pipeline.



LevelDB?

Glorified file manager


written by Google engineers

utilizes similar file organization to BigTable

NO SERVER out-of-the-box (runs in-process)

VERY node.js friendly
VERY stream friendly

how do it know?


  • Sorted key/value file store (sst)
  • in-memory write log
  • out-of-process compaction/sorting
  • bloom filters

fast!



yeah...fast
depends a bit on value size
still...come on...it's just fast

STOP talking!

Fun with Streams and LevelDB

By Michael Atkins

Made with Slides.com

Fun with Streams and LevelDB

  • 202

More from Michael Atkins