Fun with Streams and LevelDB

node.js intro


  • node.js is Google's V8 javascript VM + libuv
  • libuv provides I/O functionality not normally available to javascript developers
  • event-driven functional programming with asynchronous I/O
  • single threaded (in developer space, libuv takes care of threading I/O)
  • obey the law of the event loop...don't block, finish fast
  • currently in the .10 version
  • rabid community
  • https://github.com/joyent/node/wiki/Projects,-Applications,-and-Companies-Using-Node

Streams?

The way to push data through a functional pipeline


node core:
var s = new Stream();

WritableStream
ReadableStream

Sugar

  • Separates IO concerns from logic concerns

  • Allows modules to control "backpressure".
    • e.g. return false on write() and then emit 'drain'

  • use pipe() to send data to the next function in the pipeline.

https://github.com/dominictarr/event-stream


LevelDB?

Glorified file manager


written by Google engineers

utilizes similar file organization to BigTable

NO SERVER out-of-the-box (runs in-process)

VERY node.js friendly
VERY stream friendly

how do it know?


  • Sorted key/value file store (sst)
  • in-memory write log
  • out-of-process compaction/sorting
  • bloom filters

fast!


http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html

yeah...fast
depends a bit on value size
still...come on...it's just fast

STOP talking!

Made with Slides.com