Mongodb

aggregation & Map-Reduce

Tomasz Ducin
16th April 2014, Warsaw 

plan

  • who doesn't know mongo?
  • motivation (big data)
  • simple aggregation operations 
  • aggregation framework
  • map-reduce

Simple aggregation operations

TOOLS SIMILAR TO SQL (docs)

Aggregation framework


aggregating operations

api

PIPELINE

db.collection.aggregate([ step_1, step_2, ..., step_n ])
  • each step is one of 7 operations
  • combining (repetative) operations in any order
  • output of one becomes input of another 

demo

let's try it out

aggregation fw limitations

https://www.youtube.com/watch?v=8BQzKXI-_wE#t=65

  • result set limited to 16MB of memory (the maximum BSON document size is 16MB as well)
  • 10% of memory on a machine
  • sharding

map-reduce


algorithm

  • 3 steps
  • map function: scope
  • reduce function: ~recursion, common key