Mongodb
aggregation & Map-Reduce
Tomasz Ducin
16th April 2014, Warsaw
plan
who doesn't know mongo?
motivation (big data)
simple aggregation operations
aggregation framework
map-reduce
Simple aggregation operations
TOOLS SIMILAR TO SQL (
docs
)
count (
dbcmd
,
collection
)
distinct (
dbcmd
,
collection
)
group (
dbcmd
,
collection
)
Aggregation framework
aggregating operations
project
- select, reshape - 1:1
match
- filter - n:1
group
- aggregate - n:1
unwind
- split - 1:n
sort
- 1:1
skip
- n:1
limit
- n:1
others (
redact
,
geoNear
,
out
)
api
PIPELINE
db.collection.aggregate([ step_1, step_2, ..., step_n ])
each step is one of 7 operations
combining (repetative) operations in any order
output of one becomes input of another
demo
let's try it out
aggregation fw limitations
https://www.youtube.com/watch?v=8BQzKXI-_wE#t=65
result set limited to 16MB of memory (the maximum BSON document size is 16MB as well)
10% of memory on a machine
sharding
map-reduce
algorithm
3 steps
map function: scope
reduce function: ~recursion, common key