KVS

Abstract

Erlang Term 

Database

@5HT

How not to 

store a cat?



     SQL -- relational algebra for cats

     ORM -- trying to do OLAP in Java

     XML -- suck <and> blow waste

     JSON -- JavaScript database


How our fathers stored cats?



        BerkleyDB -- 1986 

        LDAP -- X.500 Interface to KV

CAP



CA -- SQL RDBMS, Mnesia (non-distr)
CP -- BDB,Redis,BigTable,MongoDB 
AP -- Dynamo,Riak,Couch,Cassandra

Consistency



GOSSIP AP -- Eventual

PAXOS CP -- Hadoop

RAFT CP -- etcd

ZAB CP -- ZooKeeper HBase Storm

2PC CP -- SQL RDBMS


Erlang AP KV Boom



CouchDB Couchbase Riak

Mnesia KAI LeoDB 

Hibari HanoiDB

Pitfalls

How to store a 

cat-to-cat subscriptions?


Secondary Indexes


1. LevelDB -- HanoiDB, Riak

2. DETS -- Mnesia, KAI

Pitfalls

How to put each cat in a box?


Consistency


1. Reconcile -- Editing History

2. Sequence Lock -- Erlang Process

REST

Riak/WebMachine 

CouchDB/Own

FarWest/Cowboy

Abstractions

Feuerlabs/kvdb 

project-fifo/fifo-db

KVS is...

Statically Typed

Mutable

Social Schema Based

Key-Value

Term Store

with Secondary Indexes

and REST ... Abstract DB

Social Schema




Users Products Reviews Feeds

Comments Groups Likes Acl

Teams Tags Messages

Payments Accounts

Transactions Purchases

KVS



Riak / Mnesia / KAI Backends

Cowboy/N2O REST

JSON/BERT/...

Records <-> Maps <-> Proplists

1500 LOC

KVS



#user { id, mail, name }

#product { id, name }

#review { id, body, product }





KVS



#user { id, next, prev, ... }

#product { id, next, prev, ... }

#review { id, next, prev, ... }

KVS core

  

     #container { id, top, count } 

         #iterator { id, container, 

                               prev, next

                               feeds = [] }


KVS Core



           #subscription { who,
                                            whom,
                                            what }

Macros


-define(CONTAINER, id, top, count=0).

-define(ITERATOR(Container),

id, container=Container,

feed_id, prev, next, feeds=[]).


-record(feed, {?CONTAINER, aclver}).

-record(user, {?ITERATOR(feed), name, email}).

-record(product, {?ITERATOR(feed), name}).


Schema



-define(CONTAINERS, [
    {feed, record_info(fields, feed)},
    {acl, record_info(fields, acl)},
    {transaction,  record_info(fields, transaction)},
    {payment,  record_info(fields, payment)} ]).


API


   POST/PUT kvs:add(#user{}).

        DELETE kvs:remove(user,2).

PUT/PATCH kvs:put(#user{id=2}).

               GET kvs:entries(Feed,user).

               GET kvs:get(user,2).

               GET kvs:all(user).

Feed Server



For each Container

we spawn Erlang Lock Process 

for Feed Write Operations 

ordering and consistensy


Console


> kvs:join('root@synrc.com').> kvs:initialize().> kvs:init_db().> kvs:dir().[{table,"user"}, {table,"feed"}, {table,"acl"}, {table,"transaction"}, ... ]

Console


> rr(kvs).> kvs:add(#user{id="mes@ua.fm"}).> kvs:entries(      kvs:get(feed,users),      user,undefined).[#user{id="mes@ua.fm"}]> kvs:get(user,"mes@ua.fm").{ok,#user{id="mes@ua.fm"}



1500 LOC


@5HT @doxtop


https://github.com/synrc/kvs

KVS

By Maxim Sokhatsky

Made with Slides.com

KVS

KVS is statically typed, Erlang Term, abstract database with secondary indexes support and REST handler. It has default schema that was designed for social applications, blog engines, app stores.

  • 8,514

More from Maxim Sokhatsky