Expensive Bugs

Knight Capital Group

Expensive Bugs

Expensive Bugs

Ariane 5 Crash

Expensive Bugs

Flexcoin

Expensive Bugs

By sending thousands of simultaneous requests, the attacker was able to "move" coins from one user account to another until the sending account was overdrawn, before balances were updated.

Flexcoin

Expensive Bugs

Data consistency is YOUR responsibility!

NOT of the technology you use...

Expensive Bugs

Polyglot Persistence

- Relational vs distributed

Marten

- PostgreSQL

- Document database

- Event Store

Content

Data storage used to be (more) simple

RDBMS

RDBMS

ACID

RDBMS

ACID

TRANSACTIONS

RDBMS

ACID

Atomicity

Consistency

Isolation

Durability

RDBMS

All operations in a transaction are fully completed, or not completed at all.

Atomicity

ACID 

The data updated by a transaction should keep the database in a valid state.

Consistency

ACID 

Transactions are processed as if they run one-by-one after each other.

Isolation

ACID 

Isolation

ACID 

Transfer 500€ from A to B

single transaction

Isolation

ACID 

Transfer 500€ from A to B

Transfer 350€ from A to C

concurrent transactions - serialization

Isolation

ACID 

Isolation Levels

When a transaction is completed successfuly the result is stored permanently.

Durability

ACID 

Data storage is more complicate these days

Database revolution

Scalability Issues

Database revolution

Development productivity

Relational

Key/value

Database revolution

Document

Graph

Event Log

Search

Use the database that is best suited for the job

Polyglot Persistence

Polyglot Persistence

Distributed data

CAP Theorem

Consistency

Availability

Partitions

Distributed data

CAP Theorem

Consistency

All data is consistent on all the nodes

CAP Theorem

Consistency in Non-Transactional Distributed Storage Systems

CAP Theorem

High availability

of that data (for updates)

CAP Theorem

Partitions

Tolerance to network partitions

CAP Theorem

Only 2 out of 3

C

A

P

CAP Theorem

CA

C

A

RDBMS

PostgreSQL

MySql

SQL Server

Oracle

CAP Theorem

CP

C

P

MongoDB

BigTable

Redis

MemcachedDB

CAP Theorem

AP

A

P

Cassandra

Riak

CouchDB

SimpleDB

CAP Theorem

The modern CAP goal should be to maximize combinations of consistency and availability that make sense for the specific application.

CAP Twelve Years Later: How the "Rules" Have Changed

Developer Productivity

Developer Productivity

Aggregate Oriented Database

{  
   "ID":"3f86959a-6970-485c-9a1a-ba168cb77212",
   "Date":"2016-05-26T00:00:00",
   "Stops":{  
      "$type":"System.Collections.Generic.List`1[[Soloco.Talks.PolyglotPersistence.Stop, Soloco.Talks.PolyglotPersistence]], mscorlib",
      "$values":[  
         {  
            "Name":"Home",
            "Position":{ "Latitude":4.481736, "Longitude":51.197894 }
         },
         {  
            "Name":"WooRank",
            "Position":{ "Latitude":4.400963, "Longitude":50.828417 }
         },
         {  
            "Name":"Home",
            "Position":{ "Latitude":4.481736, "Longitude":51.197894 }
         }
      ]
   },
   "Status":"Planned"
}

Polyglot Persistence

Each database system added is added complexity

Polyglot Persistence

Many databases are not mature yet

Polyglot Persistence

Integration between data-stores is hard to get right

Polyglot Persistence

Shared database integration

Integration

Application database

Integration

Integration

Services

Event Driven Architecture

Integration

CQRS

synchronous

Integration

CQRS

synchronous

Integration

CQRS

asynchronous

Integration

CQRS+ES

(a)synchronyous

Single Log

Integration

The Log: What every software engineer should know... (LinkedIn)

Great power comes with great responsibility

Distributed data

Marten

Marten + PostgreSQL

PostgreSQL

You are in good company

PostgreSQL

And many more...

Marten

Document database

Event Store

> Demo

Marten

Based on proven technlogy

ACID Support

Scalability options

Developer productivity

Marten

Code: Github

Chat: Gitter

Documentation

Package: Nuget

Questions?

Tim Cools ( @timcoolsnet )