Expensive Bugs

Knight Capital Group

Expensive Bugs

Expensive Bugs

Ariane 5 Crash

Expensive Bugs

Flexcoin

Expensive Bugs

Polyglot Persistence

- Relational databases

- Distributed databases

- Pittfalls & integration

Content

RDBMS

ACID

RDBMS

ACID

TRANSACTIONS

RDBMS

ACID

Atomicity

Consistency

Isolation

Durability

RDBMS

All operations in a transaction are fully completed, or not completed at all.

Atomicity

ACID 

The data updated by a transaction should keep the database in a valid state.

Consistency

ACID 

Transactions are processed as if they run one-by-one after each other.

Isolation

ACID 

Isolation

ACID 

Transfer 500€ from A to B

single transaction

Isolation

ACID 

Transfer 500€ from A to B

Transfer 350€ from A to C

concurrent transactions - serialization non-lock

Isolation

ACID 

Isolation Levels

When a transaction is completed successfuly the result is stored permanently.

Durability

ACID 

Durability

ACID 

Scalability Issues

Database revolution

Database revolution

Graph

Tim Cools ( @timcoolsnet )

Use the database that is best suited for the job

Polyglot Persistence

Polyglot Persistence

Distributed data

CAP Theorem

Consistency

Availability

Partitions

Distributed data

CAP Theorem

Consistency

All data is consistent on all the nodes

CAP Theorem

Consistency in Non-Transactional Distributed Storage Systems

CAP Theorem

High availability

of that data (for updates)

CAP Theorem

Partitions

Tolerance to network partitions

CAP Theorem

Only 2 out of 3

C

A

P

CAP Theorem

CA

C

A

RDBMS

Single-node
NoSql

Martin Kleppmann, 2014

CAP Theorem

CA

C

A

RDBMS

Single-node
NoSql

Martin Kleppmann, 2014

CAP Theorem

CP

C

P

MongoDB

BigTable

Redis

MemcachedDB

CAP Theorem

AP

A

P

Cassandra

Riak

CouchDB

SimpleDB

CAP Theorem

The modern CAP goal should be to maximize combinations of consistency and availability that make sense for the specific application.

CAP Twelve Years Later: How the "Rules" Have Changed

Developer Productivity

Developer Productivity

Model

ORM

Developer Productivity

Model

public class Route
{
    private readonly List<Stop> _stops = new List<Stop>();

    public Guid Id { get; private set; }
    public RouteStatus Status { get; private set; }
    public DateTime? Date { get; private set; }

    public IEnumerable Stops => _stops;

    public void Plan(DateTime date)
    {
        if (date < DateTime.Today.AddDays(1))
        {
            throw new InvalidOperationException("Route can only plan from tomorrow.");
        }

        Status = RouteStatus.Planned;
        Date = date;
    }

    ...

Developer Productivity

Document Oriented Database

{  
   "ID":"3f86959a-6970-485c-9a1a-ba168cb77212",
   "Date":"2016-05-26T00:00:00",
   "Stops": [  
     {  
       "Name":"Home",
       "Position":{ "Latitude":4.481736, "Longitude":51.197894 }
     },
     {  
       "Name":"Build Stuff",
       "Position":{ "Latitude":4.400963, "Longitude":50.828417 }
     },
     {  
       "Name":"Home",
       "Position":{ "Latitude":4.481736, "Longitude":51.197894 }
     }
   ],
   "Status":"Planned"
}

Polyglot Persistence

Tim Cools ( @timcoolsnet )

Each database system added is added complexity

Polyglot Persistence

Tim Cools ( @timcoolsnet )

Many databases are not mature yet

Polyglot Persistence

Tim Cools ( @timcoolsnet )

Integration between data-stores is hard to get right

Polyglot Persistence

Tim Cools ( @timcoolsnet )

Shared database integration

Integration

Application database

Integration

Integration

Services

Event Driven Architecture

Integration

CQRS

synchronous

Integration

CQRS

synchronous

Integration

CQRS

synchronous

Integration

CQRS

asynchronous

Integration

CQRS+ES

(a)synchronyous

Single Log

Integration

The Log: What every software engineer should know... (LinkedIn)

Great power comes with great responsibility

Polyglot Persistence

Be responsible

Learn your current databases

Tim Cools ( @timcoolsnet )

Marten + PostgreSQL

Document

Database

Event Store

Open source .NET library

Tim Cools ( @timcoolsnet )

Thank You

https://goo.gl/p3GYNw

Tim Cools ( @timcoolsnet )

Feedback Form

Questions?

Tim Cools ( @timcoolsnet )