Which problems are we trying to solve?

Chumba Data Staging

Which problems are we trying to solve?

Chumba Data Staging

Which problems are we trying to solve?

Lossiness

Timeliness

Customer Privacy-ness

Chumba Data Staging

Which problems are we trying to solve?

Lossiness

Timeliness

Customer Privacy-ness

Which problems are we trying to solve?

Lossiness

Timeliness

Data Privacy-ness

Which problems are we trying to solve?

Lossiness

Timeliness

Data Privacy-ness

One important note...

Chumba Data Staging

Which problems are we trying to solve?

Lossiness

Timeliness

Data Privacy-ness

One important note...

Chumba Data Staging

How did we do it?

One important note...

Chumba Data Staging

How did we do it?

Which problems are we trying to solve?

One important note...

Chumba Data Staging

How do we solve these problems?

How do we solve these problems?

Append-only Audit Logging

Populators

Delivery Pipeline

How do we solve these problems?

Append-only Audit Logging

Populators

Delivery Pipelines

How do we solve these problems?

Append-only Audit Logging

Populators

Delivery Pipelines

How do we solve these problems?

Append-only Audit Logging

Populators

Delivery Pipelines

Append-only Audit Logging

Populators

Delivery Pipelines

How do we solve these problems?

Append-only Audit Logging

How do we solve these problems?

Populators

Delivery Pipelines

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 10000 150 15m ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 10000 150 15m ago

12000

170

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123

12000

170

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 170 0s ago

12000

170

0s ago

12000

170

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 170 5s ago

150

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000

150

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 150 0s ago

150

0s ago

2150

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 150 5s ago

2150

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000

2150

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 2150 0s ago

2150

0s ago

How many times did Matt purchase Gold Coins?

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 2150 0s ago

How many times did Matt purchase Gold Coins?

What are Matt's total Sweeps Coins winnings?

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 2150 0s ago

What are Matt's total Sweeps Coins winnings?

How many times did Matt purchase Gold Coins?

Does Matt have better luck on Tuesdays?

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 2150 0s ago

Does Matt have better luck on Tuesdays?

What are Matt's total Sweeps Coins winnings?

What are Matt's total Sweeps Coins winnings?

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 2150 0s ago

Does Matt have better luck on Tuesdays?

Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 12000 2150 5s ago
Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp

14000

2170

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123
Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 0s ago

14000

2170

0s ago

13800

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 2170
Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 5s ago
123 Spin 200 0 0s ago

14000

2170

0s ago

13800

0s ago

13600

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 2170
Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 10s ago
123 Spin 200 0 5s ago
123 Spin 200 0 0s ago

14000

2170

0s ago

13800

0s ago

13600

0s ago

19400

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 2170
Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 11s ago
123 Spin 200 0 6s ago
123 Spin 200 0 1s ago
123 Win 6000 0 0s ago

14000

2170

0s ago

13800

0s ago

13600

0s ago

19400

0s ago

19200

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 2170
Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 35s ago
123 Spin 200 0 30s ago
123 Spin 200 0 25s ago
123 Win 6000 0 24s ago
123 Spin 200 0 0s ago

14000

2170

0s ago

13800

0s ago

13600

0s ago

19400

0s ago

19200

0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 19200 2170 5s ago

14000

2170

0s ago

13800

0s ago

13600

0s ago

19400

0s ago

19200

0s ago

Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 35s ago
123 Spin 200 0 30s ago
123 Spin 200 0 25s ago
123 Win 6000 0 24s ago
123 Spin 200 0 0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 19200 2170 5s ago

14000

2170

0s ago

13800

0s ago

13600

0s ago

19400

0s ago

19200

0s ago

Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 35s ago
123 Spin 200 0 30s ago
123 Spin 200 0 25s ago
123 Win 6000 0 24s ago
123 Spin 200 0 0s ago

Append-only Audit Logging

How do we solve these problems?

Customer ID ​Gold Coins ​Sweeps Coins Last Updated
123 19200 2170 5s ago
Customer ID ​Type ​Gold Coins Sweeps Coins Timestamp
123 Purchase 2000 20 35s ago
123 Spin 200 0 30s ago
123 Spin 200 0 25s ago
123 Win 6000 0 24s ago
123 Spin 200 0 0s ago

14000

2170

0s ago

13800

0s ago

13600

0s ago

19400

0s ago

19200

0s ago

Append-only Audit Logging

Populators

Delivery Pipelines

How do we solve these problems?

Append-only Audit Logging

Populators

Delivery Pipelines

How do we solve these problems?

S3 Buckets

Firehose

Streams

Landing

Staging

Delivery Pipelines

How do we solve these problems?

S3 Buckets?

Firehose?

Streams?

Landing?

What is a Delivery Pipeline?

Staging?

Delivery Pipelines

How do we solve these problems?

What is a Delivery Pipeline?

S3 Buckets?

Firehose?

Streams?

Landing?

Staging?

Delivery Pipelines

How do we solve these problems?

UP

Amazon Kinesis Data Firehose

Delivery Pipelines

How do we solve these problems?

UP

Amazon Kinesis Data Firehose

Delivery Pipelines

How do we solve these problems?

UP

Amazon Kinesis Data Firehose

Delivery Pipelines

How do we solve these problems?

UP

Amazon Cloudwatch

Delivery Pipelines

How do we solve these problems?

UP

AWS Identity & Access Management

Amazon Cloudwatch

Delivery Pipelines

How do we solve these problems?

UP

AWS Identity & Access Management

Amazon Cloudwatch

Delivery Pipelines

How do we solve these problems?

UP

AWS Identity & Access Management

Delivery Pipelines

How do we solve these problems?

Chumba

DnA

Amazon S3

Amazon Simple Notification Service

Delivery Pipelines

How do we solve these problems?

Chumba

DnA

Amazon Simple Notification Service

Amazon S3

Delivery Pipelines

How do we solve these problems?

Chumba

DnA

Delivery Pipelines

Data

Delivery Pipelines

Data

Delivery Pipelines

Logs + Metrics

Data

Logs + Metrics

Data

Fewer Moving Parts

Supports Audit Logging

Improved Observability

Delivery Pipelines

Improved Observability

Fewer Moving Parts

Supports Audit Logging

Improved Observability

Delivery Pipelines

Improved Observability

Fewer Moving Parts

Supports Audit Logging

Improved Observability

Delivery Pipelines

Improved Observability

Fewer Moving Parts

Supports Audit Logging

Improved Observability

Delivery Pipelines

Append-only Audit Logging

Populators

Delivery Pipelines

How do we solve these problems?

Improved Observability

Fewer Moving Parts

Supports Audit Logging

Append-only Audit Logging

Populators

Delivery Pipelines

How do we solve these problems?

Populators

How do we solve these problems?

Populators

How do we solve these problems?

Populators

How do we solve these problems?