Data Pipelines With Apache Spark and Couchbase
Jowanza Joseph
@jowanza
www.jowanza.com
Agenda
- About Me ~ 1 Min
- What are data pipelines? ~ 3 Min
- Challenges ~ 3 Min
- Couchbase ~ 3 Min
- Spark ~ 3 Min
- Marshalling ~ 2 Min
- Demo ~ 15 Min
- Questions
About Me
- Software Engineer at One Click Retail
- Spark, Scala / Java
- Apache Spark Field Book
- Cycling / Golf
- Father
What are data pipelines?
Challenges
- Volume
- Drowning
- Latency
- Swiss Army Knife
- Flexibility
Volume
Drowning
Latency
Swiss Army Knife
Flexibility
Cap Theorem
Why Spark?
- Glue Framework
- Type Safety
- Distributed
- Fault Tolerant
Why Couchbase?
- Batch & Stream
- Full-text search
- Flexible Schema
- N1QL
- Easy Distribution
- Minimal Operational Overhead
Marshall
