Distributed Cache with Alluxio

Jowanza Joseph

@jowanza

Agenda

• About Me
• Problem Space
• Explored Solutions
• Alluxio
• Future Directions
• Questions

• Software Engineer at Pluralsight
• Speaking too much
• Open Source
• IoT/Model Rocketry

Problems

• Multi-tenant set up
• 300 TB of Data
• ~16,000 Concurrent Requests
• 1 TB Max User Query
• p99 latency of 20 Seconds
• 200 Virtual CPUs
• 200 TB of RAM

Latency

Alluxio

Improvements

• p99 Latency of 7 Seconds
• Support ticket decreased by 90%
• Scaled to hundreds of TBs of data

Cost Profile

Cost Profile

Future Directions

Try Alluxio

• Documentation
• Docker Image
• Local Mode

Questions?