Distributed Cache with Alluxio
Jowanza Joseph
@jowanza
Agenda
- About Me
- Problem Space
- Explored Solutions
- Alluxio
- Future Directions
- Questions
- Software Engineer at Pluralsight
- Speaking too much
- Open Source
- IoT/Model Rocketry
Problems
- Multi-tenant set up
- 300 TB of Data
- ~16,000 Concurrent Requests
- 1 TB Max User Query
- p99 latency of 20 Seconds
- 200 Virtual CPUs
- 200 TB of RAM
Latency
Alluxio
Improvements
- p99 Latency of 7 Seconds
- Support ticket decreased by 90%
- Scaled to hundreds of TBs of data
Cost Profile
Cost Profile
Future Directions
Try Alluxio
- Documentation
- Docker Image
- Local Mode