on the bleeding edge
Highly scalable, distributed,
cloud-based architecture using Scala,
actor model and RESTful Web Services
Andrzej Dębski
Bartłomiej Szczepanik
Agenda
-
Project origin and goals
- Scalability problem
-
Solution building blocks
- Application architecture
- Application evaluation
- Achievements
- Future work
project timeline
- Oct 2012 - PaaSage project started
- Lufthansa Systems accepts to provide industrial use case
- Nov 2013 - Master thesis proposal
- Feb 2014 - Development started, 3 weeks on-site in Berlin
- Mar 2014 till now - Continuation of the work remotely
- Dec 2014 - PaaSage deadline for the industrial use case
project goals
-
Build highly scalable and distributed application
- Prove that chosen building blocks
live up to their expectations
our background
- Almost no previous Scala & Akka experience
- Only theoretical knowledge about distributed programming, Domain-Driven Design, NoSQL, computer systems evaluation
- Ready to dive into the topic
scalability
we need horizontal scalability
Reactive programming
Reactive programming
www.reactivemanifesto.org
building blocks
- Blends functional and object-oriented world
- Scales to programmer needs
- Fast-paced evolution
- Type safe with type inference
-
Java interoperability (runs on JVM)
-
Steep learning curve
- Actor concurrency model
- Distributed by design
- Actor location transparency
- Let it crash philosophy
- Part of Typesafe stack
for reactive applications
akka modules
- Many useful add-ons to basic Akka functionality
- Developed both by Typesafe and community
- Interesting examples:
- Akka Persistence
- Akka Cluster
- Akka Cluster Sharding
- Distributed Publish Subscribe in Cluster
- Akka Reactive Streams
- The World's Leading Graph Database
- Index free adjacency
- ACID compliance
- REST/HTTP toolkit
- Implemented on top of Akka in Scala
- Does not use servlets
- Encourages asynchronous processing
-
Will be part of Akka project soon
Domain-driven design
- Set of strategic and tactical patterns
in software engineering - More and more popular
- Fits well with reactive programming - commands & events
-
All business logic resides in the "domain model"
Command-query responsibility segregation (CQRS)
event sourcing
on the bleeding edge
Application business requirements
- Flight scheduling service
- Legs and airplane rotations
- Schedule validity checks
- Reports
- Simple domain
- Focus on architecture
Application architecture
write model Scalability
read models scalability
implementation details
- Akka-persistence
- Distributed publish-subscribe event bus
- CQRS+Event Sourcing mini-framework
- Akka-clustering
- Data sharding based on business rules
- Batch event processing
EVALUATION - THESIS
Chosen building blocks enable creating highly scalable distributed applications with way less effort than before.
evaluation facts
-
Flexiant cloud provider
- Simulating production environment
- Both synthetic and real workloads
- Lots of available specialized tools, e.g. Gatling, Kamon
- We have to consider many low level JVM details
Example test scenarios
- Responsiveness
- Time of big data sets import
- Different request rates
- Different query distribution
- Rebalancing influence on response time
- Resilience
- Mean time to recover when a node crashes
- System startup time
JVM warmup matters
data import time
current progress
- CQRS architecture
- Neo4j read model
- Most of business logic implemented
- Tackled lots of technical problems
- First evaluations done, more to come
- We are approaching the scale-out phase
- Got in touch with Typesafe
- Joined Krakow DDD and Scala Community
Ideas for future work
- CQRS + Event Sourcing framework for Scala
- Generic benchmark specification
for distributed applications
On the bleeding edge
Andrzej Dębski
andrzejdebski91 @ gmail.com
Bartłomiej Szczepanik
mequrel @ gmail.com