Distributed Products
Microservices Architecture from the Trenches
Distributed
@francmichal
www.mfranc.com
Michal Franc
ENGINEER | LEADER | SPEAKER
Wroclaw University of Technology
-
Tech 4 Good
-
Social Giving Platform
-
20+ mln users
-
3 bln $ raised
-
100k funded causes
Let me tell you a story...
Monolithic app
master
The Master Repo
Growth
Spaghetti Code
Code Entropy
- A computer program that is used will be modified
- When a program is modified, its complexity will increase, provided that one does not actively work against this.
Code Entropy
https://en.wikipedia.org/wiki/Software_entropy
time
complexity
ver1
ver2
ver3
refactor
maintenance costs
scale
problems
danger zone
timeouts
down times
smaller issues
Monolith Scaling
vertical
horizontal
Monolith Scaling
33.3%
33.3%
33.3%
Bottleneck
Data Replication :(
master
Git Force Push
Non-scaling
process
Git Flow?
2 weeks
Big Bang Deploys
-
Scalability
-
Availability
-
Productivity
Distributed Product
Scalability
Scalability is the ability of a system to expand to meet your business needs.
https://msdn.microsoft.com/en-us/library/aa578023.aspx
Scalable
Works well on small data set, number of requests as working on large ones
can handle increased loads of work
load
stability
Monolith Scaling
vertical
horizontal
( scale up}
( scale out}
Vertical
-
less maintnance
-
non-continuous upgrade
-
one point of failure
-
limited
http://epis.com/powermarketinsights/wp-content/uploads/2016/09/cpu_dev.png
Hardware Limits
CPU
AWS EC2
https://aws.amazon.com/ec2/instance-types/
Horizontal
-
more options
-
continuous upgrade
-
higher maintenance
-
more complex
-
not a silver bullet!
https://www.techopedia.com/definition/17035/amdahls-law
Scale Cube
https://thenewstack.io/from-monolith-to-microservices/
0.0.0
1.1.1
Y and Z scaling
https://thenewstack.io/from-monolith-to-microservices/
Functional Decomposition
Functional Decomposition
Functional Decomposition
Functional Decomposition
Z scaling
Data partitioning
??
Data partitioning
Different db engine
https://msdn.microsoft.com/en-gb/library/dn589795.aspx
This is a 'perfect' partitioning
Benefits
-
scalability
-
performance
-
security
-
flexibility
-
continuous upgrades
-
no single point of failure
-
availability
Problems
-
maintenance
-
costs
-
complexity
-
transactions
-
different programming model
'Joins'
Local copy
denormalized
Monolithic system
-> RDBMS
Distributed
-> freedom
http://db-engines.com/en/
http://highscalability.com/blog/2011/6/20/35-use-cases-for-choosing-your-next-nosql-database.html
Document Databases
Graph Databases
Relational Databases
Object Oriented Databases
Key-Value Stores
BigTable Clones
Data Structure Servers
Grid Databases
DB Options
https://www.thoughtworks.com/insights/blog/nosql-databases-overview
Ok ok
how to do it?
DB 'integration' layer
First Iteration
... but Single point of failure
Natural step from monolithic DB
-
data storage type bias
-
tightly coupled services
-
non-distributed data model
Maintaining existing requirements ...
Code structure follows data model
Distributed data model will encourage distributed code
-
table per service
-
schema per service
2nd Step
forcing relational :(
3rd Step
Monolithic DB is still here :(
Asynchronous Calls
Gentle 'Peaks'
Availability
availability is a measure of how often the application is available for use
Single point of failure
Cached data
Static Page
No Interaction
Monolithic UI
Hidden
Disabled - Cached Data
Distributed
UI
Introducing Message Bus
-
independent teams
-
independent daily releases
-
smaller orchestration
-
simplified processes
Productivity
team1
team2
Cloud
Release
Release
Adopting distributed architecture requires changes in the team structure and culture
Distributed system require 'distributed' teams
Distributed
Not a Free Lunch
-
more complex
-
expensive
-
different programming model
-
different patterns
-
many traps ( micromonolith )
Network Layer
Is a beast
-
latency
-
complexity
-
debbuging
-
security
-
connectivity
Assume that there will be problems
You can't beat the nature of the network
Defensive design and coding
Do it only if you fell it is needed
Micro Monolith
distributed monolith
Micro monolithic Architecture
Decoupled Dependancies
Questions!
Thank you
@francmichal
www.mfranc.com
Distributed Products Microservices Architecture from the Trenches
Distributed Products
By Michal Franc
Distributed Products
- 1,465
