Distributed Products

Microservices Architecture from the Trenches

Distributed

@francmichal

www.mfranc.com

Michal Franc

 

ENGINEER | LEADER | SPEAKER

Wroclaw University of Technology

  • Tech 4 Good

  • Social Giving Platform

  • 20+ mln users

  • 3 bln $ raised

  • 100k funded causes

Let me tell you a story...

Monolithic app

master

The Master Repo

Growth

Source

Spaghetti Code

Code Entropy

 

  1. A computer program that is used will be modified
  2. When a program is modified, its complexity will increase, provided that one does not actively work against this.

Code Entropy

https://en.wikipedia.org/wiki/Software_entropy

time

complexity

ver1

ver2

ver3

refactor

maintenance costs

scale

problems

danger zone

timeouts

down times

smaller issues

Monolith Scaling

vertical

horizontal

Monolith Scaling

33.3%

33.3%

33.3%

Bottleneck

Data Replication :(

master

Git Force Push

Non-scaling

process

Git Flow?

2 weeks

Big Bang Deploys

  • Scalability

  • Availability

  • Productivity

Distributed Product

Scalability

Scalability is the ability of a system to expand to meet your business needs.

https://msdn.microsoft.com/en-us/library/aa578023.aspx

Scalable

Works well on small data set, number of requests as working on large ones

can handle increased loads of work

load

stability

Monolith Scaling

vertical

horizontal

( scale up}

( scale out}

Vertical

  • less maintnance

  • non-continuous upgrade

  • one point of failure

  • limited

http://epis.com/powermarketinsights/wp-content/uploads/2016/09/cpu_dev.png

Hardware Limits

CPU

AWS EC2

https://aws.amazon.com/ec2/instance-types/

Horizontal

  • more options

  • continuous upgrade

  • higher maintenance

  • more complex

  • not a silver bullet!

https://www.techopedia.com/definition/17035/amdahls-law

Scale Cube

https://thenewstack.io/from-monolith-to-microservices/

0.0.0

1.1.1

Y and Z scaling

https://thenewstack.io/from-monolith-to-microservices/

Functional Decomposition

Functional Decomposition

Functional Decomposition

Functional Decomposition

Z scaling

Data partitioning

??

Data partitioning

Different db engine

https://msdn.microsoft.com/en-gb/library/dn589795.aspx

This is a 'perfect' partitioning

Benefits

  • scalability

  • performance

  • security

  • flexibility

  • continuous upgrades

  • no single point of failure

  • availability

Problems

  • maintenance

  • costs

  • complexity

  • transactions

  • different programming model

'Joins'

Local copy

denormalized

Monolithic system
-> RDBMS

Distributed
-> freedom

http://db-engines.com/en/

http://highscalability.com/blog/2011/6/20/35-use-cases-for-choosing-your-next-nosql-database.html

Document Databases

Graph Databases

Relational Databases

Object Oriented Databases

Key-Value Stores

BigTable Clones

Data Structure Servers

Grid Databases

DB Options

https://www.thoughtworks.com/insights/blog/nosql-databases-overview

Ok ok

how to do it?

DB 'integration' layer

First Iteration

... but Single point of failure

Natural step from monolithic DB

 

  • data storage type bias

  • tightly coupled services

  • non-distributed data model

Maintaining existing requirements ...

Code structure follows data model

Distributed data model will encourage distributed code

  • table per service

  • schema per service

2nd Step

forcing relational :(

3rd Step

Monolithic DB is still here :(

Asynchronous Calls

Gentle 'Peaks' 

Availability

 availability is a measure of how often the application is available for use

Single point of failure

Cached data

Static Page

No Interaction

Monolithic UI

Hidden

Disabled - Cached Data

Distributed
UI

Introducing Message Bus

  • independent teams

  • independent daily releases

  • smaller orchestration

  • simplified processes

Productivity

team1

team2

Cloud

Release

Release

Adopting distributed architecture requires changes in the team structure and culture

 

Distributed system require 'distributed' teams

Distributed

Not a Free Lunch

  • more complex

  • expensive

  • different programming model

  • different patterns

  • many traps ( micromonolith )

Network Layer

Is a beast

  • latency

  • complexity

  • debbuging

  • security

  • connectivity

Assume that there will be problems

You can't beat the nature of the network

Defensive design and coding

Do it only if you fell it is needed

Micro Monolith

distributed monolith

Micro monolithic Architecture

Decoupled Dependancies

Questions!

Thank you

@francmichal

www.mfranc.com

Distributed Products

By Michal Franc

Made with Slides.com

Distributed Products

  • 1,299

More from Michal Franc