SENG3011

Monolith vs Microservices 🪨

• Monolith: a single large application that contains the entire software solution
  • One service to rule them all
• Microservices: A series of small-scale services that communicate with one another
  • Each service does one task well

Microservices Case Study: Netflix 📖

• Cloudshift - from server to cloud (2008)
• Migrate from monolith to microservices, service by service
• Started with non customer facing; move to customer facing
• By 2013, Netflix’s API gateway was handling two billion daily API edge requests, managed by over 500 cloud-hosted microservices
• By 2017, its architecture consisted of over 700 loosely coupled microservices

Microservices Case Study: Netflix 📖

• Adopting microservice architecture is a business decision as well as an engineering decision
• Today, Netflix:
  • Makes around $8 billion a year and
  • Streams approximately six billion hours of content weekly
  • More than 220 million subscribers in 190 countries, and it continues to grow
  • Revolutionised how we consume media and started the streaming wars
• Cloud costs per streaming start a fraction of those in the data centre

Microservices Case Study: Netflix 📖

Microservices Case Study: Uber 📖

• Issues in business growth growth was related to a monolithic architecture:
  • More inertia to ship new features
  • More bugs and risk in deployments
  • Much harder to onboard new engineers to work on the system
  • The architecture couldn't meet the demand and popularity of the application - not scalable
•  Architecture
  • REST API to access monolith
  • Three adapters with embedded API functions
  • Single MySQL database
  • All features in the monolith

Microservices Case Study: Uber 📖

• One engineering team per microservice
• Reduced coupling and dependency
• Focus on scaling at the individual service level - address bottlenecks
• Improved speed of development, performance, overall quality, fault tolerance
• Need to create global standards for services to ensure compatibility and interoperability

Microservices Case Study: Uber 📖

Defining your Microservices 📡

• Let's try an example - design WebCMS as a microservice architecture
• System requirements (simplified)
  • Lecturers can create courses and put up content
  • Lecturers can create assessments
  • Students can view content and submit assessments
  • Tutors can mark assessments and students can view their marks
  • Enrolment data is synced from UNSW systems
• What are your microservices?

Defining your Microservices

• Domain driven design - modelling software to match a business domain

• Conway's Law: Organisations who design systems, are constrained to produce designs which are copies of the communication structure of these organisations

Bounded Context

• The logical boundary of a domain where particular terms and rules apply consistently
• Are browsing and purchasing separate contexts?

Bounded Context

• Are browsing and purchasing separate contexts?
  • Are they a single business process or different processes?
  • How much data do they share?

Gluing services together:

Choreography & Orchestration 🧵

• Choreography - microservices work independently, but co-ordinate with one another using cues or events
  • Method of control of the saga / workflow is determined by a predefined set of cues/events
• Orchestration - microservices are controlled by a single orchestrator or conductor service
  • Centralised control of the workflow

Choreography Example

Orchestration Example

Choreography & Orchestration: Trade-offs 🧵

• Choreography
  • Loosely coupled 😃
  • Easy to adapt 😃
  • Additional overhead to monitor the state of the process 😓
• Orchestration
  • Tightly coupled 😓
  • Single point of failure 😓
  • More difficult to change 😓
  • Easier to monitor process state 😃

Separating the data model 🔹

• Single database or distributed database?
• Trade-offs 😓
  • Databases often don't scale very well
  • Distributing databases creates more complexity
  • Single points of failure

Separating the data model 🔹

• Replication - data copied across multiple machines
  • Scale database to cope with load
  • Improved fault tolerance
  • Locate database closer to end users
• Partitioning - split the data of a system onto multiple nodes, called partitions
• Independent databases - each microservice has its own database

Dealing with distributed computing 🚎

• When we break up services into individual nodes on a network, we have to deal with problems in distributed computing
• Network reliability - you can't guarantee that the network is reliable
  • Exponential backoff - a technique to manage inter-service request failures

Dealing with distributed computing 🚎

• Network latency - it takes time for packets to travel across the wire
• Bandwidth isn't infinite
• The locked room problem - no guarantee the network is secure

Dealing with distributed computing 🚎

• Reliability - how often does your software succeed and how often does it break/fail?
• Fault tolerance - when software fails, what steps are in place to recover?
• Death, taxes and computer system failure are all inevitable to some degree - Howard and LeBlanc
• Individual computers fail all the time - spread the risk of failure over multiple computers - backups

Data Interchange: Sync vs Async 🔊

• Synchronous communication: Request and immediate response between services
• Technologies
  • REST endpoints
  • GraphQL
  • gRPC
• Trade-offs 😓
  • Increased coupling 😓

Data Interchange: Sync vs Async 🔊

• Asynchronous communication
  • Request + delayed response (promises)
  • Event-driven architecture
• Producers, consumers and streams
• Technologies
  • Apache Kafka
  • Amazon SQS
  • Amazon SNS
• Trade-offs
  • Loosely coupled 😃
  • Easy to add/remove services 😃
  • More overhead to setup 😓
  • How do we keep our data model accurate? 😓

Data Interchange and Accuracy 🔊

• Byzantine Generals Problem
  • n generals agree on a plan
  • Can only communicate via messenger
  • Messenger may be delayed / lost
  • Some generals are traitors - pretend to not receive message/send dishonest messages
• Validate format of received messages
• Sanitise inputs
• Retrieve data from multiple sources

Data Interchange and Accuracy 🔊

• Idempotency - what happens when a consumer receives the same message twice?
• Eventual Consistency - data updates take time to propagate between services
  • Reads can be stale
  • Writes can cause conflicts - need to manage this!
  • Single sources of truth?

Eventual Consistency in Action: UNSW Systems

Stateful vs Stateless Architecture 🏛️

• Two types of microservices: container and lambda
• Containerised Microservices
  • Container as a Service (CaaS)
  • In AWS, provisioned by ECS
  • A continually running instance in a container
  • Allows for persistence within the service 
  • More expensive, since it is continually active
  • Useful for heavy and continuous computation (e.g. processing a data stream) 
  • More control over the running environment of the business logic 

Stateful vs Stateless Architecture 🏛️

• Lambda Microservices
  • Function as a Service (FaaS)
  • In AWS, provisioned using lambda functions
  • A series of lambda functions that take input and produce output
  • Lambdas can also have side effects (e.g. updating a stateful component, such as S3 bucket / cache)
  • Grouped under a common API Gateway
  • Multiple/concurrent invocation is independent
  • Only active when the function is called - pay-as-you-go on AWS
  • Suitable when workload is small, operation takes a few seconds
  • Emphasis on business logic and operations

Stateful vs Stateless Architecture 🏛️

• Lambda Microservice Example

Stateful vs Stateless Architecture 🏛️

• CRUD operations on a database (deployed separately in the ecosystem)
• Generating and validating JWTs
• Instant messaging service
• Fetch and process the weather every 1 minute
• Fetch and process the weather every 12 hours
• Running machine learning on a data stream, storing the results and using the results to make predictions

Ecosystem Security 🔑

• How do we maintain authentication and authorisation when everything is split up into different services?
  • SSO
  • Authentication microservice
  • Tokens
• Centralising points of entry into the ecosystem
  • Internal microservices are on a private network
  • A single public-facing API (usually GraphQL) that handles orchestration
• Need to consider north-south and east-west traffic

Ecosystem Security 🔑

Trade-offs: Microservice Benefits 😃

• Freedom for service-specific programming languages / technology stacks
• Less responsibility, less coupling
• Easier to test
• Faster build and release cycles
• Lower risk per-microservice
• Not a single point of failure
• Easier to scale individual services

Trade-offs: Microservice Costs 😓

• Either everything breaks, or the glue breaks - how much time and money is actually saved?
• Dealing with distributed systems
• Overhead, complexity and risk in orchestrating services in an end-to-end use case
• More complex deployment
• Security - now need to authenticate for every service, not just one
• Debugging is more difficult as control flows over different services (distributed tracing)

Summary

• All Software Architecture is making trade-offs
• Monoliths grow too large, complex and risky and too difficult to scale
• Microservices present an alternative, which have their own challenges