The structure of the world and of software

From Pyramids to Ecosystems

• The two biggest changes in our society in the last 30 years
• The Pyramid Paradigm
  • Centralised power structure
  • Vertical hierarchy
  • Single point of failure in authority
  • COMP1511 C Program - top-down design

Ecosystems 🌿

The Ecosystem Paradigm

• Distributed model of power/information/interaction
• Synchronous interchange/ - things happen face to face
  • E.g. Bird catches its prey
• Asynchronous interchange
  • E.g. Excrement from an animal fertilizes soil for plants to grow
• Contains elements of centralisation, but on a smaller scale
• Businesses, universities, the human body, Marvel... all ecosystems
• The individual picks and chooses those microservices for that help them satisfy their requirements

SENG3011 Ecosystem

Monoliths to Microservices

Monolith vs Microservices 

• Monolith: a single large application that contains the entire software solution
  • One service to rule them all
• Microservices: A series of small-scale services that communicate with one another
  • Each service does one task well

Microservices Case Study: Netflix 

• Cloudshift - from server to cloud (2008)
• Migrate from monolith to microservices, service by service
• Started with non customer facing; move to customer facing
• By 2013, Netflix’s API gateway was handling two billion daily API edge requests, managed by over 500 cloud-hosted microservices
• By 2017, its architecture consisted of over 700 loosely coupled microservices

Microservices Case Study: Netflix 

• Adopting microservice architecture is a business decision as well as an engineering decision
• Today, Netflix:
  • Makes around $8 billion a year and
  • Streams approximately six billion hours of content weekly
  • More than 220 million subscribers in 190 countries, and it continues to grow
  • Revolutionised how we consume media and started the streaming wars
• Cloud costs per streaming start a fraction of those in the data centre

Microservices Case Study: Netflix 

Microservices Case Study: Uber 

• Issues in business growth growth was related to a monolithic architecture:
  • More inertia to ship new features
  • More bugs and risk in deployments
  • Much harder to onboard new engineers to work on the system
  • The architecture couldn't meet the demand and popularity of the application - not scalable
•  Architecture
  • REST API to access monolith
  • Three adapters with embedded API functions
  • Single MySQL database
  • All features in the monolith

Microservices Case Study: Uber 

• One engineering team per microservice
• Reduced coupling and dependency
• Focus on scaling at the individual service level - address bottlenecks
• Improved speed of development, performance, overall quality, fault tolerance
• Need to create global standards for services to ensure compatibility and interoperability

Microservices Case Study: Uber 

Defining your Microservices 

• Let's try an example - design WebCMS as a microservice architecture
• System requirements (simplified)
  • Lecturers can create courses and put up content
  • Lecturers can create assessments
  • Students can view content and submit assessments
  • Tutors can mark assessments and students can view their marks
  • Enrolment data is synced from UNSW systems
• What are your microservices?

Defining your Microservices

• Domain driven design - modelling software to match a business domain

• Conway's Law: Organisations who design systems, are constrained to produce designs which are copies of the communication structure of these organisations

• More broadly: Software is a product of the world in which it is created, and it is thus a reflection of the world in which is created and its structures of power and information

Bounded Context

• The logical boundary of a domain where particular terms and rules apply consistently
• Are browsing and purchasing separate contexts?

Bounded Context

• Are browsing and purchasing separate contexts?
  • Are they a single business process or different processes?
  • How much data do they share?

Business Processes

• When designing microservices, we model business processes (use cases)
  • A shopper ordering an item on Amazon
  • Processing the order (inventory, delivery)
• The design of our software will reflect the context in which it is used
• Cohesive, well designed microservices contain a single business process

1. How do we control the flow of data to produce meaningful business processes with microservices? 

Choreography and Orchestration

Choreography & Orchestration 

• Choreography - microservices work independently, but co-ordinate with one another using cues or events
  • Method of control of the saga / workflow is determined by a predefined set of cues/events
• Orchestration - microservices are controlled by a single orchestrator or conductor service
  • Centralised control of the workflow

Choreography Example

Orchestration Example

Choreography & Orchestration: Trade-offs 

• Choreography
  • Loosely coupled 😃
  • Easy to adapt 😃
  • Additional overhead to monitor the state of the process 😓
• Orchestration
  • Tightly coupled 😓
  • Single point of failure 😓
  • More difficult to change 😓
  • Easier to monitor process state 😃

2. How do we interchange data between microservices?

Synchronous and asynchronous interchange

Data Interchange: Sync vs Async 

• Synchronous communication: Request and immediate response between services
• Technologies
  • REST endpoints
  • GraphQL
  • gRPC
• Trade-offs 😓
  • Increased coupling 😓

Data Interchange: Sync vs Async 

• Asynchronous communication
  • Request + delayed response (promises)
  • Event-driven architecture
• Producers, consumers and streams
• Technologies
  • Apache Kafka
  • Amazon SQS
  • Amazon SNS
• Trade-offs
  • Loosely coupled 😃
  • Easy to add/remove services 😃
  • More overhead to setup 😓
  • How do we keep our data model accurate? 😓

Data Interchange and Accuracy 

• Byzantine Generals Problem
  • n generals agree on a plan
  • Can only communicate via messenger
  • Messenger may be delayed / lost
  • Some generals are traitors - pretend to not receive message/send dishonest messages
• Validate format of received messages
• Sanitise inputs
• Retrieve data from multiple sources

Data Interchange and Accuracy 

• Idempotency - what happens when a consumer receives the same message twice?
• Eventual Consistency - data updates take time to propagate between services
  • Reads can be stale
  • Writes can cause conflicts - need to manage this!
  • Single sources of truth?

Eventual Consistency in Action: UNSW Systems

3. How should our microservices manage state?

Stateful and Stateless Architecture

Stateful vs Stateless Architecture 

• Two types of microservices: container and lambda
• Containerised Microservices
  • Container as a Service (CaaS)
  • In AWS, provisioned by ECS
  • A continually running instance in a container
  • Allows for persistence within the service 
  • More expensive, since it is continually active
  • Useful for heavy and continuous computation (e.g. processing a data stream) 
  • More control over the running environment of the business logic 

Stateful vs Stateless Architecture 

• Lambda Microservices
  • Function as a Service (FaaS)
  • In AWS, provisioned using lambda functions
  • A series of lambda functions that take input and produce output
  • Lambdas can also have side effects (e.g. updating a stateful component, such as S3 bucket / cache)
  • Grouped under a common API Gateway
  • Multiple/concurrent invocation is independent
  • Only active when the function is called - pay-as-you-go on AWS
  • Suitable when workload is small, operation takes a few seconds
  • Emphasis on business logic and operations

Stateful vs Stateless Architecture 

• Lambda Microservice Example

Stateful vs Stateless Architecture 

• CRUD operations on a database (deployed separately in the ecosystem)
• Generating and validating JWTs
• Instant messaging service
• Fetch and process the weather every 1 minute
• Fetch and process the weather every 12 hours
• Running machine learning on a data stream, storing the results and using the results to make predictions

4. How do our microservice ecosystem store data? 

Distributed Data Models

Separating the data model 

• Single database or distributed database?
• Trade-offs 😓
  • Databases often don't scale very well
  • Distributing databases creates more complexity
  • Single points of failure

Separating the data model 

• Replication - data copied across multiple machines
  • Scale database to cope with load
  • Improved fault tolerance
  • Locate database closer to end users
• Partitioning - split the data of a system onto multiple nodes, called partitions
• Independent databases - each microservice has its own database

4. What do we have to worry about with

distributed computing?

Nothing, right?

Dealing with distributed computing 

• When we break up services into individual nodes on a network, we have to deal with problems in distributed computing
• Network reliability - you can't guarantee that the network is reliable
  • Exponential backoff - a technique to manage inter-service request failures

Dealing with distributed computing 

• Network latency - it takes time for packets to travel across the wire
• Bandwidth isn't infinite
• The locked room problem - no guarantee the network is secure

Dealing with distributed computing 

• Reliability - how often does your software succeed and how often does it break/fail?
• Fault tolerance - when software fails, what steps are in place to recover?
• Death, taxes and computer system failure are all inevitable to some degree - Howard and LeBlanc
• Individual computers fail all the time - spread the risk of failure over multiple computers - backups

4. How do we keep our microservices secure?

Protecting against rotten glue

Ecosystem Security 

• How do we maintain authentication and authorisation when everything is split up into different services?
  • SSO
  • Authentication microservice
  • Tokens
• Centralising points of entry into the ecosystem
  • Internal microservices are on a private network
  • A single public-facing API (usually GraphQL) that handles orchestration
• Need to consider north-south and east-west traffic

Ecosystem Security 

Trade-offs: Microservice Benefits 

• Freedom for service-specific programming languages / technology stacks
• Less responsibility, less coupling
• Easier to test
• Faster build and release cycles
• Lower risk per-microservice
• Not a single point of failure
• Easier to scale individual services

Trade-offs: Microservice Costs 

• Either everything breaks, or the glue breaks - how much time and money is actually saved?
• Dealing with distributed systems
• Overhead, complexity and risk in orchestrating services in an end-to-end use case
• More complex deployment
• Security - now need to authenticate for every service, not just one
• Debugging is more difficult as control flows over different services (distributed tracing)

Summary

• All Software Architecture is making trade-offs
• Monoliths grow too large, complex and risky and too difficult to scale
• Microservices present an alternative, which have their own challenges