CMSC389L

Load Balancers

Why Load Balancers?

• Architectures seen so far: single server directly handling requests
  • Guaranteed downtime (Single Point of Failure)
• Want to horizontally scale

What is a Load Balancer?

• Given a set of targets (servers, sub-networks, etc.), a load balancer has to distribute traffic uniformly
• Goals:
  • Ensure no single server is overworked
  • Detect dead servers and redirect traffic
  • Provide flexibility to dynamically add/remove servers

How to Balance Load?

• Multiple common load balancing algorithms:
  • Simplest: Round-Robin
    • ​Permute a list of targets, sending one request at a time
    • Assumes all requests have similar load characteristics
  • Most optimal: Metric-based
    • ​Least likely to overwork any single server
  • Other:
    • Hash-based (for example: hash(IP) % # targets)
    • Random

What about changes in load?

• Load is rarely uniform -- oftentimes it is cyclical (seasonal, nightly, etc.) or spiky
• Perfectly distributed load could still lead to failures if there is not enough capacity
  • Thus: load balancers need to be able to launch new capacity as demand changes

• AWS makes available Auto-Scaling for this purpose

Back to AWS: Why use AWS?

• A load balancer is oftentimes a key user-facing component of your architecture
• It has to:
  • Be available forever
  • Serve a hostname that never changes
  • Accept any amount of incoming requests
  • Distribute requests evenly across backends
  • Have perfect knowledge of failed instances to route traffic away from or new instances to start routing traffic to

AWS Load Balancers (ELB)

• CLBs: Classic Load Balancers
• ALBs: Application Load Balancers
• NLBs: Network Load Balancers
  • Brand New!

AWS Load Balancers: Concepts

• Listeners
• Targets
• Target Groups
• Health Checks
• Content-Based Routing

ALB Pricing

• Continuous Per-Hour
  • 2.25¢ / hour
    • ​$16.20 / month for a 30-day month
• LCU-hours ("Load balancer Capacity Unit")
  • 0.8¢ / LCU / hour
  • ​Goal: measure your actual usage and bill you for that
    • Netflix should pay more than you for an ALB
  • 1 LCU is measured on the largest dimension. Either:
    • New Connections (25 / second)
    • Active Connections (3k / minute)
    • Bandwidth (1GB / hour)
    • Rule Evaluations (1k / second)

ALB Demo

ALB Worksheet

Complete the worksheet section on ALB.