load balancing algorithms

irving cordova


what is heroku?

  • platform as a service
  • in reality a quick way to deploy apps
  • all you need to do is        
            git commit
            git push heroku master

platform as a service

heroku's popularity

it even got acquired

why am i talking about it?

heroku broke 

its load balancing




load balancing is hard

what have others done?

it turns out a lot

Parallel Randomized Load Balancing (1995)

Parallel Randomized Load Balancing (1995)

Parallel Randomized Load Balancing (1995)

Balanced Allocations (1999)

 if the balls choose sequentially from d > 2 u.i.r. bins greedily the currently least loaded one, the maximum load is

They prove that this bound is stochastically optimal in the sense that any other strategy to assign the balls majorizes their approach.

Tight Bounds for Parallel Randomized Load Balancing (2011)

Comparison of efficiency


the algorithm

how to simulate the environment? 
  • it must be parallel
  • must have message passing
  • concurrency is hard


what are actors?


The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.


Balls: 300 x producer
Producers: 20
Bins: 20

the  naive algorithm

tights bound algorithm


  • high cpu usage
  • eventually the stack was blown
  • tasks take too long to be taken

improving the

tights bound algorithm

the algorithm is already parallel and randomized

sample and modify

this technique can still be used

if the bin is already known to be busy

let's not query it again

in effect the working set is reduced

the stack is still being blown

still being blown....


final algorithm

  • Set k(1) = 1, i = i, available bins = all bins.
  • Balls contact k(1) bins requesting permission to be placed in them, selected u.i.r from the set of available bins.
  • Each bin admits permission to one of the requesting balls (if it received any) and declines all other requests.
  • Removed bins that declined requests from the set of available bins.
  • Any ball receiving at least one permission chooses and arbitrary set of the bins to be placed into, informs it, and terminates 
  • If i>= 10 set k(i) = sqrt ( log ( n )) else set k(i) = k(i) e ^ (k(i)/5)
  • If the set of available bins == 0 sleep k(i), restart algorithm




where to go from here?

  • mathematically model the new approach
  • test with extended loads
  • have good implementations



  • M. Adler, S. Chakrabarti, M. Mitzenmacher, and L. Rasmussen. Parallel Randomized Load Balancing. In Proc. 27th Symposium on Theory of Computing (STOC), pages 238{247, 1995
  • Y. Azar, A. Z. Broder, A. R. Karlin, and E. Upfal. Balanced Allocations. SIAM Journal on Computing, 29(1):180–200, 1999.
  • C Lenzen, R. Wattenhofer, Tight Bounds for Parallel Randomized Load Balancing, STOC'11 June 6-8 2011


  • http://outre-monde.com/2012/04/12/keep-on-asking-questions/
  • http://www.root-image-analysis.org/wp-content/uploads/2011/09/choice.jpg
  • http://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2
  • http://www.essex1.com/pages/paul/Uh-oh-title.gif
  • http://spin.atomicobject.com/2011/10/27/generic-memoization-in-c/
  • http://www.google.com
  • http://3.bp.blogspot.com/-bxj9LtU6bJE/UGl0Idls8_I/AAAAAAAAAFo/ld8Pk5OWGGE/s1600/heroku-logo-white.jpg
  • http://www.heroku.com
  • http://techcrunch.com/2010/12/08/breaking-salesforce-buys-heroku-for-212-million-in-cash/
  • http://rapgenius.com/James-somers-herokus-ugly-secret-lyrics
  • http://www.myconfinedspace.com/2010/04/07/go-ask-alice-jefferson-airplane/tell-you-where-to-go-jpg/