load balancing algorithms


irving cordova

http://irvingc.com

what is heroku?



  • platform as a service
  • in reality a quick way to deploy apps
  • all you need to do is        
     .
            git commit
            git push heroku master
            

platform as a service


heroku's popularity



it even got acquired



why am i talking about it?





heroku broke 

its load balancing

algorithm



changes...



results...




load balancing is hard



what have others done?





it turns out a lot
(unsurprisingly)

Parallel Randomized Load Balancing (1995)



Parallel Randomized Load Balancing (1995)



Parallel Randomized Load Balancing (1995)



Balanced Allocations (1999)



 if the balls choose sequentially from d > 2 u.i.r. bins greedily the currently least loaded one, the maximum load is



They prove that this bound is stochastically optimal in the sense that any other strategy to assign the balls majorizes their approach.

Tight Bounds for Parallel Randomized Load Balancing (2011)



Comparison of efficiency




implementing

the algorithm


    
how to simulate the environment? 
 
  • it must be parallel
  • must have message passing
  • concurrency is hard

actors!



what are actors?


Wikipedia:

The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.

results


Balls: 300 x producer
Producers: 20
Bins: 20

the  naive algorithm



tights bound algorithm


problems




  • high cpu usage
  • eventually the stack was blown
  • tasks take too long to be taken

improving the

tights bound algorithm


the algorithm is already parallel and randomized


sample and modify


this technique can still be used

if the bin is already known to be busy

let's not query it again

in effect the working set is reduced

the stack is still being blown




still being blown....




simplify




final algorithm


  • Set k(1) = 1, i = i, available bins = all bins.
  • Balls contact k(1) bins requesting permission to be placed in them, selected u.i.r from the set of available bins.
  • Each bin admits permission to one of the requesting balls (if it received any) and declines all other requests.
  • Removed bins that declined requests from the set of available bins.
  • Any ball receiving at least one permission chooses and arbitrary set of the bins to be placed into, informs it, and terminates 
  • If i>= 10 set k(i) = sqrt ( log ( n )) else set k(i) = k(i) e ^ (k(i)/5)
  • If the set of available bins == 0 sleep k(i), restart algorithm

demo

results

compare


where to go from here?


    
 
  • mathematically model the new approach
  • test with extended loads
  • have good implementations

questions?



sources


  • M. Adler, S. Chakrabarti, M. Mitzenmacher, and L. Rasmussen. Parallel Randomized Load Balancing. In Proc. 27th Symposium on Theory of Computing (STOC), pages 238{247, 1995
  • Y. Azar, A. Z. Broder, A. R. Karlin, and E. Upfal. Balanced Allocations. SIAM Journal on Computing, 29(1):180–200, 1999.
  • C Lenzen, R. Wattenhofer, Tight Bounds for Parallel Randomized Load Balancing, STOC'11 June 6-8 2011

images

  • http://outre-monde.com/2012/04/12/keep-on-asking-questions/
  • http://www.root-image-analysis.org/wp-content/uploads/2011/09/choice.jpg
  • http://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2
  • http://www.essex1.com/pages/paul/Uh-oh-title.gif
  • http://spin.atomicobject.com/2011/10/27/generic-memoization-in-c/
  • http://www.google.com
  • http://3.bp.blogspot.com/-bxj9LtU6bJE/UGl0Idls8_I/AAAAAAAAAFo/ld8Pk5OWGGE/s1600/heroku-logo-white.jpg
  • http://www.heroku.com
  • http://techcrunch.com/2010/12/08/breaking-salesforce-buys-heroku-for-212-million-in-cash/
  • http://rapgenius.com/James-somers-herokus-ugly-secret-lyrics
  • http://www.myconfinedspace.com/2010/04/07/go-ask-alice-jefferson-airplane/tell-you-where-to-go-jpg/