load balancing algorithms

irving cordova

what is heroku?

platform as a service
in reality a quick way to deploy apps

all you need to do is

 .
        git commit
        git push heroku master

platform as a service

heroku's popularity

it even got acquired

why am i talking about it?

heroku broke

its load balancing

algorithm

changes...

results...

load balancing is hard

what have others done?

it turns out a lot

(unsurprisingly)

Parallel Randomized Load Balancing (1995)

Balanced Allocations (1999)

if the balls choose sequentially from d > 2 u.i.r. bins greedily the currently least loaded one, the maximum load is

They prove that this bound is stochastically optimal in the sense that any other strategy to assign the balls majorizes their approach.

Tight Bounds for Parallel Randomized Load Balancing (2011)

Comparison of efficiency

implementing

the algorithm

how to simulate the environment?

it must be parallel
must have message passing
concurrency is hard

actors!

what are actors?

Wikipedia:

The Actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-oriented programming languages, but differs in that object-oriented software is typically executed sequentially, while the Actor model is inherently concurrent.

results

Balls: 300 x producer

Producers: 20

Bins: 20

the naive algorithm

tights bound algorithm

problems

high cpu usage
eventually the stack was blown
tasks take too long to be taken

improving the

tights bound algorithm

the algorithm is already parallel and randomized

sample and modify

this technique can still be used

if the bin is already known to be busy

let's not query it again

in effect the working set is reduced

the stack is still being blown

still being blown....

simplify

final algorithm

Set k(1) = 1, i = i, available bins = all bins.
Balls contact k(1) bins requesting permission to be placed in them, selected u.i.r from the set of available bins.
Each bin admits permission to one of the requesting balls (if it received any) and declines all other requests.
Removed bins that declined requests from the set of available bins.
Any ball receiving at least one permission chooses and arbitrary set of the bins to be placed into, informs it, and terminates
If i>= 10 set k(i) = sqrt ( log ( n )) else set k(i) = k(i) e ^ (k(i)/5)
If the set of available bins == 0 sleep k(i), restart algorithm

demo

results

compare

where to go from here?

mathematically model the new approach
test with extended loads
have good implementations

questions?

sources

M. Adler, S. Chakrabarti, M. Mitzenmacher, and L. Rasmussen. Parallel Randomized Load Balancing. In Proc. 27th Symposium on Theory of Computing (STOC), pages 238{247, 1995
Y. Azar, A. Z. Broder, A. R. Karlin, and E. Upfal. Balanced Allocations. SIAM Journal on Computing, 29(1):180–200, 1999.
C Lenzen, R. Wattenhofer, Tight Bounds for Parallel Randomized Load Balancing, STOC'11 June 6-8 2011

images

http://outre-monde.com/2012/04/12/keep-on-asking-questions/
http://www.root-image-analysis.org/wp-content/uploads/2011/09/choice.jpg
http://letitcrash.com/post/29044669086/balancing-workload-across-nodes-with-akka-2
http://www.essex1.com/pages/paul/Uh-oh-title.gif
http://spin.atomicobject.com/2011/10/27/generic-memoization-in-c/
http://www.google.com
http://3.bp.blogspot.com/-bxj9LtU6bJE/UGl0Idls8_I/AAAAAAAAAFo/ld8Pk5OWGGE/s1600/heroku-logo-white.jpg
http://www.heroku.com
http://techcrunch.com/2010/12/08/breaking-salesforce-buys-heroku-for-212-million-in-cash/
http://rapgenius.com/James-somers-herokus-ugly-secret-lyrics
http://www.myconfinedspace.com/2010/04/07/go-ask-alice-jefferson-airplane/tell-you-where-to-go-jpg/