Building more reliable web applications with magic
Hubert Łępicki
2015-10-10
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815646/programistok.png)
About me
- Hubert Łępicki
- local guy
- backup guy (not so great)
- > 10 years programming
- 2 daughters
- part of awesome team at AmberBit
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815659/me.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815667/amberbit-logo.png)
I work with web apps
Web apps suck
Why?
JavaScript
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815734/trollface.png)
Server-side
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815737/servers.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815753/tomcat500.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815756/rails500.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815758/django500.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815760/php500.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816209/github.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815761/h500.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815762/500-error.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815775/error.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815776/images__1_.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815777/500_internal_error.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815778/500.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815779/jira_500.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815783/530_.gif)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815785/wildfire-500-internal-server-error.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815788/maxresdefault.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815790/server-error.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815805/bomb.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816214/heroku.png)
Speaking of the devil...
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816220/notifications.png)
Some users have 13k notifications
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1815805/bomb.jpg)
What exactly happened?
- One of the requests took time to process
- it allocated a lot of memory
- slow requests blocked web server
- app become unresponsive for other users
The deployment
- 2 x 2x Heroku Dyno (VM)
- 8 unicorn web servers per dyno
- 16 concurrent connections
- this list sucks big time
- 16 concurrent connections
- 16 requests to block it all
Ruby reality
- Ruby is slow
- unicorn vs. puma
- Ruby cannot into threads
- there is no way to scale
- this must be rocket science
- must it?
- why so painful?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816227/ruby_on_rails-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816229/live_on_this_planet_futurama.gif)
Why making it work is hard?
- shared mutable data
- no built-in crash recovery mechanisms
- GC is hard
- threading is hard
- preempting is hard
- monitoring is hard
if only was there better VM
taking care of all those issues...
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816254/java.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816262/dotnet.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816267/python.png)
Edsger Dijkstra says...
- famous dining philosophers problem
- mutexes/semaphores
- deadlocks
- race conditions
- spending too much CPU time in critical sections (wasting CPU power resources)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816286/dijkstra.jpg)
Let's discuss memory allocation
Shared heap is the source of all evil
Stack
- LIFO structure
- each function called adds stuff on top
- each function finished removes stuff from top
- VMs handle 1 stack per thread
Heap
- shared memory between all threads
- classes and types metadata
- dynamically allocated memory
Heap of problems
- global state
- slow access by different CPUs/cores
- need to synchronize access (locks, mutexes)
- single point of failure
- GC is a rocket science
Entirely different VM
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816279/Erlang_logo.png)
Erlang/OTP
- exists since 1986
- free and open source
- proven in many battles
- your telecom runs it
Simple concurrency
There is no shared memory
There is no problem synchronizing access to one
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816302/erts_mem.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816306/ericsson.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816308/ericsson.png)
Multitask like a telecom box
- multiple CPU cores required
- message passing between processes
- supervisors and crash recovery
Your web server in 2015
- has multiple CPU cores
- has tones of memory
- is perfect hardware to run Erlang on
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816355/erlang_mess.jpg)
Why programmers avoid Erlang?
- unfriendly syntax
- unfamiliar terminology
- different, functional mindset
- requires writing a lot of code
José Valim
- creator of many Ruby gems
- long-time Rails contributor
- frustrated with state of things
- looked at Erlang
- created new language that runs on top of Erlang VM
- programmer-friendly language
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816361/jose.jpg)
Elixir
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816371/mascott.jpeg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816375/elixir-logo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816378/brewing.jpg)
Elixir is
- dynamic programming language (with optional type checking!)
- running on top of Erlang VM (beam)
- integrates nicely into Erlang/OTP ecosystem
- clean, easy to learn Ruby-like syntax
- pattern matching
- functional
- immutable data structures
- compiled
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816383/elixir-example.png)
Great Unicode support
Source code documentation built-in
ExDoc
Testing framework
ExUnit
Automates common tasks
mix
Manages dependencies
hex
Talks to the databases
Ecto
Serves web requests
Plug
Serves modern web apps
Phoenix Framework
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816454/phoenixframework-logo.png)
Erlang tools
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816474/apps.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816475/processes.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816476/observer_2.png)
Metaprogramming
Writing code that writes code
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816482/plug.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816494/supervisor.png)
Writing supervised code
Hardware failure
OTP app cluster
- connect N nodes via (fast) web
- concurrency / message passing just works
- minimal changes to your code
- yay, you wrote distributed app by accident
- production-ready (remember dRB?)
- failover
- load balancing
- sleep well
Zero-downtime deployments
- elixir/erlang allows code hot-swapping
- your app is modular
- replace individual parts of system (applications)
- do not miss a single message
- do not loose a single HTTP request
Less infrastructure is more
Ruby on Rails
- [web] unicorn/puma + nginx/apache
- [worker] resque
- [scheduled tasks] cron + rake
- [websockets] faye/pusher
Elixir
[web] elixir
[worker] elixir
[scheduled tasks] elixir
[websockets] elixir
Application server
- bloat
+ speed
Microservices
(sorta)
Containers
(sorta)
http://elixir-lang.org
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816567/elixir.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816568/action.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/116857/images/1816569/phoenix__1_.jpg)
Thank you!
@hubertlepicki
Copy of Building more reliable web applications with magic
By Hubert Łępicki
Copy of Building more reliable web applications with magic
- 1,495