Building more reliable web applications with magic
Hubert Łępicki
2015-10-10
About me
- Hubert Łępicki
- local guy
- backup guy (not so great)
- > 10 years programming
- 2 daughters
- part of awesome team at AmberBit
I work with web apps
Web apps suck
Why?
JavaScript
Server-side
Speaking of the devil...
Some users have 13k notifications
What exactly happened?
- One of the requests took time to process
- it allocated a lot of memory
- slow requests blocked web server
- app become unresponsive for other users
The deployment
- 2 x 2x Heroku Dyno (VM)
- 8 unicorn web servers per dyno
- 16 concurrent connections
- this list sucks big time
- 16 concurrent connections
- 16 requests to block it all
Ruby reality
- Ruby is slow
- unicorn vs. puma
- Ruby cannot into threads
- there is no way to scale
- this must be rocket science
- must it?
- why so painful?
Why making it work is hard?
- shared mutable data
- no built-in crash recovery mechanisms
- GC is hard
- threading is hard
- preempting is hard
- monitoring is hard
if only was there better VM
taking care of all those issues...
Edsger Dijkstra says...
- famous dining philosophers problem
- mutexes/semaphores
- deadlocks
- race conditions
- spending too much CPU time in critical sections (wasting CPU power resources)
Let's discuss memory allocation
Shared heap is the source of all evil
Stack
- LIFO structure
- each function called adds stuff on top
- each function finished removes stuff from top
- VMs handle 1 stack per thread
Heap
- shared memory between all threads
- classes and types metadata
- dynamically allocated memory
Heap of problems
- global state
- slow access by different CPUs/cores
- need to synchronize access (locks, mutexes)
- single point of failure
- GC is a rocket science
Entirely different VM
Erlang/OTP
- exists since 1986
- free and open source
- proven in many battles
- your telecom runs it
Simple concurrency
There is no shared memory
There is no problem synchronizing access to one
Multitask like a telecom box
- multiple CPU cores required
- message passing between processes
- supervisors and crash recovery
Your web server in 2015
- has multiple CPU cores
- has tones of memory
- is perfect hardware to run Erlang on
Why programmers avoid Erlang?
- unfriendly syntax
- unfamiliar terminology
- different, functional mindset
- requires writing a lot of code
José Valim
- creator of many Ruby gems
- long-time Rails contributor
- frustrated with state of things
- looked at Erlang
- created new language that runs on top of Erlang VM
- programmer-friendly language
Elixir
Elixir is
- dynamic programming language (with optional type checking!)
- running on top of Erlang VM (beam)
- integrates nicely into Erlang/OTP ecosystem
- clean, easy to learn Ruby-like syntax
- pattern matching
- functional
- immutable data structures
- compiled
Great Unicode support
Source code documentation built-in
ExDoc
Testing framework
ExUnit
Automates common tasks
mix
Manages dependencies
hex
Talks to the databases
Ecto
Serves web requests
Plug
Serves modern web apps
Phoenix Framework
Erlang tools
Metaprogramming
Writing code that writes code
Writing supervised code
Hardware failure
OTP app cluster
- connect N nodes via (fast) web
- concurrency / message passing just works
- minimal changes to your code
- yay, you wrote distributed app by accident
- production-ready (remember dRB?)
- failover
- load balancing
- sleep well
Zero-downtime deployments
- elixir/erlang allows code hot-swapping
- your app is modular
- replace individual parts of system (applications)
- do not miss a single message
- do not loose a single HTTP request
Less infrastructure is more
Ruby on Rails
- [web] unicorn/puma + nginx/apache
- [worker] resque
- [scheduled tasks] cron + rake
- [websockets] faye/pusher
Elixir
[web] elixir
[worker] elixir
[scheduled tasks] elixir
[websockets] elixir
Application server
- bloat
+ speed
Microservices
(sorta)
Containers
(sorta)
http://elixir-lang.org
