From local to deployable : building Elixir "The Hard Way"
Thomas Depierre
@DianaO
Diana Olympos
Twitter :
Github :
- B2B
- Big Company ™
- FTSE 250
- Founded 1947
- New organisation : Ascential Makers
- Elixir for everything in the backend
What is this "Build" thing ?
How does you code go from "work on my machine" to "running" ?
- I build a docker container, i push and i run it
- PR merged, then it is automatically deployed on Heroku
- I don't know, it is an Ops problem
That may be ok, but know why you are cutting some steps
Deployment
- "Work on my machine"
- Git(hub)
- (Test)
- Target configuration
- Build step
- Artifact stored
- Generate runtime config
- Bundle with Artefact or at runtime
- Move, unbundle, set up
- Run
Today
- Target configuration
- Build step
- Artifact stored
- (Generate runtime config)
- (Bundle config with Artefact)
Docker ?
- Built to solve another problem
- Quite good at cutting through some steps
- It is a "dark debt"
- May be worth it
- But do not see it as a solution.
Mix in production ?
-
Why mix ?
-
Risk surface
-
Dead code
-
Not easy to integrate in a runtime
Special for you know who you are :
mix phx.server
-
server: true
-
mix run --no-halt
Build
Goals :
- Easy start and stop
- Logs
- Self contained
- as much as possible
- configurable target
- same code
- different artefact
- bake in some config
Answers :
Distillery
- start and stop
- Logs (runerl)
- Self contained
- BEAM ?
- NIFs ?
- OpenSSL ?
- Releases
-
rel/config.exs
Configurable target
Distillery
-
release
-
environment
-
rel/config.exs
- Full elixir script
- version
- BEAM or not
- hooks
- runtime config
- etc
https://hexdocs.pm/distillery/configuration.html#configuration
Speeeeeeeeeeed
There is only one real solution, two folded :
- caching
- incremental compilation
Speeeeeeeeeeed
Caching :
-
mix deps.get
- based on mix.lock
- per architecture
-
mix deps.compile --all
-
based on mix.lock
- per architecture
- Beware with umbrellas
-
Incremental :
-
mix deps.get
- incremental
-
mix deps.compile --all
-
incremental
-
-
mix compile
-
incremental
-
but ... caching ?
-
does it really matters...
-
macros and use...
-
Storing the Artefact
Mostly will depend of your deployment strategy :
- .deb
- RPM
- tarball
- Docker
- Zip of source code
- ... git ...
Storing the Artefact
Up to you, depends of your use case.
- S3
- GCP equivalent
- Your own file system
- Docker repository
- Artifactory
- Mix of these
- ...
Runtime config
- Secrets?
- Environment variables ?
-
sys.config
-
vm.args
Google SRE Release Engineering
- Most of the stuff are here
- Yes cutting corner is ok
- But know why
- It may be far from your usual stuff
- Avoid ad hoc build every time
- ie, "the Docker Way"
https://landing.google.com/sre/book/chapters/release-engineering.html
Bonus : Versioning
- Version your build please
- Multiple solutions here
- Whatever is your scheme, stick to it
-
git tag
-
You can version in distillery dynamically
-
rel/config.exs
-
set version
-
- File path or file on disk
- Not in an env var only please (exception for dotenv)
Bonus : Build Environment
- Docker make sense here
- But it is really hard to cache that thing...
- Especially the low levels
- Caching in Docker is really basic
- I do not have a perfect solution yet
- Nix ....
- Guix ...