From local to deployable : building Elixir "The Hard Way"

Thomas Depierre

@DianaO

Diana Olympos

 

Twitter :

Github :

  • B2B
  • Big Company ™
    • FTSE 250
    • Founded 1947
  • New organisation : Ascential Makers
  • Elixir for everything in the backend

What is this "Build" thing ?

How does you code go from "work on my machine" to "running" ? 

  • I build a docker container, i push and i run it
  • PR merged, then it is automatically deployed on Heroku
  • I don't know, it is an Ops problem

That may be ok, but know why you are cutting some steps

Deployment

  • "Work on my machine"
  • Git(hub)
  • (Test)
  • Target configuration
  • Build step
  • Artifact stored
  • Generate runtime config
  • Bundle with Artefact or at runtime
  • Move, unbundle, set up
  • Run

Today

  • Target configuration
  • Build step
  • Artifact stored
  • (Generate runtime config)
  • (Bundle config with Artefact)

Docker ?

  • Built to solve another problem
  • Quite good at cutting through some steps
  • It is a "dark debt"
  • May be worth it
  • But do not see it as a solution.

Mix in production ?

  • Why mix ?

  • Risk surface

  • Dead code

  • Not easy to integrate in a runtime

Special for you know who you are :

mix phx.server
  1. server: true
  2. mix run --no-halt

Build

Goals :

  • Easy start and stop
  • Logs
  • Self contained
    • as much as possible
  • configurable target
    • same code
    • different artefact
  • bake in some config

Answers :

Distillery

  • start and stop
  • Logs (runerl)
  • Self contained
    • BEAM ?
    • NIFs ?
    • OpenSSL ?
  • Releases
  • rel/config.exs

Configurable target

Distillery

  • release
  • environment
  • rel/config.exs
  • Full elixir script
  • version
  • BEAM or not
  • hooks
  • runtime config
  • etc

https://hexdocs.pm/distillery/configuration.html#configuration

Speeeeeeeeeeed

There is only one real solution, two folded :

  • caching
  • incremental compilation

Speeeeeeeeeeed

Caching : 

  • mix deps.get
    • based on mix.lock
    • per architecture
  • mix deps.compile --all
    • based on mix.lock

    • per architecture
    • Beware with umbrellas

Incremental : 

  • mix deps.get
    • incremental
  • mix deps.compile --all
    • incremental

  • mix compile
    • incremental

    • but ... caching ?

    • does it really matters...

    • macros and use...

Storing the Artefact

Mostly will depend of your deployment strategy :

  • .deb
  • RPM
  • tarball
  • Docker
  • Zip of source code
  • ... git ...

Storing the Artefact

Up to you, depends of your use case.

  • S3
  • GCP equivalent
  • Your own file system
  • Docker repository
  • Artifactory
  • Mix of these
  • ...

Runtime config

  • Secrets?
  • Environment variables ?
  • sys.config
  • vm.args

 

https://hexdocs.pm/distillery/runtime-configuration.html

Google SRE Release Engineering

  • Most of the stuff are here
  • Yes cutting corner is ok
  • But know why
  • It may be far from your usual stuff
  • Avoid ad hoc build every time
    • ie, "the Docker Way"

 

https://landing.google.com/sre/book/chapters/release-engineering.html

Bonus : Versioning

  • Version your build please
  • Multiple solutions here
  • Whatever is your scheme, stick to it
  • git tag
  • You can version in distillery dynamically

    • rel/config.exs
    • set version
  • File path or file on disk
  • Not in an env var only please (exception for dotenv)

Bonus : Build Environment

  • Docker make sense here
  • But it is really hard to cache that thing...
    • Especially the low levels
    • Caching in Docker is really basic
  • I do not have a perfect solution yet
    • Nix ....
    • Guix ...

Thank You

Questions ?