Lumen and Eir

Hans Elias B. Josephsen, @hansihe

Lumen

Lumen

Greenfield compiler and runtime for BEAM languages

WebAssembly

Native

True Embedded

Targets

Browser Apps

CDN Edge Compute

Server Deployments

Embedded Applications

Native GUI Applications

Current

Current

x86_64

x86_64

arm64

Future?

Your toaster?

Wide range of requirements

Startup

 

+++

+++

 

-

++

+

 

++

WASM

    Browser

    CDN Edge

Native

    Server

    GUI

    Embedded

True Embedded

    Toaster

Code Size

 

+++

-

 

-

+

+

 

+++

Performance

 

+

++

 

+++

++

+

 

-

No hot code reloading

Tradeoffs need to be made

Compiled not Interpreted

Tradeoffs need to be made

No hot code reloading

Compiled not Interpreted

  • Opportunities for cross module optimization
  • Dead code elimination
  • Type optimization
  • + more!

Tradeoffs need to be made

No hot code reloading

Compiled not Interpreted

  • Opportunities for cross module optimization
  • Dead code elimination
  • Type optimization
  • + more!
  • Improved startup time
  • Compile directly to WASM

The compiler is really important for us!

Let's look at the anatomy of a compiler

The compiler is really important for us!

Let's look at the anatomy of a compiler

Placeholder slide

Source Code

AST

IR

IR

Target

Parsing

Lowering

Optimization

Lowering

Source Code

AST

IR

IR

Target

Parsing

Lowering

Optimization

Lowering

function(a) {
  a * 2 * 2
}

function

a

*

*

a

2

2

function(a) {
  a * 2 * 2
}

Source Code

AST

IR

IR

Target

Parsing

Lowering

Optimization

Lowering

blah

function

a

*

*

a

2

2

Source Code

AST

IR

IR

Target

Parsing

Lowering

Optimization

Lowering

function($a) {
  $1 = $a * 2;
  $2 = $1 * 2;
  ret $2;
}
function(a) {
  a * 2 * 2
}

function

a

*

*

a

2

2

function($a) {
  $1 = $a * 4;
  ret $1;
}

Source Code

AST

IR

IR

Target

Parsing

Lowering

Optimization

Lowering

function($a) {
  $1 = $a * 2;
  $2 = $1 * 2;
  ret $2;
}
function(a) {
  a * 2 * 2
}

function

a

*

*

a

2

2

function($a) {
  $1 = $a * 4;
  ret $1;
}

Source Code

AST

IR

IR

Target

Parsing

Lowering

Optimization

Lowering

function(a) {
  a * 2 * 2
}

function

a

*

*

a

2

2

function($a) {
  $1 = $a * 4;
  ret $1;
}
function($a) {
  $1 = $a * 2;
  $2 = $1 * 2;
  ret $2;
}
mul $0 4
ret

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Lumen Compiler

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Eir Project

Lumen Compiler

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Eir Project

Lumen Compiler

Erlang

Erlang AST

EIR

MLIR

LLVM IR

Abstract Erlang

Elixir

WASM

x86_64

Others

Optimization passes

Eir Project

Lumen Compiler

EIR

IR Structure

a'my_function'/1 {
  entry(%ret, %thr, %arg):
    %tup = {a'ok', %arg};
    %ret(%tup);
}
def my_function(arg) do
  {:ok, arg}
end
a'foo'/1 {
  entry(%ret, %thr, %arg):
    match %arg {
      value 0 => branch1;
      _ => branch2;
    };
  
  branch1():
    %ret(a'bar');
  branch2():
    %ret(a'baz');
}
def foo(0), do: :bar
def foo(_), do: :baz
a'bar'/1 {
  entry(%ret, %thr, %a):
    %ret(clos);
  
  clos(%cret, %cthr, %b):
    %cret({%a, %b});
}
def bar(a) do
  fn (b) -> {a, b} end
end
  • Extremely uniform representation of control flow
    • Return, Throw, Branching, Calls all represented in same way
  • Whole spectrum of control flow can be handled identically by optimization passes
  • Almost all IR modifications are handled by a single primitive, Lambda Mangling
def bar(a) do
  :mod.foo(a)
end
def bar(a) do
  :mod.foo(a)
end
a'bar'/1 {
  entry(%ret, %thr, %a):
    b1();
  b1():
    %fun = a'mod':a'foo'/1;
    %fun(%a) => b3 except b2;
  b2(%t1, %t2, %t3):
    %thr(%t1, %t2, %t3);
  b3(%r1):
    %ret(%r1);
}
a'bar'/1 {
  entry(%ret, %thr, %a):
    b1();
  b1():
    %fun = a'mod':a'foo'/1;
    %fun(%a) => b3 except b2;
  b2(%t1, %t2, %t3):
    %thr(%t1, %t2, %t3);
  b3(%r1):
    %ret(%r1);
}
a'bar'/1 {
  entry(%ret, %thr, %a):
    %fun = a'mod':a'foo'/1;
    %fun(%a) => %ret except %thr;
}

Compiler Passes

  • Graph simplification pass
    • Control flow simplification
    • Tail call elimination
    • Constant propagation
    • Simple constant folding
    • Branch elimination
  • Closure inlining pass
  • Pattern matching compilation pass

Community

  • #lumen on elixir-lang slack
  • Open standups every week! See Slack channel
  • Archives on Youtube channel
  • getlumen.org

Lumen and Eir

By Hans Elias Bukholm Josephsen

Lumen and Eir

  • 338

More from Hans Elias Bukholm Josephsen