Elixir programming language evaluation for IoT

Geovane Fedrecheski

Laisa Costa

Marcelo Zuffo

School of Engineering

University of São Paulo

Introduction

The Internet

The Internet of Things

The Swarm at the edge of the cloud

Edward Lee, Jan Rabey. et al. "The Swarm at the Edge of the Cloud", 2014

Main challenges

Resource constrained
Power efficiency
Communication unreliability
Context sensitivity
Remote management
Concurrency

Popular languages for IoT

C
C++
Java
Javascript
Python

http://www.techworm.net/2016/06/best-programming-language-iot.html

Erlang

context

Created in the Telecommunication industry

...

~1980

Erlang

requirements

Concurrency
Soft real-time
Distribution
Hardware Interaction
Large Software Systems
Complex Functionality
Continuous Operation
Quality Requirements
Fault Tolerance

Erlang

nowadays

HTTP

Functional

Concurreny

Fault tolerance

Data structures

Consistent libs

Modern tooling

Elixir

~2011

Elixir

Goals

Extensibility
Productivity
Compatibility

Elixir

Erlang

Industry

Nerves Framework

Method

Swarm Broker

Authorizator

YES

consumer

producer

http://localhost:4000/broker/authorizator

NO

GET

Comparison

Lines of Code
CPU
Memory
Requests / sec.

Platform

Quad-core computer (x86)
6GB of RAM
Java 1.8
Erlang 18.2
Elixir 1.2
Load test tool: Tsung 1.6

Results

Lines of Code

Elixir

Java

551

176

	Lines of Code
	CPU
	Memory
	Requests / sec.

Discussion

Elixir used more CPU, but had more requests per second
Elixir has good potential to run IoT/Swarm micro-services

Next steps

Test in embedded platforms
Explore persistent connections (e.g websockets)
Explore reliability and fault-tolerance

Obrigado

Thanks

Geovane Fedrecheski

Laisa Costa

Marcelo Zuffo

School of Engineering

University of São Paulo

School of Engineering

University of São Paulo

The Internet of Things
Functional Programming
Elixir
IoT + Elixir: 4 Case Studies

Functionaλ Programming

f(g(x), h(...))

Steve Vinoski, IEEE Computing Society 2009

"... ironically, the resurgence of interest in FP languages owes a lot to Java’s popularity."

"Another reason [...] is the move toward multicore architectures."

"... developers look for the opportunity to
get more done with less."

Welcome to the Functional Web

Functional Programming Languages

Lisp

Elm

Elixir

José Valim

Core committer

Unsatisfied with thread-based concurrency

~2010

Unsatisfied with thread-based concurrency

Repetitive strain

injury

by Bruce A. Tate

So what's interesting about

this Erlang thing?

What even an "Erlang" is?

A bit of History

Cases

I loved everything that Erlang had

but I hated everything that it didn't had

- José Valim

Features

Pattern Matching

> {method, path} = {:get, "temp"}
{:get, "temp"}
> method
:get
> path
"temp"

b = :binary.encode_unsigned(258)
# b has value <<1, 2>>

<<x, 2>> = b
# x has value 1

i = :binary.decode_unsigned(<<x, 2>>)
# i has value 258

request = {:get, "temp"}



case request do
  {:post, path, payload} ->
    send_resp 200
  {:get, path} ->
    send_resp 200, "{'value':17.5, 'unit': 'celsius'}"
end

defmodule Math do
  def fib(0), do: 0
  def fib(1), do: 1
  def fib(n), do: fib(n-1) + fib(n-2)
end

Math.fib(8) # => 21

Anonymous Functions

> add = fn(a, b) -> a + b end
#Function<12.71889879/2 in :erl_eval.expr/5>
> add.(1, 2)
3

> Enum.map([17.9, 20.2, 18.4], fn(x) -> x * 2 end)
[35.8, 40.4, 36.8]

Can be passed as parameters

Processes

spawn

A lightweight thread of execution

(The key Erlang/Elixir abstraction)

How is this different from threads?

No shared memory
VM Processes

Do not share memory

Communicate via

Message Passing

message

VM Processes

Excuse me, have you ever heard of (VM) processes?

Erlang VM

Scheduler

Sched.

OS Processes

OS Threads

VM Processes

pid = spawn(fn ->
  :timer.sleep(3000)
  IO.puts "hi"
end)

fn ..

spawn

self()

sleep

puts

A lightweight thread of execution

Messages

pid = spawn(...)
send(pid, "hello")

receive do
  msg ->
    IO.puts msg
end

fn ..

spawn

self()

receive

send

Node.spawn(
  :"app@computer-2",
  fn -> Hello.world end)

send

receive

app@computer-1

app@computer-2

Distributed

Same syntax

We have:

Isolated Processes
Message passing
Distribution

We want:

Fault Tolerance

dangling processes!

spawn

spawn

call the supervisor

supervisor

worker

worker

Supervision Tree

import Supervisor.Spec

children = [
  worker(Cache, []),
  supervisor(DatabaseSupervisor, []),
  supervisor(TCPSupervisor, [4040])
]

Supervisor.start_link(children, strategy: :one_for_one)

Applications

your app

http
server

logger

app@computer-1

Distributed Applications

app@computer-2

Fault Tolerance

Open Telecom Platform

Supervisor
Application
GenServer
GenEvent

OOP

Objects

Elixir

Processes

Production-ready?

https://github.com/doomspork/elixir-companies

Making the IoT
Functional with Elixir:
Four scenarios

Easy CoAP binary parser
Fault-tolerant microservices

Elixir and Java load test
Automatic deployment of Elixir images

CoAP

Constrained Application Protocol
kind of a "smaller HTTP, for IoT"

request

response

CoAP - Codes

(binary header)

0.01 (GET) (Request)

0.02 (POST) (Request)

2.05 (Content) (Response)

4.04 (Not Found) (Response)

000.00001

000.00010

010.00101

100.00100

CoAP

parser

0.01

123

01000000 00000001 0000000001111011 11111111

255

64 1 0 123 255

(empty)

%Message(type: :con,
         code: :get,
         msg_id: 123,
         options: [])

int coap_parseHeader(coap_header_t *hdr, 
                     const uint8_t *buf,
                     size_t buflen)
{
    if (buflen < 4)
        return COAP_ERR_HEADER_TOO_SHORT;
    hdr->ver = (buf[0] & 0xC0) >> 6;
    if (hdr->ver != 1)
        return COAP_ERR_VERSION_NOT_1;
    hdr->t = (buf[0] & 0x30) >> 4;
    hdr->tkl = buf[0] & 0x0F;
    hdr->code = buf[1];
    hdr->id[0] = buf[2];
    hdr->id[1] = buf[3];
    return 0;
}

https://github.com/1248/microcoap/blob/master/coap.c

C

def decode(<<1::2, type::2, token_len::4,
             code_class::3, code_detail::5,
             msg_id::16,
             token::binary-size(token_len),
             rest::binary>>) do

def decode(<<1::2, type::2, token_len::4,
             code_class::3, code_detail::5,
             msg_id::16,
             token::binary-size(token_len),
             rest::binary>>) do
  code = decode_code(code_class, code_detail)
  token = decode_token(token, token_len)
  {options, payload} =
    decode_options_and_payload(rest)

  %Message{
    version: 1,
    type: Registry.from(:types, type),
    code: code,
    msg_id: msg_id, token: token,
    options: options, payload: payload
  }
end

int coap_parseHeader(coap_header_t *hdr, 
                     const uint8_t *buf,
                     size_t buflen)
{
    if (buflen < 4)
        return COAP_ERR_HEADER_TOO_SHORT;
    hdr->ver = (buf[0] & 0xC0) >> 6;
    if (hdr->ver != 1)
        return COAP_ERR_VERSION_NOT_1;
    hdr->t = (buf[0] & 0x30) >> 4;
    hdr->tkl = buf[0] & 0x0F;
    hdr->code = buf[1];
    hdr->id[0] = buf[2];
    hdr->id[1] = buf[3];
    return 0;
}

Microservices

Source: http://martinfowler.com/articles/microservices.html

Microservices features

Source: http://martinfowler.com/articles/microservices.html

Services as components
Organized as business capabilites
Smart endpoints and dumb pipes
Decentralized Governance
Decentralized Data Management
Infrastructure Automation
Design for failure
Evolutionary Design

Microservices

"... my primary guideline would be:
don't even consider microservices unless you have a system that's too complex to manage as a monolith

Martin Fowler, MicroservicePremium, May 2015

The Swarm at the edge of the cloud

Elixir in times of microservices*

Processes as nanoservices
Processes packaged in Applications
Fault-tolerant processes
Process supervisors
Distributed Erlang

*http://blog.plataformatec.com.br/2015/06/elixir-in-times-of-microservices/

Elixir in the embedded world

Raspberry Pi

Beaglebone Black

sudo vi /etc/apt/sources.list

# install erlang stuff...

# clone elixir
make && make install

But..

Repeat the process for each board?
- Maybe write a script
What if want to use another board? (will need a different setup)

Nerves

Craft and deploy bulletproof embedded software in Elixir

Based on

https://buildroot.org/

Buildroot is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation.

Code

Package

Run

Prepare

VM

ARM

Toolchain

=

+

|>

{

,

}

=

Code

Package

Run

Prepare

$ mix nerves.new hello_nerves --target rpi3
$ cd hello_nerves
$ mix deps.get
$ mix compile

(write your project in Elixir)

$ mix firmware
$ mix firmware.burn

$ NERVES_TARGET=bbb mix firmware

FOTA (Firmware Over the Air) Upgrades!

$ http POST "192.168.0.121:8988/firmware" \
content-type:application/x-firmware \
x-reboot:true < firmware.fw

Run

How does it compare?

Image sizes

Nerves

Raspbian

~300MB

~20MB

Boot time

Nerves

Raspbian

~30 sec.

~3 sec.

Multiple Boards

Nerves

Raspbian

All drivers included

Nerves

Raspbian

For more Information

http://nerves-project.org/

Appendix

:observer.start

(slide intentionally left blank)

(this is the appendix!)

IoT Architectures and Implementations

Industry case

Code

Compiler

Erlang

VM

(other OS processes)

defmodule Math do
  # this is a comment.
  # below is a function
  def sum(a, b) do
    a + b
  end
end

# pattern matching is like method
# overloading on steroids!

defmodule Math do
  def zero?(0) do
    true
  end
  def zero?(x) when is_integer(x) do
    false
  end
end

{a, b, c} = {:hello, "world", 42} 
a # has value :hello
b # "world"
c # 42

b = :binary.encode_unsigned(258)
# b has value <<1, 2>>
<<x, 2>> = b
# x has value 1
i = :binary.decode_unsigned(<<x, 2>>)
# i has value 258

Elixir

Some Drips

my = self() #PID<0.141.0>

pid = spawn(fn -> 
  :timer.sleep(3000)
  send(my, {:get, "temp"})
end)

receive do # will block for 3 secs!
  {:post, path} ->
    IO.puts "was a post on path '#{path}'"
  {:get, path} ->
    IO.puts "was a get on path '#{path}'"
end

Pattern Matching

> x = 1
1
> 1 = x
1
> 2 = x
** (MatchError)

Elixir programming language evaluation for IoT

Introduction

The Internet

The Internet of Things

The Swarm at the edge of the cloud

Main challenges

Popular languages for IoT

Erlang

Erlang

Erlang

Elixir

~2011

Elixir

Extensibility

Productivity

Compatibility

Elixir

Erlang

Industry

Nerves Framework

Method

Swarm Broker

Swarm Broker

Authorizator

Comparison

Comparison

Platform

Results

Lines of Code

Elixir

Java

551

176

Discussion

Next steps

Obrigado

Thanks

Contents

Functionaλ Programming

Welcome to the Functional Web

Functional Programming Languages

Elixir

José Valim

So what's interesting about

this Erlang thing?

What even an "Erlang" is?

A bit of History

Cases

Features

Pattern Matching

Anonymous Functions

Processes

A lightweight thread of execution

How is this different from threads?

No shared memory

VM Processes

Do not share memory

Communicate via

Message Passing

VM Processes

Erlang VM

A lightweight thread of execution

Messages

Distributed

Same syntax

We have:

Isolated Processes

Message passing

Distribution

We want:

Fault Tolerance

dangling processes!

call the supervisor

supervisor

Supervision Tree

Applications

your app

httpserver

logger

Distributed Applications

http
server

Making the IoT
Functional with Elixir:
Four scenarios

"... my primary guideline would be:
don't even consider microservices unless you have a system that's too complex to manage as a monolith