Making the IoT Functional with Elixir

Geovane Fedrecheski - geovane@lsi.usp.br

Pablo C. Ccori - pablo@lsi.usp.br

School of Engineering

University of São Paulo

The Internet of Things
Functional Programming
Elixir
IoT + Elixir: 4 Case Studies

The Internet

The Internet of Things

IoT applications

Smart infrastructure
Healthcare
Logistics
Social applications

Industry case

IoT Architectures and Implementations

Main challenges

Resource constrained
Power efficiency
Communication unreliability
Context sensitivity
Remote management
Concurrency

Popular languages for IoT

C
C++
Java
Javascript
Python

http://www.techworm.net/2016/06/best-programming-language-iot.html

Functionaλ Programming

f(g(x), h(...))

Steve Vinoski, IEEE Computing Society 2009

"... ironically, the resurgence of interest in FP languages owes a lot to Java’s popularity."

"Another reason [...] is the move toward multicore architectures."

"... developers look for the opportunity to
get more done with less."

Welcome to the Functional Web

Functional Programming Languages

Lisp

Elm

Elixir

José Valim

Core committer

Unsatisfied with thread-based concurrency

~2010

Unsatisfied with thread-based concurrency

Repetitive strain

injury

by Bruce A. Tate

So what's interesting about

this Erlang thing?

What even an "Erlang" is?

A bit of History

Erlang

context

Created in the Telecommunication industry

...

~1980

Erlang

requirements

Concurrency
Soft real-time
Distribution
Hardware Interaction
Large Software Systems
Complex Functionality
Continuous Operation
Quality Requirements
Fault Tolerance

Programming language

Virtual Machine - BEAM

Framework - OTP

-module(fact).
-export([fac/1]).

fac(0) -> 1;
fac(N) -> N * fac(N-1).

Cases

The VM

Concurreny

Fault tolerance

Data structures

Consistent libs

Modern tooling

I loved everything that Erlang had

but I hated everything that it didn't had

- José Valim

Elixir

~2011

Elixir

Goals

Extensibility
Productivity
Compatibility

Elixir

Erlang

Features

Pattern Matching

> {method, path} = {:get, "temp"}
{:get, "temp"}
> method
:get
> path
"temp"

b = :binary.encode_unsigned(258)
# b has value <<1, 2>>

<<x, 2>> = b
# x has value 1

i = :binary.decode_unsigned(<<x, 2>>)
# i has value 258

request = {:get, "temp"}



case request do
  {:post, path, payload} ->
    send_resp 200
  {:get, path} ->
    send_resp 200, "{'value':17.5, 'unit': 'celsius'}"
end

defmodule Math do
  def fib(0), do: 0
  def fib(1), do: 1
  def fib(n), do: fib(n-1) + fib(n-2)
end

Math.fib(8) # => 21

Anonymous Functions

> add = fn(a, b) -> a + b end
#Function<12.71889879/2 in :erl_eval.expr/5>
> add.(1, 2)
3

> Enum.map([17.9, 20.2, 18.4], fn(x) -> x * 2 end)
[35.8, 40.4, 36.8]

Can be passed as parameters

Processes

spawn

A lightweight thread of execution

(The key Erlang/Elixir abstraction)

How is this different from threads?

No shared memory
VM Processes

Do not share memory

Communicate via

Message Passing

message

VM Processes

Excuse me, have you ever heard of (VM) processes?

Erlang VM

Scheduler

Sched.

OS Processes

OS Threads

VM Processes

pid = spawn(fn ->
  :timer.sleep(3000)
  IO.puts "hi"
end)

fn ..

spawn

self()

sleep

puts

A lightweight thread of execution

Messages

pid = spawn(...)
send(pid, "hello")

receive do
  msg ->
    IO.puts msg
end

fn ..

spawn

self()

receive

send

Node.spawn(
  :"app@computer-2",
  fn -> Hello.world end)

send

receive

app@computer-1

app@computer-2

Distributed

Same syntax

We have:

Isolated Processes
Message passing
Distribution

We want:

Fault Tolerance

dangling processes!

spawn

spawn

call the supervisor

supervisor

worker

worker

Supervision Tree

import Supervisor.Spec

children = [
  worker(Cache, []),
  supervisor(DatabaseSupervisor, []),
  supervisor(TCPSupervisor, [4040])
]

Supervisor.start_link(children, strategy: :one_for_one)

Applications

your app

http
server

logger

app@computer-1

Distributed Applications

app@computer-2

Fault Tolerance

Open Telecom Platform

Supervisor
Application
GenServer
GenEvent

OOP

Objects

Elixir

Processes

Production-ready?

https://github.com/doomspork/elixir-companies

Making the IoT
Functional with Elixir:
Four scenarios

Easy CoAP binary parser
Fault-tolerant microservices

Elixir and Java load test
Automatic deployment of Elixir images

CoAP

Constrained Application Protocol
kind of a "smaller HTTP, for IoT"

request

response

CoAP - Codes

(binary header)

0.01 (GET) (Request)

0.02 (POST) (Request)

2.05 (Content) (Response)

4.04 (Not Found) (Response)

000.00001

000.00010

010.00101

100.00100

CoAP

parser

0.01

123

01000000 00000001 0000000001111011 11111111

255

64 1 0 123 255

(empty)

%Message(type: :con,
         code: :get,
         msg_id: 123,
         options: [])

int coap_parseHeader(coap_header_t *hdr, 
                     const uint8_t *buf,
                     size_t buflen)
{
    if (buflen < 4)
        return COAP_ERR_HEADER_TOO_SHORT;
    hdr->ver = (buf[0] & 0xC0) >> 6;
    if (hdr->ver != 1)
        return COAP_ERR_VERSION_NOT_1;
    hdr->t = (buf[0] & 0x30) >> 4;
    hdr->tkl = buf[0] & 0x0F;
    hdr->code = buf[1];
    hdr->id[0] = buf[2];
    hdr->id[1] = buf[3];
    return 0;
}

https://github.com/1248/microcoap/blob/master/coap.c

C

def decode(<<1::2, type::2, token_len::4,
             code_class::3, code_detail::5,
             msg_id::16,
             token::binary-size(token_len),
             rest::binary>>) do

def decode(<<1::2, type::2, token_len::4,
             code_class::3, code_detail::5,
             msg_id::16,
             token::binary-size(token_len),
             rest::binary>>) do
  code = decode_code(code_class, code_detail)
  token = decode_token(token, token_len)
  {options, payload} =
    decode_options_and_payload(rest)

  %Message{
    version: 1,
    type: Registry.from(:types, type),
    code: code,
    msg_id: msg_id, token: token,
    options: options, payload: payload
  }
end

int coap_parseHeader(coap_header_t *hdr, 
                     const uint8_t *buf,
                     size_t buflen)
{
    if (buflen < 4)
        return COAP_ERR_HEADER_TOO_SHORT;
    hdr->ver = (buf[0] & 0xC0) >> 6;
    if (hdr->ver != 1)
        return COAP_ERR_VERSION_NOT_1;
    hdr->t = (buf[0] & 0x30) >> 4;
    hdr->tkl = buf[0] & 0x0F;
    hdr->code = buf[1];
    hdr->id[0] = buf[2];
    hdr->id[1] = buf[3];
    return 0;
}

Microservices

Source: http://martinfowler.com/articles/microservices.html

Microservices features

Source: http://martinfowler.com/articles/microservices.html

Services as components
Organized as business capabilites
Smart endpoints and dumb pipes
Decentralized Governance
Decentralized Data Management
Infrastructure Automation
Design for failure
Evolutionary Design

Microservices

"... my primary guideline would be:
don't even consider microservices unless you have a system that's too complex to manage as a monolith

Martin Fowler, MicroservicePremium, May 2015

The Swarm at the edge of the cloud

Elixir in times of microservices*

Processes as nanoservices
Processes packaged in Applications
Fault-tolerant processes
Process supervisors
Distributed Erlang

*http://blog.plataformatec.com.br/2015/06/elixir-in-times-of-microservices/

Elixir and Java

load test

Swarm Broker

Comparison

Lines of Code
CPU
Memory
Requests / sec.

Platform

Quad-core computer (x86)
6GB of RAM
Java 1.8
Erlang 18.2
Elixir 1.2
Load test tool: Tsung 1.6

Lines of Code

Elixir

Java

551

176

	Lines of Code
	CPU
	Memory
	Requests / sec.

Elixir in the embedded world

Raspberry Pi

Beaglebone Black

sudo vi /etc/apt/sources.list

# install erlang stuff...

# clone elixir
make && make install

But..

Repeat the process for each board?
- Maybe write a script
What if want to use another board? (will need a different setup)

Nerves

Craft and deploy bulletproof embedded software in Elixir

Based on

https://buildroot.org/

Buildroot is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation.

Code

Package

Run

Prepare

VM

ARM

Toolchain

=

+

|>

{

,

}

=

Code

Package

Run

Prepare

$ mix nerves.new hello_nerves --target rpi3
$ cd hello_nerves
$ mix deps.get
$ mix compile

(write your project in Elixir)

$ mix firmware
$ mix firmware.burn

$ NERVES_TARGET=bbb mix firmware

FOTA (Firmware Over the Air) Upgrades!

$ http POST "192.168.0.121:8988/firmware" \
content-type:application/x-firmware \
x-reboot:true < firmware.fw

Run

How does it compare?

Image sizes

Nerves

Raspbian

~300MB

~20MB

Boot time

Nerves

Raspbian

~30 sec.

~3 sec.

Multiple Boards

Nerves

Raspbian

All drivers included

Nerves

Raspbian

For more Information

http://nerves-project.org/

Obrigado

Gracias

Merci

Thanks

Appendix

:observer.start

(slide intentionally left blank)

(this is the appendix!)

Code

Compiler

Erlang

VM

(other OS processes)

defmodule Math do
  # this is a comment.
  # below is a function
  def sum(a, b) do
    a + b
  end
end

# pattern matching is like method
# overloading on steroids!

defmodule Math do
  def zero?(0) do
    true
  end
  def zero?(x) when is_integer(x) do
    false
  end
end

{a, b, c} = {:hello, "world", 42} 
a # has value :hello
b # "world"
c # 42

b = :binary.encode_unsigned(258)
# b has value <<1, 2>>
<<x, 2>> = b
# x has value 1
i = :binary.decode_unsigned(<<x, 2>>)
# i has value 258

Elixir

Some Drips

my = self() #PID<0.141.0>

pid = spawn(fn -> 
  :timer.sleep(3000)
  send(my, {:get, "temp"})
end)

receive do # will block for 3 secs!
  {:post, path} ->
    IO.puts "was a post on path '#{path}'"
  {:get, path} ->
    IO.puts "was a get on path '#{path}'"
end

Pattern Matching

> x = 1
1
> 1 = x
1
> 2 = x
** (MatchError)

Making the IoT Functional with Elixir

Contents

The Internet

The Internet of Things

IoT applications

Industry case

IoT Architectures and Implementations

Main challenges

Popular languages for IoT

Functionaλ Programming

Welcome to the Functional Web

Functional Programming Languages

Elixir

José Valim

So what's interesting about

this Erlang thing?

What even an "Erlang" is?

A bit of History

Erlang

Erlang

Cases

Elixir

~2011

Elixir

Extensibility

Productivity

Compatibility

Elixir

Erlang

Features

Pattern Matching

Anonymous Functions

Processes

A lightweight thread of execution

How is this different from threads?

No shared memory

VM Processes

Do not share memory

Communicate via

Message Passing

VM Processes

Erlang VM

A lightweight thread of execution

Messages

Distributed

Same syntax

We have:

Isolated Processes

Message passing

Distribution

We want:

Fault Tolerance

dangling processes!

call the supervisor

supervisor

Supervision Tree

Applications

your app

httpserver

logger

Distributed Applications

Fault Tolerance

Open Telecom Platform

Supervisor

Application

GenServer

GenEvent

OOP

Objects

Elixir

Processes

Production-ready?

Making the IoT Functional with Elixir: Four scenarios

CoAP

CoAP - Codes

CoAP

parser

C

Microservices

Microservices features

http
server

Making the IoT
Functional with Elixir:
Four scenarios

"... my primary guideline would be:
don't even consider microservices unless you have a system that's too complex to manage as a monolith