Elixir 1.6
Chapter 15
Working with
Multiple Processes
Actors
Elixir uses the actor model of concurrency.
An actor is an independent process that shares nothing with any other process.
You can spawn new processes, send them messages, and receive messages back.
Elixir uses process support in Erlang. These processes will run across all your CPUs (just like native processes), but they have very little overhead.
A Simple Process
The spawn function kicks off a new process and returns a Process Identifier, normally called a PID.
When we call spawn, it creates a new process to run the code we specify. We don’t know exactly when it will execute—we know only that it is eligible to run.
defmodule SpawnBasic do
def greet do
IO.puts "Hello"
end
end
iex> c("spawn-basic.ex")
[SpawnBasic]
iex> SpawnBasic.greet
Hello
:ok
iex> spawn(SpawnBasic, :greet, [])
Hello
#PID<0.42.0>Sending Messages Between Processes
We send a message using the send function. It takes a PID and the message to send .
You can send anything you want, but most Elixir developers seem to use atoms and tuples.
defmodule Spawn1 do
def greet do
receive do
{sender, msg} ->
send sender, { :ok, "Hello, #{msg}" }
end
end
end
# here's a client
pid = spawn(Spawn1, :greet, [])
>> send pid, {self, "World!"}
receive do
{:ok, message} ->
IO.puts message
end
Sending Messages Between Processes
We wait for messages using receive. This acts like case with the message body as the parameter.
Inside the block associated with the receive call, you can specify any number of patterns and associated actions. The first pattern that matches the function is run.
defmodule Spawn1 do
def greet do
>> receive do
{sender, msg} ->
send sender, { :ok, "Hello, #{msg}" }
end
end
end
# here's a client
pid = spawn(Spawn1, :greet, [])
send pid, {self, "World!"}
receive do
{:ok, message} ->
IO.puts message
end
Sending Messages Between Processes
We use send sender, ... to send a formatted string back to the original message sender. We package that string into a tuple, with :ok as its first element.
defmodule Spawn1 do
def greet do
receive do
{sender, msg} ->
>> send sender, { :ok, "Hello, #{msg}" }
end
end
end
# here's a client
pid = spawn(Spawn1, :greet, [])
send pid, {self, "World!"}
receive do
{:ok, message} ->
IO.puts message
end
Sending Messages Between Processes
We call spawn to create a process, and send it a tuple.
The function self returns its caller’s PID. We use it to pass our PID to the greet function.
In our receive,we do a pattern match on {:ok, message}.
defmodule Spawn1 do
def greet do
receive do
{sender, msg} ->
send sender, { :ok, "Hello, #{msg}" }
end
end
end
# here's a client
>> pid = spawn(Spawn1, :greet, [])
send pid, {self, "World!"}
receive do
{:ok, message} ->
IO.puts message
end
Handling Multiple Messages
This will wait forever because once receive has been process it exits. greet only handles one message.
defmodule Spawn2 do
def greet do
receive do
{sender, msg} ->
send sender, { :ok, "Hello, #{msg}" }
end
end
end
# here's a client
pid = spawn(Spawn2, :greet, [])
send pid, {self, "World!"}
receive do
{:ok, message} ->
IO.puts message
end
send pid, {self, "Kermit!"}
receive do
{:ok, message} ->
IO.puts message
end
Handling Multiple Messages (Take Two)
Now we will end after 500 milliseconds
but greet still only handles one message.
defmodule Spawn3 do
def greet do
receive do
{sender, msg} ->
send sender, { :ok, "Hello, #{msg}" }
end
end
end
# here's a client
pid = spawn(Spawn3, :greet, [])
send pid, {self, "World!"}
receive do
{:ok, message} ->
IO.puts message
end
send pid, {self, "Kermit!"}
receive do
{:ok, message} ->
IO.puts message
after 500 ->
IO.puts "The greeter has gone away"
end
Handling Multiple Messages (Take Three)
Yay! Now both messages get handled.
defmodule Spawn4 do
def greet do
receive do
{sender, msg} ->
send sender, { :ok, "Hello, #{msg}" }
greet()
end
end
end
# here's a client
pid = spawn(Spawn4, :greet, [])
send pid, {self, "World!"}
receive do
{:ok, message} ->
IO.puts message
end
send pid, {self, "Kermit!"}
receive do
{:ok, message} ->
IO.puts message
after 500 ->
IO.puts "The greeter has gone away"
end
Recursion, Looping, and the Stack
Elixir implements tail-call optimization to efficiently handle recursion but beware—the recursive call must be the very last thing executed.
# tail-call optimization not used
def factorial(0), do: 1
def factorial(n), do: n * factorial(n-1)
# tail-call optimization
defmodule TailRecursive do
def factorial(n), do: _fact(n, 1)
defp _fact(0, acc), do: acc
defp _fact(n, acc), do: _fact(n-1, acc*n)
endProcess Overhead
defmodule Chain do
def counter(next_pid) do
receive do
n ->
send next_pid, n + 1
end
end
def create_processes(n) do
last = Enum.reduce 1..n, self,
fn (_,send_to) ->
spawn(Chain, :counter, [send_to])
end
send last, 0 # start the count by sending a zero to the last process
receive do # and wait for the result to come back to us
final_answer when is_integer(final_answer) ->
"Result is #{inspect(final_answer)}"
end
end
def run(n) do
IO.puts inspect :timer.tc(Chain, :create_processes, [n])
end
end
Process Overhead
def counter(next_pid) do
receive do
n ->
send next_pid, n + 1
end
endThe counter function is the code that will be run in separate processes.
It is passed the PID of the next process in the chain. When it receives a number, it increments it and sends it on to that next process
Process Overhead
def create_processes(n) do
last = Enum.reduce 1..n, self,
fn (_,send_to) ->
spawn(Chain, :counter, [send_to])
end
send last, 0 # start the count by sending a zero to the last process
receive do # and wait for the result to come back to us
final_answer when is_integer(final_answer) ->
"Result is #{inspect(final_answer)}"
end
end
create_processes is passed a number of processes to create. Each processed is passed
the PID of the previous process.
reduce will iterate passing the PID to the next call. We pass it self, our PID. Each time we spawn a new process we pass it to the previous process's PID to send_to.
Process Overhead
def create_processes(n) do
last = Enum.reduce 1..n, self,
fn (_,send_to) ->
spawn(Chain, :counter, [send_to])
end
send last, 0 # start the count by sending a zero to the last process
receive do # and wait for the result to come back to us
final_answer when is_integer(final_answer) ->
"Result is #{inspect(final_answer)}"
end
end
We pass 0 to the last process which gets the ball rolling.
We use the receive block to capture the final answer and format it.
We are using a guard clause on final_answer
to make sure it's an integer.
Process Overhead
def run(n) do
IO.puts inspect :timer.tc(Chain, :create_processes, [n])
end
The run function starts the whole thing off.
It uses a built-in Erlang library, tc,
which times a function’s execution.
We pass it the module, name, and parameters, and it responds with a tuple. The first element is the
execution time in microseconds and
the second is the result the function returns.
Process Overhead
defmodule Chain do
def counter(next_pid) do
receive do
n ->
send next_pid, n + 1
end
end
def create_processes(n) do
last = Enum.reduce 1..n, self,
fn (_,send_to) ->
spawn(Chain, :counter, [send_to])
end
send last, 0 # start the count by sending a zero to the last process
receive do # and wait for the result to come back to us
final_answer when is_integer(final_answer) ->
"Result is #{inspect(final_answer)}"
end
end
def run(n) do
IO.puts inspect :timer.tc(Chain, :create_processes, [n])
end
end
Process Overhead
$ elixir -r chain.ex -e "Chain.run(10)"
{4015, "Result is 10"}
$ elixir -r chain.ex -e "Chain.run(100)"
{4562, "Result is 100"}
$ elixir -r chain.ex -e "Chain.run(1_000)"
{8458, "Result is 1000"}
$ elixir -r chain.ex -e "Chain.run(10_000)"
{66769, "Result is 10000"}
$ elixir --erl "+P 1000000" -r chain.ex -e "Chain.run(400_000)"
{2249466, "Result is 400000"}
$ elixir --erl "+P 1000000" -r chain.ex -e "Chain.run(1_000_000)"
{5470945, "Result is 1000000"}
# We ran a million processes (sequentially) in about 5½ seconds
When Processes Die
Who gets told when a process dies? By default, no one.
defmodule Link1 do
import :timer, only: [ sleep: 1 ]
def sad_function do
sleep 500
exit(:boom)
end
def run do
spawn(Link1, :sad_function, [])
receive do
msg ->
IO.puts "MESSAGE RECEIVED: #{inspect msg}"
after 1000 ->
IO.puts "Nothing happened as far as I am concerned"
end
end
end
Link1.run
$ elixir -r link1.exs
Nothing happened as far as I am concerned
Linking Two Processes
We can link processes so that each can receive information when the other exits.
To link them we call spawn_link.
Linked processes die together.
defmodule Link2 do
import :timer, only: [ sleep: 1 ]
def sad_function do
sleep 500
exit(:boom)
end
def run do
spawn_link(Link2, :sad_function, [])
receive do
msg ->
IO.puts "MESSAGE RECEIVED: #{inspect msg}"
after 1000 ->
IO.puts "Nothing happened as far as I am concerned"
end
end
end
Link2.run
$ elixir -r link2.exs
** (EXIT from #PID<0.35.0>) :boomLinking Two Processes
You can tell Elixir to convert the exit signals from a linked process into a message you can handle.
Do this by trapping the exit.
defmodule Link3 do
import :timer, only: [ sleep: 1 ]
def sad_function do
sleep 500
exit(:boom)
end
def run do
Process.flag(:trap_exit, true)
spawn_link(Link3, :sad_function, [])
receive do
msg ->
IO.puts "MESSAGE RECEIVED: #{inspect msg}"
after 1000 ->
IO.puts "Nothing happened as far as I am concerned"
end
end
end
Link3.run
$ elixir -r link3.exs
MESSAGE RECEIVED: {:EXIT, #PID<0.41.0>, :boom}
Monitoring a Process
Monitoring lets a process spawn another and be notified of its termination, but without the reverse notification—it is one-way only.
When you monitor a process, you receive a :DOWN message when it exits or fails, or if it doesn’t exist.
You can use spawn_monitor to turn on monitoring when you spawn a process.
Monitoring a Process
defmodule Monitor1 do
import :timer, only: [ sleep: 1 ]
def sad_function do
sleep 500
exit(:boom)
end
def run do
res = spawn_monitor(Monitor1, :sad_function, [])
IO.puts inspect res
receive do
msg ->
IO.puts "MESSAGE RECEIVED: #{inspect msg}"
after 1000 ->
IO.puts "Nothing happened as far as I am concerned"
end
end
end
Monitor1.run
$ elixir -r monitor1.exs
{#PID<0.37.0>,#Reference<0.0.0.53>}
MESSAGE RECEIVED: {:DOWN,#Reference<0.0.0.53>,:process,#PID<0.37.0>,:boom}Parallel Map - The Erlang "Hello, World"
defmodule Parallel do
def pmap(collection, fun) do
me = self
collection
|> Enum.map(fn (elem) ->
spawn_link fn -> (send me, { self, fun.(elem) }) end
end)
|> Enum.map(fn (pid) ->
receive do { ^pid, result } -> result end
end)
end
end
iex> c("pmap.exs")
[Parallel]
iex> Parallel.pmap 1..10, &(&1 * &1)
[1,4,9,16,25,36,49,64,81,100]The first transformation maps into a list of PIDs.
The second transformation converts the PIDs into the result corresponding to the PID. The ^pid allows us to get the results in the correct order.
A Fibonacci Server
Let's write a parallel Fibonacci calculator. We’ll write a trivial server process that does the calculation, and a scheduler that assigns work to a calculation process when it becomes free.
When the calculator is ready for the next number,
it sends a :ready message to the scheduler.
If there is still work to do, the scheduler sends it to the calculator in a :fib message;
otherwise it sends the calculator a :shutdown.
When a calculator receives a :fib message, it calculates the given Fibonacci number and returns it in an :answer. If it gets a :shutdown, it simply exits.
A Fibonacci Server
Text
A Fibonacci Server
defmodule FibSolver do
def fib(scheduler) do
send scheduler, { :ready, self }
receive do
{ :fib, n, client } ->
send client, { :answer, n, fib_calc(n), self }
fib(scheduler)
{ :shutdown } ->
exit(:normal)
end
end
# very inefficient, deliberately
defp fib_calc(0), do: 0
defp fib_calc(1), do: 1
defp fib_calc(n), do: fib_calc(n-1) + fib_calc(n-2)
end
A Fibonacci Server
When it starts, it sends a :ready message
and waits for a message back.
If it gets a :fib message, it calculates the answer and sends it back to the client.
It then loops by calling itself recursively.
This will send another :ready message, telling the client it is ready for more work.
If it gets a :shutdown it simply exits.
A Fibonacci Server - The Task Scheduler
defmodule Scheduler do
def run(num_processes, module, func, to_calculate) do
(1..num_processes)
|> Enum.map(fn(_) -> spawn(module, func, [self()]) end)
|> schedule_processes(to_calculate, [])
end
defp schedule_processes(processes, queue, results) do
receive do
{:ready, pid} when length(queue) > 0 ->
[ next | tail ] = queue
send pid, {:fib, next, self()}
schedule_processes(processes, tail, results)
{:ready, pid} ->
send pid, {:shutdown}
if length(processes) > 1 do
schedule_processes(List.delete(processes, pid), queue, results)
else
Enum.sort(results, fn {n1,_}, {n2,_} -> n1 <= n2 end)
end
{:answer, number, result, _pid} ->
schedule_processes(processes, queue, [ {number, result} | results ])
end
end
end
A Fibonacci Server - The Task Scheduler
It receives the number of processes to spawn, the module and function to spawn, and a list of things to process. The run function spawns the correct number of processes and records their PIDs and then calls schedule_processes.
If it gets a :ready message from a server, it sees if there is more work in the queue. If there is, it passes the next number to the calculator and then recurses with one fewer number in the queue.
If the work queue is empty when it receives a :ready message, it sends a shutdown to the server.
If it gets an :answer message, it records the answer and recurses to handle the next message.
A Fibonacci Server
The to_process list contains the numbers we’ll be passing to our fib servers.
We give it the same number, 37, six times.
The intent here is to load each of our processors.
to_process = [ 37, 37, 37, 37, 37, 37 ]
Enum.each 1..10, fn num_processes ->
{time, result} = :timer.tc(
Scheduler, :run,
[num_processes, FibSolver, :fib, to_process]
)
if num_processes == 1 do
IO.puts inspect result
IO.puts "\n # time (s)"
end
:io.format "~2B ~.2f~n", [num_processes, time/1000000.0]
endA Fibonacci Server
We run the code a total of 10 times, varying the number of spawned processes from 1 to 10. We use :timer.tc to determine the elapsed time of each iteration, reporting the result in seconds. The first time around the loop, we also display the numbers we calculated.
to_process = [ 37, 37, 37, 37, 37, 37 ]
Enum.each 1..10, fn num_processes ->
{time, result} = :timer.tc(
Scheduler, :run,
[num_processes, FibSolver, :fib, to_process]
)
if num_processes == 1 do
IO.puts inspect result
IO.puts "\n # time (s)"
end
:io.format "~2B ~.2f~n", [num_processes, time/1000000.0]
endA Fibonacci Server
We see time decrease until we load up all our cores.
dustinmccraw@Dustins-Teamsnap-MacBook-Pro spawn $ elixir fib.exs
[{37, 24157817}, {37, 24157817}, {37, 24157817}, {37, 24157817}, {37, 24157817}, {37, 24157817}]
# time (s)
1 6.38
2 3.07
3 2.09
4 2.22
5 2.56
6 1.91
7 1.86
8 1.90
9 1.85
10 1.89Agents—A Teaser
Let's cache our fibonacci calculations.
defmodule FibAgent do
def start_link do
Agent.start_link(fn -> %{ 0 => 0, 1 => 1 } end)
end
def fib(pid, n) when n >= 0 do
Agent.get_and_update(pid, &do_fib(&1, n))
end
defp do_fib(cache, n) do
case cache[n] do
nil ->
{ n_1, cache } = do_fib(cache, n-1)
result = n_1 + cache[n-2]
{ result, Map.put(cache, n, result) }
cached_value ->
{ cached_value , cache }
end
end
end
{:ok, agent} = FibAgent.start_link()
IO.puts FibAgent.fib(agent, 2000)Thinking in Processes
Just about every decent Elixir program will have many, many processes, and by and large they’ll be just as easy to create and manage as the
objects were in object-oriented programming.
Thank you!
Programming Elixir 1.6 Chapter 16
By Dustin McCraw
