Unison: a language for distributed, elastic computation
Paul Chiusano (@pchiusano)
Cofounder, Unison Computing
joint work with:
Rúnar Bjarnason (@runarorama)
Arya Irani (@aryairani)
DISCLAIMER!!!1!1! : UNDER CONSTRUCTION
Unison
-
Statically-typed, functional
-
Open source, MIT licensed
-
In R&D since 2013... now working toward spring 2019 release
WHY?
Programming = FUN!! 🦄 🌈 🎉
Goal: make every aspect of software creation delightful... or at least REASONABLE
λ> "ARI""ARI"λ> replicate 6 "ARI"["ARI","ARI","ARI","ARI","ARI","ARI"]
λ> replicate 92 "ARI"["ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI","ARI","ARI","ARI","ARI",
"ARI","ARI","ARI","ARI"]
λ> replicate 92 "MAMA"["MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA","MAMA",
"MAMA","MAMA","MAMA","MAMA"]
1 OS process
EVERYTHING ELSE
Docker
Kubernetes
Terraform
Kafka
DynamoDB
S3
EC2
ElasticSearch
Kibana
Prometheus
Grafana
PagerDuty
etcd
ELB
Route 53
Consul
systemd
Flannel
Weave
Lambda
App Engine
rkt
CoreOS
Zookeeper
Redis
memcached
Protobufs
Thrift
Envoy
Mesos
Nomad
ASGs
←JSON→
Haskell
Java
"Just set up an ASG connected to ELB"
Chef
Puppet
Distributed, elastic compute: the default
Cheaper/faster
Why?
Data deps of compute
Netflix, nytimes.com...
Also: people sleep, world popl not evenly distributed...
Before the internet:
1 OS process
✅ 👍
Unison
increment : Nat -> Nat
increment n = n + 1
> x = 1 + 40
> y = increment xUnison basics
👀 Now evaluating any watch expressions (lines starting with `>`) ...
4 | > x = 1 + 40
⧩
41
5 | > increment x
⧩
42⬇️
Unison distributed programming
doDistributedStuff lotsOfData =
alice : Node
alice = spawn
job1 = at alice '(hugeComputation1 lotsOfData)
job2 = at spawn '(hugeComputation2 lotsOfData)
force job1 + force job2
spawn : {Remote} Node
at : Node -> '{Remote} a ->{Remote} (Future a)
force : Future a ->{Remote} adoDistributedStuff : Stuff -> Nat
doDistributedStuff : Stuff ->{Remote} Nat
👇
runLocal : '{Remote} a -> a
runLocalChaosMonkey : '{Remote} a -> a
Cloud.run : Params -> '{Remote} a -> a
> runLocal '(doDistributedStuff lotsOfData)
> runLocalChaosMonkey '(doDistributedStuff lotsOfData)
> Cloud.run usEast '(doDistributedStuff lotsOfData)👈
merge : (a -> a -> Boolean) -> [a] -> [a] -> [a]
merge lte a b =
go out a b = case (uncons a, uncons b) of
(None,_) -> out ++ b
(_,None) -> out ++ a
(Some (hA, tA), Some (hB, tB)) ->
if lte hA hB then go (snoc hA out) tA b
else go (snoc hB out) a tB
go [] a b
> merge (<) [1,3,4,99,504,799] [0,19,22,23]
Ex: distributed mergesort in 17 LOC
93 | > merge (<) [1,3,4,99,504,799] [0,19,22,23]
⧩
[0, 1, 3, 4, 19, 22, 23, 99, 504, 799]⬇️
sort : (a -> a -> Boolean) -> [a] -> [a]
sort lte s =
if size s < 2 then s
else case halve s of (left, right) ->
r1 = sort lte left
r2 = sort lte right
merge lte r1 r2
> sort (<) [1,2,3,234,6,2,4,66,2,33,4,2,57]Ex: distributed mergesort in 17 LOC
112 | > sort (<) [1,2,3,234,6,2,4,66,2,33,4,2,57]
⧩
[1, 2, 2, 2, 2, 3, 4, 4, 6, 33, 57, 66, 234]⬇️
dsort : (a -> a -> Boolean) -> [a] ->{Remote} [a]
dsort lte s =
if size s < 2 then s
else case halve s of (left, right) ->
r1 = at spawn '(dsort lte left)
r2 = at spawn '(dsort lte right)
merge lte (force r1) (force r2)
> runLocal '(dsort (<) [1,2,3,234,6,2,4,66,2,33,4,2,57])
Ex: distributed mergesort in 17 LOC
114 | > Remote.runLocal '(dsort (<) [1,2,3,234,6,2,4,66,2,33,4,2,57])
⧩
[1, 2, 2, 2, 2, 3, 4, 4, 6, 33, 57, 66, 234]⬇️
dsort : (a -> a -> Boolean) -> [a] ->{Remote} [a]
dsort lte s =
if size s < 2 then s
else case halve s of (left, right) ->
r1 = at spawn '(dsort lte left)
r2 = at spawn '(dsort lte right)
merge lte (force r1) (force r2)
Ex: distributed mergesort in 17 LOC
sort : (a -> a -> Boolean) -> [a] -> [a]
sort lte s =
if size s < 2 then s
else case halve s of (left, right) ->
r1 = sort lte left
r2 = sort lte right
merge lte r1 r2DEMO
Provisioning, deployment, and scaling are part of the programming model
Deployment
Scaling, failover...
Provisioning
doDistributedStuff lotsOfData =
alice : Node
alice = spawn
job1 = at alice '(hugeComputation1 lotsOfData)
job2 = at spawn '(hugeComputation2 lotsOfData)
force job1 + force job2
Composability for distributed programs
dsort : (a -> a -> Boolean) -> [a] ->{Remote} [a]
dsort lte s =
if size s < 2 then s
else case halve s of (left, right) ->
r1 = at spawn '(dsort lte left)
r2 = at spawn '(dsort lte right)
merge lte (force r1) (force r2)
dreduce : Monoid a -> [a] ->{Remote} a
dreduce m s =
if size s < 2 then Monoid.orElse m (Sequence.head s)
else case halve s of (left, right) ->
r1 = at spawn '(dreduce m left)
r2 = at spawn '(dreduce m right)
Monoid.op m (force r1) (force r2)
dsort : (a -> a -> Boolean) -> [a] ->{Remote} [a]
dsort lte s =
dreduce (Monoid (merge lte) [])
(Sequence.map single s)dreduce : Monoid a -> [a] ->{Remote} a
dsort : (a -> a -> Boolean) -> [a] ->{Remote} [a]
dsort lte s =
dreduce (Monoid (merge lte) [])
(Sequence.map single s)Roadmap
meetup.com/boston-unison
Target: public release Spring 2019
A pre-alpha sooner than that
Also
🙏 Arya & Rúnar, for being such amazing coworkers 🦄 🌈
🙏 Chris Gibbs, new contributor, for work on pretty-printer + design reviews ❤️ 🎉
@unisonweb, unisonweb.org
Lots of cool work to do, impact can be HUGE
Appendix
bob = spawn
at bob '(factorial n)... but how?
.
.
factorial n =
foldLeft (*) 1 (range 1 (n + 1))
blah z =
foldLeft (*) 1 (range 1 (z + 1))Key idea: identify code via content hash
#Q82jfkasdf823jbc192at bob '(factorial n)at bob '(#Q82jfkasdf823jbc192 n).
.
.
.
.
.
.
.
factorial n =
foldLeft (*) 1 (range 1 (n + 1))Implication: codebase is a purely functional data structure
#Q82jfkasdf823jbc192at bob '(factorial n)at bob '(#Q82jfkasdf823jbc192 n).
.
factorial n = 42#Zhsdf238jsfjasdWjd3