Parallelism and Concurrency in Clojure
Richard Whaling
richard@spantree.net
Spantree Technology Group LLC
Parallel Programs are Hard!
(def counter 0)
(dotimes [i 100000]
(.start (Thread. (fn []
;; (println counter)
(def counter (inc counter))))))
(println counter)
N.B.: This is not idiomatic Clojure!
Don't ever do this in Java either!
Atomic operations help
(def counter (atom 0))
(dotimes [i 100000]
(.start (Thread. (fn []
(do
(swap! counter inc))))))
(println @counter)This works, but limited to a single atom.
Synchronized Transactions:
(def counter (ref 0))
(dotimes [i 100000]
(.start(Thread. (fn []
(dosync
(alter counter inc))))))
(println @counter)
This is more flexible, but expensive.
And things can still go wrong.
Doing this with i = 1,000,000 locks the JVM!
What is Concurrency?
"Concurrency is the composition of independently executing computations.
Concurrency is a way to structure software, particularly as a way to write clean code that interacts well with the real world." -- Rob Pike
Concurrency vs Parallelism
"Concurrency is not parallelism, although it enables parallelism.
If you have only one processor, your program can still be concurrent but it cannot be parallel.
On the other hand, a well-written concurrent program might run efficiently in parallel on a multiprocessor." -- Rob Pike (again)
Concurrency as Style
An effective concurrent language or framework allows one to:
- Understand a complex system as a set of discrete processes
- Reason about each process as a single serial flow of execution
- Eliminate memory safety issues by encapsulating all access in a single process
- Write programs that can be implicitly parallelized, or entirely serialized
Varieties of Concurrency
- Communicating Sequential Processes (CSP)
- Actors
- Erlang
- Concurrent ML
- Go
- Scala/Akka
- Clojure/core.async
Why concurrency?
Software is moving from monoliths to microservices.
Monoliths, like MS Windows, consist of many millions of lines of code, developed by hundreds or thousands of engineers.
Microservices, like Netflix and Facebook's back-ends, are usually a few tens of thousands of lines of code each, but there are dozens of them!
Microservices are much easier to write and test, but network I/O becomes a bottleneck.
Asynchronous I/O
Asynchronous I/O is a technique for making programs that do not waste CPU cycles while they are waiting for things to happen!
What components are slower than CPU?
- Databases
- Remote servers
- Disk
- Graphics
Instead of waiting for these to return, you write event-driven code that defines actions to take when the task is completed.
Case Study: PayPal
In 2013, PayPal rewrote a large multithreaded Java application to node.js. The node.js app:
- Was written in 40% fewer lines of code
- Served requests 200ms faster
- Served twice as many requests per second!
- Uses a single-threaded event-loop architecture
2x requests per second means 50% reduction in infrastructure costs. If you get a 7-figure Amazon bill every month, that's significant.
clojure.core.async
A library that:
- Enables lightweight processes via goroutines
- Allows processes to communicate via channels
- Works on dedicate threads, thread pools, or single-threaded event loops
- Is implemented as a pure Clojure library
- Runs on both the Java and Javascript Clojure runtimes
- Provides both blocking and asynchronous (non-blocking) channel operations
- Design based on CSP and Go
Channels and Goroutines
;; create an unbuffered channel
(def counter (chan))
;; create a sender goroutine
(go (dotimes [i 100000]
;; send a value onto the channel
(>! counter 1)))
;; create a receiver goroutine
(go-loop [n 0]
(println n)
;; receive, increment, loop
(recur (+ n (<! counter))))
A Simple Distributed System
API
Authentication
Database
User
Simple Microservices
(defn db [q]
(println ["querying" q])
(Thread/sleep (rand-int 1000))
(join ":" ["db", q]))
(defn auth [u]
(println ["authenticating" u])
(Thread/sleep (rand-int 1000))
true)
Core business logic is functional, and contains no concurrent operations.
Take 1: Read/Write Channel
(defn auth-service-1 []
(let [auth-ch (chan)]
(go-loop []
(let [req (<! auth-ch)]
(>! auth-ch (auth req)))
(recur))
auth-ch))
(defn auth-query-1 [auth-chan q]
(go (>! auth-chan q))
auth-chan)
Note that the service reads and writes to the same channel.
Take 1: Read/Write Channel
;; blocking read
(let [auth-ch (auth-service-1)
queries (range 3)]
(doseq [q queries]
(auth-query-1 auth-ch q)
(println (<!! auth-ch))))
;; concurrent read
(let [auth-ch (auth-service-1)
queries (range 3)]
(doseq [q queries]
(auth-query-1 auth-ch q)
(go (println (<! auth-ch)))))Reading one value off at a time works, concurrent reads are broken. Why?
Take 2: Request/Response Channels
(defn auth-service-2 []
(let [auth-ch (chan)]
(go-loop []
(let [ [q output] (<! auth-ch) ]
(>! output (auth q)))
(recur))
auth-ch))
(defn auth-query-2 [auth-chan q]
(let [resp-ch (chan)]
(go (>! auth-chan [q resp-ch]))
resp-ch))
Note that the query function creates a channel to receive the response.
Take 2: Request/Response Channels
;concurrent read
(let [auth-ch (auth-service-2)
queries (range 3)
responses (for [q queries] (auth-query-2 auth-ch q))]
(doseq [r responses]
(go (println (<! r)))))
;blocking read
(let [auth-ch (auth-service-2)
queries (range 3)
responses (for [q queries] (auth-query-2 auth-ch q))]
(doseq [r responses]
(println (<!! r))))
Now the concurrent read works, but the blocking read stalls!
Take 3: Request/Response Channels, Buffered
(defn auth-service-3 []
(let [auth-ch (chan)]
(go-loop []
(let [[req resp-ch] (<! auth-ch)]
(>! resp-ch (auth req)))
(recur))
auth-ch))
(defn auth-query-3 [auth-ch q]
(let [resp-ch (chan 1)]
(go (>! auth-ch [q resp-ch]))
resp-ch))
The only difference is that resp-ch is now (chan 1) instead of (chan)
Take 3: Request/Response Channels, Buffered
; blocking read
(let [auth-ch (auth-service-3)
queries (range 3)
responses (for [q queries] (auth-query-3 auth-ch q))]
(doseq [r responses]
(println (<!! r))))
; concurrent read
(let [auth-ch (auth-service-3)
queries (range 3)
responses (for [q queries] (auth-query-3 auth-ch q))]
(doseq [r responses]
(go (println (<!! r)))))
It works!
A Simple Distributed System
API
Authentication
Database
User
Applying the pattern:
(defn db-service []
(let [db-ch (chan)]
(go-loop []
(let [[req resp-ch] (<! db-ch) ]
(>! resp-ch (db req)))
(recur))
db-ch))
(defn db-query [db-chan q]
(let [resp-ch (chan 1)]
(go (>! db-chan [q resp-ch]))
resp-ch))
Applying the pattern:
(defn api-service [auth-ch db-ch]
(let [input (chan)]
(go-loop []
(let [[u q output] (<! input)
auth-output (auth-query-3 auth-ch u)
db-output (db-query db-ch q)]
(if (<! auth-output)
(>! output
(<! db-output))))
(recur))
input))
(defn api-query [input u q]
(let [output (chan 1)]
(go (>! input [u q output]))
output))
Testing:
;; blocking read
(let [auth-ch (auth-service-3)
db-ch (db-service)
api-ch (api-service auth-ch db-ch)
queries (range 3)
responses (for [q queries] (api-query api-ch q q))]
(doseq [r responses]
(println ["api sent" (<!! r)])))
;; concurrent read
(let [auth-ch (auth-service-3)
db-ch (db-service)
api-ch (api-service auth-ch db-ch)
queries (range 3)
responses (for [q queries] (api-query api-ch q q))]
(doseq [r responses]
(go (println ["api sent" (<! r)]))))
Conclusions
- Concurrency is easier and safer than low-level parallelism, but the patterns are still subtle
- New techniques and frameworks are being developed all the time!
- Clojure, Scala, Rust, JavaScript, and Python are innovating concurrent styles in different ways.
- LISPs are always going to have an advantage in expressive new language features.
- Everyone who writes software for the web needs to understand concurrency.
concurrency_in_clojure
By Richard Whaling
concurrency_in_clojure
- 1,037
