Richard Whaling
richard@spantree.net
Spantree Technology Group LLC
(def counter 0)
(dotimes [i 100000]
(.start (Thread. (fn []
;; (println counter)
(def counter (inc counter))))))
(println counter)
N.B.: This is not idiomatic Clojure!
Don't ever do this in Java either!
(def counter (atom 0))
(dotimes [i 100000]
(.start (Thread. (fn []
(do
(swap! counter inc))))))
(println @counter)
This works, but limited to a single atom.
(def counter (ref 0))
(dotimes [i 100000]
(.start(Thread. (fn []
(dosync
(alter counter inc))))))
(println @counter)
This is more flexible, but expensive.
And things can still go wrong.
Doing this with i = 1,000,000 locks the JVM!
"Concurrency is the composition of independently executing computations.
Concurrency is a way to structure software, particularly as a way to write clean code that interacts well with the real world." -- Rob Pike
"Concurrency is not parallelism, although it enables parallelism.
If you have only one processor, your program can still be concurrent but it cannot be parallel.
On the other hand, a well-written concurrent program might run efficiently in parallel on a multiprocessor." -- Rob Pike (again)
An effective concurrent language or framework allows one to:
Software is moving from monoliths to microservices.
Monoliths, like MS Windows, consist of many millions of lines of code, developed by hundreds or thousands of engineers.
Microservices, like Netflix and Facebook's back-ends, are usually a few tens of thousands of lines of code each, but there are dozens of them!
Microservices are much easier to write and test, but network I/O becomes a bottleneck.
Asynchronous I/O is a technique for making programs that do not waste CPU cycles while they are waiting for things to happen!
What components are slower than CPU?
Instead of waiting for these to return, you write event-driven code that defines actions to take when the task is completed.
In 2013, PayPal rewrote a large multithreaded Java application to node.js. The node.js app:
2x requests per second means 50% reduction in infrastructure costs. If you get a 7-figure Amazon bill every month, that's significant.
A library that:
;; create an unbuffered channel
(def counter (chan))
;; create a sender goroutine
(go (dotimes [i 100000]
;; send a value onto the channel
(>! counter 1)))
;; create a receiver goroutine
(go-loop [n 0]
(println n)
;; receive, increment, loop
(recur (+ n (<! counter))))
API
Authentication
Database
User
(defn db [q]
(println ["querying" q])
(Thread/sleep (rand-int 1000))
(join ":" ["db", q]))
(defn auth [u]
(println ["authenticating" u])
(Thread/sleep (rand-int 1000))
true)
Core business logic is functional, and contains no concurrent operations.
(defn auth-service-1 []
(let [auth-ch (chan)]
(go-loop []
(let [req (<! auth-ch)]
(>! auth-ch (auth req)))
(recur))
auth-ch))
(defn auth-query-1 [auth-chan q]
(go (>! auth-chan q))
auth-chan)
Note that the service reads and writes to the same channel.
;; blocking read
(let [auth-ch (auth-service-1)
queries (range 3)]
(doseq [q queries]
(auth-query-1 auth-ch q)
(println (<!! auth-ch))))
;; concurrent read
(let [auth-ch (auth-service-1)
queries (range 3)]
(doseq [q queries]
(auth-query-1 auth-ch q)
(go (println (<! auth-ch)))))
Reading one value off at a time works, concurrent reads are broken. Why?
(defn auth-service-2 []
(let [auth-ch (chan)]
(go-loop []
(let [ [q output] (<! auth-ch) ]
(>! output (auth q)))
(recur))
auth-ch))
(defn auth-query-2 [auth-chan q]
(let [resp-ch (chan)]
(go (>! auth-chan [q resp-ch]))
resp-ch))
Note that the query function creates a channel to receive the response.
;concurrent read
(let [auth-ch (auth-service-2)
queries (range 3)
responses (for [q queries] (auth-query-2 auth-ch q))]
(doseq [r responses]
(go (println (<! r)))))
;blocking read
(let [auth-ch (auth-service-2)
queries (range 3)
responses (for [q queries] (auth-query-2 auth-ch q))]
(doseq [r responses]
(println (<!! r))))
Now the concurrent read works, but the blocking read stalls!
(defn auth-service-3 []
(let [auth-ch (chan)]
(go-loop []
(let [[req resp-ch] (<! auth-ch)]
(>! resp-ch (auth req)))
(recur))
auth-ch))
(defn auth-query-3 [auth-ch q]
(let [resp-ch (chan 1)]
(go (>! auth-ch [q resp-ch]))
resp-ch))
The only difference is that resp-ch is now (chan 1) instead of (chan)
; blocking read
(let [auth-ch (auth-service-3)
queries (range 3)
responses (for [q queries] (auth-query-3 auth-ch q))]
(doseq [r responses]
(println (<!! r))))
; concurrent read
(let [auth-ch (auth-service-3)
queries (range 3)
responses (for [q queries] (auth-query-3 auth-ch q))]
(doseq [r responses]
(go (println (<!! r)))))
It works!
API
Authentication
Database
User
(defn db-service []
(let [db-ch (chan)]
(go-loop []
(let [[req resp-ch] (<! db-ch) ]
(>! resp-ch (db req)))
(recur))
db-ch))
(defn db-query [db-chan q]
(let [resp-ch (chan 1)]
(go (>! db-chan [q resp-ch]))
resp-ch))
(defn api-service [auth-ch db-ch]
(let [input (chan)]
(go-loop []
(let [[u q output] (<! input)
auth-output (auth-query-3 auth-ch u)
db-output (db-query db-ch q)]
(if (<! auth-output)
(>! output
(<! db-output))))
(recur))
input))
(defn api-query [input u q]
(let [output (chan 1)]
(go (>! input [u q output]))
output))
;; blocking read
(let [auth-ch (auth-service-3)
db-ch (db-service)
api-ch (api-service auth-ch db-ch)
queries (range 3)
responses (for [q queries] (api-query api-ch q q))]
(doseq [r responses]
(println ["api sent" (<!! r)])))
;; concurrent read
(let [auth-ch (auth-service-3)
db-ch (db-service)
api-ch (api-service auth-ch db-ch)
queries (range 3)
responses (for [q queries] (api-query api-ch q q))]
(doseq [r responses]
(go (println ["api sent" (<! r)]))))