ExecutionContext
Demystified
Petra Bierleutgeb
@pbvie
ExecutionContext
Tapas
Life is Async
...and we don't like waiting
Async in real life
- asynchrony is actually much more natural than synchrony
- we're experts in context switching
- do we ever really wait?
Concurrency != Parallelism !=Asynchrony
Concurrency
- imagine we have three tasks: Task A, Task B and Task C
- start working on A
- switch to and start working on B
- switch back to and finish A
- switch to and start working on C
- switch back to and finish B
- switch back to and finish C
Concurrency
- opposite of working sequentially
- i.e. start and finish A, start and finish B,...
- concurrency is possible even if there's only one CPU
The term Concurrency refers to techniques that make programs more usable. If you can load multiple documents simultaneously in the tabs of your browser and you can still open menus and perform other actions, this is concurrency.
https://wiki.haskell.org/Parallelism_vs._Concurrency
Parallelism
- imagine we have three tasks: Task A, Task B and Task C
- we work on A, B and C at exactly the same time
(by using multiple CPUs) - common pattern: split task in subtasks that can be processed in parallel
The term Parallelism refers to techniques to make programs faster by performing several computations at the same time. This requires hardware with multiple processing units.
https://wiki.haskell.org/Parallelism_vs._Concurrency
Asynchrony
"An asynchronous operation is an operation which continues in the background after being initiated, without forcing the caller to wait for it to finish before running other code."
https://blog.slaks.net/2014-12-23/parallelism-async-threading-explained/
ExecutionContext
in a nutshell
Motivation and Responsibilities
Why not use Threads directly?
- creation of threads is expensive
- number of threads should (usually) be bounded
- starting up too many threads will lead to out-of-memory errors
- meaning the application is not able to degrade gracefully
decouple implementation logic from execution logic
ExecutionContext Basics
- similar to Executor and ExecutorService (Java)
- accept tasks to be executed
- usually - but not necessarily - backed by a thread pool
Two main responsibilities
- execute a given Runnable
- handle/report failures that happened during execution
trait ExecutionContext {
def execute(runnable: Runnable): Unit
def reportFailure(cause: Throwable): Unit
}
Providing an ExecutionContext
- Future is not lazy
- whenever we do something with a Future we need an EC
- we can either
- import an implicit EC
- create an implicit val storing the EC
- let the caller pass the EC to our component/function
ExecutionContext.global
a.k.a. Scala's default ExecutionContext
Quick Facts
- good general-purpose ExecutionContext
- backed by a work-stealing ForkJoinPool
- since Scala 2.12 the java.util.concurrent.ForkJoinPool is used
- work-stealing: idle threads can steal tasks from other thread's work queues => keep CPUs busy
- support for BlockContext (more on that shortly!)
How many threads will this start?
def doSomethingAsync()(ec: ExecutionContext): Future[Int] = ...
implicit val ec: ExecutionContext = ExecutionContext.global
Future.traverse(1 to 25)(i => doSomethingAsync())
it depends!
Configuration Options
scala.concurrent.context.minThreads = 1
scala.concurrent.context.numThreads = "x1"
scala.concurrent.context.maxThreads = "x1"
scala.concurrent.context.maxExtraThreads = 256
- scala.concurrent.context.minThreads (default: 1)
- scala.concurrent.context.numThreads (default: number of CPUs)
- scala.concurrent.context.maxThreads (default: number of CPUs)
- scala.concurrent.context.maxExtraThreads (default: 256)
Why blocking is actually bad
- blocked threads can't be used to perform other tasks even though they are not doing actual work
- other tasks have to wait because no free threads are available
- risk of deadlocks
Meet BlockContext
- lets the ExecutionContext know that a thread is about to block
- ExecutionContext will spin up another thread to compensate for blocked thread
- starts up to scala.concurrent.context.maxExtraThreads additional threads
- before Scala 2.12 that number was unbounded
BlockContext Example
def slowOp(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = Future {
logger.debug("slowOp")
Thread.sleep(80)
a + b
}
def combined(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = {
val seqF = for (i <- 1 to 100) yield slowOp(a + i, b + i + 1)
Future.sequence(seqF).map(_.sum)
}
0.950 ops/s
BlockContext Example
[DEBUG] Thread[20,scala-execution-context-global-20,5] - slowOp {}
[DEBUG] Thread[18,scala-execution-context-global-18,5] - slowOp {}
[DEBUG] Thread[15,scala-execution-context-global-15,5] - slowOp {}
[DEBUG] Thread[17,scala-execution-context-global-17,5] - slowOp {}
[DEBUG] Thread[22,scala-execution-context-global-22,5] - slowOp {}
[DEBUG] Thread[16,scala-execution-context-global-16,5] - slowOp {}
[DEBUG] Thread[19,scala-execution-context-global-19,5] - slowOp {}
[DEBUG] Thread[21,scala-execution-context-global-21,5] - slowOp {}
[DEBUG] Thread[18,scala-execution-context-global-18,5] - slowOp {}
[DEBUG] Thread[15,scala-execution-context-global-15,5] - slowOp {}
[DEBUG] Thread[20,scala-execution-context-global-20,5] - slowOp {}
[DEBUG] Thread[17,scala-execution-context-global-17,5] - slowOp {}
[DEBUG] Thread[16,scala-execution-context-global-16,5] - slowOp {}
BlockContext Example
def slowOpBlocking(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = Future {
logger.debug("slowOpBlocking")
blocking {
Thread.sleep(80)
}
a + b
}
def combined(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = {
val seqF = for (i <- 1 to 100) yield slowOpBlocking(a + i, b + i + 1)
Future.sequence(seqF).map(_.sum)
}
6.861 ops/s
BlockContext Example
...
[DEBUG] Thread[3177,scala-execution-context-global-3177,5] - slowOpBlocking {}
[DEBUG] Thread[3178,scala-execution-context-global-3178,5] - slowOpBlocking {}
[DEBUG] Thread[3179,scala-execution-context-global-3179,5] - slowOpBlocking {}
[DEBUG] Thread[3180,scala-execution-context-global-3180,5] - slowOpBlocking {}
[DEBUG] Thread[3181,scala-execution-context-global-3181,5] - slowOpBlocking {}
[DEBUG] Thread[3182,scala-execution-context-global-3182,5] - slowOpBlocking {}
[DEBUG] Thread[3183,scala-execution-context-global-3183,5] - slowOpBlocking {}
[DEBUG] Thread[3184,scala-execution-context-global-3184,5] - slowOpBlocking {}
...
Choosing an ExecutionContext
Questions to be asked
- Computation or IO?
- Is the executed code blocking?
- How long are the executed operations expected to run?
- Are threads starting new (child) threads?
When not to use the global EC
- many, long-blocking operations
- have separate ECs for computational and IO work
- small and fast tasks (optional)
- context switches add overhead
- when you're sure those tasks are fast and non-blocking they could be executed in the calling thread directly
Thank you!
Slides and further reading
coming soon
(will be posted on meetup.com)
ExecutionContext Tapas
By Petra Bierleutgeb
ExecutionContext Tapas
- 1,341