ExecutionContext
Demystified

Petra Bierleutgeb

@pbvie

ExecutionContext
Tapas

Life is Async

...and we don't like waiting

Async in real life

  • asynchrony  is actually much more natural than synchrony
  • we're experts in context switching
  • do we ever really wait?

Concurrency != Parallelism !=Asynchrony

Concurrency

  • imagine we have three tasks: Task A, Task B and Task C
  • start working on A
  • switch to and start working on B
  • switch back to and finish A
  • switch to and start working on C
  • switch back to and finish B
  • switch back to and finish C

Concurrency

  • opposite of working sequentially
    • i.e. start and finish A, start and finish B,...
  • concurrency is possible even if there's only one CPU

The term Concurrency refers to techniques that make programs more usable. If you can load multiple documents simultaneously in the tabs of your browser and you can still open menus and perform other actions, this is concurrency.

https://wiki.haskell.org/Parallelism_vs._Concurrency

Parallelism

  • imagine we have three tasks: Task A, Task B and Task C
  • we work on A, B and C at exactly the same time
    (by using multiple CPUs)
  • common pattern: split task in subtasks that can be processed in parallel

The term Parallelism refers to techniques to make programs faster by performing several computations at the same time. This requires hardware with multiple processing units.

https://wiki.haskell.org/Parallelism_vs._Concurrency

Asynchrony

"An asynchronous operation is an operation which continues in the background after being initiated, without forcing the caller to wait for it to finish before running other code."

https://blog.slaks.net/2014-12-23/parallelism-async-threading-explained/

ExecutionContext
in a nutshell

Motivation and Responsibilities

Why not use Threads directly?

  • creation of threads is expensive
  • number of threads should (usually) be bounded
  • starting up too many threads will lead to out-of-memory errors
    • meaning the application is not able to degrade gracefully

decouple implementation logic from execution logic

ExecutionContext Basics

  • similar to Executor and ExecutorService (Java)
  • accept tasks to be executed
  • usually - but not necessarily - backed by a thread pool

Two main responsibilities

  • execute a given Runnable
  • handle/report failures that happened during execution
trait ExecutionContext {

  def execute(runnable: Runnable): Unit

  def reportFailure(cause: Throwable): Unit

}

Providing an ExecutionContext

  • Future is not lazy
    • whenever we do something with a Future we need an EC
  • we can either
    • import an implicit EC
    • create an implicit val storing the EC
    • let the caller pass the EC to our component/function

ExecutionContext.global

a.k.a. Scala's default ExecutionContext

Quick Facts

  • good general-purpose ExecutionContext
  • backed by a work-stealing ForkJoinPool
    • since Scala 2.12 the java.util.concurrent.ForkJoinPool is used
    • work-stealing: idle threads can steal tasks from other thread's work queues => keep CPUs busy
  • support for BlockContext (more on that shortly!) 

How many threads will this start?

def doSomethingAsync()(ec: ExecutionContext): Future[Int] = ...

implicit val ec: ExecutionContext = ExecutionContext.global
Future.traverse(1 to 25)(i => doSomethingAsync())

it depends!

Configuration Options

scala.concurrent.context.minThreads = 1
scala.concurrent.context.numThreads = "x1"
scala.concurrent.context.maxThreads = "x1"
scala.concurrent.context.maxExtraThreads = 256
  • scala.concurrent.context.minThreads (default: 1)
  • scala.concurrent.context.numThreads (default: number of CPUs)
  • scala.concurrent.context.maxThreads (default: number of CPUs)
  • scala.concurrent.context.maxExtraThreads (default: 256)

Why blocking is actually bad

  • blocked threads can't be used to perform other tasks even though they are not doing actual work
  • other tasks have to wait because no free threads are available
  • risk of deadlocks

Meet BlockContext

  • lets the ExecutionContext know that a thread is about to block
  • ExecutionContext will spin up another thread to compensate for blocked thread
  • starts up to scala.concurrent.context.maxExtraThreads additional threads
    • ​before Scala 2.12 that number was unbounded

BlockContext Example

def slowOp(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = Future {
  logger.debug("slowOp")
  Thread.sleep(80)
  a + b
}
def combined(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = {
  val seqF = for (i <- 1 to 100) yield slowOp(a + i, b + i + 1)
  Future.sequence(seqF).map(_.sum)
}

0.950 ops/s

BlockContext Example

[DEBUG] Thread[20,scala-execution-context-global-20,5] - slowOp {} 
[DEBUG] Thread[18,scala-execution-context-global-18,5] - slowOp {} 
[DEBUG] Thread[15,scala-execution-context-global-15,5] - slowOp {} 
[DEBUG] Thread[17,scala-execution-context-global-17,5] - slowOp {} 
[DEBUG] Thread[22,scala-execution-context-global-22,5] - slowOp {} 
[DEBUG] Thread[16,scala-execution-context-global-16,5] - slowOp {} 
[DEBUG] Thread[19,scala-execution-context-global-19,5] - slowOp {} 
[DEBUG] Thread[21,scala-execution-context-global-21,5] - slowOp {} 
[DEBUG] Thread[18,scala-execution-context-global-18,5] - slowOp {} 
[DEBUG] Thread[15,scala-execution-context-global-15,5] - slowOp {} 
[DEBUG] Thread[20,scala-execution-context-global-20,5] - slowOp {} 
[DEBUG] Thread[17,scala-execution-context-global-17,5] - slowOp {} 
[DEBUG] Thread[16,scala-execution-context-global-16,5] - slowOp {}

BlockContext Example

def slowOpBlocking(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = Future {
  logger.debug("slowOpBlocking")
  blocking {
    Thread.sleep(80)
  }
  a + b
}
def combined(a: Int, b: Int)(implicit ec: ExecutionContext): Future[Int] = {
  val seqF = for (i <- 1 to 100) yield slowOpBlocking(a + i, b + i + 1)
  Future.sequence(seqF).map(_.sum)
}

6.861 ops/s

BlockContext Example

...
[DEBUG] Thread[3177,scala-execution-context-global-3177,5] - slowOpBlocking {} 
[DEBUG] Thread[3178,scala-execution-context-global-3178,5] - slowOpBlocking {} 
[DEBUG] Thread[3179,scala-execution-context-global-3179,5] - slowOpBlocking {} 
[DEBUG] Thread[3180,scala-execution-context-global-3180,5] - slowOpBlocking {} 
[DEBUG] Thread[3181,scala-execution-context-global-3181,5] - slowOpBlocking {} 
[DEBUG] Thread[3182,scala-execution-context-global-3182,5] - slowOpBlocking {} 
[DEBUG] Thread[3183,scala-execution-context-global-3183,5] - slowOpBlocking {} 
[DEBUG] Thread[3184,scala-execution-context-global-3184,5] - slowOpBlocking {} 
...

Choosing an ExecutionContext

Questions to be asked

  • Computation or IO?
  • Is the executed code blocking?
  • How long are the executed operations expected to run?
  • Are threads starting new (child) threads?

When not to use the global EC

  • many, long-blocking operations
    • have separate ECs for computational and IO work
  • small and fast tasks (optional)
    • context switches add overhead
    • when you're sure those tasks are fast and non-blocking they could be executed in the calling thread directly

Thank you!

Slides and further reading
coming soon
(will be posted on meetup.com)

Made with Slides.com