Dispelling Magic behind Concurrency in FP

Piotr Gawryś

Who am I?

  • One of the maintainers of Monix
  • Contributor to Typelevel ecosystem
  • Kraków Scala User Group co-organizer
    ​(let me know if you'd like to speak!)
  • Super excited to be here!

https://github.com/Avasil

twitter.com/p_gawrys

What's the talk about

  • Concurrency fundamentals on JVM
  • Focused around purely functional programming (Cats-Effect, FS2, Monix, ZIO)
  • No overview/marketing of any of them

twitter.com/p_gawrys

Concurrency

  • Processing multiple tasks interleaved
  • Doesn't require more than one thread to work

twitter.com/p_gawrys

Concurrency

twitter.com/p_gawrys

Parallelism

  • Executing multiple tasks at the same time to finish them faster

twitter.com/p_gawrys

Parallelism

twitter.com/p_gawrys

Threads

  • Basic unit of CPU utilization
  • Typically part of Operating System
  • Threads can share memory
  • Processors can usually run up to 1 thread per CPU's core at the same time

twitter.com/p_gawrys

Threads on JVM

  • Map 1:1 to native OS threads
  • Each thread takes around 1 MB of memory on 64-bit systems by default

twitter.com/p_gawrys

JVM Memory Model

Heap

Thread Stack

Objects

call stack

local variables

Thread Stack

call stack

local variables

Thread Stack

call stack

local variables

twitter.com/p_gawrys

CPU

CPU's Core 1

Main Memory

registers

CPU

Cache

JVM

CPU's Core 3

registers

CPU

Cache

CPU's Core 2

registers

CPU

Cache

twitter.com/p_gawrys

Context Switch

  • Happens when new thread starts working on CPU's core
  • OS needs to store the state of old task and restore the state of the new one
  • Cache locality is lost
  • Synchronous => no context switches => best performance

twitter.com/p_gawrys

Context Switch

twitter.com/p_gawrys

Thread Pools

  • Take care of managing threads for us
  • Can reuse threads
  • Several types, e.g. Cached, SingleThreaded, WorkStealing etc.
  • Think ExecutionContext (Future), ContextShift (cats.effect.IO), Scheduler (Monix Task)

twitter.com/p_gawrys

Thread Pools

import java.util.concurrent.Executors
import monix.execution.Scheduler
import scala.concurrent.ExecutionContext

val ec = ExecutionContext.fromExecutor(Executors.newCachedThreadPool())
val scheduler = Scheduler(ec)

twitter.com/p_gawrys

Extra capabilities

import monix.execution.schedulers.TestScheduler
import scala.concurrent.duration._
import cats.implicits._

val sc = TestScheduler()

val failedTask: Task[Int] = Task.sleep(2.days) >> 
  Task.raiseError[Int](new Exception("boom"))

val f: CancelableFuture[Int] = failedTask.runToFuture(sc)
  
println(f.value) // None
sc.tick(10.hours)
println(f.value) // None
sc.tick(1000.days)
println(f.value) // Some(Failure(java.lang.Exception: boom))

twitter.com/p_gawrys

Asynchronous Boundary

  • Task returns to Thread Pool to be scheduled again
  • Many scenarios can occur, e.g.:
    • Cancelation
    • Another Task starts executing
    • The same Task executes on the same thread it was before
    • New Thread is created
    • and more!

twitter.com/p_gawrys

Asynchronous Boundary

val s: Scheduler = Scheduler.global

def repeat: Task[Unit] =
  for {
    _ <- Task.shift
    _ <- Task(println(s"Shifted to: ${Thread.currentThread().getName}"))
    _ <- repeat
  } yield ()

repeat.runToFuture(s)

twitter.com/p_gawrys

Asynchronous Boundary

// Output:
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-14
// Shifted to: scala-execution-context-global-14
// Shifted to: scala-execution-context-global-14
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-13
// Shifted to: scala-execution-context-global-13
// Shifted to: scala-execution-context-global-13
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// ...

twitter.com/p_gawrys

val s1: Scheduler = Scheduler(
  ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor()),
  ExecutionModel.SynchronousExecution)

def repeat(id: Int): Task[Unit] =
    Task(print(id)).flatMap(_ => repeat(id))

val prog = (repeat(1), repeat(2)).parTupled

prog.runToFuture(s1)

// Output:
// 1111111111111111111111111111111111111111111111111111111111...

Task Scheduling

twitter.com/p_gawrys

val s1: Scheduler = Scheduler(
  ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor()),
  ExecutionModel.SynchronousExecution)

def repeat(id: Int): Task[Unit] =
  Task(print(id)).flatMap(_ => Task.shift >> repeat(id))

val prog = (repeat(1), repeat(2)).parTupled

prog.runToFuture(s1)

// Output:
// 121212121212121212121212121212121212121212121 ...

Task Scheduling

twitter.com/p_gawrys

val s1: Scheduler = Scheduler( // 4 = number of cores on my laptop
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4)),
  ExecutionModel.SynchronousExecution)
val s2: Scheduler = Scheduler(
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1)),
  ExecutionModel.SynchronousExecution)

def repeat(id: Int): Task[Unit] =
  Task(print(id)).flatMap(_ => repeat(id))

val program = (repeat(1), repeat(2), repeat(3), repeat(4), repeat(5),
  repeat(6).executeOn(s2)).parTupled

program.runToFuture(s1)

// Output:
// 143622331613424316134424316134243161342431613424316134243 ...
// no '5' !

Task Scheduling

twitter.com/p_gawrys

Light Async Boundary

  • Continues on the same thread by means of a trampoline
  • Checks with thread pool for cancelation status (in Monix & Cats-Effect IO)
  • Can help with stack safety
  • Low level!

twitter.com/p_gawrys

Light Async Boundary

implicit val s = Scheduler.global
  .withExecutionModel(ExecutionModel.SynchronousExecution)

def task(i: Int): Task[Unit] =
  Task(println(s"$i: ${Thread.currentThread().getName}")) >> 
    Task.shift(TrampolineExecutionContext.immediate) >> task(i + 1)

val t =
  for {
    fiber <- task(0)
      .doOnCancel(Task(println("cancel")))
      .start
    _ <- fiber.cancel
  } yield ()

t.runToFuture 

// Output: 
// 0: scala-execution-context-global-11
// 1: scala-execution-context-global-11
// 2: scala-execution-context-global-11
// cancel

twitter.com/p_gawrys

Light Async Boundary

val immediate: TrampolineExecutionContext =
  TrampolineExecutionContext(new ExecutionContext {
    def execute(r: Runnable): Unit = r.run()
    def reportFailure(e: Throwable): Unit = throw e
  })

twitter.com/p_gawrys

Blocking Threads

  • Operation that takes entire thread without doing any useful work
  • Don't do it if you have a choice
  • Thread is being wasted, might prevent other tasks from being scheduled if the thread pool is limited
  • Use dedicated thread pool for blocking operations 
  • Use timeouts

twitter.com/p_gawrys

Dealing with blocking ops

implicit val globalScheduler = Scheduler.global
val blockingScheduler = Scheduler.io()

val blockingOp = Task {
  Thread.sleep(1000)
  println(s"${Thread.currentThread().getName}: done blocking")
}

val cpuBound = Task {
  // keep cpu busy
  println(s"${Thread.currentThread().getName}: done calculating")
}

val t =
  for {
    _ <- cpuBound
    _ <- blockingOp.executeOn(blockingScheduler)
    _ <- cpuBound
  } yield ()

t.runSyncUnsafe()

// scala-execution-context-global-11: done calculating
// monix-io-12: done blocking
// scala-execution-context-global-11: done calculating

Semantic / Asynchronous Blocking

  • Task waits for certain signal before it can complete
  • Doesn't really block any threads, other tasks can execute in the meantime

twitter.com/p_gawrys

Semantic / Asynchronous Blocking

val ec = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor())
implicit val scheduler = Scheduler(ec)

val otherTask = Task.sleep(50.millis) >> Task(println("Running concurrently"))

val t =
  for {
    sem <- Semaphore[Task](0L)
    // start means it will run asynchronously "in the background"
    // and the next step in for comprehension will begin
    _ <- (Task.sleep(100.millis) >>
      Task(println("Releasing semaphore")) >> sem.release).start
    _ <- Task(println("Waiting for permit")) >> sem.acquire
    _ <- Task(println("Done!"))
  } yield ()

(t, otherTask).parTupled.runSyncUnsafe()

// Waiting for permit
// Running concurrently
// Releasing semaphore
// Done!

Fairness

  • Likelihood that different tasks are able to advance 
  • Without fairness you could get high disparity of latencies between processing different requests
  • Monix and ZIO introduce async boundaries from time to time (configurable)

twitter.com/p_gawrys

Fairness

Cancelation / Interruption

  • Ability to stop running Tasks
  • Not all tasks are cancelable
  • Cancelation might not happen immediately or even at all

twitter.com/p_gawrys

What makes Task cancelable?

  • Asynchronous boundaries
  • flatMaps (and more methods depending on implementation)
  • Handling java.lang.InterruptedException (opt-in)

twitter.com/p_gawrys

Canceling Task

twitter.com/p_gawrys

implicit val s = Scheduler.global

def foo(i: Int): Task[Unit] =
  for {
    _ <- Task(println(s"start $i"))
    _ <- if (i % 2 == 0) Task.raiseError(DummyException("error"))
         else Task.sleep(10.millis)
    _ <- Task(println(s"end $i"))
  } yield ()

val tasks: List[Task[Unit]] = List(foo(1), foo(2), foo(3), foo(4))
val result: Task[List[Unit]] = Task.gather(tasks)

println(result.attempt.runSyncUnsafe())

// start 4
// start 1
// start 2
// start 3
// Left(monix.execution.exceptions.DummyException: error)

Canceling Task

twitter.com/p_gawrys

// (...) code from the previous slide

val tasks: List[Task[Unit]] = List(foo(1), foo(2), foo(3), foo(4))
val result: Task[List[Either[Throwable, Unit]]] = Task.wander(tasks)(_.attempt)

println(result.runSyncUnsafe())

// start 2
// start 1
// start 3
// start 4
// end 1
// end 3
// List(
//  Right(()), Left(monix.execution.exceptions.DummyException: error), 
//  Right(()), Left(monix.execution.exceptions.DummyException: error)
// )

Canceling Task

twitter.com/p_gawrys

implicit val s = Scheduler.global

val t1 =
  for {
    _ <- Task(println("t1: start"))
    _ <- Task.sleep(100.millis)
    _ <- Task(println("t1: middle"))
    _ <- Task.sleep(100.millis)
    _ <- Task(println("t1: end"))
  } yield ()

t1.timeout(150.millis).runSyncUnsafe()

// t1: start
// t1: middle
// Exception in thread "main" java.util.concurrent.TimeoutException: 
// Task timed-out after 150 milliseconds of inactivity

Canceling Task

twitter.com/p_gawrys

val ec = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor())
implicit val scheduler = Scheduler(ec)

def middle: Task[Unit] = Task {
  while(true){}
}

val t1 =
  for {
    _ <- Task(println("t1: start"))
    _ <- Task.sleep(100.millis)
    _ <- middle >> Task(println("t1: middle"))
    _ <- Task.sleep(100.millis)
    _ <- Task(println("t1: end"))
  } yield ()

t1.timeout(150.millis).runSyncUnsafe()

// t1: start
// ... never stops running

Green Threads and Fibers

Green Thread

Thread is scheduled by VM instead of OS. Cheap to start and we can map M Green Threads to N OS Threads.

Fiber

"Lightweight thread". Fibers voluntarily yield control to the scheduler.

twitter.com/p_gawrys

Is Task like a Green Thread/Fiber?

  • We can map thousands of Tasks to very few OS threads
  • Async boundaries give a chance to other tasks from the thread pool to execute, very much like cooperative multitasking
  • "Blocking" a Task itself is OK since it doesn't block underlying Threads

twitter.com/p_gawrys

Fiber in Cats-Effect

twitter.com/p_gawrys

trait Fiber[F[_], A] {

  def cancel: F[Unit]

  def join: F[A]
}


sealed abstract class Task[+A] {
  // (...)

  final def start: Task[Fiber[A]]

  // (...)
}

Thank you !

https://gitter.im/typelevel/cats-effect

https://gitter.im/functional-streams-for-scala/fs2

https://gitter.im/monix/monix

https://gitter.im/ZIO/core

I will appreciate any feedback. :)

​If you're looking for help/discussion:

twitter.com/p_gawrys

Dispelling Magic behind Concurrency in FP (flatMap(Oslo) 2019)

By Piotr Gawryś

Made with Slides.com

Dispelling Magic behind Concurrency in FP (flatMap(Oslo) 2019)

  • 1,644

More from Piotr Gawryś