Dispelling Magic behind Concurrency in FP

Piotr Gawryś

Who am I?

One of the maintainers of Monix
Contributor to Typelevel ecosystem
Kraków Scala User Group co-organizer
(let me know if you'd like to speak!)
Super excited to be here!

https://github.com/Avasil

twitter.com/p_gawrys

What's the talk about

Concurrency fundamentals on JVM
Focused around purely functional programming (Cats-Effect, FS2, Monix, ZIO)
No overview/marketing of any of them

twitter.com/p_gawrys

Concurrency

Processing multiple tasks interleaved
Doesn't require more than one thread to work

twitter.com/p_gawrys

Concurrency

twitter.com/p_gawrys

Parallelism

Executing multiple tasks at the same time to finish them faster

twitter.com/p_gawrys

Parallelism

twitter.com/p_gawrys

Threads

Basic unit of CPU utilization
Typically part of Operating System
Threads can share memory
Processors can usually run up to 1 thread per CPU's core at the same time

twitter.com/p_gawrys

Threads on JVM

Map 1:1 to native OS threads
Each thread takes around 1 MB of memory on 64-bit systems by default

twitter.com/p_gawrys

JVM Memory Model

Heap

Thread Stack

Objects

call stack

local variables

Thread Stack

call stack

local variables

Thread Stack

call stack

local variables

twitter.com/p_gawrys

CPU

CPU's Core 1

Main Memory

registers

CPU

Cache

JVM

CPU's Core 3

registers

CPU

Cache

CPU's Core 2

registers

CPU

Cache

twitter.com/p_gawrys

Context Switch

Happens when new thread starts working on CPU's core
OS needs to store the state of old task and restore the state of the new one
Cache locality is lost
Synchronous => no context switches => best performance

twitter.com/p_gawrys

Context Switch

twitter.com/p_gawrys

Thread Pools

Take care of managing threads for us
Can reuse threads
Several types, e.g. Cached, SingleThreaded, WorkStealing etc.
Think ExecutionContext (Future), ContextShift (cats.effect.IO), Scheduler (Monix Task)

twitter.com/p_gawrys

Thread Pools

import java.util.concurrent.Executors
import monix.execution.Scheduler
import scala.concurrent.ExecutionContext

val ec = ExecutionContext.fromExecutor(Executors.newCachedThreadPool())
val scheduler = Scheduler(ec)

twitter.com/p_gawrys

Extra capabilities

import monix.execution.schedulers.TestScheduler
import scala.concurrent.duration._
import cats.implicits._

val sc = TestScheduler()

val failedTask: Task[Int] = Task.sleep(2.days) >> 
  Task.raiseError[Int](new Exception("boom"))

val f: CancelableFuture[Int] = failedTask.runToFuture(sc)
  
println(f.value) // None
sc.tick(10.hours)
println(f.value) // None
sc.tick(1000.days)
println(f.value) // Some(Failure(java.lang.Exception: boom))

twitter.com/p_gawrys

Asynchronous Boundary

Task returns to Thread Pool to be scheduled again
Many scenarios can occur, e.g.:
- Cancelation
- Another Task starts executing
- The same Task executes on the same thread it was before
- New Thread is created
- and more!

twitter.com/p_gawrys

Asynchronous Boundary

val s: Scheduler = Scheduler.global

def repeat: Task[Unit] =
  for {
    _ <- Task.shift
    _ <- Task(println(s"Shifted to: ${Thread.currentThread().getName}"))
    _ <- repeat
  } yield ()

repeat.runToFuture(s)

twitter.com/p_gawrys

Asynchronous Boundary

// Output:
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-14
// Shifted to: scala-execution-context-global-14
// Shifted to: scala-execution-context-global-14
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-13
// Shifted to: scala-execution-context-global-13
// Shifted to: scala-execution-context-global-13
// Shifted to: scala-execution-context-global-12
// Shifted to: scala-execution-context-global-12
// ...

twitter.com/p_gawrys

val s1: Scheduler = Scheduler(
  ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor()),
  ExecutionModel.SynchronousExecution)

def repeat(id: Int): Task[Unit] =
    Task(print(id)).flatMap(_ => repeat(id))

val prog = (repeat(1), repeat(2)).parTupled

prog.runToFuture(s1)

// Output:
// 1111111111111111111111111111111111111111111111111111111111...

Task Scheduling

twitter.com/p_gawrys

val s1: Scheduler = Scheduler(
  ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor()),
  ExecutionModel.SynchronousExecution)

def repeat(id: Int): Task[Unit] =
  Task(print(id)).flatMap(_ => Task.shift >> repeat(id))

val prog = (repeat(1), repeat(2)).parTupled

prog.runToFuture(s1)

// Output:
// 121212121212121212121212121212121212121212121 ...

Task Scheduling

twitter.com/p_gawrys

val s1: Scheduler = Scheduler( // 4 = number of cores on my laptop
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4)),
  ExecutionModel.SynchronousExecution)
val s2: Scheduler = Scheduler(
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(1)),
  ExecutionModel.SynchronousExecution)

def repeat(id: Int): Task[Unit] =
  Task(print(id)).flatMap(_ => repeat(id))

val program = (repeat(1), repeat(2), repeat(3), repeat(4), repeat(5),
  repeat(6).executeOn(s2)).parTupled

program.runToFuture(s1)

// Output:
// 143622331613424316134424316134243161342431613424316134243 ...
// no '5' !

Task Scheduling

twitter.com/p_gawrys

Light Async Boundary

Continues on the same thread by means of a trampoline
Checks with thread pool for cancelation status (in Monix & Cats-Effect IO)
Can help with stack safety
Low level!

twitter.com/p_gawrys

Light Async Boundary

implicit val s = Scheduler.global
  .withExecutionModel(ExecutionModel.SynchronousExecution)

def task(i: Int): Task[Unit] =
  Task(println(s"$i: ${Thread.currentThread().getName}")) >> 
    Task.shift(TrampolineExecutionContext.immediate) >> task(i + 1)

val t =
  for {
    fiber <- task(0)
      .doOnCancel(Task(println("cancel")))
      .start
    _ <- fiber.cancel
  } yield ()

t.runToFuture 

// Output: 
// 0: scala-execution-context-global-11
// 1: scala-execution-context-global-11
// 2: scala-execution-context-global-11
// cancel

twitter.com/p_gawrys

Light Async Boundary

val immediate: TrampolineExecutionContext =
  TrampolineExecutionContext(new ExecutionContext {
    def execute(r: Runnable): Unit = r.run()
    def reportFailure(e: Throwable): Unit = throw e
  })

twitter.com/p_gawrys

Blocking Threads

Operation that takes entire thread without doing any useful work
Don't do it if you have a choice
Thread is being wasted, might prevent other tasks from being scheduled if the thread pool is limited
Use dedicated thread pool for blocking operations
Use timeouts

twitter.com/p_gawrys

Dealing with blocking ops

implicit val globalScheduler = Scheduler.global
val blockingScheduler = Scheduler.io()

val blockingOp = Task {
  Thread.sleep(1000)
  println(s"${Thread.currentThread().getName}: done blocking")
}

val cpuBound = Task {
  // keep cpu busy
  println(s"${Thread.currentThread().getName}: done calculating")
}

val t =
  for {
    _ <- cpuBound
    _ <- blockingOp.executeOn(blockingScheduler)
    _ <- cpuBound
  } yield ()

t.runSyncUnsafe()

// scala-execution-context-global-11: done calculating
// monix-io-12: done blocking
// scala-execution-context-global-11: done calculating

Semantic / Asynchronous Blocking

Task waits for certain signal before it can complete
Doesn't really block any threads, other tasks can execute in the meantime

twitter.com/p_gawrys

Semantic / Asynchronous Blocking

val ec = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor())
implicit val scheduler = Scheduler(ec)

val otherTask = Task.sleep(50.millis) >> Task(println("Running concurrently"))

val t =
  for {
    sem <- Semaphore[Task](0L)
    // start means it will run asynchronously "in the background"
    // and the next step in for comprehension will begin
    _ <- (Task.sleep(100.millis) >>
      Task(println("Releasing semaphore")) >> sem.release).start
    _ <- Task(println("Waiting for permit")) >> sem.acquire
    _ <- Task(println("Done!"))
  } yield ()

(t, otherTask).parTupled.runSyncUnsafe()

// Waiting for permit
// Running concurrently
// Releasing semaphore
// Done!

Fairness

Likelihood that different tasks are able to advance
Without fairness you could get high disparity of latencies between processing different requests
Monix and ZIO introduce async boundaries from time to time (configurable)

twitter.com/p_gawrys

Fairness

Cancelation / Interruption

Ability to stop running Tasks
Not all tasks are cancelable
Cancelation might not happen immediately or even at all

twitter.com/p_gawrys

What makes Task cancelable?

Asynchronous boundaries
flatMaps (and more methods depending on implementation)
Handling java.lang.InterruptedException (opt-in)

twitter.com/p_gawrys

Canceling Task

twitter.com/p_gawrys

implicit val s = Scheduler.global

def foo(i: Int): Task[Unit] =
  for {
    _ <- Task(println(s"start $i"))
    _ <- if (i % 2 == 0) Task.raiseError(DummyException("error"))
         else Task.sleep(10.millis)
    _ <- Task(println(s"end $i"))
  } yield ()

val tasks: List[Task[Unit]] = List(foo(1), foo(2), foo(3), foo(4))
val result: Task[List[Unit]] = Task.gather(tasks)

println(result.attempt.runSyncUnsafe())

// start 4
// start 1
// start 2
// start 3
// Left(monix.execution.exceptions.DummyException: error)

Canceling Task

twitter.com/p_gawrys

// (...) code from the previous slide

val tasks: List[Task[Unit]] = List(foo(1), foo(2), foo(3), foo(4))
val result: Task[List[Either[Throwable, Unit]]] = Task.wander(tasks)(_.attempt)

println(result.runSyncUnsafe())

// start 2
// start 1
// start 3
// start 4
// end 1
// end 3
// List(
//  Right(()), Left(monix.execution.exceptions.DummyException: error), 
//  Right(()), Left(monix.execution.exceptions.DummyException: error)
// )

Canceling Task

twitter.com/p_gawrys

implicit val s = Scheduler.global

val t1 =
  for {
    _ <- Task(println("t1: start"))
    _ <- Task.sleep(100.millis)
    _ <- Task(println("t1: middle"))
    _ <- Task.sleep(100.millis)
    _ <- Task(println("t1: end"))
  } yield ()

t1.timeout(150.millis).runSyncUnsafe()

// t1: start
// t1: middle
// Exception in thread "main" java.util.concurrent.TimeoutException: 
// Task timed-out after 150 milliseconds of inactivity

Canceling Task

twitter.com/p_gawrys

val ec = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor())
implicit val scheduler = Scheduler(ec)

def middle: Task[Unit] = Task {
  while(true){}
}

val t1 =
  for {
    _ <- Task(println("t1: start"))
    _ <- Task.sleep(100.millis)
    _ <- middle >> Task(println("t1: middle"))
    _ <- Task.sleep(100.millis)
    _ <- Task(println("t1: end"))
  } yield ()

t1.timeout(150.millis).runSyncUnsafe()

// t1: start
// ... never stops running

Green Threads and Fibers

Green Thread

Thread is scheduled by VM instead of OS. Cheap to start and we can map M Green Threads to N OS Threads.

Fiber

"Lightweight thread". Fibers voluntarily yield control to the scheduler.

twitter.com/p_gawrys

Is Task like a Green Thread/Fiber?

We can map thousands of Tasks to very few OS threads
Async boundaries give a chance to other tasks from the thread pool to execute, very much like cooperative multitasking
"Blocking" a Task itself is OK since it doesn't block underlying Threads

twitter.com/p_gawrys

Fiber in Cats-Effect

twitter.com/p_gawrys

trait Fiber[F[_], A] {

  def cancel: F[Unit]

  def join: F[A]
}


sealed abstract class Task[+A] {
  // (...)

  final def start: Task[Fiber[A]]

  // (...)
}

Thank you !

https://gitter.im/typelevel/cats-effect

https://gitter.im/functional-streams-for-scala/fs2

https://gitter.im/monix/monix

https://gitter.im/ZIO/core

I will appreciate any feedback. :)

If you're looking for help/discussion:

twitter.com/p_gawrys