Sleeping well in the lion's den with Monix Catnap

Piotr Gawryś

About me

An open source contributor for fun
One of the maintainers of Monix
Kraków Scala User Group co-organizer
(let me know if you'd like to speak!)

https://github.com/Avasil

twitter.com/p_gawrys

Monix

Scala / Scala.js library for asynchronous programming
Multiple modules exposing Task, Observable, Iterant, Coeval and many concurrency utilities
Favors purely functional programming but provides for all
2.0.0 released August 31, 2016
3.0.0 released September 11, 2019

twitter.com/p_gawrys

Monix Niche

Mixed codebases
Reactive Programming
Good integration and consistency with Cats ecosystem
Performance-sensitive applications
Stability

twitter.com/p_gawrys

Monix Modules

monix-execution - low level concurrency abstractions, companion to scala.concurrent
monix-catnap - purely functional abstractions, Cats-Effect friendly
monix-eval - Task and Coeval
monix-reactive - Observable, functional take on RxObservable
monix-tail - Iterant, purely functional pull-based stream
monix-bio - bifunctor implementation

twitter.com/p_gawrys

Cats-Effect

Library which abstracts over different effect type implementations
Opens doors to the entire ecosystem regardless of your choice, i.e. http4s, finch, doobie, fs2, ...

twitter.com/p_gawrys

Problem: Limiting parallelism

object SharedResource {
  private val counter = AtomicInt(2)

  def access(i: Int): Unit = {
    if (counter.decrementAndGet() < 0)
      throw new IllegalStateException("counter less than 0")
    Thread.sleep(100)
    counter.increment()
  }
}

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(Future.traverse(List(1, 2, 3, 4, 5))(f), 60.second)

Exception in thread "main" java.lang.IllegalStateException: counter less than 0

Semaphore

Synchronization primitive
A counter is incremented when semaphore's permit is released
A counter is decremented when permit is acquired
acquire blocks until there is a permit available

twitter.com/p_gawrys

java.util.concurrent.Semaphore

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // java.util.concurrent.Semaphore
  val semaphore = new Semaphore(n)

  Future.traverse(list) { i =>
    val future = Future(semaphore.acquire()).flatMap(_ => f(i))
    future.onComplete(_ => semaphore.release())
    future
  }
}

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(traverseN(2, List.range(1, 5))(f), Duration.Inf) // works!
Await.result(traverseN(2, List.range(1, 10))(f), Duration.Inf) // hangs forever...

Semantic/Asynchronous blocking

Blocks a fiber instead of an underlying thread
Can we do it for a Future?

twitter.com/p_gawrys

Let's see how!

Acquire

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])],
  awaitReleases: List[(Long, Listener[Unit])])
  
def unsafeAcquireN(n: Long, await: Listener[Unit]): Cancelable

Check state
Are n permits available?
- NO => add Listener to awaitPermits queue
- YES => decrement permits and call Listener callback

Acquire Cancelation

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])],
  awaitReleases: List[(Long, Listener[Unit])])

def cancelAcquisition(n: Long, isAsync: Boolean): (Listener[Unit] => Unit)

Check state
find Listener in awaitPermits and remove it
release n permits

Release

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])],
  awaitReleases: List[(Long, Listener[Unit])])
  
def unsafeReleaseN(n: Long): Unit

Check state
Is anything awaiting permit?
- NO => add permit, go through awaitReleases
- YES => go through queue and give permits

Implementing with Future

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])],
  awaitReleases: List[(Long, Listener[Unit])])


def acquireN(n: Long): CancelableFuture[Unit] = {
  if (unsafeTryAcquireN(n)) {
    CancelableFuture.unit
  } else {
    val p = Promise[Unit]()
    unsafeAcquireN(n, Callback.fromPromise(p)) match {
      case Cancelable.empty => CancelableFuture.unit
      case c => CancelableFuture(p.future, c)
    }
  }
}

monix.execution.AsyncSemaphore

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // monix.execution.AsyncSemaphore
  val semaphore = AsyncSemaphore(n)

  Future.traverse(list) { i =>
    semaphore.withPermit(() => f(i))
  }
}

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(traverseN(2, List.range(1, 10))(f), Duration.Inf) // works!

object LocalExample extends App with StrictLogging {
  implicit val ec = ExecutionContext.global

  def req(requestId: String, userName: String): Future[Unit] = Future {
    logger.info(s"Received a request to create a user $userName")
    // do sth
  }.flatMap(_ => registerUser(userName))
  
  def registerUser(name: String): Future[Unit] = {
    // business logic
    logger.info(s"Registering a new user named $name")
    Future.unit
  }

  val requests = List(req("1", "Clark"), req("2", "Bruce"), req("3", "Diana"))
  Await.result(Future.sequence(requests), Duration.Inf)
}

Received a request to create a user Bruce
Registering a new user named Bruce
Received a request to create a user Diana
Registering a new user named Diana
Received a request to create a user Clark
Registering a new user named Clark

Problem: Logging Requests

def req(requestId: String, userName: String): Future[Unit] = Future {
  logger.info(s"$requestId: Received a request to create a user $userName")
  // do sth
}.flatMap(_ => registerUser(requestId, userName))

def registerUser(requestId: String, name: String): Future[Unit] = {
  // business logic
  logger.info(s"$requestId: Registering a new user named $name")
  Future.unit
}

3: Received a request to create a user Diana
3: Registering a new user named Diana
1: Received a request to create a user Clark
1: Registering a new user named Clark
2: Received a request to create a user Bruce
2: Registering a new user named Bruce

Logging Requests

logger.info("Logging something.")
MDC.put("requestId", "1")
logger.info("Logging something with MDC.")


: Logging something.
1: Logging something with MDC.

Propagating context with MDC

def req(requestId: String, userName: String): Future[Unit] = Future {
  MDC.put("requestId", requestId)
  logger.info(s"Received a request to create a user $userName")
  // more flatmaps to add async boundaries
}.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))

def registerUser(name: String): Future[Unit] = {
  // business logic
  logger.info(s"Registering a new user named $name")
  Future.unit
}

3: Received a request to create a user Diana
2: Received a request to create a user Bruce
1: Received a request to create a user Clark
1: Registering a new user named Clark
2: Registering a new user named Bruce
2: Registering a new user named Diana

MDC and concurrency

monix.execution.misc.Local

ThreadLocal with a flexible scope which can be propagated over async boundaries
Supports Future and Monix Task
Good for context propagation like MDC nad OpenTracing without manually passing parameters
Quite low level and still have rough edges
First version introduced in 2017

twitter.com/p_gawrys

Local Model

Local is shared unless told otherwise
Needs TracingScheduler for Future
TaskLocal is a pure version just for a Task
Task is a bit smarter about it and does not always require manual isolation

implicit val s = Scheduler.traced

// from https://github.com/mdedetrich/monix-mdc
MonixMDCAdapter.initialize()

def req(requestId: String, userName: String): Future[Unit] = Local.isolate {
  Future {
    MDC.put("requestId", requestId)
    logger.info(s"Received a request to create a user $userName")
    // more flatmaps to add async boundaries
  }.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))
}

1: Received a request to create a user Clark
3: Received a request to create a user Diana
2: Received a request to create a user Bruce
3: Registering a new user named Diana
1: Registering a new user named Clark
2: Registering a new user named Bruce

MDC with Monix Local

Blackbox Asynchronous Code

implicit val ec = Scheduler.traced

val local = Local(0)

def blackbox: Future[Unit] = {
  val p = Promise[Unit]()
  new Thread {
    override def run(): Unit = {
      Thread.sleep(100)
      p.success(())
    }
  }.start()
  p.future
}

val f = Local.isolate {
  for {
    _ <- Future { local := local.get + 100 }
    _ <- blackbox
    _ <- Future { local := local.get + 100 }
  // can print 100 if blackbox is not isolated!
  } yield println(local.get) 
}

Await.result(f, Duration.Inf)

Asynchronous Queue

A collection which allows to add elements to one end of the sequence and remove them from the other end
Producer is backpressured on offer if a queue is full
Consumer is backpressured on poll if a queue is empty
Useful for decoupling producer and consumer, distributing work

twitter.com/p_gawrys

Monix Queues

ConcurrentQueue[F[_], A] - a purely functional asynchronous queue for any Cats-Effect compliant effect
AsyncQueue[A] - impure asynchronous queue for scala.concurrent.Future

twitter.com/p_gawrys

Example - Fast Producer

implicit val s = Scheduler.singleThread("example-pool")

def consumer(queue: ConcurrentQueue[Task, Int]): Task[Unit] = {
  queue.poll
    .flatMap(i => Task(println(s"Consuming $i")))
    .delayExecution(1.second)
    .loopForever
}
def producer(queue: ConcurrentQueue[Task, Int], n: Int = 0): Task[Unit] = {
  for {
    _ <- queue.offer(n)
    _ <- Task(println(s"Produced $n"))
    _ <- producer(queue, n + 1)
  } yield ()
}

val t =
  for {
    queue <- ConcurrentQueue.bounded[Task, Int](2)
    _     <- consumer(queue).startAndForget
    _     <- producer(queue)
  } yield ()

t.executeAsync.runSyncUnsafe()

== Output ==
Produced 0
Produced 1
Produced 2
Consuming 0
Produced 3
Consuming 1
Produced 4
...

Streaming with Queue

implicit val s = Scheduler.singleThread("example-pool")

def consumer(queue: ConcurrentQueue[Task, Long]): Task[Unit] = {
  Observable
    .repeatEvalF(queue.poll)
    .consumeWith(Consumer.foreachTask(i => Task(println(s"Consumed $i"))))
}

def producer(queue: ConcurrentQueue[Task, Long]): Task[Unit] = {
  Observable.intervalAtFixedRate(1.second)
    .doOnNext(i => queue.offer(i))
    .consumeWith(Consumer.foreachTask(i => Task(println(s"Produced $i"))))
}

val t =
  for {
    queue <- ConcurrentQueue.bounded[Task, Long](2)
    _     <- consumer(queue).startAndForget
    _     <- producer(queue)
  } yield ()

t.executeAsync.runSyncUnsafe()

== Output ==
Produced 0
Consumed 0
Consumed 1
Produced 1
Consumed 2
Produced 2
Consumed 3
...

Other implementations

twitter.com/p_gawrys

	MONIX	FS2	ZIO
Effects	Cats-Effect and Future-native	Cats-Effect native	ZIO-native
API	basic, lacks termination	a lot of different types of queues	adds methods like map, filter, contramap, etc.
Fairness	no	yes	yes
Performance	3 - 10x faster	baseline	0,6 - 2x faster

Fairness

twitter.com/p_gawrys

implicit val s = Scheduler.global

def consumer(id: Int, queue: ConcurrentQueue[Task, Int]): Task[Unit] = {
  queue.poll
    .flatMap(i => Task(println(s"$id: Consuming $i")))
    .loopForever
}

def producer(id: Int, queue: ConcurrentQueue[Task, Int], n: Int = 0): Task[Unit] = {
  for {
    _ <- queue.offer(n)
    _ <- producer(id, queue, n + 1).delayExecution(1.second)
  } yield ()
}

val t =
  for {
    queue <- ConcurrentQueue.bounded[Task, Int](2)
    _     <- consumer(1, queue).startAndForget
    _     <- consumer(2, queue).startAndForget
    _     <- consumer(3, queue).startAndForget
    _     <- producer(1, queue)
  } yield ()

t.executeAsync.runSyncUnsafe()

Fairness

twitter.com/p_gawrys

== Monix Output ==
1: Consuming 0
1: Consuming 1
3: Consuming 2
3: Consuming 3
2: Consuming 4
1: Consuming 5
3: Consuming 6
2: Consuming 7
2: Consuming 8
3: Consuming 9
...

== FS2 Output ==
1: Consuming 0
3: Consuming 1
1: Consuming 2
2: Consuming 3
3: Consuming 4
1: Consuming 5
2: Consuming 6
3: Consuming 7
1: Consuming 8
2: Consuming 9
...

Benchmark Results

twitter.com/p_gawrys

[info] Benchmark                                   Mode  Cnt      Score    Error  Units
[info] QueueBackPressureBenchmark.fs2Queue        thrpt   30    541.506 ± 78.125  ops/s
[info] QueueBackPressureBenchmark.monixQueue      thrpt   30   2906.331 ± 64.291  ops/s
[info] QueueBackPressureBenchmark.zioQueue        thrpt   30   1011.617 ± 13.278  ops/s

[info] QueueParallelBenchmark.fs2Queue            thrpt   30   1622.631 ± 21.835  ops/s
[info] QueueParallelBenchmark.monixQueue          thrpt   30   6073.337 ± 75.890  ops/s
[info] QueueParallelBenchmark.zioQueue            thrpt   30   2585.049 ± 76.588  ops/s

[info] QueueSequentialBenchmark.fs2Queue          thrpt   30   2679.532 ± 11.624  ops/s
[info] QueueSequentialBenchmark.monixQueue        thrpt   30  12829.954 ± 61.343  ops/s
[info] QueueSequentialBenchmark.zioQueue          thrpt   30   1685.545 ± 11.178  ops/s

// Source of benchmarks
// https://github.com/zio/zio/tree/master/benchmarks

monix.catnap.ConcurrentChannel

Created for the sole purpose of modeling complex producer-consumer scenarios
Supports multicasting / broadcasting to multiple consumers and workers
Sort of like ConcurrentQueue per Consumer with higher level API which allows termination, waiting on consumers etc.
Inspired by Haskell's ConcurrentChannel

twitter.com/p_gawrys

monix.catnap.ConcurrentChannel

twitter.com/p_gawrys

final class ConcurrentChannel[F[_], E, A] {
  def push(a: A): F[Boolean]
  def pushMany(seq: Iterable[A]): F[Boolean]
  def halt(e: E): F[Unit]
  def consume: Resource[F, ConsumerF[F, E, A]]
  def consumeWithConfig(config: ConsumerF.Config): Resource[F, ConsumerF[F, E, A]]
  def awaitConsumers(n: Int): F[Boolean]
}

trait ConsumerF[F[_], E, A] {
  def pull: F[Either[E, A]]
  def pullMany(minLength: Int, maxLength: Int): F[Either[E, Seq[A]]]
}

Usage Example

def consume(consumerId: Int, consumer: ConsumerF[Task, String, Int]): Task[Unit] = {
  consumer.pull.flatMap {
    case Right(element) =>
      Task(println(s"$consumerId: is processing $element"))
        .flatMap(_ => consume(consumerId, consumer))
    case Left(msg) =>
      Task(println(s"$consumerId: is done with msg: $msg"))
  }
}

val simple: Task[Unit] =
  for {
    channel <- ConcurrentChannel.of[Task, String, Int]
    _ <- channel.consume.use(consume(1, _)).startAndForget
    _ <- channel.consume.use(consume(2, _)).startAndForget
    _ <- channel.consume.use(consume(3, _))
      .delayExecution(100.millis).startAndForget
    _ <- channel.awaitConsumers(2)
    _ <- channel.pushMany(List(1, 2))
    _ <- channel.awaitConsumers(3)
    _ <- channel.push(3)
    _ <- channel.halt("good job!")
  } yield ()

2: is processing 1
1: is processing 1
1: is processing 2
2: is processing 2
3: is processing 3
2: is processing 3
1: is processing 3
3: is done with msg: good job!
2: is done with msg: good job!
1: is done with msg: good job!

def consume(consumerId: Int, workerId: Int, consumer: ConsumerF[Task, String, Int]): Task[Unit] = {
  consumer.pull.flatMap {
    case Right(element) =>
      Task(println(s"Worker $consumerId-$workerId is processing $element"))
        .flatMap(_ => consume(consumerId, workerId, consumer))
    case Left(msg) =>
      Task(println(s"Worker $consumerId-$workerId is done with msg: $msg"))
  }.delayExecution(100.millis)
}

def parallelConsumer(consumerId: Int, workers: Int, consumer: ConsumerF[Task, String, Int]): Task[Unit] = {
  Task.wander(List.range(0, workers))(i => consume(consumerId, i, consumer)).map(_ => ())
}

val app =
  for {
    channel <- ConcurrentChannel.of[Task, String, Int]
    _ <- channel.consume.use(consumer => parallelConsumer(1, 4, consumer)).startAndForget
    _ <- channel.consume.use(consumer => consume(2, 0, consumer)).startAndForget
    _ <- channel.awaitConsumers(2)
    _ <- channel.pushMany(List(1, 2, 3, 4))
    _ <- Task.sleep(1.second)
    _ <- channel.halt("good job!")
  } yield ()

monix.catnap.ConcurrentChannel

twitter.com/p_gawrys

Worker 2-0 is processing 1
Worker 1-0 is processing 1
Worker 1-2 is processing 2
Worker 1-1 is processing 3
Worker 1-3 is processing 4
Worker 2-0 is processing 2
Worker 2-0 is processing 3
Worker 2-0 is processing 4
Worker 1-2 is done with msg: good job!
Worker 1-0 is done with msg: good job!
Worker 1-1 is done with msg: good job!
Worker 2-0 is done with msg: good job!
Worker 1-3 is done with msg: good job!

val backpressure: Task[Unit] = for {
  channel <- ConcurrentChannel.of[Task, String, Int]
  customConfig = ConsumerF.Config.default
    .copy(capacity = Some(BufferCapacity.Bounded(2))
  )
  fiber <- channel.consumeWithConfig(customConfig)
    .use(consumer => consume(1, 1, consumer)).start
  _ <- channel.awaitConsumers(1)
  _ <- Task.traverse(List.range(1, 10))(i => 
      channel.push(i) >> Task(println(s"push($i)"))
    )
  _ <- fiber.join
} yield ()

twitter.com/p_gawrys

Backpressure

push(1)
push(2)
push(3)
Worker 1-1 is processing 1
Worker 1-1 is processing 2
push(4)
Worker 1-1 is processing 3
push(5)
push(6)
Worker 1-1 is processing 4
Worker 1-1 is processing 5
push(7)
push(8)
Worker 1-1 is processing 6
Worker 1-1 is processing 7
push(9)
Worker 1-1 is processing 8
Worker 1-1 is processing 9

What about Observable?

def parallelConsumer(consumerId: Int, n: Int): Consumer[Either[String, Int], Unit] = {
  val workers: List[Consumer[Either[String, Int], Unit]] =
    List.range(0, n).map(i => Consumer.foreachTask[Either[String, Int]](consume(consumerId, i, _)))
  Consumer.loadBalance(workers: _*).map(_ => ())
}

val app: Task[Unit] =
  for {
    queue  <- ConcurrentQueue.unbounded[Task, Either[String, Int]]()
    signal <- Semaphore[Task](0)
    _      <-  Observable
          .repeatEvalF(queue.poll)
          .takeWhileInclusive(_.isRight)
          .publishSelector { sharedSource =>
            val c1 = sharedSource
                .doOnSubscribe(signal.release)
                .mapEval(elem => consume(consumerId = 1, workerId = 0, elem))
            val c2 = sharedSource
                .doOnSubscribe(signal.release)
                .consumeWith(parallelConsumer(consumerId = 2, n = 4))
            
            Observable.fromTask(Task.parZip2(c1.completedL, c2))
          }.completedL.startAndForget
    _     <- signal.acquireN(2) // awaitConsumers
    _     <- queue.offerMany(List(1, 2, 3, 4).map(i => Right[String, Int](i)))
    _     <- Task.sleep(1.second)
             // only one worker per consumer will get it
    _     <- queue.offer(Left[String, Int]("good job!")) 
    _     <- Task.sleep(1.second)
  } yield ()

...And there's more!

CircuitBreaker, Cancelables, CancelableFuture, Future utils, TestScheduler, Future-based MVar, ...
If you have any questions or more ideas, make sure to let us know at https://github.com/monix/monix or https://gitter.im/monix/monix
Contributions are very welcome!

twitter.com/p_gawrys

Thank you !

https://github.com/typelevel/cats-effect

https://github.com/functional-streams-for-scala/fs2

https://github.com/monix/monix

https://github.com/zio/zio

Some of the projects worth checking out:

twitter.com/p_gawrys

Monix Catnap ( Hamburg & Berlin User Group )

By Piotr Gawryś

Monix Catnap ( Hamburg & Berlin User Group )

1,199

Piotr Gawryś

p_gawrys

Sleeping well in the lion's den with Monix Catnap

About me

Monix

Monix Niche

Monix Modules

Cats-Effect

Problem: Limiting parallelism

Semaphore

java.util.concurrent.Semaphore

Semantic/Asynchronous blocking

Let's see how!

Acquire

Acquire Cancelation

Release

Implementing with Future

monix.execution.AsyncSemaphore

Problem: Logging Requests

Logging Requests

Propagating context with MDC

MDC and concurrency

monix.execution.misc.Local

Local Model

MDC with Monix Local

Blackbox Asynchronous Code

Asynchronous Queue

Monix Queues

Example - Fast Producer

Streaming with Queue

Other implementations

Fairness

Fairness

Benchmark Results

monix.catnap.ConcurrentChannel

monix.catnap.ConcurrentChannel

Usage Example

monix.catnap.ConcurrentChannel

Backpressure

What about Observable?

...And there's more!

Thank you !

Monix Catnap ( Hamburg & Berlin User Group )

More from Piotr Gawryś