Sleeping well in the lion's den with Monix Catnap

Piotr Gawryś

Who am I?

  • One of the maintainers of Monix
  • Contributor to Typelevel ecosystem
  • Kraków Scala User Group co-organizer
    (let me know if you'd like to speak!)

https://github.com/Avasil

twitter.com/p_gawrys

Monix

  • Scala / Scala.js library for asynchronous programming
  • Multiple modules exposing Task, Observable, Iterant, Coeval and many concurrency utilities
  • Favors purely functional programming but doesn't discriminate
  • 3.0.0 released September 11, 2019

twitter.com/p_gawrys

Monix Modules

  • monix-execution - low level concurrency abstractions, companion to scala.concurrent
  • monix-catnap - purely functional abstractions, Cats-Effect friendly
  • monix-eval - Task and Coeval
  • monix-reactive - Observable, functional take on RxObservable
  • monix-tail - Iterant, purely functional pull-based stream

twitter.com/p_gawrys

Cats-Effect

  • Library which abstracts over different effect type implementations (currently cats.effect.IO, Monix Task, ZIO)
  • Opens doors to the entire ecosystem regardless of your choice, i.e. http4s, finch, doobie, fs2, ...

twitter.com/p_gawrys

Task vs Future

  • referentially transparent
  • lazily evaluated
  • cancelable
  • resource safe
  • more convenient thread pool management
  • much easier to write higher-level combinators
  • luxury of not being a part of the standard library :)

twitter.com/p_gawrys

What is a pure function?

  • Referential transparency:
    • Replacing an expression by its bound value doesn't alter the behavior of your program

twitter.com/p_gawrys

val a: Int = 10
val b: Int = a + 1 // 11

val c: Int = a + b // 21
val d: Int = 10 + (10 + 1) // 21

What is a pure function?

twitter.com/p_gawrys

val r = scala.util.Random

val a: Int = r.nextInt // generates X
val b: Int = a + 1     // X + 1

val c: Int = a + b // X + X + 1
val d: Int = r.nextInt + (r.nextInt + 1) // X + Y + 1

What is a pure function?

twitter.com/p_gawrys

// generates X
val a: Future[Int] = Future(r.nextInt) 
// X + 1
val b: Future[Int] = a.map(_ + 1)
// X + X + 1
val c: Future[Int] = for { a0 <- a; b0 <- b } yield a0 + b0
// X + Y + 1
val d: Future[Int] = for { 
    a1 <- Future(r.nextInt)
    a2 <- Future(r.nextInt + 1) 
  } yield a1 + a2
val r = scala.util.Random

val a: Int = r.nextInt // generates X
val b: Int = a + 1     // X + 1

val c: Int = a + b // X + X + 1
val d: Int = r.nextInt + (r.nextInt + 1) // X + Y + 1

What is a pure function?

twitter.com/p_gawrys

// generates X when executed
val a: Task[Int] = Task(r.nextInt) 
// generates X + 1 when executed
val b: Task[Int] = a.map(_ + 1)
// X + Y + 1
val c: Task[Int] = (a, b).mapN(_ + _)
// X + Y + 1
val d: Task[Int] = (Task(r.nextInt), Task(r.nextInt + 1)).mapN(_ + _) 
val r = scala.util.Random

val a: Int = r.nextInt // generates X
val b: Int = a + 1     // X + 1

val c: Int = a + b // X + X + 1
val d: Int = r.nextInt + (r.nextInt + 1) // X + Y + 1

monix.execution.atomic.Atomic

  • Specialized AtomicReference implementations
  • A bunch of convenient helpers on top of compareAndSet
  • For safe concurrent access to shared variables without any locking
  • Allows customizing padding to reduce cache contention (for JVM hackers out there)

twitter.com/p_gawrys

val counter = AtomicInt(0)

counter.compareAndSet(expect = 0, update = 2)
println(counter.get()) // 2

counter.compareAndSet(expect = 0, update = 4)
println(counter.get()) // 2

val update = counter.transformAndGet(_ + 2)
println(update) // 4
println(counter.get()) // 4

val extract: String = counter.transformAndExtract { oldValue =>
  val newValue = 0
  val returnValue = s"OldValue: $oldValue"
  (returnValue, newValue)
}

println(extract) // oldValue: 4
println(counter.get) // 0

monix.execution.atomic.Atomic

monix.execution.AsyncSemaphore

  • Non-blocking semaphore for Future
  • Waiting on permit is cancelable

twitter.com/p_gawrys

Limiting parallelism

object SharedResource {
  private val counter = AtomicInt(2)

  def access(i: Int): Unit = {
    if (counter.decrementAndGet() < 0)
      throw new IllegalStateException("counter less than 0")
    Thread.sleep(100)
    counter.increment()
  }
}

implicit val ec = ExecutionContext.global

val list = List.range(1, 5)

val future: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

val f = Future.traverse(list)(future)
Await.result(f, 60.second)

Exception in thread "main" java.lang.IllegalStateException: counter less than 0

Limiting parallelism

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // java.util.concurrent.Semaphore
  val semaphore = new Semaphore(n)

  Future.traverse(list) { i =>
    val future = Future(semaphore.acquire()).flatMap(_ => f(i))
    future.onComplete(_ => semaphore.release())
    future
  }
}

implicit val ec = ExecutionContext.global

val list = List.range(1, 5)

val future: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

traverseN(2, list)(future) // works!

Limiting parallelism

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // java.util.concurrent.Semaphore
  val semaphore = new Semaphore(n)

  Future.traverse(list) { i =>
    val future = Future(semaphore.acquire()).flatMap(_ => f(i))
    future.onComplete(_ => semaphore.release())
    future
  }
}

implicit val ec = ExecutionContext.global

val list = List.range(1, 10)

val future: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

traverseN(2, list)(future) // hangs forever ...

Limiting parallelism

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // monix.execution.AsyncSemaphore
  val semaphore = AsyncSemaphore(n)

  Future.traverse(list) { i =>
    semaphore.withPermit(() => f(i))
  }
}

implicit val ec = ExecutionContext.global

val list = List.range(1, 10)

val future: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

traverseN(2, list)(future) // works!

monix.execution.misc.Local

  • ThreadLocal with a flexible scope which can be propagated over async boundaries
  • Supports Future and Monix Task
  • Integrates with MDC nad OpenTracing
  • Quite low level and still have rough edges
  • First version introduced in 2017 and it was successfully used in multiple big companies

twitter.com/p_gawrys

object LocalExample extends App with StrictLogging {
  implicit val ec = ExecutionContext.global

  def registerUser(name: String): Future[Unit] = {
    // business logic
    logger.info(s"Registering new user named $name")
    Future.unit
  }

  def req(requestId: String, userName: String): Future[Unit] = Future {
    logger.info(s"$requestId: Received request to create user $userName")
    // do sth
  }.flatMap(_ => registerUser(userName))

  val requests = List(req("1", "Wiem"), req("2", "Michał"), req("3", "Piotrek"))
  Await.result(Future.sequence(requests), Duration.Inf)
}

2: Received request to create user Michał
Registering new user named Michał
3: Received request to create user Piotrek
Registering new user named Piotrek
1: Received request to create user Wiem
Registering new user named Wiem

Example: MDC

def registerUser(requestId: String, name: String): Future[Unit] = {
  // business logic
  logger.info(s"$requestId: Registering new user named $name")
  Future.unit
}

def req(requestId: String, userName: String): Future[Unit] = Future {
  logger.info(s"$requestId: Received request to create user $userName")
  // do sth
}.flatMap(_ => registerUser(requestId, userName))

3: Received request to create user Piotrek
3: Registering new user named Piotrek
1: Received request to create user Wiem
1: Registering new user named Wiem
2: Received request to create user Michał
2: Registering new user named Michał

Example: MDC

logger.info("Logging something.")
MDC.put("requestId", "1")
logger.info("Logging something with MDC.")


: Logging something.
1: Logging something with MDC.

Example: MDC

def registerUser(name: String): Future[Unit] = {
  // business logic
  logger.info(s"Registering new user named $name")
  Future.unit
}

def req(requestId: String, userName: String): Future[Unit] = Future {
  MDC.put("requestId", requestId)
  logger.info(s"Received request to create user $userName")
  // more flatmaps to add async boundaries
}.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))

3: Received request to create user Piotrek
2: Received request to create user Michał
1: Received request to create user Wiem
1: Registering new user named Wiem
2: Registering new user named Michał
2: Registering new user named Piotrek

Example: MDC

implicit val s = Scheduler.traced

// from https://github.com/mdedetrich/monix-mdc
MonixMDCAdapter.initialize()

def req(requestId: String, userName: String): Future[Unit] = Local.isolate {
  Future {
    MDC.put("requestId", requestId)
    logger.info(s"Received request to create user $userName")
    // more flatmaps to add async boundaries
  }.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))
}

1: Received request to create user Wiem
3: Received request to create user Piotrek
2: Received request to create user Michał
3: Registering new user named Piotrek
1: Registering new user named Wiem
2: Registering new user named Michał

Example: MDC

Local Model

  • Local is shared unless told otherwise
  • Needs TracingScheduler for Future
  • TaskLocal is a pure version just for a Task
  • Task is a bit smarter about it and does not always require manual isolation

monix.catnap.ConcurrentQueue

  • A high-performance, back-pressured, generic concurrent queue implementation
  • Backpressures on offer if a queue is full
  • Waits for an element if polled on empty queue
  • Purely functional, works for any Concurrent
  • monix.execution.AsyncQueue is a Future-based variant
  • Internally backed by JCTools Queues

twitter.com/p_gawrys

Backpressure - Fast Producer

implicit val s = Scheduler.singleThread("example-pool")

def consumer(queue: ConcurrentQueue[Task, Int]): Task[Unit] = {
  queue.poll
    .flatMap(i => Task(println(s"Consuming $i")))
    .delayExecution(1.second)
    .loopForever
}
def producer(queue: ConcurrentQueue[Task, Int], n: Int = 0): Task[Unit] = {
  for {
    _ <- queue.offer(n)
    _ <- Task(println(s"Produced $n"))
    _ <- producer(queue, n + 1)
  } yield ()
}

val t =
  for {
    queue <- ConcurrentQueue.bounded[Task, Int](2)
    _     <- consumer(queue).startAndForget
    _     <- producer(queue)
  } yield ()

t.executeAsync.runSyncUnsafe()
== Output ==
Produced 0
Produced 1
Produced 2
Consuming 0
Produced 3
Consuming 1
Produced 4
...

Backpressure - Slow Producer

implicit val s = Scheduler.singleThread("example-pool")

def consumer(queue: ConcurrentQueue[Task, Int]): Task[Unit] = {
  queue.poll
    .flatMap(i => Task(println(s"Consuming $i")))
    .loopForever
}

def producer(queue: ConcurrentQueue[Task, Int], n: Int = 0): Task[Unit] = {
  for {
    _ <- queue.offer(n)
    _ <- Task(println(s"Produced $n"))
    _ <- producer(queue, n + 1).delayExecution(1.second)
  } yield ()
}

val t =
  for {
    queue <- ConcurrentQueue.bounded[Task, Int](2)
    _     <- consumer(queue).startAndForget
    _     <- producer(queue)
  } yield ()

t.executeAsync.runSyncUnsafe()
== Output ==
Produced 0
Consuming 0
Consuming 1
Produced 1
Consuming 2
Produced 2
Consuming 3
...

monix.catnap.ConcurrentChannel

  • Created for the sole purpose of modeling complex producer-consumer scenarios
  • Supports multicasting / broadcasting to multiple consumers and workers
  • Sort of like ConcurrentQueue per Consumer with higher level API which allows termination, waiting on consumers etc.
  • Inspired by Haskell's ConcurrentChannel

twitter.com/p_gawrys

monix.catnap.ConcurrentChannel

twitter.com/p_gawrys

final class ConcurrentChannel[F[_], E, A] {
  def push(a: A): F[Boolean]
  def pushMany(seq: Iterable[A]): F[Boolean]
  def halt(e: E): F[Unit]
  def consume: Resource[F, ConsumerF[F, E, A]]
  def consumeWithConfig(config: ConsumerF.Config): Resource[F, ConsumerF[F, E, A]]
  def awaitConsumers(n: Int): F[Boolean]
}

trait ConsumerF[F[_], E, A] {
  def pull: F[Either[E, A]]
  def pullMany(minLength: Int, maxLength: Int): F[Either[E, Seq[A]]]
}
implicit val s = Scheduler.global

def consume(id: Int, consumer: ConsumerF[Task, String, Int]): Task[Unit] = {
  consumer.pull.flatMap {
    case Right(element) =>
      Task(println(s"Worker $id is processing $element"))
        .flatMap(_ => consume(id, consumer))
    case Left(msg) =>
      Task(println(s"Worker $id is done with msg: $msg"))
  }
}

def parallelConsumer(n: Int, consumer: ConsumerF[Task, String, Int]): Task[Unit] = {
  Task.wander(List.range(1, n))(i => consume(i, consumer)).void
}

val app: Task[Unit] =
  for {
    channel <- ConcurrentChannel.of[Task, String, Int]
    _ <- channel.consume.use(consumer => parallelConsumer(4, consumer)).startAndForget
    _ <- channel.consume.use(consumer => consume(5, consumer)).startAndForget
    _ <- channel.awaitConsumers(2)
    _ <- channel.pushMany(List(1, 2, 3, 4))
    _ <- Task.sleep(1.second)
    _ <- channel.halt("good job!")
    _ <- Task.sleep(1.second)
  } yield ()

app.runSyncUnsafe()

monix.catnap.ConcurrentChannel

twitter.com/p_gawrys

Worker 5 is processing 1
Worker 5 is processing 2
Worker 5 is processing 3
Worker 5 is processing 4
Worker 2 is processing 3
Worker 1 is processing 1
Worker 3 is processing 2
Worker 2 is processing 4
Worker 2 is done with msg: good job!
Worker 5 is done with msg: good job!
Worker 1 is done with msg: good job!
Worker 3 is done with msg: good job!

Can't we use ConcurentQueue?

val app: Task[Unit] =
  for {
    queue  <- ConcurrentQueue.unbounded[Task, Either[String, Int]]()
    signal <- Semaphore[Task](0)
    _      <-  Observable.repeatEvalF(queue.poll)
      .takeWhileInclusive(_.isRight) // terminate on "halt"
      .publishSelector { sharedSource =>
        val c1 = sharedSource
            .doOnSubscribe(signal.release)
            .mapEval(elem => consume(5, elem))
        val c2 = sharedSource
          .doOnSubscribe(signal.release)
          // separating workers would require even more machinery
          .mapParallelUnordered(4)(elem => consume(1, elem)) 
        c1.zip(c2)
      }.completedL.startAndForget
    _ <- signal.acquireN(2) // awaitConsumers
    _ <- queue.offerMany(List(1, 2, 3, 4).map(i => Right[String, Int](i)))
    _ <- Task.sleep(1.second)
    _ <- queue.offer(Left[String, Int]("good job!"))
    _ <- Task.sleep(1.second)
  } yield ()

...And there's more!

  • CircuitBreaker, Cancelables, CancelableFuture, Future conversions, TestScheduler, Future-based MVar, ...
  • If you have any questions or more ideas, make sure to let us know at https://github.com/monix/monix or https://gitter.im/monix/monix
  • Contributions are very welcome!

twitter.com/p_gawrys

Thank you !

https://github.com/typelevel/cats-effect

https://github.com/functional-streams-for-scala/fs2

https://github.com/monix/monix

https://github.com/ZIO/core

​Some of the projects worth checking out:

twitter.com/p_gawrys

Monix Catnap

By Piotr Gawryś

Monix Catnap

  • 53
Loading comments...

More from Piotr Gawryś