Sleeping well in the lion's den with Monix Catnap

Piotr Gawryś

About me

  • An open source contributor for fun
  • One of the maintainers of Monix
  • Kraków Scala User Group co-organizer
    ​(let me know if you'd like to speak!)

https://github.com/Avasil

twitter.com/p_gawrys

Monix

  • Scala / Scala.js library for asynchronous programming
  • Multiple modules exposing Task, Observable, Iterant, Coeval and many concurrency utilities
  • Favors purely functional programming but provides for all

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 
  • Good integration and consistency with Cats ecosystem

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 
  • Good integration and consistency with Cats ecosystem
  • Maturity/Stability

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 
  • Good integration and consistency with Cats ecosystem
  • Maturity/Stability
  • Performance-sensitive applications

twitter.com/p_gawrys

This talk

twitter.com/p_gawrys

  • monix-execution -low-level concurrency abstractions, companion to scala.concurrent
  • monix-catnap - purely functional abstractions, Cats-Effect friendly

Problem: Limiting parallelism

object SharedResource {
  private val counter = AtomicInt(2)

  def access(i: Int): Unit = {
    if (counter.decrementAndGet() < 0)
      throw new IllegalStateException("counter less than 0")
    Thread.sleep(100)
    counter.increment()
  }
}

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(Future.traverse(List(1, 2, 3, 4, 5))(f), 60.second)

Exception in thread "main" java.lang.IllegalStateException: counter less than 0

Semaphore

  • Synchronization primitive
  • A counter is incremented when semaphore's permit is released
  • A counter is decremented when permit is acquired
  • acquire blocks until there is a permit available

twitter.com/p_gawrys

java.util.concurrent.Semaphore

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // java.util.concurrent.Semaphore
  val semaphore = new Semaphore(n)

  Future.traverse(list) { i =>
    val future = Future(semaphore.acquire()).flatMap(_ => f(i))
    future.onComplete(_ => semaphore.release())
    future
  }
}

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(traverseN(2, List.range(1, 5))(f), Duration.Inf) // works!
Await.result(traverseN(2, List.range(1, 10))(f), Duration.Inf) // hangs forever...

Semantic/Asynchronous blocking

  • Blocks a fiber instead of an underlying thread
  • Can we do it for a Future?

twitter.com/p_gawrys

Let's see how!

Acquire

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)
  
def unsafeAcquireN(n: Long, await: Listener[Unit]): Cancelable
  • Check state
  • Are n permits available?
    • NO => add Listener to awaitPermits queue
    • YES => decrement permits and call Listener callback

Acquire Cancelation

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)

def cancelAcquisition(n: Long, isAsync: Boolean): (Listener[Unit] => Unit)
  • Check state
  • find Listener in awaitPermits and remove it
  • release n permits

Release

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)
  
def unsafeReleaseN(n: Long): Unit
  • Check state
  • Is anything awaiting permit?
    • NO => add permit
    • YES => go through queue and give permits

Implementing with Future

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)


def acquireN(n: Long): CancelableFuture[Unit] = {
  if (unsafeTryAcquireN(n)) {
    CancelableFuture.unit
  } else {
    val p = Promise[Unit]()
    unsafeAcquireN(n, Callback.fromPromise(p)) match {
      case Cancelable.empty => CancelableFuture.unit
      case c => CancelableFuture(p.future, c)
    }
  }
}

monix.execution.AsyncSemaphore

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // monix.execution.AsyncSemaphore
  val semaphore = AsyncSemaphore(n)

  Future.traverse(list) { i =>
    semaphore.withPermit(() => f(i))
  }
}

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(traverseN(2, List.range(1, 10))(f), Duration.Inf) // works!
object LocalExample extends App with StrictLogging {
  implicit val ec = ExecutionContext.global

  def req(requestId: String, userName: String): Future[Unit] = Future {
    logger.info(s"Received a request to create a user $userName")
    // do sth
  }.flatMap(_ => registerUser(userName))
  
  def registerUser(name: String): Future[Unit] = {
    // business logic
    logger.info(s"Registering a new user named $name")
    Future.unit
  }

  val requests = List(req("1", "Clark"), req("2", "Bruce"), req("3", "Diana"))
  Await.result(Future.sequence(requests), Duration.Inf)
}

Received a request to create a user Bruce
Registering a new user named Bruce
Received a request to create a user Diana
Registering a new user named Diana
Received a request to create a user Clark
Registering a new user named Clark

Problem: Logging Requests

def req(requestId: String, userName: String): Future[Unit] = Future {
  logger.info(s"$requestId: Received a request to create a user $userName")
  // do sth
}.flatMap(_ => registerUser(requestId, userName))

def registerUser(requestId: String, name: String): Future[Unit] = {
  // business logic
  logger.info(s"$requestId: Registering a new user named $name")
  Future.unit
}

3: Received a request to create a user Diana
3: Registering a new user named Diana
1: Received a request to create a user Clark
1: Registering a new user named Clark
2: Received a request to create a user Bruce
2: Registering a new user named Bruce

Logging Requests

logger.info("Logging something.")
MDC.put("requestId", "1")
logger.info("Logging something with MDC.")


: Logging something.
1: Logging something with MDC.

Propagating context with MDC

def req(requestId: String, userName: String): Future[Unit] = Future {
  MDC.put("requestId", requestId)
  logger.info(s"Received a request to create a user $userName")
  // more flatmaps to add async boundaries
}.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))

def registerUser(name: String): Future[Unit] = {
  // business logic
  logger.info(s"Registering a new user named $name")
  Future.unit
}

3: Received a request to create a user Diana
2: Received a request to create a user Bruce
1: Received a request to create a user Clark
1: Registering a new user named Clark
2: Registering a new user named Bruce
2: Registering a new user named Diana

MDC and concurrency

monix.execution.misc.Local

  • ThreadLocal with a flexible scope which can be propagated over async boundaries
  • Supports Future and Monix Task
  • Good for context propagation like MDC nad OpenTracing without manually passing parameters
  • Quite low level and still have rough edges
  • First version introduced in 2017

twitter.com/p_gawrys

Local Model

  • Local is shared unless told otherwise
  • Needs TracingScheduler for Future
  • TaskLocal is a pure version just for a Task
  • Task is a bit smarter about it and does not always require manual isolation
implicit val s = Scheduler.traced

// from https://github.com/mdedetrich/monix-mdc
MonixMDCAdapter.initialize()

def req(requestId: String, userName: String): Future[Unit] = Local.isolate {
  Future {
    MDC.put("requestId", requestId)
    logger.info(s"Received a request to create a user $userName")
    // more flatmaps to add async boundaries
  }.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))
}

1: Received a request to create a user Clark
3: Received a request to create a user Diana
2: Received a request to create a user Bruce
3: Registering a new user named Diana
1: Registering a new user named Clark
2: Registering a new user named Bruce

MDC with Monix Local

Gotcha: Blackbox Asynchronous Code

implicit val ec = Scheduler.traced

val local = Local(0)

def blackbox: Future[Unit] = {
  val p = Promise[Unit]()
  new Thread {
    override def run(): Unit = {
      Thread.sleep(100)
      p.success(())
    }
  }.start()
  p.future
}

val f = Local.isolate {
  for {
    _ <- Future { local := local.get + 100 }
    _ <- blackbox
    _ <- Future { local := local.get + 100 }
  // can print 100 if blackbox is not isolated!
  } yield println(local.get) 
}

Await.result(f, Duration.Inf)

Tracking Parcel Delivery

COURIER

LOCATION UPDATE

SHIPPING SYSTEM

REGISTER PARCEL

PARCEL

CHECK STATUS

UPDATE STATUS

case class ParcelStatus(estimatedDelivery: OffsetDateTime, route: String)

case class ParcelDelivery(id: Long, getLatestStatus: Task[Option[ParcelStatus]])

class ShippingSystem {
  // adds new parcel to the system
  def registerParcel(
    id: Long, 
    destination: String
  ): Task[ParcelDelivery] = ???

  // sends a new location of the parcel
  def updateLocation(id: Long, location: String): Task[Unit] = ???
}

Tracking Parcel Delivery

case class ParcelStatus(estimatedDelivery: OffsetDateTime, route: String)

case class ParcelDelivery(id: Long, getLatestStatus: Task[Option[ParcelStatus]])

class ShippingSystem {
  // adds new parcel to the system
  def registerParcel(
    id: Long, 
    destination: String
  ): Task[ParcelDelivery] = ???

  // sends a new location of the parcel
  def updateLocation(id: Long, location: String): Task[Unit] = ???
}

Tracking Parcel Delivery

could use synchronization and/or back-pressure

Parcel Delivery with Queue

COURIER

LOCATION UPDATE

QUEUE

SHIPPING SYSTEM

REGISTER PARCEL

PARCEL

CHECK STATUS

UPDATE STATUS

Asynchronous Queue

  • A collection which allows to add elements to one end of the sequence and remove them from the other end
  • Producer is backpressured on offer if a queue is full
  • Consumer is backpressured on poll if a queue is empty
  • Useful for decoupling producer and consumer, distributing work

twitter.com/p_gawrys

Monix Queues

  • ConcurrentQueue[F[_], A] - a purely functional asynchronous queue for any Cats-Effect compliant effect
  • AsyncQueue[A] - impure asynchronous queue for scala.concurrent.Future

twitter.com/p_gawrys

case class ParcelLocation(id: Long, location: String)
type ParcelLocationQueue = ConcurrentQueue[Task, ParcelLocation]

class ShippingSystem private[example] (
    queue: ParcelLocationQueue,
    // impure data structure to cut boilerplate for the sake of example
    deliveries: TrieMap[Long, ParcelStatus]
) {
  def registerParcel(id: Long, destination: String): Task[ParcelDelivery] =
    Task {
      val parcelStatus = ParcelStatus(OffsetDateTime.now().plusYears(1L), s"route-to-$destination")
      deliveries.addOne(id -> parcelStatus)
      ParcelDelivery(id, Task(deliveries.get(id)))
    }

  def updateLocation(id: Long, location: String): Task[Unit] =
    queue.offer(ParcelLocation(id, location))

  def run(): Task[Unit] =
    queue.poll.flatMap(updateStatus).loopForever

  private def updateStatus(parcel: ParcelLocation): Task[Unit] = {
    Task(deliveries.get(parcel.id)).flatMap {
      case Some(lastStatus) =>
        // imagine some calculations here
        val newStatus = ParcelStatus(
          OffsetDateTime.now().plusHours(4L), lastStatus.route + s"-at-${parcel.location}")

        Task(deliveries.update(parcel.id, newStatus))
          .flatMap(_ => Task(println(s"Updated parcel ${parcel.id} with new status $newStatus")))
      case None =>
        Task(println(s"Received missing parcel ${parcel.id}"))
    }
  }
}
override def run(args: List[String]): Task[ExitCode] =
  for {
    // will back-pressure updateLocation if it's full
    queue <- ConcurrentQueue.bounded[Task, ParcelLocation](1024)
    shippingSystem = new ShippingSystem(
      queue, TrieMap.empty[Long, ParcelStatus])
    // run shippingSystem in the background
    _ <- shippingSystem.run().startAndForget
    parcel <- shippingSystem.registerParcel(0L, "NYC")
    _ <- shippingSystem.updateLocation(0L, "WAW")
    _ <- Task.sleep(100.millis)
    latestStatus <- parcel.getLatestStatus
  } yield {
    ExitCode.Success
  }

Other implementations

  • fs2: F[_] support, extra methods for fs2.Stream, lots of types of queues, fairness 
  • ZIO: ZIO support, profunctor queue (has contramap, map, filter etc.), fairness
  • Monix: F[_] + Future support, best performance

twitter.com/p_gawrys

What if there are more systems which need parcel location?

COURIER

LOCATION UPDATE

???

SHIPPING SYSTEM

REGISTER PARCEL

PARCEL

CHECK STATUS

UPDATE STATUS

ANALYTICS SYSTEM

monix.catnap.ConcurrentChannel

  • Created for the sole purpose of modeling complex producer-consumer scenarios
  • Supports multicasting / broadcasting to multiple consumers and workers
  • Sort of like ConcurrentQueue per Consumer with higher level API which allows termination, waiting on consumers etc.
  • Inspired by Haskell's Control.Concurrent.Chan

twitter.com/p_gawrys

monix.catnap.ConcurrentChannel

twitter.com/p_gawrys

final class ConcurrentChannel[F[_], E, A] {
  def push(a: A): F[Boolean]
  def pushMany(seq: Iterable[A]): F[Boolean]
  def halt(e: E): F[Unit]
  def consume: Resource[F, ConsumerF[F, E, A]]
  def consumeWithConfig(config: ConsumerF.Config): Resource[F, ConsumerF[F, E, A]]
  def awaitConsumers(n: Int): F[Boolean]
}

trait ConsumerF[F[_], E, A] {
  def pull: F[Either[E, A]]
  def pullMany(minLength: Int, maxLength: Int): F[Either[E, Seq[A]]]
}
type ParcelLocationChannel = ConcurrentChannel[Task, Unit, ParcelLocation]
  
class ShippingSystem private[example] (
    channel: ParcelLocationChannel,
    // impure data structure to cut boilerplate for the sake of example
    deliveries: TrieMap[Long, ParcelStatus]
) {

  // the same as queue version
  def registerParcel(id: Long, destination: String): Task[ParcelDelivery]

  def updateLocation(id: Long, location: String): Task[Unit] =
    channel.push(ParcelLocation(id, location)).map(_ => ())

  def run(): Task[Unit] = channel.consume.use(consumeChannel)

  private def consumeChannel(
      consumer: ConsumerF[Task, Unit, ParcelLocation]): Task[Unit] = {
    consumer.pull.flatMap {
      case Left(halt) =>
        Task(println(s"Closing ShippingSystem"))
      case Right(parcel) =>
        updateStatus(parcel).flatMap(_ => consumeChannel(consumer))
    }
  }
  
  // the same as queue version
  private def updateStatus(parcel: ParcelLocation): Task[Unit]
}

twitter.com/p_gawrys

override def run(args: List[String]): Task[ExitCode] =
  for {
    channel <- ConcurrentChannel.of[Task, Unit, ParcelLocation]
    shippingSystem = new ShippingSystem(
      channel, TrieMap.empty[Long, ParcelStatus])
    analytics = new AnalyticsSystem(channel)
    // run both systems in the background
    _ <- Task.parZip2(shippingSystem.run(), analytics.run()).startAndForget
    // wait until we have 2 subscribers
    _ <- channel.awaitConsumers(2)
    parcel <- shippingSystem.registerParcel(0L, "NYC")
    _ <- shippingSystem.updateLocation(0L, "new location")
    _ <- Task.sleep(100.millis)
    latestStatus <- parcel.getLatestStatus
  } yield {
    println(latestStatus)
    ExitCode.Success
  }

Parcel Delivery

Alternative solutions

  • monix.reactive.Observable: lots of options and control but sharing streams is not 100% pure 
  • ConcurrentChannel: simple, pure API based on F[_] + Resource
  • fs2.concurrent.Topic: simple, pure API based on fs2.Stream
  • Akka: Hubs for AkkaStreams and distributed PubSub with Actors

twitter.com/p_gawrys

...And there's more!

  • CircuitBreaker, Cancelables, CancelableFuture, Future utils, TestScheduler, Future-based MVar, ...
  • If you have any questions or more ideas, make sure to let us know at https://github.com/monix/monix or https://gitter.im/monix/monix
  • Contributions are very welcome!

twitter.com/p_gawrys

Thank you !

https://github.com/typelevel/cats-effect

https://github.com/functional-streams-for-scala/fs2

https://github.com/monix/monix

https://github.com/zio/zio

​Some of the projects worth checking out:

twitter.com/p_gawrys

Sleeping well in the lion's den with Monix Catnap (Typelevel Summit 2020)

By Piotr Gawryś

Made with Slides.com

Sleeping well in the lion's den with Monix Catnap (Typelevel Summit 2020)

  • 2,682

More from Piotr Gawryś