Sleeping well in the lion's den with Monix Catnap

Piotr Gawryś

About me

  • An open source contributor for fun
  • One of the maintainers of Monix
  • Kraków Scala User Group co-organizer
    ​(let me know if you'd like to speak!)

https://github.com/Avasil

twitter.com/p_gawrys

Monix

  • Scala / Scala.js library for asynchronous programming
  • Multiple modules exposing Task, Observable, Iterant, Coeval and many concurrency utilities
  • Favors purely functional programming but provides for all

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 
  • Good integration and consistency with Cats ecosystem

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 
  • Good integration and consistency with Cats ecosystem
  • Maturity/Stability

twitter.com/p_gawrys

Monix Niche

  • Mixed codebases 
  • Good integration and consistency with Cats ecosystem
  • Maturity/Stability
  • Performance-sensitive applications

twitter.com/p_gawrys

This talk

twitter.com/p_gawrys

  • monix-execution -low-level concurrency abstractions, companion to scala.concurrent
  • monix-catnap - purely functional abstractions, Cats-Effect friendly

Problem: Limiting parallelism

object SharedResource {
  private val counter = AtomicInt(2)

  def access(i: Int): Unit = {
    if (counter.decrementAndGet() < 0)
      throw new IllegalStateException("counter less than 0")
    Thread.sleep(100)
    counter.increment()
  }
}

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(Future.traverse(List(1, 2, 3, 4, 5))(f), 60.second)

Exception in thread "main" java.lang.IllegalStateException: counter less than 0

Semaphore

  • Synchronization primitive
  • A counter is incremented when semaphore's permit is released
  • A counter is decremented when permit is acquired
  • acquire blocks until there is a permit available

twitter.com/p_gawrys

java.util.concurrent.Semaphore

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // java.util.concurrent.Semaphore
  val semaphore = new Semaphore(n)

  Future.traverse(list) { i =>
    val future = Future(semaphore.acquire()).flatMap(_ => f(i))
    future.onComplete(_ => semaphore.release())
    future
  }
}

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(traverseN(2, List.range(1, 5))(f), Duration.Inf) // works!
Await.result(traverseN(2, List.range(1, 10))(f), Duration.Inf) // hangs forever...

Semantic/Asynchronous blocking

  • Blocks a fiber instead of an underlying thread
  • Can we do it for a Future?

twitter.com/p_gawrys

Let's see how!

Acquire

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)
  
def unsafeAcquireN(n: Long, await: Listener[Unit]): Cancelable
  • Check state
  • Are n permits available?
    • NO => add Listener to awaitPermits queue
    • YES => decrement permits and call Listener callback

Acquire Cancelation

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)

def cancelAcquisition(n: Long, isAsync: Boolean): (Listener[Unit] => Unit)
  • Check state
  • find Listener in awaitPermits and remove it
  • release n permits

Release

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)
  
def unsafeReleaseN(n: Long): Unit
  • Check state
  • Is anything awaiting permit?
    • NO => add permit
    • YES => go through queue and give permits

Implementing with Future

type Listener[A] = Either[Throwable, A] => Unit

private final case class State(
  available: Long,
  awaitPermits: Queue[(Long, Listener[Unit])]
)


def acquireN(n: Long): CancelableFuture[Unit] = {
  if (unsafeTryAcquireN(n)) {
    CancelableFuture.unit
  } else {
    val p = Promise[Unit]()
    unsafeAcquireN(n, Callback.fromPromise(p)) match {
      case Cancelable.empty => CancelableFuture.unit
      case c => CancelableFuture(p.future, c)
    }
  }
}

monix.execution.AsyncSemaphore

implicit val ec = 
  ExecutionContext.fromExecutor(Executors.newFixedThreadPool(4))

def traverseN(n: Int, list: List[Int])(
  f: Int => Future[Unit]
): Future[List[Unit]] = {
  // monix.execution.AsyncSemaphore
  val semaphore = AsyncSemaphore(n)

  Future.traverse(list) { i =>
    semaphore.withPermit(() => f(i))
  }
}

val f: Int => Future[Unit] = i => Future {
  SharedResource.access(i)
}

Await.result(traverseN(2, List.range(1, 10))(f), Duration.Inf) // works!
object LocalExample extends App with StrictLogging {
  implicit val ec = ExecutionContext.global

  def req(requestId: String, userName: String): Future[Unit] = Future {
    logger.info(s"Received a request to create a user $userName")
    // do sth
  }.flatMap(_ => registerUser(userName))
  
  def registerUser(name: String): Future[Unit] = {
    // business logic
    logger.info(s"Registering a new user named $name")
    Future.unit
  }

  val requests = List(req("1", "Clark"), req("2", "Bruce"), req("3", "Diana"))
  Await.result(Future.sequence(requests), Duration.Inf)
}

Received a request to create a user Bruce
Registering a new user named Bruce
Received a request to create a user Diana
Registering a new user named Diana
Received a request to create a user Clark
Registering a new user named Clark

Problem: Logging Requests

def req(requestId: String, userName: String): Future[Unit] = Future {
  logger.info(s"$requestId: Received a request to create a user $userName")
  // do sth
}.flatMap(_ => registerUser(requestId, userName))

def registerUser(requestId: String, name: String): Future[Unit] = {
  // business logic
  logger.info(s"$requestId: Registering a new user named $name")
  Future.unit
}

3: Received a request to create a user Diana
3: Registering a new user named Diana
1: Received a request to create a user Clark
1: Registering a new user named Clark
2: Received a request to create a user Bruce
2: Registering a new user named Bruce

Logging Requests

logger.info("Logging something.")
MDC.put("requestId", "1")
logger.info("Logging something with MDC.")


: Logging something.
1: Logging something with MDC.

Propagating context with MDC

def req(requestId: String, userName: String): Future[Unit] = Future {
  MDC.put("requestId", requestId)
  logger.info(s"Received a request to create a user $userName")
  // more flatmaps to add async boundaries
}.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))

def registerUser(name: String): Future[Unit] = {
  // business logic
  logger.info(s"Registering a new user named $name")
  Future.unit
}

3: Received a request to create a user Diana
2: Received a request to create a user Bruce
1: Received a request to create a user Clark
1: Registering a new user named Clark
2: Registering a new user named Bruce
2: Registering a new user named Diana

MDC and concurrency

monix.execution.misc.Local

  • ThreadLocal with a flexible scope which can be propagated over async boundaries
  • Supports Future and Monix Task
  • Good for context propagation like MDC nad OpenTracing without manually passing parameters
  • Quite low level and still have rough edges
  • First version introduced in 2017

twitter.com/p_gawrys

Local Model

  • Local is shared unless told otherwise
  • Needs TracingScheduler for Future
  • TaskLocal is a pure version just for a Task
  • Task is a bit smarter about it and does not always require manual isolation
implicit val s = Scheduler.traced

// from https://github.com/mdedetrich/monix-mdc
MonixMDCAdapter.initialize()

def req(requestId: String, userName: String): Future[Unit] = Local.isolate {
  Future {
    MDC.put("requestId", requestId)
    logger.info(s"Received a request to create a user $userName")
    // more flatmaps to add async boundaries
  }.flatMap(_ => Future(()).flatMap(_ => Future())).flatMap(_ => registerUser(userName))
}

1: Received a request to create a user Clark
3: Received a request to create a user Diana
2: Received a request to create a user Bruce
3: Registering a new user named Diana
1: Registering a new user named Clark
2: Registering a new user named Bruce

MDC with Monix Local

Gotcha: Blackbox Asynchronous Code

implicit val ec = Scheduler.traced

val local = Local(0)

def blackbox: Future[Unit] = {
  val p = Promise[Unit]()
  new Thread {
    override def run(): Unit = {
      Thread.sleep(100)
      p.success(())
    }
  }.start()
  p.future
}

val f = Local.isolate {
  for {
    _ <- Future { local := local.get + 100 }
    _ <- blackbox
    _ <- Future { local := local.get + 100 }
  // can print 100 if blackbox is not isolated!
  } yield println(local.get) 
}

Await.result(f, Duration.Inf)

Tracking Parcel Delivery

COURIER

LOCATION UPDATE

SHIPPING SYSTEM

REGISTER PARCEL

PARCEL

CHECK STATUS

UPDATE STATUS

case class ParcelStatus(estimatedDelivery: OffsetDateTime, route: String)

case class ParcelDelivery(id: Long, getLatestStatus: Task[Option[ParcelStatus]])

class ShippingSystem {
  // adds new parcel to the system
  def registerParcel(
    id: Long, 
    destination: String
  ): Task[ParcelDelivery] = ???

  // sends a new location of the parcel
  def updateLocation(id: Long, location: String): Task[Unit] = ???
}

Tracking Parcel Delivery

case class ParcelStatus(estimatedDelivery: OffsetDateTime, route: String)

case class ParcelDelivery(id: Long, getLatestStatus: Task[Option[ParcelStatus]])

class ShippingSystem {
  // adds new parcel to the system
  def registerParcel(
    id: Long, 
    destination: String
  ): Task[ParcelDelivery] = ???

  // sends a new location of the parcel
  def updateLocation(id: Long, location: String): Task[Unit] = ???
}

Tracking Parcel Delivery

could use synchronization and/or back-pressure

Parcel Delivery with Queue

COURIER

LOCATION UPDATE

QUEUE

SHIPPING SYSTEM

REGISTER PARCEL

PARCEL

CHECK STATUS

UPDATE STATUS

Asynchronous Queue

  • A collection which allows to add elements to one end of the sequence and remove them from the other end
  • Producer is backpressured on offer if a queue is full
  • Consumer is backpressured on poll if a queue is empty
  • Useful for decoupling producer and consumer, distributing work

twitter.com/p_gawrys

Monix Queues

  • ConcurrentQueue[F[_], A] - a purely functional asynchronous queue for any Cats-Effect compliant effect
  • AsyncQueue[A] - impure asynchronous queue for scala.concurrent.Future

twitter.com/p_gawrys

case class ParcelLocation(id: Long, location: String)
type ParcelLocationQueue = ConcurrentQueue[Task, ParcelLocation]

class ShippingSystem private[example] (
    queue: ParcelLocationQueue,
    // impure data structure to cut boilerplate for the sake of example
    deliveries: TrieMap[Long, ParcelStatus]
) {
  def registerParcel(id: Long, destination: String): Task[ParcelDelivery] =
    Task {
      val parcelStatus = ParcelStatus(OffsetDateTime.now().plusYears(1L), s"route-to-$destination")
      deliveries.addOne(id -> parcelStatus)
      ParcelDelivery(id, Task(deliveries.get(id)))
    }

  def updateLocation(id: Long, location: String): Task[Unit] =
    queue.offer(ParcelLocation(id, location))

  def run(): Task[Unit] =
    queue.poll.flatMap(updateStatus).loopForever

  private def updateStatus(parcel: ParcelLocation): Task[Unit] = {
    Task(deliveries.get(parcel.id)).flatMap {
      case Some(lastStatus) =>
        // imagine some calculations here
        val newStatus = ParcelStatus(
          OffsetDateTime.now().plusHours(4L), lastStatus.route + s"-at-${parcel.location}")

        Task(deliveries.update(parcel.id, newStatus))
          .flatMap(_ => Task(println(s"Updated parcel ${parcel.id} with new status $newStatus")))
      case None =>
        Task(println(s"Received missing parcel ${parcel.id}"))
    }
  }
}
override def run(args: List[String]): Task[ExitCode] =
  for {
    // will back-pressure updateLocation if it's full
    queue <- ConcurrentQueue.bounded[Task, ParcelLocation](1024)
    shippingSystem = new ShippingSystem(
      queue, TrieMap.empty[Long, ParcelStatus])
    // run shippingSystem in the background
    _ <- shippingSystem.run().startAndForget
    parcel <- shippingSystem.registerParcel(0L, "NYC")
    _ <- shippingSystem.updateLocation(0L, "WAW")
    _ <- Task.sleep(100.millis)
    latestStatus <- parcel.getLatestStatus
  } yield {
    ExitCode.Success
  }

Other implementations

  • fs2: F[_] support, extra methods for fs2.Stream, lots of types of queues, fairness 
  • ZIO: ZIO support, profunctor queue (has contramap, map, filter etc.), fairness
  • Monix: F[_] + Future support, best performance

twitter.com/p_gawrys

What if there are more systems which need parcel location?

COURIER

LOCATION UPDATE

???

SHIPPING SYSTEM

REGISTER PARCEL

PARCEL

CHECK STATUS

UPDATE STATUS

ANALYTICS SYSTEM

monix.catnap.ConcurrentChannel

  • Created for the sole purpose of modeling complex producer-consumer scenarios
  • Supports multicasting / broadcasting to multiple consumers and workers
  • Sort of like ConcurrentQueue per Consumer with higher level API which allows termination, waiting on consumers etc.
  • Inspired by Haskell's Control.Concurrent.Chan

twitter.com/p_gawrys

monix.catnap.ConcurrentChannel

twitter.com/p_gawrys

final class ConcurrentChannel[F[_], E, A] {
  def push(a: A): F[Boolean]
  def pushMany(seq: Iterable[A]): F[Boolean]
  def halt(e: E): F[Unit]
  def consume: Resource[F, ConsumerF[F, E, A]]
  def consumeWithConfig(config: ConsumerF.Config): Resource[F, ConsumerF[F, E, A]]
  def awaitConsumers(n: Int): F[Boolean]
}

trait ConsumerF[F[_], E, A] {
  def pull: F[Either[E, A]]
  def pullMany(minLength: Int, maxLength: Int): F[Either[E, Seq[A]]]
}
type ParcelLocationChannel = ConcurrentChannel[Task, Unit, ParcelLocation]
  
class ShippingSystem private[example] (
    channel: ParcelLocationChannel,
    // impure data structure to cut boilerplate for the sake of example
    deliveries: TrieMap[Long, ParcelStatus]
) {

  // the same as queue version
  def registerParcel(id: Long, destination: String): Task[ParcelDelivery]

  def updateLocation(id: Long, location: String): Task[Unit] =
    channel.push(ParcelLocation(id, location)).map(_ => ())

  def run(): Task[Unit] = channel.consume.use(consumeChannel)

  private def consumeChannel(
      consumer: ConsumerF[Task, Unit, ParcelLocation]): Task[Unit] = {
    consumer.pull.flatMap {
      case Left(halt) =>
        Task(println(s"Closing ShippingSystem"))
      case Right(parcel) =>
        updateStatus(parcel).flatMap(_ => consumeChannel(consumer))
    }
  }
  
  // the same as queue version
  private def updateStatus(parcel: ParcelLocation): Task[Unit]
}

twitter.com/p_gawrys

override def run(args: List[String]): Task[ExitCode] =
  for {
    channel <- ConcurrentChannel.of[Task, Unit, ParcelLocation]
    shippingSystem = new ShippingSystem(
      channel, TrieMap.empty[Long, ParcelStatus])
    analytics = new AnalyticsSystem(channel)
    // run both systems in the background
    _ <- Task.parZip2(shippingSystem.run(), analytics.run()).startAndForget
    // wait until we have 2 subscribers
    _ <- channel.awaitConsumers(2)
    parcel <- shippingSystem.registerParcel(0L, "NYC")
    _ <- shippingSystem.updateLocation(0L, "new location")
    _ <- Task.sleep(100.millis)
    latestStatus <- parcel.getLatestStatus
  } yield {
    println(latestStatus)
    ExitCode.Success
  }

Parcel Delivery

Alternative solutions

  • monix.reactive.Observable: lots of options and control but sharing streams is not 100% pure 
  • ConcurrentChannel: simple, pure API based on F[_] + Resource
  • fs2.concurrent.Topic: simple, pure API based on fs2.Stream
  • Akka: Hubs for AkkaStreams and distributed PubSub with Actors

twitter.com/p_gawrys

...And there's more!

  • CircuitBreaker, Cancelables, CancelableFuture, Future utils, TestScheduler, Future-based MVar, ...
  • If you have any questions or more ideas, make sure to let us know at https://github.com/monix/monix or https://gitter.im/monix/monix
  • Contributions are very welcome!

twitter.com/p_gawrys

Thank you !

https://github.com/typelevel/cats-effect

https://github.com/functional-streams-for-scala/fs2

https://github.com/monix/monix

https://github.com/zio/zio

​Some of the projects worth checking out:

twitter.com/p_gawrys

Made with Slides.com