Jean-Rémi Desjardins

an alternative to for-comprehensions

Scala by the Bay 2015

def getPhone: Future[Phone]
def getAddress: Future[Address]
def getSpamScore: Future[SpamScore]
for {p <- getPhone
     a <- getAddress
     s <- getSpamScore} yield render(p, a, s)

What's wrong?

phone: Future[Phone]
address: Future[Address]
spamScore: Future[SpamScore]
for {p <- phone
     a <- address
     s <- spamScore} yield render(p, a, s)

What about now?

phone: Future[Phone] // 1 second
address: Future[Address] // 2 seconds
spamScore: Future[SpamScore] // fail after 10 milliseconds
for {p <- phone
     a <- address
     s <- spamScore} yield render(p, a, s) // fails after 2 seconds

The problem

phone: Future[Phone] // 1 second
address: Future[Address] // 2 seconds
spamScore: Future[SpamScore] // fail after 10 milliseconds
Applicative[Future].apply3(a,b,c)(combine)

A Solution

through scalaz

phone: Future[Phone] // 1 second
getAddress(phone: Phone): Future[Address] // 2 seconds
getSpamScore(phone: Phone): Future[SpamScore] // fail after 10 milliseconds
for {
    p <- phone
    address <- getAddress(phone)
    spamScore <- getSpamScore(phone)
} yield render(p, address, spamScore) // fails after 2 seconds

More Problems

phone: Future[Phone] // 1 second
getAddress(phone: Phone): Future[Address] // 2 seconds
getSpamScore(phone: Phone): Future[SpamScore] // fail after 10 milliseconds
phone.map { p =>
    Applicative[Future].apply2(getAddress(p), getSpamScore(p)) { case (address, spamScore) =>
        render(p, address, spamScore)
    }
} // fails after 10 milliseconds

A Solution

phone: Future[Phone] // 1 second
getAddress(phone: Phone): Future[Address] // 2 seconds
getSpamScore(phone: Phone): Future[SpamScore] // fail after 10 milliseconds
Expression{
    val p = extract(phone)
    render(p, extract(getAddress(p)), extract(getSpamScore(p)))
} // fails after 10 milliseconds

A Better Solution

through Expressions

phone: Future[Phone] // 1 second
getAddress(phone: Phone): Future[Address] // 2 seconds
getSpamScore(phone: Phone): Future[SpamScore] // fail after 10 milliseconds
Expression{
    val p = extract(phone)
    val address = extract(getAddress(p))
    val spamScore = extract(getSpamScore(p))
    render(p, address, spamScore)
} // fails after 10 milliseconds

Or if you prefer

Why exactly does this not fail fast?

val a = Future { sleep(1.second); 5 }
val b = Future { sleep(5.seconds); 10}
val c = Future { sleep(3.seconds); fail()}

for {aa <- a
     bb <- b
     cc <- c} yield combine(a, b, c)

See my talk at PNWScala 2014

Desugared

val a = Future { sleep(1.second); 5 }
val b = Future { sleep(5.seconds); 10}
val c = Future { sleep(3.seconds); fail()}

a.flatMap { aa =>
    b.flatMap { bb =>
        c.map { cc =>
          combine(aa, bb, cc)
        }
    }
}

flatMap()

trait Future[+A] {
    def flatMap(f: A => Future[B]): Future[B]
}

zip()

trait Future[+A] {
    def flatMap(f: A => Future[B]): Future[B]
    def zip[B](fb: Future[B]): Future[(A,B)]
}

Desugared

val a = Future { sleep(1.second); 5 }
val b = Future { sleep(5.seconds); 10}
val c = Future { sleep(3.seconds); fail()}

a.zip(b).zip(c) { case ((aa, bb), cc) =>
    combine(aa, bb, cc)
}

What about Async/await?

val a = Future { sleep(1.second); 5 }
val b = Future { sleep(5.seconds); 10}
val c = Future { sleep(3.seconds); fail()}

async { combine(await(a), await(b), await(c)) }

A notation for all the things

Expressions

Features

  • Uses the least powerful interface

    • ​Futures can fail-fast

    • supports Validation

  • Plays well with if and match statements

  • Unified notation for all abstractions

  • Customizable

Examples

Failing fast

for {aa <- a
     bb <- b
     cc <- c} yield combine(a, b, c)
Expression { combine(a,b,c) }
val a: Future[A] = wait(5)
val b: Future[B] = fail(1)
val c: Future[C] = wait(3)

wait(5)

fail(1)

Interacting with if

for {aa <- a
     bb <- b
     cc <- c} yield if (aa == something) polish(bb) else polish(cc)
 (for (aa <- a) yield
   if (aa == something) for (bb <- b) yield polish(bb)
   else for (cc <- c) yield polish(cc)).flatMap(identity)
val a: Future[A] = wait(1)
val b: Future[B] = wait(5)
val c: Future[C] = wait(2)
Expression { if(extract(a) == something) polish(b) else polish(c) }

if (aa == something) => wait(5)

if (aa != something) => wait(5)

if (aa == something) => wait(5)

if (aa != something) => wait(2)

if (aa == something) => wait(5)

if (aa == something) => wait(5)

Supports all the abstractions

val a: Option[A]
val b: Option[B]
val c: Option[C]
Expression { if(extract(a) == something) polish(b) else polish(c) }
val a: Err \/ A
val b: Err \/ B
val c: Err \/ C
val a: Writer[A]
val b: Writer[B]
val c: Writer[C]
val a: Task[A]
val b: Task[B]
val c: Task[C]
val a: IO[A]
val b: IO[B]
val c: IO[C]
val a: List[A]
val b: List[B]
val c: List[C]

Supports Validation

in theory...

val first = Validation.success(5)
val second = Validation.success(4)
def getThird(a: Int): ValidationNel[String, Int] = Validation.failureNel("too big")
val fourth: ValidationNel[String, Int] = Validation.failureNel("too small")
val result = Expression[({type l[a] = Validation[NonEmptyList[String], a]})#l, Int] {
    val one = extract2(first)
    one + extract2(second) + extract2(getThird(one)) + extract2(fourth)
}(validationMonad)
result ==== Failure(NonEmptyList("too big", "too small"))

How it works

Uses Scalaz

trait Functor[F[_]] {
    def map[A,B](fa: F[A])(f: A => B): F[B]
}

trait Apply[F[_]] extends Functor[F] {
    // looks like something we've seen before?
    def apply2[A,B,C](fa: F[A], fb: F[B])(f: (A,B) => C): F[C]
    //derived
    def map[A,B](fa: F[A])(f: A => B): F[B] = derived
}

trait Applicative[F[_]] extends Apply[F] {
    def point[A](a: A): F[A]
}

trait Monad[F[_]] extends Applicative[F] {
    // looks like something we've seen before?
    def bind[A,B](fa: F[A])(f: A => F[B]): F[B]
    //derived
    def apply2[A,B,C](fa: F[A], fb: F[B])(f: (A,B) => C): F[C] = derived
}

(but doesn't need too...)

Simple code transformation

Expression { foo(extract(a), b, extract(c) }
Applicative[_].apply2(a,c)(foo(_,b,_))
val a: Future[String]
val b: String
val c: Future[String]

def foo(one: String, two: String, three: String)

Importance of using the least powerful interface

Power of the abstraction

Functor

Applicative

Monad

Flexibility offered to implementation

Functor

Applicative

Monad

Monad is required

Expression { foo(extract(bar(extract(a))), b, extract(c)) }
Monad[_].apply(Monad[_].bind(a)(bar),c)(foo(_,b,_))
val a: Future[String]
val b: String
val c: Future[String]

def foo(one: String, two: String, three: String)
def bar(g: String): Future[String]

if statement

Expression { if (extract(a)) extract(b) else extract(c) }
Monad[_].bind(a)(if(_) b else c)
val a: Future[String]
val b: String
val c: Future[String]

match statement

Expression {
    extract(a) match {
        case 1 => extract(b)
        case 2 => extract(c) + 1
    }
}
Monad[_].bind(a)(_ match {
    case 1 => b
    case 2 => instance.map(c)(cc => cc + 1)
}
val a: Future[String]
val b: Future[String]
val c: Future[String]

match statement can get hairy

Expression {
    val fooY = extract(foo)
    extract(bar) match {
        case `fooY` => extract(b)
        case 2 => extract(c) + 1
    }
}
val fooY = getFoo
Monad[_].bind(Monad[_].apply(fooY, bar)((_,_)){ case (x$1, x$2) => x$1 match {
    case `x$2` => b
    case 2 => instance.map(c)(cc => cc + 1)
}
val foo: Future[String]
val bar: Future[String]
val b: Future[String]
val c: Future[String]

match statement can get REALLY hairy

Expression {
  val fooY = extract(foo)
  val bizY = extract(biz)
  extract(bar) match {
    case `fooY` => extract(a)
    case `bizY` => extract(c)
    case 2 => extract(c) + 1
  }
}
{
  val fooY = foo
  val bizY = biz
  Monad[_].bind(Monad[_].apply(fooY, barY)((_,_)){ case (x$1, x$2) => x$1 match {
    case `x$2` => a
    case _ => Monad[_].bind(biz){ x$3 => x$1 match {
      case `x$3` => b
      case 2 => instance.map(c)(cc => cc + 1)
    }
  }
}

Similar Projects

  • Effectful

    • ​Alternative to for-comprehension and/or generalization of Async/Await

    • Does not use the least powerful interface (cannot fail-fast)

  • Scala Workflow

    • Many features (all Expressions features + context manipulation)

    • Uses untyped macros

    • Reimplements parts of scalac including scoping which leads to minimal coverage of the language

  • Async/Await

    • Only works with Scala Futures

    • Does not fail-fast

Limitations

Known to work (somewhat)

  • function applications
  • if-else statement
  • function currying
  • string interpolation
  • blocks
  • basic match statements

Know limitations

  • pattern matching in value definitions

  • advanced match statements

When to use them

Write Async code

val response: Future[HTML] = Expression {
    val phone = extract(lookupPhone(phoneString))
    val address = extract(lookupAddress(phone))
    val rep = extract(lookupReputation(phone))
    renderPage(phone, address, rep)
}

Large blocks of code within a Monad

val response: Option[HTML] = Expression {
    val firstFruit = json.fruits(0)
    extract(firstFruit.as[String]) match {
        case "apple" => "good"
        case "peach" =>
            if (extract(firstFruit.color.as[String]) == "brown") "bad"
            else "good"
    }
}

Not a replacement for for-comprehensions

def time[A](task: Task[A]): Task[(Duration, A)] = for {
    t1 <- Task.delay(System.currentTimeMillis)
    a  <- task
    t2 <- Task.delay(System.currentTimeMillis)
} yield ((t2 - t1).milliseconds, a)

Example from Remotely

Future work

  • Use Scala Meta
  • Improve ScalaCheck function generation
  • Implement Context manipulation
  • Support nested abstractions
  • Fix known limitations
  • Generalize tests

Context Manipulation

a: Writer[String, Int] = 8.set("This is a magic value")
b: Writer[String, Int] = random().set("I am a random value")
c: Writer[String, String] = "Hello World".set("Hello World...")
Expression {
  val div = if (b == 0) {
    ctx :++> "We avoided a division by zero, yay!"
    5
  } else a / b
  c * div
}

Thank you

Questions, comments?