Ecosystèmes Scala:

Lightbend VS Typelevel

Présentation

Loïc Descotte

  • Développeur Scala
  • Architecte chez Kaizen Solutions

Lightbend

Entreprise qui offre du support autour de Scala et de librairies Scala (et avec des API Java)

 

Propose une transition simple depuis Java via des librairies mixant OO et FP

 

Orienté architectures d’entreprises : micro services, cloud, etc.

Typelevel

Communauté open source promouvant la FP pure et proposant un grand nombre de librairies basées sur ce principe

Lightbend

Akka

Play Framework

Slick

SBT

Lagom

 

Typelevel

Cats (fondation)

Shapeless

Http4s

Doobie

Circe

FS2

Monix

...

Fonctions pures : définition

Les fonctions pures sont :
 

  • Totales : elles renvoient toujours un résultat
     
  • Déterministes : elles renvoient toujours le même résultat pour les mêmes paramètres d’entrée
     
  • Sans effet de bord : elles ne modifient rien qui leur est extérieur

Fonctions pures : avantages

  • Prédictibles
  • Transparence référentielle (RT), donc refactoring plus aisé
  • Testables

RT (Wikipedia) :La transparence référentielle  est une propriété des expressions d'un langage de programmation qui fait qu'une expression peut être remplacée par sa valeur sans changer le comportement du programme

Effets

Définition

Fonctions à effet de bord (wikipedia) :

Les fonctions qui modifient une variable  ou ses arguments, qui lèvent une exception, qui écrivent des données vers un écran ou un fichier, qui lisent les données d'un clavier ou d'un fichier, les fonctions appelant d'autres fonctions à effet de bord


Plus simplement : l'absence de transparence référentielle.

Future

  • Traitements asynchrones
  • Non bloquant
  • Non lazy (strict)
  • Threads natifs

IO


  • Encapsulation des effets
  • Lazy
  • Synchrones ou asynchrones
  • Fibers (threads légers)

Future : exemples

def futureComputation(x: Int): Future[Int] = ???


//parallel computation

val f1 = futureComputation(1)
val f2 = futureComputation(2)

for {
  v1 <- f1
  v2 <- f2
} yield (v1 + v2)


//sequential computation

def f1 = futureComputation(1)
def f2 = futureComputation(2)

for {
  v1 <- f1
  v2 <- f2
} yield (v1 + v2)

Ok mais attention au refactoring!

IO : exemples

def futureComputation(x: Int): IO[Int] = ??? //async IO

val f1 = futureComputation(1)
val f2 = futureComputation(2)

for {
  v1 <- f1
  v2 <- f2
} yield (v1 + v2)

def f1 = futureComputation(1)
def f2 = futureComputation(2)

for {
  v1 <- f1
  v2 <- f2
} yield (v1 + v2)


for {
  v1 <- futureComputation(1)
  v2 <- futureComputation(2)
} yield (v1 + v2)

Ces 3 for comprehension sont équivalents.

Les traitements sont séquentiels.

IO est lazy et permet d'assurer la transparence référentielle (RT).

 

Exemple avec Cats Effect

IO : exemples

def futureComputation(x: Int): IO[Int] = ??? //async IO

val f1 = futureComputation(1)
val f2 = futureComputation(2)

// traitement parralèle
val program = (f1 , f2).parMapN { 
  (v1, v2) => v1 + v2 
}

program.unsafeRunSync()

//ou

object Main extends IOApp {
  //...
  def run(args: List[String]): IO[ExitCode] = program.as(ExitCode.Success)
}

Avant le unsafeRunSync rien n'est exécuté, tout est lazy

On sait où les effets vont se produire et où les erreurs peuvent apparaître

 

Type classes

 

Polymorphisme ad hoc

Massivement utilisées dans Cats

 

case class Color(r: Int, g: Int, b: Int)

implicit val colorNumeric: Numeric[Color] = ??? //implement numeric methods for Color

List(Color(10,10,10), Color(200, 255, 10), Color(20, 55, 1)).sum
// Color(62,66,32)

// List scaladoc :

def sum[B >: A](implicit num: Numeric[B]): B 
List(1,2,3).sum //6

Type classes et IO

import cats.effect.IO
import cats.implicits._


//traverse

val ioList: List[IO[Int]] = List(1, 2, 3).map(x => IO.pure(x+1))

val ioList2: IO[List[Int]] = List(1, 2, 3).map(x => IO.pure(x+1)).sequence

val ioList3: IO[List[Int]] = List(1, 2, 3).traverse(x => IO.pure(x+1))


//cartesian

// execute both, keep only dbCall result in IO
val dbResult: IO[DBResult] = IO(println("go!")) *> IO(dbCall(x)) 

// execute both, keep only println result in IO (IO[Unit])
val logMessageIO: IO[Unit] = IO(println("go!")) <* IO(dbCall(x))

Cats Effect bénéficie des type classes de Cats.

Streaming : exemples

case class Message(text: String, author: String)
object Message { implicit val messageReader = Json.reads[Message] }

object MixedStream extends App {

  implicit val system = ActorSystem("MixedStream")
  implicit val materializer = ActorMaterializer()
  val wsClient = StandaloneAhcWSClient()

  def queryToSource(keyword: String): Source[Message, NotUsed] = {
    val request = wsClient
      .url(s"http://localhost:3000?keyword=$keyword")

    Source.fromFuture(request.stream()).flatMapConcat(_.bodyAsSource)
      .via(Framing.delimiter(ByteString("\n"), 
        maximumFrameLength = 100, allowTruncation = true))
      .map(byteString => Json.parse(byteString.utf8String).as[Message])
  }

  // can mix n streams
  val keywordSources: Source[String, NotUsed] = Source(List("Akka", "FS2")) 
  val done: Future[Done] = keywordSources.flatMapMerge(10, queryToSource)
    .runWith(Sink.foreach(println))
  Await.result(done, Duration.Inf)
}

Akka Stream / Play

Streaming : exemples

case class Message(text: String, author: String)
object Message { implicit val messageReader = Json.reads[Message] }

object MixedStream extends App {

  implicit val system = ActorSystem("MixedStream")
  implicit val materializer = ActorMaterializer()
  val wsClient = StandaloneAhcWSClient()

  def queryToSource(keyword: String): Source[Message, NotUsed] = {
    val request = wsClient
      .url(s"http://localhost:3000?keyword=$keyword")

    Source.fromFuture(request.stream()).flatMapConcat(_.bodyAsSource)
      .via(Framing.delimiter(ByteString("\n"), 
        maximumFrameLength = 100, allowTruncation = true))
      .map(byteString => Json.parse(byteString.utf8String).as[Message])
  }

  // can mix n streams
  val keywordSources: Source[String, NotUsed] = Source(List("Akka", "FS2")) 
  val done: Future[Done] = keywordSources.flatMapMerge(10, queryToSource)
    .runWith(Sink.foreach(println))
  Await.result(done, Duration.Inf)
}

Akka Stream / Play

Streaming : exemples

case class Message(text: String, author: String)
object Message { implicit val messageReader = Json.reads[Message] }

object MixedStream extends App {

  implicit val system = ActorSystem("MixedStream")
  implicit val materializer = ActorMaterializer()
  val wsClient = StandaloneAhcWSClient()

  def queryToSource(keyword: String): Source[Message, NotUsed] = {
    val request = wsClient
      .url(s"http://localhost:3000?keyword=$keyword")

    Source.fromFuture(request.stream()).flatMapConcat(_.bodyAsSource)
      .via(Framing.delimiter(ByteString("\n"), 
        maximumFrameLength = 100, allowTruncation = true))
      .map(byteString => Json.parse(byteString.utf8String).as[Message])
  }

  // can mix n streams
  val keywordSources: Source[String, NotUsed] = Source(List("Akka", "FS2")) 
  val done: Future[Done] = keywordSources.flatMapMerge(10, queryToSource)
    .runWith(Sink.foreach(println))
  Await.result(done, Duration.Inf)
}

Akka Stream / Play

Streaming : exemples

case class Message(text: String, author: String)
object Message { implicit val messageReader = Json.reads[Message] }

object MixedStream extends App {

  implicit val system = ActorSystem("MixedStream")
  implicit val materializer = ActorMaterializer()
  val wsClient = StandaloneAhcWSClient()

  def queryToSource(keyword: String): Source[Message, NotUsed] = {
    val request = wsClient
      .url(s"http://localhost:3000?keyword=$keyword")

    Source.fromFuture(request.stream()).flatMapConcat(_.bodyAsSource)
      .via(Framing.delimiter(ByteString("\n"), 
        maximumFrameLength = 100, allowTruncation = true))
      .map(byteString => Json.parse(byteString.utf8String).as[Message])
  }

  // can mix n streams
  val keywordSources: Source[String, NotUsed] = Source(List("Akka", "FS2")) 
  val done: Future[Done] = keywordSources.flatMapMerge(10, queryToSource)
    .runWith(Sink.foreach(println))
  Await.result(done, Duration.Inf)
}

Akka Stream / Play

Streaming : exemples

case class Message(text: String, author: String)
object Message { implicit val messageReader = Json.reads[Message] }

object MixedStream extends App {

  implicit val system = ActorSystem("MixedStream")
  implicit val materializer = ActorMaterializer()
  val wsClient = StandaloneAhcWSClient()

  def queryToSource(keyword: String): Source[Message, NotUsed] = {
    val request = wsClient
      .url(s"http://localhost:3000?keyword=$keyword")

    Source.fromFuture(request.stream()).flatMapConcat(_.bodyAsSource)
      .via(Framing.delimiter(ByteString("\n"), 
        maximumFrameLength = 100, allowTruncation = true))
      .map(byteString => Json.parse(byteString.utf8String).as[Message])
  }

  // can mix n streams
  val keywordSources: Source[String, NotUsed] = Source(List("Akka", "FS2")) 
  val done: Future[Done] = keywordSources.flatMapMerge(10, queryToSource)
    .runWith(Sink.foreach(println))
  Await.result(done, Duration.Inf)
}

Akka Stream / Play

Streaming : exemples

object MixedStream extends App {
  implicit val sttpBackend = AsyncHttpClientFs2Backend[cats.effect.IO]()

  def queryToStream(keyword: String): IO[Stream[IO,Message]] = {
    val responseIO: IO[Response[Stream[IO,ByteBuffer]]] =
      sttp.post(uri"http://localhost:3000?keyword=$keyword").response(asStream[Stream[IO, ByteBuffer]])
        .readTimeout(Duration.Inf).send()

    responseIO.map { response =>
      response.body match {
        case Right(stream) => stream.flatMap { bytes =>
          val s = new String(bytes.array(), "UTF-8")
            Stream(s).through(stringStreamParser[IO]).through(decoder[IO, Message])
        }
        case Left(_) => Stream(Message( "http error", "app"))
      }
    }
  }

  val streamsIO: IO[List[Stream[IO,Message]]] = List("Akka", "FS2").traverse(queryToStream)
  val mergedIO: IO[Stream[IO, Message]] = streamsIO.map(streams => streams.reduceLeft(_.merge(_)))
  val printStreamIO: IO[Stream[IO, Unit]] = mergedIO.map(merged => merged.map(println))
  val printIO: IO[Unit] = printStreamIO.flatMap(_.compile.drain)
  printIO.unsafeRunSync()
}

FS2 / Circe

Streaming : exemples

object MixedStream extends App {
  implicit val sttpBackend = AsyncHttpClientFs2Backend[cats.effect.IO]()

  def queryToStream(keyword: String): IO[Stream[IO,Message]] = {
    val responseIO: IO[Response[Stream[IO,ByteBuffer]]] =
      sttp.post(uri"http://localhost:3000?keyword=$keyword").response(asStream[Stream[IO, ByteBuffer]])
        .readTimeout(Duration.Inf).send()

    responseIO.map { response =>
      response.body match {
        case Right(stream) => stream.flatMap { bytes =>
          val s = new String(bytes.array(), "UTF-8")
            Stream(s).through(stringStreamParser[IO]).through(decoder[IO, Message])
        }
        case Left(_) => Stream(Message( "http error", "app"))
      }
    }
  }

  val streamsIO: IO[List[Stream[IO,Message]]] = List("Akka", "FS2").traverse(queryToStream)
  val mergedIO: IO[Stream[IO, Message]] = streamsIO.map(streams => streams.reduceLeft(_.merge(_)))
  val printStreamIO: IO[Stream[IO, Unit]] = mergedIO.map(merged => merged.map(println))
  val printIO: IO[Unit] = printStreamIO.flatMap(_.compile.drain)
  printIO.unsafeRunSync()
}

FS2 / Circe

Streaming : exemples

object MixedStream extends App {
  implicit val sttpBackend = AsyncHttpClientFs2Backend[cats.effect.IO]()

  def queryToStream(keyword: String): IO[Stream[IO,Message]] = {
    val responseIO: IO[Response[Stream[IO,ByteBuffer]]] =
      sttp.post(uri"http://localhost:3000?keyword=$keyword").response(asStream[Stream[IO, ByteBuffer]])
        .readTimeout(Duration.Inf).send()

    responseIO.map { response =>
      response.body match {
        case Right(stream) => stream.flatMap { bytes =>
          val s = new String(bytes.array(), "UTF-8")
            Stream(s).through(stringStreamParser[IO]).through(decoder[IO, Message])
        }
        case Left(_) => Stream(Message( "http error", "app"))
      }
    }
  }

  val streamsIO: IO[List[Stream[IO,Message]]] = List("Akka", "FS2").traverse(queryToStream)
  val mergedIO: IO[Stream[IO, Message]] = streamsIO.map(streams => streams.reduceLeft(_.merge(_)))
  val printStreamIO: IO[Stream[IO, Unit]] = mergedIO.map(merged => merged.map(println))
  val printIO: IO[Unit] = printStreamIO.flatMap(_.compile.drain)
  printIO.unsafeRunSync()
}

FS2 / Circe

Streaming : exemples

object MixedStream extends App {
  implicit val sttpBackend = AsyncHttpClientFs2Backend[cats.effect.IO]()

  def queryToStream(keyword: String): IO[Stream[IO,Message]] = {
    val responseIO: IO[Response[Stream[IO,ByteBuffer]]] =
      sttp.post(uri"http://localhost:3000?keyword=$keyword").response(asStream[Stream[IO, ByteBuffer]])
        .readTimeout(Duration.Inf).send()

    responseIO.map { response =>
      response.body match {
        case Right(stream) => stream.flatMap { bytes =>
          val s = new String(bytes.array(), "UTF-8")
            Stream(s).through(stringStreamParser[IO]).through(decoder[IO, Message])
        }
        case Left(_) => Stream(Message( "http error", "app"))
      }
    }
  }

  val streamsIO: IO[List[Stream[IO,Message]]] = List("Akka", "FS2").traverse(queryToStream)
  val mergedIO: IO[Stream[IO, Message]] = streamsIO.map(streams => streams.reduceLeft(_.merge(_)))
  val printStreamIO: IO[Stream[IO, Unit]] = mergedIO.map(merged => merged.map(println))
  val printIO: IO[Unit] = printStreamIO.flatMap(_.compile.drain)
  printIO.unsafeRunSync()
}

FS2 / Circe

Streaming : exemples

object MixedStream extends App {
  implicit val sttpBackend = AsyncHttpClientFs2Backend[cats.effect.IO]()

  def queryToStream(keyword: String): IO[Stream[IO,Message]] = {
    val responseIO: IO[Response[Stream[IO,ByteBuffer]]] =
      sttp.post(uri"http://localhost:3000?keyword=$keyword").response(asStream[Stream[IO, ByteBuffer]])
        .readTimeout(Duration.Inf).send()

    responseIO.map { response =>
      response.body match {
        case Right(stream) => stream.flatMap { bytes =>
          val s = new String(bytes.array(), "UTF-8")
            Stream(s).through(stringStreamParser[IO]).through(decoder[IO, Message])
        }
        case Left(_) => Stream(Message( "http error", "app"))
      }
    }
  }

  val streamsIO: IO[List[Stream[IO,Message]]] = List("Akka", "FS2").traverse(queryToStream)
  val mergedIO: IO[Stream[IO, Message]] = streamsIO.map(streams => streams.reduceLeft(_.merge(_)))
  val printStreamIO: IO[Stream[IO, Unit]] = mergedIO.map(merged => merged.map(println))
  val printIO: IO[Unit] = printStreamIO.flatMap(_.compile.drain)
  printIO.unsafeRunSync()
}

FS2 / Circe

Slick

  • Permet de manipuler les requêtes SQL comme des collections Scala
  • Oblige à écrire soit même le mapping ou d'utiliser des annotations
  • Interface basée sur Future

 

Doobie

  • Tire parti de shapeless pour dériver les modèles (mapping) à la compilation
  • Utilise les IO (ou équivalent)

Slick : exemple

 

case class Country(code: String, name: String, pop: Int, gnp: Option[Double])

class Countries(tag: Tag) extends Table[Country](tag, "country") {
  def code = column[String]("code", O.PrimaryKey)
  def name = column[String]("name")
  def pop= column[Int]("population")
  def gnp= column[Double]("gnp")

  def * = (code , name, pop, gnp.?) <> (Country.tupled, Country.unapply)
}

val countries = TableQuery[Countries]

def populationIn(range: Range): Future[Seq[Country]] = {
  val query = for {
    c <- countries if (c.pop > range.min && c.pop < range.max)
  } yield c
  db.run(query.result)
}

Doobie: exemple

 

case class Country(code: String, name: String, pop: Int, gnp: Option[Double])

def populationIn(range: Range) = 
sql"""
     |   select code, name, population, gnp
     |   from country
     |   where population > ${range.min}
     |   and   population < ${range.max}
     | """.query[Country]




val xa = Transactor.fromDriverManager[IO](
  "org.postgresql.Driver", "jdbc:postgresql:world", "postgres", ""
)


sql"select name from country"
          .query[String]    // Query0[String]
          .to[List]         // ConnectionIO[List[String]]
          .transact(xa)     // IO[List[String]]
          .unsafeRunSync    // List[String]
          .take(5)          // List[String]
          .foreach(println) // Unit

Lightbend

  • Plus facile d'accès sans background FP
  • Librairies plus matures (pour l'instant)
  • Plus de features/plugins disponibles (modules Play, connecteurs Akka Streams, clustering ...)
  • Plus d'exemples sur Internet

Typelevel

  • Ouvre la porte à une nouvelle façon de penser
  • La FP facilite l'évolutivité du code
  • Refactoring plus sûr (RT)
  • Bon support de la communauté

Conclusion :Points forts

Scala Lightbend <-> Typelevel

By loicd

Made with Slides.com

Scala Lightbend <-> Typelevel

  • 2,024

More from loicd