shapeless cats and kittens

Data-type generic function combination 

Kai(luo) Wang 
@iHeartRadio (we'r hiring)
github, twitter: @kailuowang

with real-world examples

A quick survey

Heard of typelevel/cats?

 Has the jar in the class path?

Using it directly?

Using scalaz?

A tiny application

  object HRService {
    val numOfEmployees: DepartmentId => Int = ...
    val employeeDepartment: EmployeeId => DepartmentId = ...
  }

  object FinanceService {
    val budget: DepartmentId => Double = ...
  }

  object DirectoryService {
    val extensionNum: DepartmentId => Int = ...
  }

  case class Department(
    numOfEmployees  : Int,
    budget          : Double,
    extensionNum    : Int
  )

  def department(id: DepartmentId) : Department = ???
  def extension(employee: EmployeeId) : Int = ???

Function composed

  
  def department(id: DepartmentId): Department = 
    Department(
      numOfEmployees = HRService.numOfEmployees(id),
      budget         = FinanceService.budget(id),
      extensionNum   = DirectoryService.extensionNum(id))


  def employExtension(employeeId: EmployeeId): Int = {
    val departmentId = HRService.employeeDepartment(employeeId)
    DirectoryService.extensionNum(departmentId)
  }

Bound variables :(

Point free

  val employExtension =  
    HRService.employeeDepartment andThen DirectoryService.extensionNum
  
  def department(id: DepartmentId): Department = 
    Department(
      numOfEmployees = HRService.numOfEmployees(id),
      budget         = FinanceService.budget(id),
      extensionNum   = DirectoryService.extensionNum(id))

But what about this?

Function combinator

typelevel/cats

  • a library which provides abstractions for functional programming in Scala."
  • to provide a foundation for an ecosystem of pure, typeful libraries.
  • a respectful, helpful, and kind community
  • Applicative

  • Foldable

  • Functor

  • Monad

  • Semigroup

  • Monoid

  • Traverse

cats

Typeclasses

Date types

  • EitherT
  • Free
  • Kleisli
  • OptionT
  • StateT
  • WriterT
  • Validated

Instances for Stand lib

cats.instance.function

   val department: = 
       (HRService.numOfEmployees |@|
        FinanceService.budget |@|
        DirectoryService.extensionNum) map Department.apply
    

Associative combination

  val department = 
    for {
      numOfEmployee <- HRService.numOfEmployees
      budget        <- FinanceService.budget
      extensionNum  <- DirectoryService.extensionNum
    } yield Department(numOfEmployee, budget, extensionNum) 

Monadic sequential combination

 What if the function is async?

  object DirectoryService {
    val extensionNum: DepartmentId => Future[Int] = ...
  }

Given

   def department(id: DepartmentId): Future[Department] = 
       (HRService.numOfEmployees(id) |@|
        FinanceService.budget(id) |@|
        DirectoryService.extensionNum(id)) map Department.apply
    

Fallback to bound variable?

cats.data.Kleisli

case class Kleisli[F[_], A, B](run: A => F[B]) {
  def map[C](f: B => C)(implicit F: Functor[F]): Kleisli[F, A, C] =
    Kleisli(a => F.map(run(a))(f))
}
val department: DeparmentId => Future[Department] = 
      (( Kleisli(HRService.numOfEmployees) |@|
         Kleisli(FinanceService.budget) |@|
         Kleisli(DirectoryService.extensionNum)) map Department.apply).run
val employeeExtension: Klesili[EmployeeId, Int] = 
  Kleisli(HRService.employDepartment) andThen Kleisli(DirectoryService.extensionNum)

Associative parellel combination

Sequential combination

  object HRService {
    val numOfEmployees: DepartmentId => Future[Either[Reason, Int]] = ...
  }
 

Function with error handling

Monad Transformer - cats.data.EitherT

   case class EitherT[F[_], A, B](value: F[Either[A, B]]){
      def bimap[C, D](fa: A => C, fb: B => D)(implicit F: Functor[F])
       : EitherT[F, C, D]
       = EitherT(F.map(value)(_.bimap(fa, fb)))
   } 
  object HRService {
    type ReasonableFuture[T] = EitherT[Future, Reason, T]
    def numOfEmployees(id: DepartmentId): ReasonableFuture[Int] = ...
  }

Real-world functions have heterogeneous arguments 

  def assignDorm(req: AssignDormRoomRequest): Future[Either[Reason, DormRoom]]
  
  case class AssignDormRoomRequest(
    name: String,
    sex: Sex, 
    mentor: Mentor)

  
  def registerClass(req: ClassRegRequest): Future[Either[Reason, ClassRegistration]]
   
  case class ClassRegRequest(
    studentId: StudentId, 
    classId: ClassId,
    semester: Semester)

a type class and dependent type based generic programming library

automatic type class derivation for Cats and generic utility functions

  Map("numberOfEmployee" -> 52, "budget" -> 10000, "extensionNumber" -> 30815)

  Map("name" -> "Mike", "sex" -> "male", "address" -> "123 ABC Ave")
  case class Department(numOfEmployees: Int, budget: Double, extensionNum: String)
  
  case class Student(name: String, sex: Sex, address: Address)

Genericity by Shape

Genericity by Structure

Generic value combinator

List(Future(1), Future(2), Future(3)).sequence

 //gives a Future(List(1, 2, 3))

val r: Future[Department] = ???(numOfEmployees = Future(52),
                                budget         = Future(100000.0),
                                extensionNum   = Future(30185)).sequence
  

I wish I can do...

With cats.Traverse

    for {
      numOfEmployee <- HRService.numOfEmployees(id)
      budget        <- FinanceService.budget(id)
      extensionNum  <- DirectoryService.extensionNum(id)
    } yield Department(numOfEmployee, budget, extensionNum)
    

For this code

we can do

def sequence[G[_]: Applicative, A](fga: F[G[A]]): G[F[A]]

Generic value combinator

with kitten.sequence

    val seq = sequenceGeneric[Department]

    def department(id: DepartmentId) : Future[Department] =
      seq(numOfEmployees = HRService.numOfEmployees(id),
          budget         = FinanceService.budget(id),
          extensionNum   = DirectoryService.extensionNum(id))
sequence: HList of A[_] => A[HList]
E.g.
Future[Int] :: Future[String] :: Future[Boolean] :: HNil => 
  Future[Int :: String :: Boolean :: HNil]

Now we can do

Bonus

kittens type class instance generation

    import cats.Semigroup, cats.syntax.semigroup._
    import semigroup._, legacy._

    val department1 = Department(23, 20000.0, 23911)
    val department2 = Department(4, 1000.0, 92711)

    val Department(totalEmployee, totalBudget, _) = department1 |+| department2
  
    assert(totalEmployee == 27)
    assert(totalBudget == 21000.0)

Before we go on to generic function composition, introducing

henkan

  • automatic and type safe data structure transformation
  • some building blocks for data-type generic FP 

Experiments with shapless cats and kittens

henkan.k

trait Definitions {
  
  type Result[_]

  implicit def resultMonad: Monad[Result]

  type K[A, B] = Kleisli[Result, A, B]
  
  ...
}

Building blocks for generic function combinator

Built on abstract types

caveat: SI-2712 partial fix required

henkan.k

trait DefinitionWithEitherT extends Definitions {

  type Reason //reason ADT for failure

  /**
   * Effect of the operations, e.g. [[Future]]
   */
  type Effect[_]

  type Result[T] = EitherT[Effect, Reason, T]

  implicit def effectMonad: Monad[Effect]

  implicit lazy val resultMonad: Monad[Result] = 
    EitherT.catsDataMonadErrorForEitherT[Effect, Reason]

}

An example of EitherT based Result definitions

Reasonable future

object ReasonableFuture extends henkan.k.DefinitionWithEitherT {
  sealed trait MyReason

  case class ExceptionOccurred(msg: Option[String],
                               ex: Option[Throwable]) extends MyReason
  case class UserError(msg: String) extends MyReason
  case class Unavailable(missingThingy: String) extends MyReason

  type Reason = MyReason

  type Effect[X] = Future[X]

  implicit val effectMonad: Monad[Effect] = {
    import scala.concurrent.ExecutionContext.Implicits.global
    cats.std.future.futureInstance
  }
}

First let's define some types

Hogwarts Student Registration

object Domain {
  //passed in by user
  type Request = Map[String, String]

  case class RegistrationRecord(basicInfo: BasicInfo,
                                mentor: Mentor, 
                                dormRoom: DormRoom)

  case class BasicInfo(name: String, address: String, age: Int, sex: Sex)
  ...
} 
trait Services {
  def wandService: K[GetWandRequest, Wand]
  def mentorService: K[AssignMentorRequest, Mentor]
  def dormService: K[AssignDormRoomRequest, DormRoom]

  //protocols
  case class GetWandRequest(name: String, address: String)
  case class AssignMentorRequest(wand: Wand, specialty: Specialty)
  case class AssignDormRoomRequest(name: String, sex: Sex, mentor: Mentor)
}
def register: K[Request, RegistrationRecord] = ???

Services at disposal

Hogwarts Student Registration

Request Parser


  def stringExtractor(key: String): K[Request, String] = K.of((req: Request) ⇒
    req.get(key).toResult(UserError(s"missing value of $key")))

We need to extract this BasicInfo out of Request (Map[String, String])

  case class BasicInfo(name: String, address: String, age: Int, sex: Sex)

  def basicInfoExtractor: K[Request, BasicInfo] = ???

Generic funciton combinator

  private lazy val ctbi = composeTo[BasicInfo]
  lazy val baseInformationExtractor: K[Request, BasicInfo] = ctbi(
    name = stringExtractor("name"),
    address = stringExtractor("address"),
    age = stringExtractor("age") andThen (a ⇒ Try(a.toInt).toResult()),
    sex = stringExtractor("sex") andThen stringToSex
  )

Automatic Parser with Henkan

  import henkan.extractor._

  implicit val frString: FieldReader[Result, Request, String] = 
    stringExtractor _
  implicit val frInt: FieldReaderMapper[String, Result[Int]] =
    (s: String) ⇒ Try(s.toInt).toResult()

  implicit val frSex: FieldReaderMapper[String, Result[Sex]] =
    stringtoSex.run
  val baseInformationParser: K[Request, BasicInfo] = 
    Extractor[Result, Request, BasicInfo].apply()

Hogwarts Student Registration

Application


  import AutoExtractors._

  def register(services: Services): K[Request, RegistrationRecord] = {
    import services._
    for {
      bi     ← baseInformationExtractor
      mentor ← mentorService.contraMapR(
                 wand      = pure[Request](bi.to[GetWandRequest]()) andThen
                             wandService,
                 specialty = stringExtractor("specialty") andThen stringToSpecialty
      )
      room   ← pure[Request](bi.to[AssignDormRoomRequest].set(mentor = mentor)) andThen 
                  dormService
    } yield RegistrationRecord(bi, mentor, room)
  } 

def mentorService: K[AssignMentorRequest, Mentor]
case class AssignMentorRequest(wand: Wand, specialty: Specialty)
 def wandService: K[GetWandRequest, Wand]
 case class GetWandRequest(name: String, address: String)
  case class RegistrationRecord(basicInfo: BasicInfo,
                                mentor: Mentor, 
                                dormRoom: DormRoom)
 def dormService: K[AssignDormRoomRequest, DormRoom]
 case class AssignDormRoomRequest(name: String, sex: Sex, mentor: Mentor)

A community of projects and individuals organized around ...

  • Pure, typeful, functional programming in Scala
  • Independent, free/libre and open source software
  • A desire to share ideas and code
  • Accessible and idiomatic learning resources
  • An inclusive, welcoming and safe environment

Question?

Sorry, I don't know much about the Hogwarts school.

But I do have typelevel stickers...

Links

Made with Slides.com