Scalaz in SpotRight

What is Scalaz?

a Scala library
based on Haskell's library
contains
- defintions - traits/classes
- instances and implicits
- extra syntax

Why do we care?

Algebraic Data Types (ADT)
- Algebraic Data Types
- Allow the programmer to reason about the transformations being applied to data

http://www.haskellforall.com/2012/01/haskell-for-mainstream-programmers.html

Background

Category Theory

Can't do any better than H eiko's Blog

Kinds

The "type" of a type
Int - kind *
List[Int] - kind *
List - kind * -> *
Map - kind * -> * -> *
Functor - kind (* -> *) -> *
- Higher-kinded

https://hseeberger.wordpress.com/2010/11/25/introduction-to-category-theory-in-scala/

Monoid

trait Monoid[A] {
  def zero: A
  def append(a: A, b: => A): A
}

"Things that can be reduced"
append must be associative
- (a + b) + c == a + (b + c)
You find these things *everywhere*
- String, Num, List[A], Map[A, B: Monoid], etc
- A type might admit several monoids (same for Functor, Monad, Applicative)
  - Sum extends Monoid[Int]
  - Product extends Monoid[Int]
Often useful to add one to your own types

Functor

"Things that can be mapped over"
has map: (A => B) => F[A] => F[B]
Surprisingly tame vs other higher-kinded types

Monad

a monad is a monoid in the category of endofunctors. What's the problem?" -- One Div Zero

http://james-iry.blogspot.com/2009/05/brief-incomplete-and-mostly-wrong.html

Has point: A => M[A]
Has flatMap: (A => M[B]) => M[A] => M[B]
Is a functor, so has map: (A => B) => F[A] => F[B]
The effect of flatMap is particular to each monad. E.g.,
- List - concatenate
- Option - propagate None
- Reader - provide a common "fixed" environment
- Writer - track log messages
- State - thread modifiable state
- Future - transform asynchronously

Monad

Can be viewed from two (related) perspectives

As a data structure
- List - collection of items
- Option - Some or None
- Reader - Function1
- Writer - (A, List[String])
As a context with effectful transformations
- List - computation of entangled value
- Option - computation that might fail
- Reader - computation in an environment
- Writer (aka State) - audited computations

Monad

How to think about monadic action

map: (A => B) => M[A] => M[B]

join: M[M[A]] => M[A]

flatMap: map andThen join

map using A => M[B] then join result
- idName.get(uid).map{n => nameAddr.get(n)} // Option[Option[String]]
- idName.get(uid).flatMap{n => nameAddr.get(n)} // Option[String]
given M[A] compute M[B] and replace the M[A]

Free

What the hell is a "Free Monad" (and why do we care)?

An abstract syntax tree of the effects introduced by a monad.
Given a Functor you can always write a Free Monad of it
Using flatMap will then build you a data structure (which can be interpreted later) of the effectful transformations
We don't really care but I wanted you to know if you see it during self-study.
An excellent writeup.

The free monoid of type A is just List[A]
Notice how a list builds an AST of Monoid[A]'s append calls
A blog post containing the free monoid.

http://underscore.io/blog/posts/2015/04/14/free-monads-are-simple.html

Compare to the Free Monoid of A

http://underscore.io/blog/posts/2015/04/23/deriving-the-free-monad.html

Applicative Functor

has point: A => F[A]
has ap: F[A => B] => F[A] => F[B]
Is a functor so has map: (A => B) => F[A] => F[B]
Note that every Monad is an applicative functor
Called "applicative" because of Haskell syntax
- add a b
- add <$> Some(a) <*> Some(b)
Scala syntax not as pretty
- b.some <*> (a.some <*> add.curried)
Applicative Builder syntax
- (a.some |@| b.some){_ + _}
- Apply.apply2[Option](a.some, b.some)

Applicative Functor

Is not as flexible as monad
- Unpacks an effect but does not transform the result
- Apply.apply2(3.some, 2.some) // Some(5)
- (See below for strength of monad)
Often you don't need monad's flexibilty so problems can be addressed by applicative

for {
  x <- 3.some
  y <- 2.some
} yield {
  if (x == 3) "Horse" else "Cow"
}

Round up

Operations so far
- Functor: (A => B) => (F[A] => F[B])
- Monad: (A => M[B]) => (M[A] => M[B])
- Applicative: F[A => B] => (F[A] => F[B])
- ???
- Comonad: (M[B] => A) => (M[B] => M[A])
  - The 'dual' of monad
  - Too trippy for us at this point
  - Is basically OOP
  - See You Could Have Invented CoMonads

http://www.haskellforall.com/2013/02/you-could-have-invented-comonads.html

Comonads in Haskell

http://www.slideshare.net/davidoverton/comonad

Laws

ADT typeclasses (higher-kinded types) must obey certain laws.
The laws make sure our implementation is an ADT.
Sometimes a few laws can be broken but the result is still useful and "mostly" a Monad/Applicative/Functor
- Monad law left-identity: M[A].point(a) >>= f === M[A].point( f(a) )
- Try[Int] { sys.error("err") } map {_ + 1} != Try { n + 1 }
- Also Future (which uses Try underneath)
- Both still useful because they guarantee code wrapped won't expose Exceptions (and can still be reasoned with using monadic intuition)
Laws should be checked when you roll your own (compiler won't automatically check for you)
See The Typeclassopedia

https://wiki.haskell.org/Typeclassopedia

Working with Scalaz

Import needed definitions
- scalaz - Scalaz types (Monoid, Apply, etc.)
Import needed instances/implicits
- scalaz.std - Scala types (list, map, int, anyVal, function, etc.)
Import needed syntax
- scalaz.syntax.std - Syntax for Scala types (option, function, etc)
- scalaz.syntax - Syntax for Scalaz types (monoid, validation, traverse)

Rapid Prototyping (Cheating)

import scalaz._
import Scalaz._

import scalaz.syntax.id._

def strim(s: String): String = s.trim
def isCap(s: String): Boolean = s.capitalize == s

// Pipelining.
"  Hi" |> strim |> isCap  // true
"  Hi" |> {_.trim} |> {s => s.capitalize == s} // true

// Kestrel combinator.  Used for side-effects.  Returns input.
val conn = OldLibrary.conn() <| { c =>
    c.permissive()
    c.ttl(2000L)
}

// getOrElse for null
val foo1 = "foo"
val foo2: String = null

foo1 ?? "bar"  // "foo"
foo2 ?? "bar"  // "bar"

Fun with Id[A]

import scalaz.syntax.std.boolean._

true option 42  // Some(42)
false option 42 // None: Option[Int]

// Pull in defs not exposed by syntax
import scalaz.std.boolean._

test(2 > 1)  // 1
test(1 > 2)  // 0

// Pull in Monoid, Enum, Show for basic Scala types
import scalaz.std.anyVal._

true ?? 42  // 42  Needs Monoid[Int]
false ?? 42  // 0

Fun with Boolean

import scalaz.syntax.std.option._

// Note that Some(3) in vanilla Scala is Some[Int]
3.some  // Some(3): Option[Int]

import scalaz.std.option._

// compare None which is None.type
none[Int]  // None: Option[Int]

3.some.orZero  // error - we need our Monoids

import scalaz.std.anyVal._

3.some.orZero  // 3
none[Int].orZero  // 0

// Note that we've also imported
//
//   3.some | 42  // 3
//   none[Int] | 42  // 42
//
// but don't use this in SpotRight.
// Prefer .getOrElse and .fold

Fun with Option

Fun with Monoid[A]

import scalaz.std.anyVal._
import scalaz.syntax.semigroup._

List(1,2,3,4,5).foldLeft(0){_ |+| _}  // 15.  duh

// Tuple is a Monoid if elements are.
import scalaz.std.tuple._

List(1,2,3,4,5).view.map{(_,1)}.reduce{_ |+| _}  // (15,5)

// Map[A,B: Monoid] is a Monoid
import scalaz.std.map._

Map("a" -> 1, "b" -> 2) |+| Map("b" -> 1, "c" -> 2)

// Grab instances for List and Set
import scalaz.std.list._
import scalaz.std.set._

val words = List(
  "the", "quick", "sly", "fox", "jumped",
  "over", "the", "lazy", "brown", "dog"
)

words.view
  .map{w => Map(w.length -> List(w)}
  .reduce{_ |+| _}  // words collected by length

words.view
  .map{w => Map(w.length -> Set(w)}
  .reduce {_ |+| _}  // unique words by length

Fun with Apply (Applicative)

import scalaz.Apply
import scalaz.std.option._

// Apply is Applicative without point()
// Another way to say that is it allows a different
// way to apply pure functions to effectful values

val env = Map('a -> 1, 'b -> 2, 'c -> 3)

implicit class SymV(val s: Symbol) extends AnyVal {
  def v: Option[Int] = env.get(s)
}

def add2(x: Int, y: Int): Int = x + y
def add3(x: Int, y: Int, z: Int): Int = x + y + z

Apply[Option].apply2('a.v, 'b.v)(add2 _)  // Some(3)

Apply[Option].apply3('a.v, 'b.v, 'c.v)(add3 _)  // Some(6)

Apply[Option].apply2('a.v, 'e.v)(add2 _)  // None

More fun with Apply (Applicative)

// How about something useful?

import scalaz.Apply
import scalaz.ValidationNel
import scalaz.std.option._
import scalaz.syntax.std.option._
import scalaz.syntax.nel._

case class Person(name: String, age: Int, children: List[String])

type VNel[A] = ValidationNel[String, A]

// Real call is with `key`.  `value` and `fail` are allow us to pretend.
def iocall[A](key: String, value: A, fail: Boolean = false): Option[A] =
  if (fail) none[A] else value.some

def iovnel[A](key: String, value: A, fail: Boolean = false): VNel[A] =
  iocall(key, value, fail).toSuccess(s"failed to get key 'key'".wrapNel)

def iovnel2[A](key: String, value: A, fail: Boolean = false): Vnel[A] =
  if (fail) s"failed to get key '$key'".failureNel[A] else value.successNel[String]

val result =
  Apply[VNel].apply3(
    iovnel("name", "JohnQ"),
    iovnel("age", 32),
    iovnel("children", List("Jack", "Jill"))
  )(Person).fold(
      fail = {nel => sys.error(s"Errors:\n   ${nel.list.mkString("\n   ")}")},
      succ = identity
    )

More fun with Applicative

// Motivation.  Many functions taking same input (i.e., Reader)

import scalaz.syntax.traverse._  // Applicative traverseal
import scalaz.std.function._
import scalaz.std.list._

def plus1(x: Int): Int = x + 1
def double(x: Int): Int = x * 2
def toTen(x: Int): Int = (x % 10) + 10

val funs = List(plus1 _, double _, toTen _)

// List of Reader to Reader of List
val joined = funs.sequenceU

joined(3)  // List(4, 6, 13)

// sequenceU helps Scala figure out Int => Int === (Int => A): Applicative
// Try it if using just sequence complains about the shape

// Traverse applies some transform to each List element and then does
// sequence.  Compare to mapping then accumulating for a List.

val tjoined = funs.traverseU{ r => r.map{_ * 2} }

tjoined(3)  // List(8, 12, 26)

import scalaz.Id.Id  // grab Id (type Id[+X] = X)

funs.traverse { f => (f(3) + 1): Id[Int] }  // List(5, 7, 14)

Fun with Monad

// This sort of code should be somewhat familiar

import scalaz.MonadPlus
import scalaz.std.list._
import scalaz.std.option._

case class Person(name: String, age: Int, children: List[String])

def name2Person: Map[String,Person] = ???

def perFm[M[_],A](name: String, f: Person => M[A])(implicit M: MonadPlus[M]): M[A] =
  (  for {
       rec <- name2Person.get(name)
    } yield f(rec)
  ).getOrElse(M.empty[A])

def kidsAges(names: List[String]): List[Int] =
  for {
    parent <- names
    child <- perFm(parent, _.children)
    age <- perFm(child, r => List(r.age))
  } yield age 

val ages = kidsAges(List("Arnold", "Becky", "Charles", "Debra")

Other fun things

Reader, Writer, State
Tagged types
Tree (scalaz.syntax.tree._)
Lens
MonadPlus
Monad Transformers
Kleisli (>==>)
Arrow (***, &&&)
CoMonad
- Lenses are an Implementation
- "A structure modified by small transforms"
- "A finite-state machine"
- "Encapsulated State plus Behavior (aka OOP)"

Production Examples

Monoid |+|
- Models/asty/src/main/scala/com/spotright/models/asty/Person.scala:62
- Aragog/src/main/scala/com/spotright/aragog/models/TwitterCrawlDatum.scala:64
State Monad
- type State[S,A] = S => (S,A)
- point(a: A)
  - { s => (s,a) }
- ma.map(f: A => B)
  - How do you map a function? Make a new function.
  - { s => val (s1, a) = ma(s); val b = f(a); (s1, b) }
- ma.flatMap(f: A => M[B])
  - { s => val (s1, a) = ma(s); val (s2, b) = f(a)(s1); (s2, b) }
- Common/core/src/main/scala/com/spotright/common/util/urltidy/UrlNorm.scala

Production Examples

Applicative - Apply
DBConf from configuration file (search-slick; deprecated)

  import scalaz.{ValidationNel,Apply}
  import scalaz.syntax.validation._

  type ValNel[A] = ValidationNel[String,A]

  def get(path: String): ValNel[String] =
    Play.current.configuration.getString(path)
      .map(_.successNel[String])
      .getOrElse(path.failureNel[String])

  def getOr(path: String, default: => String): ValNel[String] =
    get(path) orElse default.successNel[String]

  // using ap6 instead of apply6 would expect a `Tuple6 => R`
  Apply[ValNel].apply6(
    get("slick.db.driver"),
    get("db.default.driver"),
    get("db.default.url"),
    getOr("db.default.user", ""),
    getOr("db.default.password", ""),
    Some(play.api.db.DB.getDataSource()).successNel[String]
  )(DBConf.apply).fold(
      fail = {ps => sys.error(s"Required config value not found: ${ps.list.mkString(", ")}")},
      succ = AppDB.createDAL
    )

Production Examples

Applicative - Apply
- Hawkeye/src/main/scala/com/spotright/hawkeye/ianda/HandleReporter.scala:469
- Considered `sequence` here to collect the VNel[Boolean] tests into a VNel[List[Boolean]] but List[Boolean] which could have then been Monoid[Disjuction] reduced. In a use-once situation though the use of Apply directly is simpler.
In-House Kestrel
- Nate needed a modified kestrel (scalaz's unsafeTap [aka <|] from syntax.id) for Boolean
- What got written was a folded kestrel
- com.spotright.common.util.SpinalTap (aka <||)

Scalaz in SpotRight

By Lanny Ripple

Scalaz in SpotRight

What is Scalaz?

Why do we care?

Background

Monoid

Functor

Monad

Monad

Monad

Free

Compare to the Free Monoid of A

Applicative Functor

Applicative Functor

Round up

Laws

Working with Scalaz

Rapid Prototyping (Cheating)

Production Examples

Production Examples

Production Examples

Scalaz in SpotRight

More from Lanny Ripple