What is a Programming Language?

• Syntax
• AST (maybe)
• Semantics

What is a Domain-Specific Programming Language?

• Syntax
• AST (maybe)
• Semantics
• All specific to a single domain

What is an Embedded Domain-Specific Programming Language?

• Syntax of the host language
• AST (maybe)
• Semantics of the host language (plus)
• All specific to a single domain

What is a Deeply Embedded Domain-Specific Programming Language?

• Syntax borrowed from the host language
• AST
• Semantics
• All specific to a single domain

Modular Object Oriented Programming

• Interfaces rely on other interfaces
• Implicit object graph of dependencies
• Each interface is a domain, a set of related concerns

interface ILog {
  void debug(String debug);
  void info(String info);
  void warn(String warn);
  void error(String message, 
             Exception ex);
}

interface IKVStore {
  String get(String key);
  void update(String key, 
              String value);
  void remove(String key);
}

class LoggedStubKVStore implements IKVStore {
  private final ILog logger;
  public LoggedStubKVStore(ILog log) {
    this.logger = log;
  }
  @Override
  public String get(String key) {
    logger.debug("tried to get a value at key: " + key);
    return "stub value";
  }
  @Override
  public void update(String key, String data) {
    logger.debug("tried to set the value at key: " + key + " to " + data);
  }
  @Override
  public void remove(String key) {
    logger.debug("tried to remove the value at key: " + key);
  }
}

class HashMapKVStore implements IKVStore {
  private final ILog logger;
  private final HashMap<String, String> hashMap;
  public HashMapKVStore(ILog logger, HashMap<String, String> hashMap) {
    this.logger = logger;
    this.hashMap = hashMap;
  }
  @Override
  public void get(String key) {
    logger.debug("tried to get a value at key: " + key);
    return hashMap.get(key);
  }
  @Override
  public void update(String key, String data) {
    logger.debug("tried to set the value at key: " + key + " to " + data);
    hashMap.put(key, data);  
  }
  @Override
  public void remove(String key) {
    logger.debug("tried to remove the value at key: " + key);
    hashMap.remove(key);
  }
}

// class RedisKVStore implements IKVStore {
// ...
// }

// class LogstashLogger implements ILog {
// ...
// }

Dependency Injection

• Objects are provided with other objects satisfying interfaces they need, instead of constructing them themselves
• Objects say what they can do and need done, and let the user wire them together
• Enhanced modularity via dependencies between capabilities rather than implementations

Problems

• Side-effects everywhere; can't implement the interface without side-effects in update()
• Types in the interface don't express effects: the fact that the results from remove, get and update may be null and may (e.g.) connect to a Redis server is not expressed in the interface
• Logging (and other concerns) repeated everywhere

Solutions

• Side-effects can be encapsulated in an IO type
• Logging (and other concerns) can be shared by mixing traits into classes
• But what about the types of objects that implement the IKVStore interface but don't perform side-effects?

Other (not side-) effects

• There are several other effects we might want to use that do not perform I/O
• An implementation of ILog might want to append to a list of logged strings, instead of printing them immediately
• An implementation of IKVStore might want to add an extra HashMap parameter and result to each method

Effects first

• Implementations of IKVStore and ILog should choose their own effects

The trick

• To allow IKVStore and ILog to choose their own effects, we can add a type constructor parameter:

(beware, this is where it starts being Scala)

trait IKVStore[F[_]] {
  def get(key: String): F[Option[String]]
  def update(key: String, value: String): F[Unit]
  def remove(key: String): F[Unit]
}

trait ILog[F[_]] {
  def debug(debug: String): F[Unit]
  def info(info: String): F[Unit]
  def warn(warn: String): F[Unit]
  def err(err: String, ex: Exception): F[Unit]
}

object PureMapKVStore {
  type MapManipulator[A] = Map[String, String] => (A, Map[String, String])
}

class PureMapKVStore extends IKVStore[MapManipulator] {
  override def get(key: String): MapManipulator[Option[String]] = 
    map => (map.get(key), map)

  override def remove(key: String): MapManipulator[Unit] = 
    map => ((), map.remove(key))

  override def update(key: String, data: String): MapManipulator[Option[String]] = 
    map => ((), map.update(key, data))
}

// class RedisKVStore extends IKVStore[IO] {
// ...
// }

// class LogstashLogger extends ILog[IO] {
// ...
// }

class IO[A] {
  def flatMap[B](fun: (B => IO[B])): IO[B] = ???
  def unsafePerformIO: A = ???
}

object IO {
  def now[A](value: A): IO[A] = ???
  def eval[A](fun: () => A): IO[A] = ???
}

So what's the problem?

• We now have a way to describe effectful objects that satisfy an interface
• Now we need a way to abstract over the effect chosen by the object
• We need a way to write a program in a language using only that interface's methods as primitives, so that the implementation can choose the effect later (dependency injection)

What's the F[_]uss about?

• At first it seems all we've done is add more precise types to our interfaces
• But if we constrain the F[_], we can abstract over it

What's the F[_]uss about?

• Both of the F[_]s so far have two functions in common:

pure(a: A): F[A];

bind(fa: F[A], fun: A => F[B]): F[B];

• This is found in the functional programming lexicon as the Monad typeclass, which expresses sequential composition

Building programs that use these interfaces

object ExtraKVStoreOps {
  def getOrUpdate[F[_]: Monad](store: IKVStore[F], 
                               key: String, newValue: String): F[Option[String]] =
    for {
      currentValue <- store.get(key)
      updated <- currentValue match {
        if (optionalResult.isDefined)
          Monad[F].pure(optionalResult)
        else
          store.update(key, newValue)
      }
    } yield updated
}

Our new problem: F<_>

• Most of the time, when we write a function that uses an object, we don't care about which effect it will eventually use. The type parameters will eventually explode in number

• This calls for a form of dependency injection that deals with these effects, so that we can concern ourselves with interfaces and delay choosing the effects they're implemented with until later

Encoding an interface without fixing an effect

• To encode this interface without its effects, we can do so algebraically
• For an example, see part of the algebra for ILog:

trait ILogAlgebra[A] {
   def act[F[_]: Monad](logger: ILog[F]): F[A]
}

case class Info(info: String) extends ILogAlgebra[Unit] {
  override def act[F[_]: Monad](logger: ILog[F]): F[Unit] =
    logger.info(this.info);
}

Any monad at all

• How can we encode the fact that we don't care which monad we will transform to later?

• This shows that the Free monad requires an F<_> (the algebra), but does not require it to be a monad; only for it to be possible to transform it into a monad G<_> later (the interpreted effect)

trait Transformation[F[_], G[_]] {
  def transform[A](input: F[A]): G[A]
}

class FreeMonad[F[_], A] {
  def run[G[_] : Monad](trans: Transformation[F, G]): G[A] = ???
}

object FreeMonad {
  def pure[F[_], A](a: A): FreeMonad[F[_], A] = ???
  def ofAlgebra[F[_], A](alg: F[A]): FreeMonad[F[_], A] = ???
}

Using this with our algebra

• These transformations the Free monad requires look exactly the same as our interfaces ILog and IKVStore from earlier

def toLogInterpreter[F[_]](log: ILog[F]): Transformation[ILogAlgebra, F] = 
  new Transformation[ILogAlgebra, F] { 
    override def transform[A](input: ILogAlgebra[A]): F[A] = input.act(log)
  }

def runLogProgram[F[_], A](program: FreeMonad[ILogAlgebra, A], 
                           log: ILog[F]): F[A] =
  program.run(toLogInterpreter(log))

ASTs for free

• Let's look at the kind of program that can be written using this technique, using the example from earlier:

def getOrUpdateFree(key: String, 
                    newValue: String): FreeMonad[IKVStoreAlgebra, Option[String]] =
  for {
    currentValue <- FreeMonad.ofAlgebra(Get(key))
    updated <- currentValue match {
      case Some(valueExists) => FreeMonad.pure(valueExists)
      case None => FreeMonad.ofAlgebra(Update(key, newValue))
    }
  } yield updated

(note we got rid of the F[_] type parameter and IKVStore[F] parameter)

Composing algebras

• Extracting our interfaces into algebras lets us write programs using the coproduct of the algebras:

case class Coproduct[L[_], R[_], A](run: Either[L[A], R[A]])

object Coproduct {
  def fixParams[L[_], R[_]] = new {
    type CoproductLR[A] = Coproduct[L, R, A]
    def left[A](la: L[A]): CoproductLR[A] = Coproduct(Left(la))
    def right[A](ra: R[A]): CoproductLR[A] = Coproduct(Right(ra))
  }
}

type LogOrStoreAlgebra[A] = Coproduct[ILogAlgebra, IKVStoreAlgebra, A]
val LogOrStoreAlgebra = Coproduct.fixParams[ILogAlgebra, IKVStoreAlgebra]

def getAndLogToInfo(key: String): FreeMonad[LogOrStoreAlgebra, Option[String]] =
  for {
    currentValue <- FreeMonad.ofAlgebra(LogOrStoreAlgebra.right(Get(key)))
    logMessage = s"got key $key with value $currentValue"
    _ <- FreeMonad.ofAlgebra(LogOrStoreAlgebra.left(Info(logMessage)))
  } yield currentValue

Sources of ugliness

• There are several syntactic and otherwise issues with this style of programming as presented here
• Algebras are boilerplate, hard-to-infer types, Coproduct only works with two algebras
• Fortunately most are solvable

Solutions

• Algebras generable by macros
• Coproduct easy to generalize to more types
• Types still not too easy to infer

Final product

• Algebras generable by macros
• Coproduct easy to generalize to more types
• Types still not too easy to infer

Solutions

• Algebras generable by macros
• Coproduct easy to generalize to more types
• Types still not too easy to infer

End result

type MyEffects = Fx.fx2[IKVStoreAlgebra, ILogAlgebra]

def removeKeyAtKey(key: String): Eff[MyEffects, Option[String]] = for {
  keyToRemove <- get[MyEffects](key)
  _ <- info[MyEffects]("removing key: " + keyToRemove)
  _ <- keyToRemove.traverseA(remove[MyEffects])
} yield keyToRemove

Le fin

What are programs made of?

• Instructions
• Combinators
• What is the difference?

How can we formalize this to build a DSL library?

• Provide instructions and combinators