Meta-program and/or shapeless all the things!

Chris Birchall

ScalaMatsuri

25th Feb 2017

We're hiring!

Programming

Program

Data

Side effects

副作用

データ

プログラム

Metaprogramming

Program

Side effects

Program

プログラム

副作用

プログラム

Scala macro

Function that runs at compile time
Input and output are trees (ASTs)

foo.bar()

Apply

Select

Ident

TermName("foo")

TermName("bar")

List()

=

コンパイル時に呼ばれる関数

入力・出力は抽象構文木

Hello World

Let's write a macro that doesn't do anything!

何もしないマクロを書いてみよう！

Level up

Pattern matching and code generation using quasiquotes

準クォートによるパターンマッチとコード生成

"Stop using macros because they are going away.

I'm not kidding!"

Adriaan Moors @ Scala eXchange 2016

マクロをやめなさい。そのうちなくなるからね。

うそじゃないよ！

scala.meta

Syntactic API

Tokens
Syntax trees
Quasiquotes
Available now

Semantic API

Name resolution
Typechecking
Work in progress
- see #604 for details
- try it in v1.6.0

scala> import scala.meta._
import scala.meta._

scala> val tree = q"val x = 1"
tree: meta.Defn.Val = val x = 1

scala> tree.show[Structure]
res0: String = Defn.Val(Nil, Seq(Pat.Var.Term(Term.Name("x"))), ...

リリース済み

実装中

名前解決

型検査

準クォート

構文木

トークン

meta paradise

inline/meta

inline keyword replaces @inline annotation
meta demarcates code that executes at compile time
See SIP-28 & 29 for the gory details

inline は既存の @inline アノテーションの代わり
meta はコンパイル時に走るコードを示す
超マニアックな詳細は SIP-28, SIP-29 へ

Typechecked Mustache

Hello {{firstName}}!
The weather is {{weather}} today.

case class Context(firstName: String, weather: String)

goal

Typechecked Mustache

Load a Mustache template
Parse it to find all the {{placeholders}}
Generate a corresponding case class

該当する case class を生成する

テンプレートを読み込む

パースして変数名を列挙する

At compile time:

tutorial

http://scalameta.org/tutorial/

Spectrum of Scala developer happiness

joy

misery

大吉

大凶

Spectrum of Scala developer happiness

joy

misery

write some boilerplate code

大吉

大凶

ボイラープレートコードを書く

Spectrum of Scala developer happiness

joy

misery

write some boilerplate code

use runtime reflection

大吉

大凶

ランタイム・リフレクションを使う

Spectrum of Scala developer happiness

joy

misery

write some boilerplate code

use runtime reflection

generate source code

大吉

大凶

ソースコード生成

Spectrum of Scala developer happiness

joy

misery

write some boilerplate code

use runtime reflection

generate source code

write a macro

大吉

大凶

マクロを書く

Spectrum of Scala developer happiness

joy

misery

write some boilerplate code

use runtime reflection

generate source code

write a macro

let Miles write the macro for you

大吉

大凶

マクロをMilesさんに書いてもらう

shapeless

case class Person(name: String, age: Int)

String :: Int :: HNil

("name", String) :: ("age", Int) :: HNil

Generic

LabelledGeneric

Convert case class to Map

goal

case class Foo(wow: String, yeah: Int)

val instance = Foo("hello", 123)

val map: Map[String, Any] = caseClassToMap(instance)

println(map) // Map(wow -> hello, yeah -> 123)

Convert case class to Map

with a macro

import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context

def caseClassToMap[A](a: A): Map[String, Any] =
  macro impl[A]

def impl[A: c.WeakTypeTag](c: Context)(a: c.Tree): c.Tree = {
  import c.universe._
  val A = weakTypeOf[A]
  if (!(A.typeSymbol.isClass && A.typeSymbol.asClass.isCaseClass))
    c.abort(c.enclosingPosition, "Sorry, case classes only")
  val primaryCtor = A.typeSymbol.asClass.primaryConstructor.asMethod
  val params = primaryCtor.paramLists.flatten
  val kvPairs = params.map { k =>
    q"${k.name.toString} -> $a.${k.name.toTermName}"
  }
  q"""_root_.scala.collection.immutable.Map($kvPairs: _*)"""
}

Convert case class to Map

with shapeless

import shapeless._
import shapeless.ops.hlist.ToList
import shapeless.ops.record.Fields

def caseClassToMap[A, L <: HList, F <: HList](a: A)
  (implicit 
    generic: LabelledGeneric.Aux[A, L],
    fields: Fields.Aux[L, F],
    toList: ToList[F, (Symbol, Any)]
  ): Map[String, Any] = {
  val labelledGen = generic.to(a)
  val fieldsHlist = fields(labelledGen)
  toList(fieldsHlist)
    .map { case (symbol, value) => (symbol.name, value) }
    .toMap
}

Convert one case class to another

goal

case class Input(foo: Int, baz: String)
case class Output(foo: Int, bar: Double, baz: String)

val input = Input(123, "wow")

val output = transform[Output](input, "bar" -> 4.56)

println(output) // Output(123, 4.56, wow)

Convert one case class to another

with a macro

def caseClassToCaseClass[A, B](a: A, extraParams: (String, Any)*): B =
  macro Bundle.impl[A, B]

class Bundle(val c: Context) {
  import c.universe._

  private case class TreeWithActualType(tree: Tree, actualType: Type)

  private def fail(msg: String) =
    c.abort(c.enclosingPosition, msg)

  def impl[A: c.WeakTypeTag, B: c.WeakTypeTag](a: c.Tree, extraParams: c.Expr[Tuple2[String, Any]]*): c.Tree = {
    val A = weakTypeOf[A]
    val B = weakTypeOf[B]

    if (!(A.typeSymbol.isClass && A.typeSymbol.asClass.isCaseClass))
      fail("Sorry, case classes only")
    if (!(B.typeSymbol.isClass && B.typeSymbol.asClass.isCaseClass))
      fail("Sorry, case classes only")

    val inputPrimaryCtor = A.typeSymbol.asClass.primaryConstructor.asMethod
    val inputParams = inputPrimaryCtor.paramLists.flatten
    val inputParamsMap: Map[TermName, TreeWithActualType] = inputParams.map { k =>
      val termName = k.name.toTermName
      termName -> TreeWithActualType(q"$a.$termName", k.info)
    }.toMap

    val extraParamsMap: Map[TermName, TreeWithActualType] = extraParams.map { expr =>
      val (key, value) = expr.tree match {
        case q"scala.Predef.ArrowAssoc[$_]($k).->[$_]($v)" => (k, v)
        case q"($k, $v)" => (k, v)
        case other => fail("You must pass extra params as either key -> value or (key, value)")
      }
      val q"${keyAsString: String}" = key
      val keyName = TermName(keyAsString)
      val actualValueType = expr.actualType.typeArgs(1)
      keyName -> TreeWithActualType(value.asInstanceOf[Tree], actualValueType)
    }.toMap

    val allParams: Map[TermName, TreeWithActualType] = inputParamsMap ++ extraParamsMap

    val outputPrimaryCtor = B.typeSymbol.asClass.primaryConstructor.asMethod
    val paramLists: List[List[Tree]] = 
      for (ps <- outputPrimaryCtor.paramLists) yield {
        for (p <- ps) yield {
          val termName = p.name.toTermName
          allParams.get(termName) match {
            case Some(t) if t.actualType weak_<:< p.typeSignature => t.tree
            case Some(t) => fail(s"Parameter ${termName.toString} has wrong type. Expected ${p.typeSignature} but got ${t.actualType}")
            case None => fail(s"Missing parameter of type ${termName.toString}")
          }
        }
      }

    q"new ${B.typeSymbol}(...$paramLists)"
  }
}

Convert one case class to another

with shapeless

import shapeless._
import shapeless.ops.hlist._

trait Transform[A, B, E <: HList] {
  def apply(a: A, extraFields: E): B
}

object Transform {
  implicit def genericTransform[
    A,
    B,
    ARepr         <: HList,
    BRepr         <: HList,
    CommonFields  <: HList,
    ExtraFields   <: HList,
    Unaligned     <: HList
  ](implicit
    aGen    : LabelledGeneric.Aux[A, ARepr],
    bGen    : LabelledGeneric.Aux[B, BRepr],
    inter   : Intersection.Aux[ARepr, BRepr, CommonFields],
    diff    : Diff.Aux[BRepr, CommonFields, ExtraFields],
    prepend : Prepend.Aux[ExtraFields, CommonFields, Unaligned],
    align   : Align[Unaligned, BRepr]
  ): Transform[A, B, ExtraFields] =
    new Transform[A, B, ExtraFields] {
      def apply(a: A, extra: ExtraFields): B = {
        val aRepr     = aGen.to(a)
        val common    = inter(aRepr)
        val unaligned = prepend(extra, common)
        val bRepr     = align(unaligned)
        bGen.from(bRepr)
      }
    }
}

learning shapeless

https://github.com/underscoreio/shapeless-guide

Also see Dave's talk at London Scala

Summary

Macros are fun and useful
Do the scala.meta tutorial
Read the shapeless book

Code

https://github.com/cb372/meta-prog-shapeless-talk

マクロは楽しくて役に立つ
scala.meta チュートリアルをやってみよう
shapeless 本を読もう

Slides

https://slides.com/cb372/metaprog-shapeless-matsuri-2017

Meta-program and/or shapeless all the things!

We're hiring!

Programming

Metaprogramming

Scala macro

Function that runs at compile time

Input and output are trees (ASTs)

=

Hello World

Level up

scala.meta

Syntactic API

Semantic API

meta paradise

inline/meta

Typechecked Mustache

goal

Typechecked Mustache

At compile time:

tutorial

http://scalameta.org/tutorial/

Spectrum of Scala developer happiness

Spectrum of Scala developer happiness

Spectrum of Scala developer happiness

Spectrum of Scala developer happiness

Spectrum of Scala developer happiness

Spectrum of Scala developer happiness

shapeless

Convert case class to Map

goal

Convert case class to Map

with a macro

Convert case class to Map

with shapeless

Convert one case class to another

goal

Convert one case class to another

with a macro

Convert one case class to another

with shapeless

learning shapeless

Summary