Dotty

A next generation Scala compiler

Singapore Scala Meetup - 12 April 2017

About Me

  • Senior Software Engineer at Citadel Technology Solutions

 

  • Currently working in:
    • Scala
    • Kotlin

 

  • The eternal student :)
    • Elixir
    • Elm
  • Find me on the internet:

_hhandoko

hhandoko

hhandoko

hhandoko.com

Motivation

Source:

https://pbs.twimg.com/media/CthyVVWWcAAIq-a.jpg:large

Scala Matsuri1

25 - 26 February 2017

Tokyo, Japan

Talks2:

  • Dotty
  • Free monads
  • Scala native
  • Scala.js
  • and many, many more...

Notes:

[1] - http://2017.scalamatsuri.org/index_en.html

[2] - https://www.youtube.com/channel/UC6DFHglgVXQUVDbUmQHtxDQ

@scala_lang1

1 RT = $3

1 fav = $1

... but seriously, people are excited about Dotty!

Notes:

[1] - https://twitter.com/_hhandoko/status/835764141843333121

Agenda

Source:

https://pbs.twimg.com/media/Ctd6sdMWYAEaBGX.jpg:large

Agenda

DOT and Dotty

 

Dotty Developer Tooling

 

Dropped, Implemented, and Proposed Features

 

Release and Timeline

 

Get Involved!

DOT

Source:

https://pbs.twimg.com/media/CprNb0PWEAAFS3F.jpg

DOT

  • (D)ependent (O)bject (T)ypes

 

  • A calculus of dependent object types that underlines Scala.

 

  • Why?1,2
    • Ensuring type soundness
    • Feedback loop for language design
    • Detect hidden connection between language features
    • Foundations are formative

Notes:

[1] - http://www.scala-lang.org/blog/2016/02/17/scaling-dot-soundness.html

[2] - https://www.youtube.com/channel/UC6DFHglgVXQUVDbUmQHtxDQ

DOT1 (cont'd)

  • vObj calculus (Odersky et al., 2003)
    • Rich type language, but unwieldly in practice
    • Subtyping lacks lower or upper bounds

 

  • Featherweight Scala (Cremet et al., 2006)
    • Attempt at a calculus with decidable type checking
  • Scalina (Moors et al., 2008)
    • Formal underpinning for introducing higher-kinded types in Scala

 

 

"A common theme among earlier systems is their complexity, which makes them hard to reason about and extend with new features."

Notes:

[1] - https://infoscience.epfl.ch/record/215280

DOT1 (cont'd)

  • Formalises System D <:, a simple calculus for path-dependent types

 

  • D <: becomes a basis for a full programming language through three extensions:
    • Records
    • Type labels
    • Recursion

Notes:

[1] - https://infoscience.epfl.ch/record/215280

DOT1 (cont'd)

  • Several extensions of DOT being studied with the aim of bridging the gaps between the calculus and the programming language:
    • Type parameters
    • Variance
    • Traits
    • Classes
    • Inheritance

Notes:

[1] - https://infoscience.epfl.ch/record/215280

Dotty

Source:

https://pbs.twimg.com/media/CfSQdwUW8AErog1.jpg

Dotty1

  • New Scala compiler developed at LAMP, EPFL

 

  • Based on DOT as a theoretical foundation

 

  • Completely redesigned internals, familiar externals
  • Its code base is almost half the size of the current nsc compiler and promises to be twice as fast

Notes:

[1] - https://www.lightbend.com/company/news/after-a-quiet-2015-martin-odersky-outlined-significant-plans-for-scala-at-scala-days-new-york

Dotty1 (cont'd)

Notes:

[1] - https://www.infoq.com/news/2016/08/scala-development-is-heating-up

TASTY

  • Typed AST serialization format
    • Introduced in scala.meta
    • Persist AST at compile time after typechecking, and load them at runtime

 

  • One of the main targets is to become an interchange formats between the language implementing it1

 

  • Aims2,3,4:
    • Compact
    • Lazy
    • Extensible
    • Precise

Notes:

[1] - https://github.com/VladimirNik/tasty

[2] - https://infoscience.epfl.ch/record/226194

[3] - https://docs.google.com/document/d/1Wp86JKpRxyWTqUU39H40ZdXOlacTNs20aTj7anZLQDw

[4] - https://docs.google.com/document/d/1h3KUMxsSSjyze05VecJGQ5H2yh7fNADtIf3chD3_wr0

scala.meta1

  • Why?2
    • Simpler
    • Fewer implementation dependencies
    • Safer, since interpretation allows sandboxing
    • Restrict syntactic freedom, since no whitebox macros3

Notes:

[1] - http://scalameta.org/

[2] - https://www.slideshare.net/Odersky/scala-days-nyc-2016

[3] - http://docs.scala-lang.org/overviews/macros/blackbox-whitebox.html

inline def m(inline x: Int, y: Float): Float =
  meta { ... }

scala.meta1 (cont'd)

Notes:

[1] - https://www.scala-lang.org/resources/img/scalameta-sketch.jpg

Dotty Linker

  • Whole program optimiser

 

  • Making your Scala application smaller and faster
    • Leverages TASTY
    • Call-graph analysis
  • What can it do?1,2
    • Smart specialization
    • Smart dead code elimination
    • Convert classes to value classes
    • Infer more precise types
    • Eliminate virtual dispatch
    • Replace vars by vals
    • Remove duplicate vals

Notes:

[1] - https://d-d.me/talks/scaladays2015/

[2] - https://d-d.me/talks/scaladays2016-linker/#/5

Dotty Developer Tooling

Source:

https://pbs.twimg.com/media/CgGkN6YVAAAZ3_1.jpg

Getting Started

  • Scastie1, pastebin for Scala (GitHub OAuth)

 

  • sbt-dotty2 SBT plugin

 

  • Dotty SDK (nightly) in Scala's IntelliJ IDEA3 plugin

Notes:

[1] - https://scastie.scala-lang.org/

[2] - https://github.com/lampepfl/sbt-dotty

[3] - https://www.jetbrains.com/help/idea/2017.1/getting-started-with-dotty.html

IDE Support

  • Future work1:
    • Easy startup
    • Automated testing
    • Optimisations
    • Build tool integration

Notes:

[1] - http://guillaume.martres.me/project1_report.pdf

[2] - https://github.com/Microsoft/language-server-protocol

[3] - https://github.com/eclipse/lsp4j

REPL

  • Simple REPL with syntax highlighting

 

  • Awesome error messages1
    • Verbose explanation via -explain
    • Mistyping members
    • Type diff

Notes:

[1] - http://www.scala-lang.org/blog/2016/10/14/dotty-errors.html

Dottydoc

  • A new code documentation generator1,2

 

  • Started as a Google Summer of Code (GSoC) 2016 project1

 

  • Part of this year's Scala's GSoC projects3
  • Similar to Ruby's Jekyll4
    • Mustache templates
    • Markdown rendering
    • Ad-hoc pages
    • YAML sidebar

Notes:

[1] - https://summerofcode.withgoogle.com/archive/2016/projects/5280919451598848/

[2] - https://github.com/lampepfl/dotty/pull/1453

[3] - https://www.scala-lang.org/gsoc/2017.html

[4] - http://dotty.epfl.ch/docs/usage/dottydoc.html

Scalafix

  • Scala code rewriting tool and library1

 

  • Helps to automate migration to future Scala versions (incl. 2.13+ and Dotty)2

 

  • Heavy-lifting done by scala.meta

Notes:

[1] - https://scalacenter.github.io/scalafix/

[2] - http://www.scala-lang.org/blog/2016/10/24/scalafix.html

Dropped Features

Source:

https://pbs.twimg.com/media/ChJzv9lUYAA9D5E.jpg:large

(Scala 2) Macros1

  • Scala 2 macros support will be dropped

 

package scalaworld.macros

import scala.meta._

class Main extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    defn match {
      case q"object $name { ..$stats }" =>
        val main = q"def main(args: Array[String]): Unit = { ..$stats }"
        q"object $name { $main }"
      case _ =>
        abort("@main must annotate an object.")
    }
  }
}
  • scala.meta has support for new-style macro annotations

Notes:

[1] - http://scalameta.org/tutorial/

Procedural Syntax1

  • Procedural syntax support is no longer supported

 

 

  • Use Unit returning method signature instead
object Test {
  def foo() { // procedure syntax
    ???
  }
}

Notes:

[1] - https://d-d.me/talks/scalaworld2015/

object Test {
  def foo(): Unit = {
    ???
  }
}

forSome1,2

  • forSome is no longer supported

 

  • Use wildcards instead

Notes:

[1] - http://www.cakesolutions.net/teamblogs/2012/09/03/existential-types

[2] - https://www.reddit.com/r/scala/comments/4o36r1/what_are_the_simplifications_that_come_with_dotty/

def foo(as: Seq[A[X]] forSome { type X })
def foo(as: Seq[A[_]])

Implemented Features

Source:

https://pbs.twimg.com/media/Cf7eHZ1W4AEeZJA.jpg:large

Function Arity Adaption

  • Function with very many parameters (> 22)1
    • Uses FunctionXXL2 which takes an array of params
    • Removing Tuple limitations is in the works

Notes:

[1] - https://github.com/lampepfl/dotty/pull/1758

[2] - https://github.com/lampepfl/dotty/blob/master/library/src/scala/FunctionXXL.scala

package scala

/** A function with all parameters grouped in an array. */
trait FunctionXXL {

  def apply(xs: Array[Object]): Object

  override def toString() = "<functionXXL>"
}

Intersection & Union Types1

  • Intersection types (A & B)

 

 

  • Union types (A | B)
    • Scala 2 will resolve to Any
    • Dotty will resolve to its union type, e.g. Int | String
trait A { def foo: Int }
trait B { def bar: Int }

def baz(ab: A & B) = ab.foo + ab.bar
trait HasX { def x: Int }

case class A(x: Int) extends HasX
case class B(x: Int, y: Int) extends HasX

def getX(ab: A | B): Int = ab.x

Notes:

[1] - http://felixmulder.com/talks/deconstructing-dotty

Trait Parameters

  • Trait parameters (SIP-251)
    • Scala 2 workaround using early definitions2
    • Dotty allows trait params

Notes:

[1] - http://docs.scala-lang.org/sips/pending/trait-parameters.html

[2] - http://felixmulder.com/talks/deconstructing-dotty

trait Super {
  def x: String
  println(x)
}
class Foo extends Super {
  val x = "hello"
}

new Foo // prints: null
trait Super(x: String) {
  println(x)
}
class Foo extends Super("hello")

new Foo // prints: "hello"

Others

  • Fast compilation (phased fusion)1,2

 

  • Non-blocking lazy val (SIP203)

 

  • @static methods and fields4

 

  • Multiversal Equality5
  • Literal Singleton Types (SIP-236)

 

  • Pattern matching improvements:
    • Option-less pattern matching7
    • Exhaustivity checks

Notes:

[1] - http://felixmulder.com/talks/the-new-scala-dev-exp/#/18

[2] - https://d-d.me/talks/scalaworld2015/#/62

[3] - http://docs.scala-lang.org/sips/pending/improved-lazy-val-initialization.html

[4] - https://github.com/scala/scala.github.com/pull/491

[5] - http://www.scala-lang.org/blog/2016/05/06/multiversal-equality.html

[6] - http://docs.scala-lang.org/sips/pending/42.type.html

[7] - https://github.com/scala/scala/pull/2848

Proposed Features

Source:

https://s-media-cache-ak0.pinimg.com/564x/2c/4c/9b/2c4c9b3070518409dd3bb60866376e94.jpg

Enums1

  • Syntactic sugar with sealed trait and case classes

 

  • Compiler will enforce necessary restrictions for interop with Java Enums.

Notes:

[1] - https://github.com/lampepfl/dotty/issues/1970

enum Color { case Red, Green, Blue }
sealed abstract class Color extends scala.Enum
object Color {
  private val $values = new scala.runtime.EnumValues[Color]
  def enumValue: Map[Int, Color] = $values.fromInt
  def enumValueNamed: Map[String, Color] = $values.fromName
  def enumValues: Iterable[Color] = $values.values

  def $new(tag: Int, name: String): Color = new Color {
    def enumTag: Int = tag
    override def toString: String = name
    $values.register(this)
  }

  final case val Red: Color = $new(0, "Red")
  final case val Green: Color = $new(1, "Green")
  final case val Blue: Color = $new(2, "Blue")
}

Non-nullable Types

  • Why?2
    • Type soundness
    • Conceptually easy once we have union types and effects

Notes:

[1] - https://github.com/lampepfl/dotty/issues/1495

[2] - https://www.slideshare.net/Odersky/scala-days-nyc-2016

  • Model nullable types as union types (T? = T | Null)
    • Currently null has a type of Null, which is a subtype of every non-value class1
    • Null should be a subtype of no other type1

Others

  • Better records1

 

  • Effect system (with implicit)1,2

 

  • Generic programming support1,3

 

  • Implicit Functions1,4

Notes:

[1] - https://www.slideshare.net/Odersky/scala-days-nyc-2016

[2] - https://www.reddit.com/r/scala/comments/2zff6d/effects_with_implicit_how_would_that_work/

[3] - https://github.com/lampepfl/dotty/issues/1347

[4] - https://www.scala-lang.org/blog/2016/12/07/implicit-function-types.html

Release and Timeline

Source:

https://pbs.twimg.com/media/Cg-zWsMWwAEJ-Zn.jpg:large

Release and Timeline1

Notes:

[1] - https://www.slideshare.net/Odersky/from-dot-to-dotty

Get Involved

Source:

https://pbs.twimg.com/media/Cp6WqQdWAAEpvFt.jpg:large

Get Involved

  • Dotty is on GitHub!

 

  • Check "Open Issues" with "Help Wanted" tag
    • Docs
    • Dottydoc
    • Infrastructure
    • Reporting
    • REPL

Source:

https://github.com/lampepfl/dotty/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22

The End

Source:

http://i.imgur.com/simMMnz.png

Dotty

By Herdy Handoko

Dotty

Presentation on Scala's Dotty, prepared for Singapore Scala Meetup on 12 April 2017. Link: https://engineers.sg/video/dotty-singapore-scala-programmers--1653

  • 4,225