Compiler-First Development:

A Fresh Take on Scala in IntelliJ

- Scala3 Compiler Engineer

- IntelliJ Plugin Developer

- Scala.js Frontend Developer

- Tooling Engineer

- Backend Developer

I work on tooling.

Scastie

Metals

IntelliJ

Thank you!

But this is not your average project

Right ?

import quoted.*

class Config(map: Map[String, Any]) extends Selectable:
  def selectDynamic(name: String): Any = map(name)

transparent inline def typesafeConfig(inline pairs: (String, Any)*) = 
  ${ typesafeConfigImpl('pairs) }

def typesafeConfigImpl(pairs: Expr[Seq[(String, Any)]])(using Quotes): Expr[Any] =
  import quotes.reflect.*
  val unpacked = Varargs.unapply(pairs).getOrElse(Nil).map:
    case '{ ($k: String) -> ($v: t) } => k.valueOrError -> v

  val typ = unpacked.foldLeft(TypeRepr.of[Config]): (acc, entry) =>
    Refinement(acc, entry._1, entry._2.asTerm.tpe.widen)

  val params = unpacked.map: (k, v) =>
    '{ ${Expr(k)} -> $v }
  typ.asType match
    case '[t] => '{ Config(Map(${Varargs(params)}: _*)).asInstanceOf[t] }

MACROS

🥁

Metals

IntelliJ

LSP with the rescue

But why did IntelliJ fail ?

Compilation 101*

* The following slides are simplification

Parser

Typer

PostTyper

SetRootTree

Frontend

Pickler

Transform

Backend

val godfatherSeries: List[String] = ???
godfatherSeries.head

Parser

Ident

"godfatherSeries"

val godfatherSeries: List[String] = ???
godfatherSeries.head

Parser

godfatherSeries

Select

"head"

Ident

"godfatherSeries"

val godfatherSeries: List[String] = ???
godfatherSeries.head

Parser

godfatherSeries.head

Parser

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

Select

"get"

Ident

"godfatherSeries"

godfatherSeries.get

Select

"get"

Ident

Apply

Literal

Parser

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

godfatherSeries.get(0)

Select

"get"

Ident

Apply

Literal

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

Select

"get"

Ident

Apply

Literal

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(...)

Select

"get"

Ident

Apply

Literal

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(...)

?.get

Select

"get"

Ident: godfatherSeries.type

Apply

Literal

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(...)

?.get

Select

"get"

Ident: godfatherSeries.type

Apply

Literal

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(...)

Select: godfatherSeries.get.type

"get"

Ident: godfatherSeries.type

Apply

Literal

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(...)

Select: godfatherSeries.get.type

"get"

Ident: godfatherSeries.type

Apply

Literal: 0.type

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(...)

Select: godfatherSeries.get.type

"get"

Ident: godfatherSeries.type

Apply

Literal: 0.type

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(Int)

Select: godfatherSeries.get.type (Int => T)

"get"

Ident: godfatherSeries.type

Apply

Literal: 0.type

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(Int)

Select: godfatherSeries.get.type (Int => String)

"get"

Ident: godfatherSeries.type

Apply

Literal: 0.type

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

?(Int)

Select: godfatherSeries.get.type

"get"

Ident: godfatherSeries.type

Apply: String

Literal: 0.type

Typer

val godfatherSeries: List[String] = ???
godfatherSeries.get(0)

"godfatherSeries"

Abstract Syntax Trees

Select

"get"

Ident

Apply

Literal

"godfatherSeries"

ReferenceExpression

"get"

ReferenceExpression

MethodCall

ArgumentList

"godfatherSeries"

PsiElement(.)

PSI - Program Structure Interface

AST - Abstract Syntax Tree

Tool	Parser	Type Checker
Python Language Server	built-in 'ast' module	custom based on type hints or heuristics
Rust analyzer	custom incremental parsing	custom type inference engine
TS language server	custom parser in the compiler	compiler's built-in type checker
Gopls	standard parser from Go stdlib	go type checker from Go stdlib
Kotlin	compiler parser	compiler's type checker
IntelliJ Scala Plugin	custom parser	custom type inference engine
IntelliJ IDEA Java	custom parser	custom type inference engine
Scala Metals	compiler parser + scalameta parser	compiler's type checker

This is not bad

It's just a tradeoff

Error recovery

Correctness

Error recovery

Correctness

Compiler diagnostics + Types from compiler

Error recovery

Correctness

Compiler diagnostics + Types from compiler

Error recovery

Correctness

BTasty format

Error recovery

Correctness

BTasty format

But what If Metals could help IntelliJ and IntelliJ could help Metals ?

Not all features require typer

But those that do, should rely on true representation

Completions

Hovers

Signature help

And others are more reliant on indices

Go-to definition

Find usages

I did a thing

But this has a very big overhead

Resource usage

Completions

public abstract CompletableFuture<CompletionList> complete(OffsetParams params);

Hovers

public abstract CompletableFuture<Optional<HoverSignature>> hover(OffsetParams params);

Signature help

public abstract CompletableFuture<CompletionList> complete(OffsetParams params);

Go-to definition

public abstract CompletableFuture<SignatureHelpResult> signatureHelp(OffsetParams params);

IntelliJ plugin creation 101

build.sbt

import org.jetbrains.sbtidea.IntelliJPlatform.IdeaCommunity

ThisBuild / intellijPluginName := "intellij-metals"
ThisBuild / intellijPlatform := IdeaCommunity
ThisBuild / intellijBuild := "241.14494.240"

lazy val root = project
  .enablePlugins(SbtIdeaPlugin)
  .settings(
    name := "intellij-next",
    scalaVersion := 3.6.4,
    intellijPlugins := Seq(
      "com.intellij.java".toPlugin,
      "org.intellij.scala".toPlugin
    )
  )

build.sbt

import org.jetbrains.sbtidea.IntelliJPlatform.IdeaCommunity

ThisBuild / intellijPluginName := "intellij-metals"
ThisBuild / intellijPlatform := IdeaCommunity
ThisBuild / intellijBuild := "241.14494.240"

lazy val root = project
  .enablePlugins(SbtIdeaPlugin)
  .settings(
    name := "intellij-next",
    scalaVersion := 3.6.4,
    intellijPlugins := Seq(
      "com.intellij.java".toPlugin,
      "org.intellij.scala".toPlugin
    ),
    libraryDependencies += 
      "org.scala-lang" % "scala3-presentation-compiler_3" % "3.6.4",
  )

Built-in IntelliJ LSP support

build.sbt

import org.jetbrains.sbtidea.IntelliJPlatform.IdeaCommunity

ThisBuild / intellijPluginName := "intellij-metals"
ThisBuild / intellijPlatform := IdeaCommunity
ThisBuild / intellijBuild := "241.14494.240"

lazy val root = project
  .enablePlugins(SbtIdeaPlugin)
  .settings(
    name := "intellij-next",
    scalaVersion := 3.6.4,
    intellijPlugins := Seq(
      "com.intellij.java".toPlugin,
      "org.intellij.scala".toPlugin,      
      "com.redhat.devtools.lsp4ij".toPlugin
    ),
    libraryDependencies += 
      "org.scala-lang" % "scala3-presentation-compiler_3" % "3.6.4",
  )

<idea-plugin>
    <id>intellij-metals</id>
    <name>Scala LSP (Metals) for IntelliJ</name>
    <version>replaced-by-build</version>

    <idea-version since-build="241" until-build="241.*"/>

    <depends>com.intellij.modules.platform</depends>
    <depends>org.intellij.scala</depends>
    <depends>com.redhat.devtools.lsp4ij</depends>

    <extensions defaultExtensionNs="com.intellij">

    </extensions>

</idea-plugin>

plugin.xml

<idea-plugin>
    <id>intellij-metals</id>
    <name>Scala LSP (Metals) for IntelliJ</name>
    <version>replaced-by-build</version>

    <idea-version since-build="241" until-build="241.*"/>

    <depends>com.intellij.modules.platform</depends>
    <depends>org.intellij.scala</depends>
    <depends>com.redhat.devtools.lsp4ij</depends>

    <extensions defaultExtensionNs="com.intellij">

    </extensions>
  
    <extensions defaultExtensionNs="com.redhat.devtools.lsp4ij">
        <server id="ScalaLanguageServer"
                name="Scala Language Server"
                factoryClass="intelli.scala.next.ScalaLanguageServerFactory">
        </server>

        <languageMapping language="Scala" serverId="ScalaLanguageServer"/>
        <languageMapping language="Scala" 
                         serverId="ScalaLanguageServer" languageId="scala"/>
    </extensions>

</idea-plugin>

plugin.xml

final class ScalaLanguageServer(project: Project) 
  extends lsp4j.LanguageServer 
  with lsp4j.TextDocumentService 
  with lsp4j.WorkspaceService 
  with lsp4j.LanguageClientAware { }

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = ???

    override def exit(): Unit = ???

    override def getTextDocumentService: TextDocumentService = ???

    override def getWorkspaceService: WorkspaceService = ???

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = ???

    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit = ???

    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit = ???

    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = ???
 
    override def didChangeConfiguration(didChangeConfigurationParams: DidChangeConfigurationParams): Unit = ???
 
    override def didChangeWatchedFiles(didChangeWatchedFilesParams: DidChangeWatchedFilesParams): Unit = ???

    override def connect(languageClient: LanguageClient): Unit = ???

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = ???

    override def exit(): Unit = ()

    override def getTextDocumentService: TextDocumentService = this

    override def getWorkspaceService: WorkspaceService = this

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = ???

    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit = ???

    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit = ???

    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = ???
 
    override def didChangeConfiguration(didChangeConfigurationParams: DidChangeConfigurationParams): Unit = ()
 
    override def didChangeWatchedFiles(didChangeWatchedFilesParams: DidChangeWatchedFilesParams): Unit = ()

    override def connect(languageClient: LanguageClient): Unit = ???

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = ???

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = ???

    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit = ???

    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit = ???

    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = ???

    override def connect(languageClient: LanguageClient): Unit = ???

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = ???

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = ???

    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit = ???

    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit = ???

    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = ???

    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val pc = PresentationCompiler()
    
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = pc.shutdown()

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = 
      pc.didOpen(didOpenTextDocumentParams)
      
    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit = 
      pc.didChange(didChangeTextDocumetParams)
      
    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit = 
      pc.didClose(didCloseTextDocumentParams)
      
    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = 
      pc.didSave(didSaveTextDocumentParams)
      
    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = pc.shutdown()

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = ()
      // pc.didOpen(didOpenTextDocumentParams)
      
    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit = ()
      // pc.didChange(didChangeTextDocumetParams)
      
    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit = ()
      // pc.didClose(didCloseTextDocumentParams)
      
    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = ()
      // pc.didSave(didSaveTextDocumentParams)
      
    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class Compilers(val project: Project) {
  private val compilers: TrieMap[module.Module, PresentationCompiler] = new TrieMap()

  def startPc(module0: module.Module): PresentationCompiler = {

    val pc = new ScalaPresentationCompiler()
      .newInstance(module0.getName, Nil.asJava, Nil.asJava)
    compilers.addOne(module0 -> pc)
    pc

  }

  def getPresentationCompiler(module0: module.Module): PresentationCompiler =
    compilers.getOrElse(module0, startPc(module0))
}

Compilers

java.util.concurrent.CompletionException: 
  dotty.tools.dotc.MissingCoreLibraryException: 
    Could not find package scala from compiler core libraries.
Make sure the compiler core libraries are on the classpath.

Compilers

final class Compilers(val project: Project) {
  private val compilers: TrieMap[module.Module, PresentationCompiler] = new TrieMap()

  def startPc(module0: module.Module): PresentationCompiler = {

    val pc = new ScalaPresentationCompiler()
      .newInstance(module0.getName, Nil.asJava, Nil.asJava)
    compilers.addOne(module0 -> pc)
    pc

  }

  def getPresentationCompiler(module0: module.Module): PresentationCompiler =
    compilers.getOrElse(module0, startPc(module0))
}

Compilers

final class Compilers(val project: Project) {
  private val compilers: TrieMap[module.Module, PresentationCompiler] = new TrieMap()

  def startPc(module0: module.Module): PresentationCompiler = {
    val originalClasspath = OrderEnumerator.orderEntries(module0).recursively()
      .withoutSdk().getClassesRoots.map(_.getPresentableUrl)
    val fullClasspath = originalClasspath.toList.map(Paths.get(_))

    val pc = new ScalaPresentationCompiler()
      .newInstance(module0.getName, fullClasspath.asJava, Nil.asJava)
    compilers.addOne(module0 -> pc)
    pc

  }

  def getPresentationCompiler(module0: module.Module): PresentationCompiler =
    compilers.getOrElse(module0, startPc(module0))
}

Compilers

libraryDependencies:
/Users/jrochala/.../scala-library-2.13.12.jar
/Users/jrochala/.../scala3-library_3/3.3.3/scala3-library_3-3.3.3.jar

moduleDependencies:
/Users/jrochala/IdeaProjects/testProject/target/scala-3.3.3/classes

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = pc.shutdown()

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = ()
      // pc.didOpen(didOpenTextDocumentParams)
      
    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit = ()
      // pc.didChange(didChangeTextDocumetParams)
      
    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit = ()
      // pc.didClose(didCloseTextDocumentParams)
      
    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = ()
      // pc.didSave(didSaveTextDocumentParams)
      
    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def shutdown(): CompletableFuture[AnyRef] = compilersService.shutdown()

    override def didOpen(didOpenTextDocumentParams: DidOpenTextDocumentParams): Unit = 
      compilerService.getPresentationCompiler(didOpenTextDocumentParams.module)
        .foreach(_.didOpen(didOpenTextDocumentParams))
      
    override def didChange(didChangeTextDocumentParams: DidChangeTextDocumentParams): Unit =       
      compilerService.getPresentationCompiler(didChangeTextDocumentParams.module)
        .foreach(_.didChange(didChangeTextDocumentParams))
      
    override def didClose(didCloseTextDocumentParams: DidCloseTextDocumentParams): Unit =
      compilerService.getPresentationCompiler(didCloseTextDocumentParams.module)
        .foreach(_.didChange(didCloseTextDocumentParams))
      
    override def didSave(didSaveTextDocumentParams: DidSaveTextDocumentParams): Unit = 
      compilerService.getPresentationCompiler(didSaveTextDocumentParams.module)
        .foreach(_.didChange(didSaveTextDocumentParams))
      
    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???

    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = ???
    
    override def completion(
      position: CompletionParams
    ): CompletableFuture[messages.Either[util.List[CompletionItem], CompletionList]] =
      ???

    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] = 
      CompletableFuture.completedFuture(new InitializeResult().withCompletionProvider(true))
    
    override def completion(
      position: CompletionParams
    ): CompletableFuture[messages.Either[util.List[CompletionItem], CompletionList]] =
      compilersService.getPresentationCompiler(position.module).map(_.complete(position))
      
    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

Wait, it was that simple ?

Integrating symbol index

/**
 * The interface for the presentation compile
 * to extract symbol documentation and perform fuzzy symbol search.
 */
public interface SymbolSearch {

    Optional<SymbolDocumentation> documentation(String symbol, ParentSymbols parents);

    List<Location> definition(String symbol, URI sourceUri);

    List<String> definitionSourceToplevels(String symbol, URI sourceUri);

    Result search(String query,
                  String buildTargetIdentifier,
                  SymbolSearchVisitor visitor);
                  
    Result searchMethods(String query,
                  String buildTargetIdentifier,
                  SymbolSearchVisitor visitor);
    
    enum Result {
        COMPLETE,
        INCOMPLETE
    }
}

Symbol search

/**
 * Consumer of symbol search results.
 *
 * Search results can come from two different sources: classpath or workspace.
 * Classpath results are symbols defined in library dependencies while workspace
 * results are symbols that are defined by the user.
 */
public abstract class SymbolSearchVisitor {

   
    abstract public boolean shouldVisitPackage(String pkg);
    
    abstract public int visitClassfile(String pkg, String filename);

    abstract public int visitWorkspaceSymbol(
      Path path, String symbol, SymbolKind kind, Range range
    );

    abstract public boolean isCancelled();

}

Symbol Search Visitor


public abstract class SymbolSearchVisitor {

    abstract public int visitWorkspaceSymbol(
      Path path, String symbol, SymbolKind kind, Range range
    );
}

public abstract class SymbolSeach {
    Result search(String query,
                  String buildTargetIdentifier,
                  SymbolSearchVisitor visitor);
}

Implementation


public abstract class SymbolSearchVisitor {

    abstract public int visitWorkspaceSymbol(
      Path path, String symbol, SymbolKind kind, Range range
    );
}

public abstract class SymbolSeach {
    Result search(String query,
                  String buildTargetIdentifier,
                  SymbolSearchVisitor visitor) =
    
    val symbols: List[PsiElement] = IntelliJCache.getSymbols(query)
    symbols.foreach(visitor.visitWorkspaceSymbol(null, _, null, null))
}

Implementation

SemanticDB

object Test {
  def main(args: Array[String]): Unit = {
    println("hello world")
  }
}


Symbols:
_empty_/Test.
_empty_/Test.main().
_empty_/Test.main().(args)

Occurrences:
[0:7..0:11) <= _empty_/Test.
[1:6..1:10) <= _empty_/Test.main().
[1:11..1:15) <= _empty_/Test.main().(args)
[1:17..1:22) => scala/Array#
[1:23..1:29) => scala/Predef.String#
[1:33..1:37) => scala/Unit#
[2:4..2:11) => scala/Predef.println(+1).

scala/Predef.String#

scala/Predef.String#

IntelliJ Symbol

scala/Predef.String#

PsiElement

scala/Predef.String#

PsiElement

Compiler Internal Symbol

  private def toSemanticdbSymbol(psiClass: PsiClass): String = {
    val suffix = if (psiClass.getLanguage.is(ScalaLanguage.INSTANCE)) "."
    else "#"
    Option(psiClass.getQualifiedName)
      .map(_.replace(".", "/") + suffix)
      .getOrElse("")
  }

* this method will not work in most cases, it is simplification for presentation purposes

final class IntelliJSymbolSearch(
  cache: PsiShortNamesCache, searchScope: GlobalSearchScope) extends SymbolSearch {

  override def search(
    query: String, buildTargetIdentifier: String, visitor: SymbolSearchVisitor
  ): SymbolSearch.Result = {

    if (query.length > 3) {
      val allMatchingClasses = cache.getAllClassNames.filter(_.startsWith(query))
      allMatchingClasses.map { classname =>
        val classes = cache.getClassesByName(classname, searchScope)
        classes.flatMap { psiClass =>
          SemanticDbSymbolCreator.createSemanticDbSymbol(psiClass)
        }.map { semanticDbSymbol =>
          visitor.visitWorkspaceSymbol(
            Paths.get(""), semanticDbSymbol, SymbolKind.Null, new lsp4j.Range()
          )
        }
      }
    }
    SymbolSearch.Result.COMPLETE
  }
}

IntelliJ Symbol Search

* this code handles only class names, it won't provide us with methods, extensions etc

final class Compilers(val project: Project) {
  private val compilers: TrieMap[module.Module, PresentationCompiler] = new TrieMap()

  def startPc(module0: module.Module): PresentationCompiler = {
    val originalClasspath = OrderEnumerator.orderEntries(module0).recursively()
      .withoutSdk().getClassesRoots.map(_.getPresentableUrl)
    val fullClasspath = originalClasspath.toList.map(Paths.get(_))

    val pc = new ScalaPresentationCompiler()
      .newInstance(module0.getName, fullClasspath.asJava, Nil.asJava)
    compilers.addOne(module0 -> pc)
    pc

  }

  def getPresentationCompiler(module0: module.Module): PresentationCompiler =
    compilers.getOrElse(module0, startPc(module0))
}

Compilers

final class Compilers(val project: Project) {
  private val compilers: TrieMap[module.Module, PresentationCompiler] = new TrieMap()

  def startPc(module0: module.Module): PresentationCompiler = {
    val originalClasspath = OrderEnumerator.orderEntries(module0).recursively()
      .withoutSdk().getClassesRoots.map(_.getPresentableUrl)
    val fullClasspath = originalClasspath.toList.map(Paths.get(_))

    val cache = PsiShortNamesCache.getInstance(project)
    val searchScope = GlobalSearchScope
      .moduleWithDependenciesAndLibrariesScope(module0, false)
    val intelliJSymbolSearch = new IntelliJSymbolSearch(cache, searchScope)

    val pc = new ScalaPresentationCompiler()
      .withSearch(intelliJSymbolSearch)
      .newInstance(module0.getName, fullClasspath.asJava, Nil.asJava)
    compilers.addOne(module0 -> pc)
    pc

  }

  def getPresentationCompiler(module0: module.Module): PresentationCompiler =
    compilers.getOrElse(module0, startPc(module0))
}

Compilers

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] =
      CompletableFuture.completedFuture(new InitializeResult().withCompletionProvider(true))
    
    override def completion(
      position: CompletionParams
    ): CompletableFuture[messages.Either[util.List[CompletionItem], CompletionList]] =
      compilersService.getPresentationCompiler(position.module).map(_.complete(position))
      
    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

final class ScalaLanguageServer(project: Project) 
  extends LanguageServer 
  with TextDocumentService 
  with WorkspaceService 
  with LanguageClientAware:
    var myClient = uninitialized
    val compilersService = Compilers.getInstance(project)
     
    override def initialize(initializeParams: InitializeParams): CompletableFuture[InitializeResult] =
      CompletableFuture.completedFuture(new InitializeResult()
        .withCompletionProvider(true)
        .withSignatureHelpProvider(true)
        .withHoverProvider(true)
        ...
      )

    
    override def completion(
      position: CompletionParams
    ): CompletableFuture[messages.Either[util.List[CompletionItem], CompletionList]] =
      compilersService.getPresentationCompiler(position.module).map(_.complete(position))
      
    override def signatureHelp(params: SignatureHelpParams): CompletableFuture[SignatureHelp] =
      compilersService.getPresentationCompiler(params.module).map(_.signatureHelp(params))
      
    override def hover(params: HoverParams): CompletableFuture[Hover] = 
      compilersService.getPresentationCompiler(params.module).map(_.hover(params))
    
    ...


    override def connect(languageClient: LanguageClient): Unit = myClient = languageClient

Scala Language Server

The API

public abstract class PresentationCompiler {
    // type CF = CompletableFuture
    CF<List<Node>> semanticTokens(VirtualFileParams params);
    CF<CompletionList> complete(OffsetParams params);
    CF<CompletionItem> completionItemResolve(CompletionItem item, String symbol);
    CF<SignatureHelp> signatureHelp(OffsetParams params);
    CF<Optional<HoverSignature>> hover(OffsetParams params);
    CF<Optional<Range>> prepareRename(OffsetParams params);
    CF<List<TextEdit>> rename(OffsetParams params, String name);
    CF<DefinitionResult> definition(OffsetParams params);
    CF<DefinitionResult> typeDefinition(OffsetParams params);
    CF<List<DocumentHighlight>> documentHighlight(OffsetParams params);
    CF<String> getTasty(URI targetUri, boolean isHttpEnabled);
    CF<List<AutoImportsResult>> autoImports(String name, OffsetParams params, Boolean isExtension);
    CF<List<TextEdit>> implementAbstractMembers(OffsetParams params);
    CF<List<TextEdit>> insertInferredType(OffsetParams params);
    CF<List<TextEdit>> inlineValue(OffsetParams params)
    CF<List<TextEdit>> extractMethod(RangeParams range, OffsetParams extractionPos);
    CF<List<TextEdit>> convertToNamedArguments(OffsetParams params, List<Integer> argIndices);
    CF<List<Diagnostic>> didChange(VirtualFileParams params);
    void didClose(URI uri);
    CF<byte[]> semanticdbTextDocument(URI filename, String code);
    CF<List<SelectionRange>> selectionRange(List<OffsetParams> params);
}

Thank you

https://slides.com/rochala/compiler-first-development

https://github.com/rochala/compiler-first-development

Compiler First Development

By rochala

Compiler First Development

Compiler-First Development:

A Fresh Take on Scala in IntelliJ

Thank you!

The API

Compiler First Development

More from rochala