Metals: "Fast, Correct — choose two"

 

Tomasz Godzik, Krzysztof Romanowski

@ VirtusLab

Virtuslab is a company focused on improving companies workflows via improvements to the Developer Experience

What do we do?

Most prominent Metals issues

01

02

03

04

MBT - Metals Build Tool

Fast IDE features

Improved Java Support

05

Questions

What is Metals?

Metals is a language server protocol server for Scala and Java.

Uses Language Server Protocol to provide IDE features in any editor that implements the LSP client

 

Based on JSON RPC

 

Widely supported by multiple editors, de facto standard for most languages

Metals LSP Server

Our goal

 

  • A lot of the editors' IDE features only available after a successful build or import
  • Large monorepo setups can have even a whole host of problems

Most prominent problems

 

Our Goal for Metals v2

 

Make the IDE useful from the moment you clone a repo.

 

 

Main Metals v2 was worked on in Databricks by Ólafur Páll Geirsson, Iulian Dragos and others in the team.

 

"How many of you have waited 5+ minutes for IDE features after opening a project?"

 

  1. Metals Build Tool symbol index

  2. IDE Features available from the start

  3. Better Java Language Support

 

 

 

Main changes

Each feature reinforces the others — together they make Metals viable for large, mixed-language monorepos.

Metals Build Tool

Wait! Another build tool!?

 

It's not really a build tool

It's about getting the minimal amount of information from the build to somewhere Metals can access it directly.

Symbol search depended on the BSP build server. We needed to connect to the build server (could be different from the build tool)

 

We had to wait to get all `src/main/scala` etc.

Before (current main)

Before (current main)

Editor opened

Build import

(1-5 min)

Build import 

Build import 

Build import 

Build server start (20s-2min)

Build import 

Build import 

Build import 

Not imported

Already imported

Indexing

(30s - 3min)

Build import 

Build import 

Build import 

Ready to serve requests

Build import 

Build import 

Build import 

Compilation (30s - 10min)

Build import 

Build import 

Build import 

Future (main-v2)

Self-contained index, zero build dependency. Can work as soon as it's indexed.

Future (main-v2)

Editor opened

Build import

(1-5 min)

Build import 

Build import 

Build import 

Build server start (20s-2min)

Build import 

Build import 

Build import 

Not imported

Already imported

Indexing

(30s - 3min)

Build import 

Build import 

Build import 

Ready to serve requests

Build import 

Build import 

Build import 

Compilation (30s - 10min)

Build import 

Build import 

Build import 

How MBT works

git ls-files --stage  

~200-300ms  

Each file checked if modified by OID 

Index each file using existing fast indexers

Create IndexedDocument

Overwrite existing index.mbt on close

Bloom filters

  Query: "UserSer"
        │
        ├─ File A → MAYBE match → compile it
        ├─ File B → NO match    → skip         
        ├─ File C → NO match    → skip
        └─ File D → MAYBE match → compile it

  Result: only 2 of 4 files opened instead of all 4

Bloom filters

At scale (10k+ files), this means orders of magnitude fewer files scanned.

  Query: "UserSer"
        │
        ├─ File A → MAYBE match → compile it
        ├─ File B → NO match    → skip         
        ├─ File C → NO match    → skip
        └─ File D → MAYBE match → compile it

  Result: only 2 of 4 files opened instead of all 4

No startup time IDE - interactive features

git clone

 Full IDE: 

  • Hover
  • Complete
  • Navigate
  • References

 

Build import 

Build import 

Build import 

Open editor

Ideal scenario

If we don't have anything we put all sources into the presentation compiler.

 

This means that even in projects with broken build tool you should be able to work on your code.

How does it work?

And it does work in a lot of cases.

How does it work?

Java support 

One of the main goals in Metals 2 to was to make it viable for large mixed codebases.

 

Hence we needed to step up out Java support as well.

Java support

We decided to implement most of the basic features of LSP for Java.

 

No complex refactors, since most likely LLM era makes them redundant.

Java support

Feature Status Powered by.
Completions ✅  Presentation compiler
Hover ✅  Presentation compiler
Signature Help  ✅  Presentation compiler
Diagnostics ✅  Presentation compiler
Semantic Highlighting ✅  Java Semanticdb 
Selection Range ✅  Javac AST
Document Symbols ✅  Javac AST
Go to Definition ✅  SemanticDB + mbt
Find References ✅  mbt + Bloom filters

But how to make it efficient in large codebase?

We use google turbine which allows us to define stubs for each file without compiling method bodies.

 

We get in-memory classfiles which the javac compiler can use as a normal classpath

But how to make it efficient in large codebase?

 

Caveat: It does not support project Lombok or annotation processors currently.

Google Turbine

example/File.java (edited)

example/Other.java

...

Classfiles stubs: example/Other.class, example/Other2.class

File.java, 

Google Turbine

Presentation compiler

Javac outline compiler, compiles the file fully

example/File.java (edited)

example/Other.java

...

Classfiles stubs: example/Other.class, example/Other2.class

File.java

Google Turbine

Presentation compiler

Javac outline compiler, compiles the file fully

Outline compiler, compiles without method bodies

AlsoEdited.java

By default turbine will run every minute, so we are back to classfiles once that completes

Google Turbine

Demo

Summary

 

Java Support

  • mostly feature complete
  • turbine
  • outline compiler

 

Fast IDE features

  • sourcepath fallback
  • mbt fallback
  • better crash resilience

 

MBT Index

  • bloom filters
  • protobuf
  • persists to disk
  • can use mbt.json

How do I actually use it?

Build from source on main-v2 

 

or

 

​metals.serverVersion : "2.0.0-M8"

Thank You

Tomasz Godzik

Bluesky:      @tgodzik.bsky.social 

Mastodon:  fosstodon.org/@tgodzik

Discord:       tgodzik

 

tgodzik@virtuslab.com