Efficient programming with Scala

and LLMs

Tomasz Godzik

VirtusLab

Maintainer of multiple Scala tools including Metals, Bloop, Scalameta, Munit, Mdoc and parts of the Scala 3 compiler.

Release officer for Scala LTS versions

Part of the Scala Core team as coordinator and VirtusLab representative

Part of the moderation team

What do I do?

How smart are LLMs really

01

02

03

04

Prompting 101

Context Engineering

Your new work environment

05

Approaches to working with LLMs

06

Questions

This presentation draws on work by myself, Łukasz Biały, and Piotr Oramus

I will not cover everything that you might need, but just give you a taste of what is currently happening

How smart are LLMs really?

For sure it's easy to fool them!

That was a few months ago

You can still fool it.

LLM has fallen into well known paths

What is it that modern LLMs really generate?

text that COULD be the answer

Is AGI already here?

Opinions differ, but LLMs are getting better.

What are modern LLMs, really?

autocomplete on steroids super-soldier serum

Something more?

What are modern LLMs, really?

autocomplete on steroids super-soldier serum

The hype can get too much at times though.

It probably is

One thing is certain, AI will not replace good engineers.

Not yet!

We need to understand what LLMs are capable of and how to make them work for us.

This is a new tool in your toolbox and there's a learning curve to using it.

I think it was already shown on previous presentations.

But just take a look at the recent AWS outages where coding agents were heavily involved.

Using languages with strong typing helps to reduce the risk of using LLMs in your development

Why Scala and LLMs?

What outages?

Prompting 101

> Create a Scala LSP server

Is this a good prompt?

> Create a Scala LSP server

Is this a good prompt?

Obviously not

You are a Scala compiler and tooling expert tasked with creating a Scala LSP server. Use scala-cli to set up, run and test the project.

Structure of a prompt

You are a Scala compiler and tooling expert tasked with creating a Scala LSP server. Use scala-cli to set up, run and test the project.

Structure of a prompt

persona

task

context

You are a Scala compiler and tooling expert tasked with creating a Scala LSP server. Use scala-cli to set up, run and test the project. Here is some compiler API you can use.

// Aggregate operator
val r1, r2, r3 = new Rectangle()
r1.width <== max(r2.width, r3.width)
// Conditional binding
r2.fill <== when (r2.hover) choose Red otherwise Blue
// Complex boolean + string concat
val tf = new TextField { text = "Hello" }
val lbl = new Label()
lbl.text <== when (r1.hover || r2.hover)

...

Structure of a prompt

Persona competence sizing: expert vs regular - allows to limit the amount of concerns the LLM will try to juggle with.
In-context learning: few-shot prompting > zero-shot prompting for output fitting the precise requirements
Bias management: it’s always good to watch how your prompt is biasing the model - maybe the solution you have in mind is not the best one?

Properties

Multimodal prompting: Different types of input, not just text, but voice, images etc.
Prompt chaining: Each conversation produces artifacts for the next prompt.
Reasoning models: Ask the model to explain its reasoning, default in some models.

Prompting techniques

Tree of Thoughts: Explore possible options, make model offer them and choose each time one that suits you better.
Meta prompting: Prompt to create new prompts for later models.
Agentic simulations: Make agents fulfill some persona and simulate things like recruitment calls etc.

Prompting techniques

It's a bit of an alchemy

Some of the existing techniques made it into tools.

The more people come up with the more the tooling catches up.

The ideas are basically endless.

Prompting

Context engineering

Context is king.

Should we just put a lot of information

into the prompt’s context?

Obviously not

Should we just put all the information

into the prompt’s context?

Context drift

When the conversation gets reused for new tasks, the previous results end up confusing the model’s attention mechanisms

Context overload

When the input provided with the initial prompt is big enough, it will confuse the model and lead it astray from the original task.

Costs!

More data in context gets attached before the prompt and uses more tokens, which cost money.

Rules

1. Provide only the information (and capabilities) necessary to execute the task at hand.

Rules

2. Prefer to work in shorter conversations that don’t reach the full size of the available context window.

Rules

3. Use compaction and external progress tracking to distill and preserve important decisions and insights.

Context compaction

> Summarize the current decisions and progress so that we can use it in another conversation

Progress tracking

Use local files for tracking specification and TODOs for the agents to pick up.

Optimizing the costs

Always provide the same prefix, each LLM query is cached.
If you change it, the model will need to recalculate everything.
This cache will only be available for a time.

Your new work environment

ask: does not change anything, used for exploration.

plan: create documents needed for further work, any specification timeline etc.

agent: actually apply changes, test code, compile until the results are satisfactory.

Typical usages of LLMs

ask: does not change anything, used for exploration.

plan: create documents need for further work, any specification timeline etc.

agent: actually apply changes, test code, compile until the results are satisfactory.

Typical usages of LLMs

Most of the time you will use

Agents

They can use a variety of tools to enrich their context and verify the results of their work.

Every development tool might be useful for you, but some might need work.

Model Context Protocol

One of the more popular tools to improve the output of LLMs

MCP uses JSON-RPC, so each possible request and response is represented by a JSON schema

Model Context Protocol

Model Context Protocol is very similar to Language Server Protocol, but instead of people it's for agents.

Host with MCP Client

MCP protocol

MCP Server 1

MCP Server 2

Local Data Source 1

Local Data Source 2

Web API

Model Context Protocol

tools: longer running tasks that agents can schedule
resources: static context available to load lazily
prompts: possible meta prompts for users of the chat

Metals is a model context protocol server.

compilation: compile-full, compile-module, compile-file
test: test
analysis: glob-search, inspect, get-docs, get-usages
utility: import-build, format-file, find-dep, run-scalafix-rule, list-scalafix-rules

Available tools

Problems with MCP

The more tools you have the more context is taken up.
Model efficiency might go down and it might start hallucinating more.
Data returned from the tools might go through the model adding to the context.

Alternative: Executable tools

Instead of ingesting the entire context, create smaller tools to change codebase or extract information.

Using typed languages like Scala makes it significantly more efficient.

Alternative: Executable tools

The agent queries each toolset it needs and creates executable code

// ./skills/getDocsFor.scala
import metals.mcp.*

@main
def main(nameToLookFor: String) =
  val allMatching = globSearch(fullyQualifiedName, None)
  val fqcns = allMatching.split("\n")
  val result = fqcns.map{
    fullName => getDocs(fullName)
  }.mkString("\n")
  println(result)
}

Security

You still need to check the code and make sure it does what you want it to do.
You cannot YOLO mode it for sure.
Martin already told you about safe capabilities, which could make it much safer.
You can also sandbox it! github.com/VirtusLab/sandcat

Tools for agents: Skills

---
name: your-skill-name
description: Brief description of what this Skill does and when to use it
---

# Your Skill Name

## Instructions
[Clear, step-by-step guidance for Claude to follow]

## Examples
[Concrete examples of using this Skill]

Tools for agents: Skills

Each skill could contain scripts or mini libraries that the agents can use

# Glob search

## Quick start

Run the script if you want to return user's entire 
documentation based on specific glob 
(part of the name user is looking for) 

```bash
scala-cli search-script.scala
```

Approaches to working with LLMs

These scripts could be added as skills to your company repo.

Some of the scripts could improve CI as it doesn't require a lot of code.

Automate any manual work even if normally writing a script would take longer than the work.

Creating small usability scripts

Brownfield projects

These are projects that were previously written using normal engineering practises and we want to add feature or fix bugs.

Brownfield projects

Might be harder to use LLMs in, tasks should be scoped, all data available that the LLM agent might need.

Provide as many tools as needed to make sure the context is spot on.

A lot depends on the complexity and the existing quality of the code.

We can also build something from scratch with LLMs in mind.

Makes it possible to create small to mid project almost automatically with guidelines from the developer

Greenfield projects

Exploration

Greenfield projects

Product planning

Component analysis

Implementation

Full product idea

Full project specification

Specification with all modules planned

Half a year ago this was not feasible, but becoming more and more so.

It is more expensive and complex to monitor.

But can for example be used to implement multiple components at the same time or make product exploration faster.

Multiagent orchestration

Summary

Verify if the code is maintainable, extensible and optimal.

Make sure your prompts are sound and context provided is well scoped.

You need to know your domain to be able to verify the output, use tools to improve that.

Thank You

Tomasz Godzik

Bluesky: @tgodzik.bsky.social

Mastodon: fosstodon.org/@tgodzik

Discord: tgodzik

tgodzik@virtuslab.com

Efficient programming with Scala and LLMs

By Tomek Godzik

Efficient programming with Scala and LLMs