Efficient programming with Scala and LLMs
Tomasz Godzik
VirtusLab
Maintainer of multiple Scala tools including Metals, Bloop, Scalameta, Munit, Mdoc and parts of the Scala 3 compiler.
Release officer for Scala LTS versions
Part of the Scala Core team as coordinator and VirtusLab representative
Part of the moderation team
What do I do?
How smart are LLMs really
01
02
03
04
Prompting 101
Context Engineering
Your new work environment
05
Approaches to working with LLMs
06
Questions
Presentation is based on the work by myself, Łukasz Biały and Piotr Oramus
I will not cover everything that you might need, but just give you a taste of what is currently happening
How smart are LLMs really?
For sure it's easy to fool them!
That was a few months ago
You can still fool it.
LLMs has fallen into well known paths
What is it that modern LLMs really generate?
text that COULD be the answer
Is AGI already here?
Is AGI already here?
Opinions differ, but LLMs are getting better.
What modern LLMs really are?
What modern LLMs really are?
autocomplete on steroids super-soldier serum
What modern LLMs really are?
autocomplete on steroids super-soldier serum
Something more?
The hype can get too much at times though.
The hype can get too much at times though.
It probably is
One thing is certain, AI will not replace good engineers.
We need to understand what LLMs are capable and how to make them work for us
Prompting 101
> Create a simple todo app
Is this a good prompt?
> Create a simple todo app
Is this a good prompt?
Obviously not
You are a javafx / scalafx expert tasked with creating a simple todo app using Scala 3. Use scala-cli to set up, run and test the project.
Structure of a prompt
You are a javafx / scalafx expert tasked with creating a simple todo app using Scala 3. Use scala-cli to set up, run and test the project.
Structure of a prompt
persona
task
context
You are a javafx / scalafx expert tasked with creating a simple todo app using Scala 3. Use scala-cli to setup, run and test the project. Here’s a snippet of Scalafx specific APIs you can use:
// Aggregate operator
val r1, r2, r3 = new Rectangle()
r1.width <== max(r2.width, r3.width)
// Conditional binding
r2.fill <== when (r2.hover) choose Red otherwise Blue
// Complex boolean + string concat
val tf = new TextField { text = "Hello" }
val lbl = new Label()
lbl.text <== when (r1.hover || r2.hover)
...
Structure of a prompt
-
Persona competence sizing: expert vs regular - allows to limit the amount of concerns the LLM will try to juggle with.
-
In-context learning: few-shot prompting > zero-shot prompting for output fitting the precise requirements
-
Bias management: it’s always good to watch how your prompt is biasing the model - maybe the solution you have in mind is not the best one?
Properties
- Multimodal prompting
-
Prompting techniques
A lot of the existing tools have some prompts and roles already
The more people come up with the more the tooling catches up
Prompting
Context engineering
Should we just put a lot of information
into the prompt’s context?
Should we just put a lot of information
into the prompt’s context?
Obviously not
Issues with large contextx
-
Context drift - when the conversation gets reused for new tasks, the previous results end up confusing the model’s attention mechanisms
Context overload - when the input provided with the initial prompt is big enough, it will confuse the model and lead it astray from the original task
Costs!
Rules
-
Provide only the information (and capabilities) necessary to execute the task at hand
Prefer to work in shorter conversations that don’t reach the full size of the available context window
Use compaction and external progress tracking to distill and preserve important decisions and insights
Optimizing the costs
-
Always provide the same prefix, each LLM query is cached
-
If you change it will recalculate everything
-
Cache available for a time
Use cases for context engineering
-
Cursor rules / Claude files
Project design docs
Style guides
Timeline.md
MCP results
Your new work envronment
How to make LLMs work for you?
Different approaches:
- ask - does not change anything
- plan - only creates plans, documents
- agent - makes changes to your code
Tools for agents: MCP
-
Tools for agents: Skills
-
Approaches to working with LLMs
Can be added as skills
Improve CI
Automize any manual work even if normally wrting a script would take longer than the work
Creating small usability scripts
Might be harder to use LLMs in, tasks should be scoped, all data available that the LLM afent might need
Brownfield projects
Existing project
Brownfield projects
Instead of ingesting the entire context, create smaller tool to change codebase or extract information
Types help!
Martin already told you about safe capabilities
Tools for agents: Executable code
Something totally new
Greenfield projects
We can switch the approach and have reliable small to mid project done automatically
Greenfield projects
Half a year ago was not feasable, but becoming more and more
It is more expensive and complex to monitor
Multiagent orchestration
Summary
Verify if the code is maintainable, extendable and optimal.
Make sure your prompts are sound and context provided is well scoped.
You need to know your domain to be able to verify the output, use tools to improve that
Thank You
Tomasz Godzik
Bluesky: @tgodzik.bsky.social
Mastodon: fosstodon.org/@tgodzik
Discord: tgodzik
tgodzik@virtuslab.com
Some links:
https://www.anthropic.com/engineering/code-execution-with-mcp
Copy of Metals V2
By Tomek Godzik
