François Lanusse

Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM

Thoughts on Coding With LLMs

Journee annuelle de l'axe astrophysique

Dec 2025

Let's start with a few questions for the audience

  • Who has already used ChatGPT to help with writing code?

     
  • Who is already using an LLM directly integrated within a IDE (e.g. GitHub Copilot in VSCode, or Cursor)?

     
  • Who is already using coding agents to write significant pieces of software libraries or analysis code? (e.g. Claude Code, Codex CLI, Gemini CLI)?
     

  • Who thinks mastering these tools will be critical to the success of their PhD?

Let me tell you a story

=> I have been working on this for 7 years, you could say I am somewhat of an expert...

2025 NeurIPS Weak Lensing Challenge

Also me

That's me

That's an LLM agent

  • A team of non-experts, aided by an LLM agent, was able in a week or two to outperform one of the most expert researcher on this problem, in the entire world.

To know more about CMBAgent and Denario:

An example of asking an agent to produce a cosmology plot

What is your role as a student?

  • You want to develop critical thinking, remain in charge of the project, make sure you understand the results.
    • Agents can still very happily generate nonsensical results
    • If your prompt is too vague, agents will make a lot of assumptions that impact the results.
    • LLMs are not actually generally intelligent, they can get stuck on some problems, if you do not understand the code or the problem, you are also stuck!

A few pro-tips

  • Have long conversations with ChatGPT (or other) to build a design document for your project.

     
  • Never trust at face value the results, ask your agent to build tests and validations for all the steps of your project. Your role is to make sure these tests are meaningful.

     
  • Question the results, ask if it is really the simplest way to reach the goal, ask what assumptions could impact the result, make sure you understand them.

So much more to discuss, but let's keep it for the roundtable