Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Writing Tests Using GitHub Copilot
Gleb Bahmutov
LA Fires
brought to you by the fossil fuel companies
image source: https://www.dailysabah.com/world/americas/more-damage-anticipated-as-california-fire-season-sets-records
Control your life
Join an organization
Vote
Greenpeace 350 Sunrise Citizen Climate Lobby
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing
Gleb Bahmutov
Sr Director of Engineering
A typical Mercari US Cypress E2E test
2022: Mercari US disallows using AI code generation assistants pending a legal review
2023: Mercari US allows using AI code generation assistants after a legal review
Profit!!!
VSCode Copilot Extension
Which model?
Mercari US isn't paying for Copilot GH CLI extension
Tip: VSCode Copilot Chat extension
can provide the same info as GitHub CLI copilot
Tip: use "/clear" command to clear the Copilot session
Paste the Copilot answer right into VSCode built-in terminal
@terminal /explain "ls -lh""chat participant"
@workspace
@github
@vscode
"command"
/clear
/doc
/tests
〞
Is GitHub Copilot
making me more productive?
〞
Measuring programming progress by lines of code is like measuring aircraft building progress by weight.
Bill Gates, MS Clippy user
Testing Productivity
🚨 "AI, write me 10 tests"
✅ "AI, help me write this test faster"
🎁 "AI, debug this failed test"
GitHub Copilot in Action
- how to ask GitHub Chat questions
- context and session
- use commands suggested by the Chat
- asking Chat for authoritative sources
- running the first Cypress test
I learned
to fly a small
plane
Copilot
"answer"
I feel shame 😔
Copilot has no shame 😊
⚠️⚠️⚠️
The Most Important Copilot Trick
⚠️⚠️⚠️
Clear Chat session and add prettier
- how to clear Copilot Chat conversion session
- how to insert suggested code snippets into a file
- format source code using Prettier
Copilot Suggestions
https://github.com/bahmutov/cypress-copilot-example
branch a3, might need "npm install"
- how to always show the Copilot suggestions bar
Copilot Suggestions
https://github.com/bahmutov/cypress-copilot-example
branch a3, might need "npm install"
- how to always show the Copilot suggestions bar
- seeing multiple suggestions at once using the Completions Panel
- tweaking the code suggestions using comments
- using Chat to suggest a better solution
Tip: Add a comment like "// format the price using USD currency"
to see better suggestions
Writing Unit Tests Using Copilot
- how to prompt Copilot to generate a unit test file
- how to adjust the generated test file to use Cypress globals and Chai assertions
- how to generate additional unit tests
Guide Copilot Using Comments
- the generated test quality is determined by the local context
- adding more comments with concrete values helps generate the correct test code
Guiding A Human QA The Same Way
- visit the page
- the page finishes loading 👎
- the body element has class "loaded" 👍
- find the input element with class "new-todo"
- type "Hello, World!" + Enter
- the list should have 1 item with "Hello, World!" text
imprecise for the computer
Test Adding Todos
- naming spec file
- accepting the full Copilot suggestion vs the next word
- guiding the Copilot to generate a comment vs a
cycommand chain - keep Cypress open to see the test result
- reset the data using beforeEach hook
Test Deleting Todos
- generating JSON fixtures
- inspecting the generated test code
- setting up the initial test data by loading a fixture and making an API call
Spy on Network Calls
- how Copilot can explain source code
- spying on the network calls using the cy.intercept command
- waiting on network calls using the cy.wait command
⚠️ if you see "cy.route" suggestion,
it is obsolete!
Modify Code In Place
- modifying the existing code using the Command-I tool
network-calls.cy.js
Fix Code In Place
- fixing the selected code using the Copilot
/fixcommand - generate the commit messages based on the changed source files
Document your code
- using
/doccommand to generate JSDoc comments - how to document your Cypress tests using
cy.logcommands
Generate Better Code By Example
- how to remove duplicated selectors in the generated queries
- how Copilot can use local variables and functions
- how you ultimately can dramatically improve the final test code quality
adding-todos.cy.js
Generate Test Data
- how you can generate arrays of objects following a simple schema
- how Copilot update the generate list to refine it further
Copilot Edits
https://github.com/bahmutov/cypress-copilot-example
branch bonus02
- a new mode for the "prompt + review proposed code diff"
Copilot Edits
https://github.com/bahmutov/cypress-copilot-example
branch bonus02
- a new mode for the "prompt + review proposed code diff"
Also: GitHub Copilot Workspace
Copilot 👍
- Easy test data generation
- Good starting point
- Follows the comments
- Speeds up typing
Copilot 🔥
- Hallucinations
- Outdated information
- Unorthodox suggestions
- Complex suggestions
- ❌ writing code that I do not know how to write myself (new language or framework)
- ❌ write big chunks of code (generate 10 tests at once)
- ✅ speeding up writing code guided by comments
- ✅ test data generation
- ✅ code or data in-place transformations
- 😒 commit message generation
GitHub Copilot
❤️ the "/explain" command
GitHub Copilot is the "Y" key when playing Mario
Local "Copilot"
- Install https://ollama.com/
Download public models (almost like Docker!)
Local "Copilot"
- Install https://ollama.com/
- Install VS Code extension "Continue"
Local "Copilot"
{
"models": [
{
"title": "DeepSeek Coder 6.7b",
"provider": "ollama",
"model": "deepseek-coder:6.7b"
},
{
"model": "claude-3-5-sonnet-latest",
"provider": "anthropic",
"apiKey": "",
"title": "Claude 3.5 Sonnet"
}
],
"tabAutocompleteModel": {
"title": "DeepSeek Coder 6.7b",
"provider": "ollama",
"model": "deepseek-coder:6.7b"
},
"embeddingsProvider": {
"provider": "ollama",
"model": "nomic-embed-text"
}
}Local "Copilot"
- Install https://ollama.com/
- Install VS Code extension "Continue"
- Configure local / remote models
- Local chat and code completions!
Local "Copilot"
- Install https://ollama.com/
- Install VS Code extension "Continue"
- Configure local / remote models
- Local chat and code completions!
I found local models
to be pretty
inaccurate
👏 Thank You 👏
Gleb Bahmutov
Writing Tests Using
GitHub Copilot
Writing Tests Using GitHub Copilot: ConFoo
By Gleb Bahmutov
Writing Tests Using GitHub Copilot: ConFoo
want to show you the secret weapon I have used for the past year. The GitHub Copilot lets me write full tests quickly, fills the gaps in my knowledge of 3rd party tools, and even writes clear descriptive commit messages. But it is not a "press the button to do it all". You need to guide the AI to do the right thing, which takes experience, but most importantly, you need to decide _what_ you want the AI to do, step by step. Presented at ConFoo 2025, 45 minutes
