Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Writing Tests Using GitHub Copilot
Gleb Bahmutov
survival is possible* but we need to act now
- change your life
- join an organization
- VOTE
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing
Gleb Bahmutov
Sr Director of Engineering
glebbahmutov.com/blog testing posts
Lots Of Testing
A typical Mercari US Cypress E2E test
2022: Mercari US disallows using AI code generation assistants pending a legal review
2023: Mercari US allows using AI code generation assistants after a legal review
Profit!!!
VSCode Copilot Extension
Which model?
Mercari US isn't paying for Copilot GH CLI extension
Tip: VSCode Copilot Chat extension
can provide the same info as GitHub CLI copilot
〞
Is GitHub Copilot
making me more productive?
〞
Measuring programming progress by lines of code is like measuring aircraft building progress by weight.
Bill Gates, MS Clippy user
Testing Productivity
🚨 "AI, write me 10 tests"
✅ "AI, help me write this test faster"
🎁 "AI, debug this failed test"
GitHub Copilot in Action
- how to ask GitHub Chat questions
- context and session
- use commands suggested by the Chat
- asking Chat for authoritative sources
- running the first Cypress test
Clear Chat session and add prettier
- how to clear Copilot Chat conversion session
- how to insert suggested code snippets into a file
- format source code using Prettier
Copilot Suggestions
- how to always show the Copilot suggestions bar
- seeing multiple suggestions at once using the Completions Panel
- tweaking the code suggestions using comments
- using Chat to suggest a better solution
Add a comment like "// format the price using USD currency" to see better suggestions
Writing Unit Tests Using Copilot
- how to prompt Copilot to generate a unit test file
- how to adjust the generated test file to use Cypress globals and Chai assertions
- how to generate additional unit tests
Guide Copilot Using Comments
- the generated test quality is determined by the local context
- adding more comments with concrete values helps generate the correct test code
Guiding A Human QA The Same Way
- visit the page
- the page finishes loading 👎
- the body element has class "loaded" 👍
- find the input element with class "new-todo"
- type "Hello, World!" + Enter
- the list should have 1 item with "Hello, World!" text
imprecise for the computer
Test Adding Todos
- naming spec file
- accepting the full Copilot suggestion vs the next word
- guiding the Copilot to generate a comment vs a
cycommand chain - keep Cypress open to see the test result
- reset the data using beforeEach hook
Test Deleting Todos
- generating JSON fixtures
- inspecting the generated test code
- setting up the initial test data by loading a fixture and making an API call
Spy on Network Calls
- how Copilot can explain source code
- spying on the network calls using the cy.intercept command
- waiting on network calls using the cy.wait command
Modify Code In Place
- modifying the existing code using the Command-I tool
Fix Code In Place
- fixing the selected code using the Copilot
/fixcommand - generate the commit messages based on the changed source files
Document your code
- using
/doccommand to generate JSDoc comments - how to document your Cypress tests using
cy.logcommands
Generate Better Code By Example
- how to remove duplicated selectors in the generated queries
- how Copilot can use local variables and functions
- how you ultimately can dramatically improve the final test code quality
Generate Test Data
- how you can generate arrays of objects following a simple schema
- how Copilot update the generate list to refine it further
Copilot Edits
https://github.com/bahmutov/cypress-copilot-example
branch bonus02
- a new mode for the "prompt + review proposed code diff"
Also: GitHub Copilot Workspace
Copilot 👍
- Easy test data generation
- Good starting point
- Follows the comments
- Speeds up typing
Copilot 🔥
- Hallucinations
- Outdated information
- Unorthodox suggestions
- Complex suggestions
- ❌ writing code that I do not know how to write myself (new language or framework)
- ❌ write big chunks of code (generate 10 tests at once)
- ✅ speeding up writing code guided by comments
- ✅ test data generation
- ✅ code or data in-place transformations
- 😒 commit message generation
GitHub Copilot
❤️ the "/explain" command
GitHub Copilot is the "Y" key when playing Mario
But wait, there is more...
- indexed repositories
- knowledge repositories
- private fine-tuned models
- Copilot extensions
Local "Copilot"
- Install https://ollama.com/
Download public models (almost like Docker!)
Local "Copilot"
- Install https://ollama.com/
- Install VS Code extension "Continue"
Local "Copilot"
{
"models": [
{
"title": "DeepSeek Coder 6.7b",
"provider": "ollama",
"model": "deepseek-coder:6.7b"
},
{
"model": "claude-3-5-sonnet-latest",
"provider": "anthropic",
"apiKey": "",
"title": "Claude 3.5 Sonnet"
}
],
"tabAutocompleteModel": {
"title": "DeepSeek Coder 6.7b",
"provider": "ollama",
"model": "deepseek-coder:6.7b"
},
"embeddingsProvider": {
"provider": "ollama",
"model": "nomic-embed-text"
}
}Local "Copilot"
- Install https://ollama.com/
- Install VS Code extension "Continue"
- Configure local / remote models
- Local chat and code completions!
Stay tuned for a blog post about it
Writing Tests Using GitHub Copilot
Gleb Bahmutov
👏 Thank You 👏
For more, see https://cypress.tips/courses
Writing Tests Using GitHub Copilot
By Gleb Bahmutov
Writing Tests Using GitHub Copilot
I want to show you the secret weapon I have used for the past year. The GitHub Copilot lets me write full tests quickly, fills the gaps in my knowledge of 3rd party tools, and even writes clear descriptive commit messages. But it is not a "press the button to do it all". You need to guide the AI to do the right thing, which takes experience, but most importantly, you need to decide _what_ you want the AI to do, step by step. Remember: AI is not going to replace you any time soon. But someone more productive with AI's help might get ahead. Let's learn Copilot for real-world test writing. It might change how you work. If you master it, Copilot becomes your super weapon. It is like playing Super Mario - in some games, you can dash quickly by pressing the "Y" button, but you need to be pointing in the right direction first. Presented at MaineJS 2024 video at https://www.youtube.com/watch?v=rTIwu8V2ujg
