Writing Tests Using GitHub Copilot

Gleb Bahmutov

gleb.dev

https://slides.com/bahmutov/writing-tests-using-github-copilot

Global temperature anomaly 2022 https://climate.nasa.gov/vital-signs/global-temperature/

survival is possible* but we need to act now

change your life
join an organization
VOTE

@bahmutov

rebellion.global www.wedonthavetime.org

www.sierraclub.org 350.org www.greenpeace.org

www.sunrisemovement.org

Speaker: Gleb Bahmutov PhD

@bahmutov

gleb.dev

github.com/bahmutov

glebbahmutov.com/blog

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing

www.youtube.com/glebbahmutov

Gleb Bahmutov

Sr Director of Engineering

glebbahmutov.com/blog testing posts

gleb.dev

https://cypress.tips/courses

gleb.dev

Lots Of Testing

https://slides.com/bahmutov/decks/mercari

A typical Mercari US Cypress E2E test

gleb.dev

2022: Mercari US disallows using AI code generation assistants pending a legal review

2023: Mercari US allows using AI code generation assistants after a legal review

Profit!!!

gleb.dev

VSCode Copilot Extension

gleb.dev

Tip: VSCode Copilot Chat extension

can provide the same info as GitHub CLI copilot

gleb.dev

〞

Is GitHub Copilot

making me more productive?

gleb.dev

〞

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

Bill Gates, MS Clippy user

gleb.dev

Testing Productivity

🚨 "AI, write me 10 tests"

✅ "AI, help me write this test faster"

🎁 "AI, debug this failed test"

gleb.dev

GitHub Copilot in Action

https://github.com/bahmutov/cypress-copilot-example

branch a1

how to ask GitHub Chat questions
context and session
use commands suggested by the Chat
asking Chat for authoritative sources
running the first Cypress test

gleb.dev

Bayeux Tapestry: Battle of Hastings (1066) King Harold is Fatally Injured When an Arrow Pierces His Eye

🖨️

Gleb

https://slides.com/bahmutov/writing-tests-using-github-copilot

Clear Chat session and add prettier

https://github.com/bahmutov/cypress-copilot-example

branch a2

how to clear Copilot Chat conversion session
how to insert suggested code snippets into a file
format source code using Prettier

gleb.dev

Copilot Suggestions

https://github.com/bahmutov/cypress-copilot-example

branch a3

how to always show the Copilot suggestions bar
seeing multiple suggestions at once using the Completions Panel
tweaking the code suggestions using comments
using Chat to suggest a better solution

gleb.dev

Add a comment like "// format the price using USD currency" to see better suggestions

gleb.dev

Writing Unit Tests Using Copilot

https://github.com/bahmutov/cypress-copilot-example

branch a4

how to prompt Copilot to generate a unit test file
how to adjust the generated test file to use Cypress globals and Chai assertions
how to generate additional unit tests

gleb.dev

Guide Copilot Using Comments

https://github.com/bahmutov/cypress-copilot-example

branch a5

the generated test quality is determined by the local context
adding more comments with concrete values helps generate the correct test code

gleb.dev

Guiding A Human QA The Same Way

visit the page
the page finishes loading 👎
the body element has class "loaded" 👍
find the input element with class "new-todo"
type "Hello, World!" + Enter
the list should have 1 item with "Hello, World!" text

gleb.dev

Test Adding Todos

https://github.com/bahmutov/cypress-copilot-example

branch a6

naming spec file
accepting the full Copilot suggestion vs the next word
guiding the Copilot to generate a comment vs a cy command chain
keep Cypress open to see the test result
reset the data using beforeEach hook

gleb.dev

Test Deleting Todos

https://github.com/bahmutov/cypress-copilot-example

branch a7

generating JSON fixtures
inspecting the generated test code
setting up the initial test data by loading a fixture and making an API call

gleb.dev

Spy on Network Calls

https://github.com/bahmutov/cypress-copilot-example

branch a8

how Copilot can explain source code
spying on the network calls using the cy.intercept command
waiting on network calls using the cy.wait command

gleb.dev

Modify Code In Place

https://github.com/bahmutov/cypress-copilot-example

branch b1

modifying the existing code using the Command-I tool

gleb.dev

Fix Code In Place

https://github.com/bahmutov/cypress-copilot-example

branch b2

fixing the selected code using the Copilot /fix command
generate the commit messages based on the changed source files

gleb.dev

Document your code

https://github.com/bahmutov/cypress-copilot-example

branch b3

using /doc command to generate JSDoc comments
how to document your Cypress tests using cy.log commands

gleb.dev

Generate Better Code By Example

https://github.com/bahmutov/cypress-copilot-example

branch b4

how to remove duplicated selectors in the generated queries
how Copilot can use local variables and functions
how you ultimately can dramatically improve the final test code quality

gleb.dev

Generate Test Data

https://github.com/bahmutov/cypress-copilot-example

branch b6

how you can generate arrays of objects following a simple schema
how Copilot update the generate list to refine it further

gleb.dev

Copilot 👍

Easy test data generation
Good starting point
Follows the comments
Speeds up typing

Good vs Bad

Copilot 🔥

Hallucinations
Outdated information
Unorthodox suggestions
Complex suggestions

❌ writing code that I do not know how to write myself (new language or framework)
❌ write big chunks of code (generate 10 tests at once)
✅ speeding up writing code guided by comments
✅ test data generation
✅ code or data in-place transformations
😒 commit message generation