Writing Tests Using GitHub Copilot

Gleb Bahmutov

survival is possible* but we need to act now

  • change your life
  • join an organization
  • VOTE

Speaker: Gleb Bahmutov PhD

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing

Gleb Bahmutov

Sr Director of Engineering

glebbahmutov.com/blog testing posts

Lots Of Testing

A typical Mercari US Cypress E2E test

2022: Mercari US disallows using AI code generation assistants pending a legal review

2023: Mercari US allows using AI code generation assistants after a legal review


VSCode Copilot Extension

Tip: VSCode Copilot Chat extension

can provide the same info as GitHub CLI copilot

Is GitHub Copilot

making me more productive?

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

Bill Gates, MS Clippy user

Testing Productivity

🚨 "AI, write me 10 tests"

✅ "AI, help me write this test faster"

🎁 "AI, debug this failed test"

GitHub Copilot in Action

  • how to ask GitHub Chat questions
  • context and session
  • use commands suggested by the Chat
  • asking Chat for authoritative sources
  • running the first Cypress test

Bayeux Tapestry: Battle of Hastings (1066) King Harold is Fatally Injured When an Arrow Pierces His Eye



Clear Chat session and add prettier

  • how to clear Copilot Chat conversion session
  • how to insert suggested code snippets into a file
  • format source code using Prettier

Copilot Suggestions

  • how to always show the Copilot suggestions bar
  • seeing multiple suggestions at once using the Completions Panel
  • tweaking the code suggestions using comments
  • using Chat to suggest a better solution

Add a comment like "// format the price using USD currency" to see better suggestions

Writing Unit Tests Using Copilot

  • how to prompt Copilot to generate a unit test file
  • how to adjust the generated test file to use Cypress globals and Chai assertions
  • how to generate additional unit tests

Guide Copilot Using Comments

  • the generated test quality is determined by the local context
  • adding more comments with concrete values helps generate the correct test code

Guiding A Human QA The Same Way

  • visit the page
  • the page finishes loading 👎
  • the body element has class "loaded" 👍
  • find the input element with class "new-todo"
  • type "Hello, World!" + Enter
  • the list should have 1 item with "Hello, World!" text

Test Adding Todos

  • naming spec file
  • accepting the full Copilot suggestion vs the next word
  • guiding the Copilot to generate a comment vs a cy command chain
  • keep Cypress open to see the test result
  • reset the data using beforeEach hook

Test Deleting Todos

  • generating JSON fixtures
  • inspecting the generated test code
  • setting up the initial test data by loading a fixture and making an API call

Spy on Network Calls

  • how Copilot can explain source code
  • spying on the network calls using the cy.intercept command
  • waiting on network calls using the cy.wait command

Modify Code In Place

  • modifying the existing code using the Command-I tool

Fix Code In Place

  • fixing the selected code using the Copilot /fix command
  • generate the commit messages based on the changed source files

Document your code

  • using /doc command to generate JSDoc comments
  • how to document your Cypress tests using cy.log commands

Generate Better Code By Example

  • how to remove duplicated selectors in the generated queries
  • how Copilot can use local variables and functions
  • how you ultimately can dramatically improve the final test code quality

Generate Test Data

  • how you can generate arrays of objects following a simple schema
  • how Copilot update the generate list to refine it further

Copilot 👍

  • Easy test data generation
  • Good starting point
  • Follows the comments
  • Speeds up typing

Copilot 🔥

  • Hallucinations
  • Outdated information
  • Unorthodox suggestions
  • Complex suggestions
  • ❌ writing code that I do not know how to write myself (new language or framework)
  • ❌ write big chunks of code (generate 10 tests at once)
  • ✅ speeding up writing code guided by comments
  • ✅ test data generation
  • ✅ code or data in-place transformations
  • 😒 commit message generation

GitHub Copilot

❤️ the "/explain" command

GitHub Copilot is the "Y" key when playing Mario

GitHub Copilot Workspace

GitHub Copilot Workspace

GitHub Copilot Workspace

Writing Tests Using GitHub Copilot

Gleb Bahmutov

👏 Thank You 👏

Writing Tests Using GitHub Copilot

By Gleb Bahmutov

Writing Tests Using GitHub Copilot

I want to show you the secret weapon I have been using for the past year. The GitHub Copilot lets me write full tests quickly, fills the gaps in my knowledge of 3rd party tools, and even writes clear descriptive commit messages. But it is not a "press the button to do it all". You need to guide the AI to do the right thing, which takes experience, but most importantly, you need to decide _what_ you want the AI to do, step by step. Remember: AI is not going to replace you any time soon. But someone being a lot more productive with AI's help might get ahead. Let's learn Copilot for real-world test writing. It might change how you work. If you master it, Copilot becomes your super weapon. It is like playing Super Mario - in some games, you can dash quickly by pressing the "Y" button, but you need to be pointing in the right direction first. Presented at iJS 2024, 30 minutes

  • 529