Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Gleb Bahmutov, PhD
VP of Engineering, Cypress.io
Test Coverage Update
This.JavaScript - State of Testing
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional / testing
EveryScape
virtual tours
MathWorks
MatLab on the web
Kensho
finance dashboards
these slides
15 people. Atlanta, Philly, Boston, LA, Chicago, NYC
Fast, easy and reliable testing for anything that runs in a browser
Do I need to write more tests?
Code Coverage
-
Instrument the code
-
Run the tests
-
See if we reached every line of code
shows code coverage by labeling code
Coverage is hard
Code coverage
is tricky
const isEmail = (s) =>
/^\w+@\w+\.\w{3,4}$/.test(s)
// 1 test = 100% code coverage
(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])
Code coverage
vs
Data coverage
mocha -r data-cover spec.jsDo I need to write more end-to-end tests?
I don't think code/data coverage is useful for end-to-end tests
You cannot test every part of your car by driving around
user interface
application code
vendor code
polyfills
there is probably a lot of app code unreachable through the UI alone
beforeEach(() => {
cy.visit('/')
})
it('works', function () {
cy.get('.new-todo').type('first todo{enter}')
cy.get('.new-todo').type('second todo{enter}')
cy.get('.todo-list li').should('have.length', 2)
})What does this test cover?
beforeEach(() => {
cy.visit('/')
})
it('works', function () {
cy.get('.new-todo').type('first todo{enter}')
cy.get('.new-todo').type('second todo{enter}')
cy.get('.todo-list li').should('have.length', 2)
})What does this test cover?
track elements test interacts with, like "type"
Cypress.Commands.overwrite('type',
(type, $el, text, options) => {
rememberSelector($el)
return type($el, text, options)
})Track "cy.type" elements
Highlight tested element
Elements NOT covered by the test
beforeEach(() => {
cy.visit('/')
})
it('works', function () {
cy.get('.new-todo').type('first todo{enter}')
cy.get('.new-todo').type('second todo{enter}')
cy.get('.todo-list li').should('have.length', 2)
.first().find(':checkbox').check()
cy.contains('.filters a', 'Active').click()
cy.url().should('include', 'active')
cy.contains('.filters a', 'Completed').click()
cy.url().should('include', 'completed')
cy.contains('.filters a', 'All').click()
cy.url().should('include', '#/')
})Extend test to cover more elements
track "type", "check", "click" elements
The test did not cover "Clear completed" button
Problem: only has noticed the first checkbox, and not all of them
big problem: only "check" this box,
but not "uncheck"
tests
code
coverage
data
coverage
element
coverage
?
Do I need to write more tests?
Gleb Bahmutov, PhD
VP of Engineering
Thank you
@cypress_io
@bahmutov
github.com/cypress-io/cypress ⭐️
www.cypress.io/blog
Test Coverage Update: State of Testing
By Gleb Bahmutov
Test Coverage Update: State of Testing
Code coverage is useful to make sure every line of code with unit tests. But it has a few shortcomings and is mostly useless for end-to-end tests. What else can we do to target out end-to-end tests? In this presentation, I will touch on element coverage that seems to be useful when covering web application tests. In blog post form at https://www.cypress.io/blog/2018/12/20/element-coverage/
