End-to-End Functional and Visual Testing for the Web

April 10th, 2019

tip: these slides are arranged in columns,

so press the Down arrow to see the next slide

Webinar video at https://www.youtube.com/watch?v=MXfZeE9RQDw

Mike Fotinakis

Co-Founder and CEO

Percy.io

Gleb Bahmutov

VP of Engineering

Cypress.io

Hosts

Mike Fotinakis

percy.io

@mikefotinakis

Co-Founder, CEO

Your all-in-one
visual review platform.

Gleb Bahmutov

Fast, easy and reliable testing for anything that runs in a browser.

gleb.dev

@bahmutov

2 years with Cypress

Cypress intro
Visual testing intro
Cypress + Percy demo
How Percy works under the hood
Tips and tricks
Q & A

Add your questions during this webinar

After the webinar

We will send an email with the video, slides and an online survey

If you complete the survey, we'll send you a Cypress T-shirt 👕

Example web application

https://github.com/cypress-io/angular-pizza-creator

fork of https://toddmotto.com/angular-pizza-creator/

Web framework does not matter

End-to-End Tests

Example web application

Functional tests

User can order a pizza

Visual tests

Application looks correct

Functional tests

Open localhost:3000
Enter delivery information
Select toppings
Click "Place Order"
- Confirmation popup

$ npm install -D cypress

Win / Mac / Linux
MIT License
OSS https://github.com/cypress-io/cypress
http://on.cypress.io/introduction-to-cypress

tip: if you are browsing the slides without demo video,

click down arrow to see the individual steps

/// <reference types="Cypress" />
context('Pizza Creator', () => {
  beforeEach(() => {
    // uses base url setting from cypress.json
    // which right now points at "localhost:3000"
    cy.visit('/')
  })

  it('orders custom pizza', function () {
    // enter delivery information
    cy.get('[formcontrolname="name"]').type('Joe')
    cy.get('[formcontrolname="email"]').type('foo@bar.com')
    cy.get('[formcontrolname="confirm"]').type('foo@bar.com')

    // without complete delivery information,
    // we should not be able to place the order
    cy.get('button[type="submit"]').should('be.disabled')

    ...
  })
})

demo cypress/integration/order-spec.js

    // add a few toppings
    cy.contains('label.pizza-topping', 'Pepperoni').click()
    cy.contains('label.pizza-topping', 'Onion').click()
    cy.contains('label.pizza-topping', 'Mozzarella').click()
    cy.contains('label.pizza-topping', 'Basil').click()

    // check the price and order pizza
    cy.contains('.pizza-summary__total-price', 'Total: $12.75')

    // let us confirm we can place our order,
    // but first, prepare for "window.alert" call
    cy.on('window:alert', cy.stub().as('alert'))

    // now the button should be enabled
    cy.get('button[type="submit"]')
      .should('be.enabled')
      .click()
    cy.get('@alert').should('have.been.calledWithExactly', 'Order placed')
  })
})

demo cypress/integration/order-spec.js

Simplify tests with Cypress custom commands

https://on.cypress.io/custom-commands

cy.get('[formcontrolname="name"]').type('Joe')

Instead of this:

const f = name => `[formcontrolname="${name}"]`

Cypress.Commands.add('enterForm', (name, text) => {
  // enter text into the form control without Command Log messages
  const quiet = { log: false }
  cy.get(f(name), quiet).type(text, quiet)
})

cy.enterForm('name', 'Joe')

Define custom command

https://on.cypress.io/custom-commands

cypress/support/index.js

cy.get('[formcontrolname="name"]').type('Joe')
cy.get('[formcontrolname="email"]').type('foo@bar.com')
cy.get('[formcontrolname="confirm"]').type('foo@bar.com')
cy.get('[formcontrolname="address"]').type('1 Pizza st')
cy.get('[formcontrolname="postcode"]').type('12345')
cy.get('[formcontrolname="phone"]').type('1234567890')

Instead of this:

Cypress.Commands.add('enterDeliveryInformation', () => {
  cy.enterForm('name', 'Joe')
  cy.enterForm('email', 'foo@bar.com')
  cy.enterForm('confirm', 'foo@bar.com')
  cy.enterForm('address', '1 Pizza st')
  cy.enterForm('postcode', '12345')
  cy.enterForm('phone', '1234567890')
})

https://on.cypress.io/custom-commands

it('orders custom pizza', function () {
  cy.enterDeliveryInformation()
  cy.get('button[type="submit"]').should('be.disabled')

  cy.pickToppings('Pepperoni', 'Onion', 'Mozzarella', 'Basil')

  // check the price and order pizza
  cy.contains('.pizza-summary__total-price', 'Total: $12.75')
 
  cy.on('window:alert', cy.stub().as('alert'))

  // now the button should be enabled
  cy.get('button[type="submit"]')
    .should('be.enabled')
    .click()
  cy.get('@alert').should('have.been.calledWithExactly', 'Order placed')
})

demo cypress/integration/dry-spec.js

Better test

All features can be tested

Ordering pizza
Edge cases (blank delivery)
Requests to the server
Internal app state

#FFDC71

#71FF71

?

Who likes their pizza crust green?

How can we catch things like

#FFDC71

#71FF71

?

🚫

Visual testing

Automated process of detecting and reviewing visual UI changes.

Gain full confidence in every visual change at every step.

Protect against visual regressions and bugs across browsers and responsive widths.

percy.io/visual-testing

Visual testing

∆

=

Visual diffs

?

Visual testing

Great for web applications, components & component libraries,

end-to-end testing frameworks,

and static sites.

Visual testing

Catch visual regressions before they reach your users
Write smaller, more effective tests
Delete/refactor CSS, layouts, and designs without fear
Make dependency upgrades safe
Test visualizations easily
Reach continuous deployment

Challenges

🤨 Hard to test complex app states

🤩 — but not with Cypress!

😳 Performance

😡 Consistent & deterministic rendering

😬 Day-to-day review process & approval workflow.

$ npm install -D @percy/cypress

import '@percy/cypress'
// ...

cypress/support/index.js

cy.percySnapshot('<name>')

@percy/cypress

tip: if you are browsing the slides without demo video,

click down arrow to see the individual steps

it('draws pizza correctly', function () {
  cy.percySnapshot('Empty Pizza')

  cy.enterDeliveryInformation()
  const toppings = ['Pepperoni', 'Chili', 'Onion']
  cy.pickToppings(...toppings)

  // make sure the web app has updated
  cy.contains('.pizza-summary__total-price', 'Total: $12.06')
  cy.percySnapshot(toppings.join(' - '))
})

demo cypress/integration/visual-spec.js

CI setup

1. Add PERCY_TOKEN to your CI environment variables.

CI setup

# instead of
npx cypress run

# wrap with percy exec:
npx percy exec -- cypress run

1. Add PERCY_TOKEN to your CI environment variables.

2. Wrap your test suite with "percy exec -- ":

see circle.yml

Demo! (setup already done)

1. Create a Percy organization & project

2. Link GitHub repo to Percy project

3. Add percySnapshot and push to "master" branch to set baseline

https://percy.io/cypress-io/angular-pizza-creator

"master" commits

example build

Visual review workflow

Make pull request with visual changes
Review & approve visual changes
Merge pull request after visual review

tip: if you are browsing the slides without demo video,

click down arrow to see the individual steps

https://github.com/cypress-io/angular-pizza-creator/pull/7

https://percy.io/cypress-io/angular-pizza-creator/builds/1696055

https://percy.io/cypress-io/angular-pizza-creator

Percy dashboard

How does Percy do cross-browser testing?

Default resolutions

Visual review workflow

<style type="text/css">
  .st0{fill:#FFD8A1;}
  .st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
  .st3{fill:#DFBA86;}
</style>

changes crust color SVG

1. Make pull request with visual changes

Visual review workflow

catch visual changes in the pull request

2. Review & approve visual changes

Visual review workflow

2. Review & approve visual changes

Visual review workflow

New snapshots become baseline images after merging into master branch

3. Merge pull request after visual review

Cypress intro
Visual testing intro
Cypress + Percy demo
How Percy works under the hood
Tips and tricks
Q & A

Add your questions during this webinar

How does Percy work?

Integrate

Set up one of Percy's SDKs for web apps, component libraries, end-to-end testing frameworks, and static sites.

Run

Percy renders and compares snapshots across browsers and screens, highlighting relevant visual diffs.

Review

Review and approve visual changes to ensure your UI is always production-ready and keep your team on the same page.

percy.io/how-it-works

cy.percySnapshot('Empty Pizza')

percy-agent

cypress runner

DOM snapshots + assets

(SHA256 fingerprinted)

Source code integration

Visual rendering & testing infrastructure

https://docs.percy.io/docs/cypress

cy.percySnapshot();
cy.percySnapshot(snapshotName);
cy.percySnapshot(snapshotName, options);

// examples
cy.percySnapshot();
cy.percySnapshot('Homepage test');
cy.percySnapshot('Homepage responsive test', 
    { widths: [768, 992, 1200] });

"cy.percySnapshot" command

Consistent snapshots

// make sure the web app has updated
cy.contains('.pizza-summary__total-price', 'Total: $12.06')
cy.percySnapshot()

// freezes the system time to Jan 1, 2018
const now = new Date(2018, 1, 1)
cy.clock(now)
... test
cy.percySnapshot()

// use the same data
cy.route('/api/users', 'fixture:users')
... test
cy.percySnapshot()

CSS animation demo

Make a pull request with very long pizza topping drop animation
Percy automatically freezes animation in rendering

branch "long-drop"

pull request #8

Percy unchanged diff

Functional tests

Application works correctly

Visual tests

Application looks right

Conclusion

Confidence in every change

More Information

🖥 slides.com/bahmutov/visual-testing-with-percy/
📃 glebbahmutov.com/blog/testing-visually/
📚 docs.percy.io/docs/cypress
⭐️ github.com/cypress-io/cypress

Mike Fotinakis

Co-Founder and CEO

Percy.io

Gleb Bahmutov

VP of Engineering

Cypress.io

End-to-End Functional and Visual Testing for the Web

Hosts

Mike Fotinakis

Gleb Bahmutov

Contents

After the webinar

Example web application

Example web application

Functional tests

Visual tests

Functional tests

Simplify tests with Cypress custom commands

Instead of this:

Define custom command

Instead of this:

Better test

All features can be tested

#FFDC71

#71FF71

?

How can we catch things like

#FFDC71

#71FF71

?

🚫

Visual testing

Automated process of detecting and reviewing visual UI changes.

Visual testing

∆

=

Visual diffs

?

Visual testing

Great for web applications, components & component libraries,

end-to-end testing frameworks,

and static sites.

Visual testing

Catch visual regressions before they reach your users

Write smaller, more effective tests

Delete/refactor CSS, layouts, and designs without fear

Make dependency upgrades safe

Test visualizations easily

Reach continuous deployment

Challenges

CI setup

CI setup

Demo! (setup already done)

Visual review workflow

Percy dashboard

Visual review workflow

Visual review workflow

Visual review workflow

Visual review workflow

Contents

How does Percy work?

"cy.percySnapshot" command

Consistent snapshots

CSS animation demo

Functional tests

Visual tests

Conclusion

Confidence in every change

More Information

Q & A