End-to-End Functional and Visual Testing for the Web

April 10th, 2019

tip: these slides are arranged in columns,

so press the Down arrow to see the next slide

Mike Fotinakis

Co-Founder and CEO

Percy.io

Gleb Bahmutov

VP of Engineering

Cypress.io

Hosts

Mike Fotinakis

Co-Founder, CEO

Your all-in-one
      visual review platform.

Gleb Bahmutov

Fast, easy and reliable testing for          anything that runs in a browser.

2 years with Cypress

Contents

  • Cypress intro
  • Visual testing intro
  • Cypress + Percy demo
  • How Percy works under the hood
  • Tips and tricks
  • Q & A 

Add your questions during this webinar

After the webinar

We will send an email with the video, slides and an online survey

If you complete the survey, we'll send you a Cypress T-shirt 👕

Example web application

Web framework does not matter

End-to-End Tests

Example web application

Functional tests

User can order a pizza

Visual tests

Application looks correct

Functional tests

  • Open localhost:3000
  • Enter delivery information
  • Select toppings
  • Click "Place Order"
    • Confirmation popup
$ npm install -D cypress

tip: if you are browsing the slides without demo video,

click down arrow to see the individual steps

/// <reference types="Cypress" />
context('Pizza Creator', () => {
  beforeEach(() => {
    // uses base url setting from cypress.json
    // which right now points at "localhost:3000"
    cy.visit('/')
  })

  it('orders custom pizza', function () {
    // enter delivery information
    cy.get('[formcontrolname="name"]').type('Joe')
    cy.get('[formcontrolname="email"]').type('foo@bar.com')
    cy.get('[formcontrolname="confirm"]').type('foo@bar.com')

    // without complete delivery information,
    // we should not be able to place the order
    cy.get('button[type="submit"]').should('be.disabled')

    ...
  })
})
    // add a few toppings
    cy.contains('label.pizza-topping', 'Pepperoni').click()
    cy.contains('label.pizza-topping', 'Onion').click()
    cy.contains('label.pizza-topping', 'Mozzarella').click()
    cy.contains('label.pizza-topping', 'Basil').click()

    // check the price and order pizza
    cy.contains('.pizza-summary__total-price', 'Total: $12.75')

    // let us confirm we can place our order,
    // but first, prepare for "window.alert" call
    cy.on('window:alert', cy.stub().as('alert'))

    // now the button should be enabled
    cy.get('button[type="submit"]')
      .should('be.enabled')
      .click()
    cy.get('@alert').should('have.been.calledWithExactly', 'Order placed')
  })
})

Simplify tests with Cypress custom commands

cy.get('[formcontrolname="name"]').type('Joe')

Instead of this:

const f = name => `[formcontrolname="${name}"]`

Cypress.Commands.add('enterForm', (name, text) => {
  // enter text into the form control without Command Log messages
  const quiet = { log: false }
  cy.get(f(name), quiet).type(text, quiet)
})

cy.enterForm('name', 'Joe')

Define custom command

cypress/support/index.js

cy.get('[formcontrolname="name"]').type('Joe')
cy.get('[formcontrolname="email"]').type('foo@bar.com')
cy.get('[formcontrolname="confirm"]').type('foo@bar.com')
cy.get('[formcontrolname="address"]').type('1 Pizza st')
cy.get('[formcontrolname="postcode"]').type('12345')
cy.get('[formcontrolname="phone"]').type('1234567890')

Instead of this:

Cypress.Commands.add('enterDeliveryInformation', () => {
  cy.enterForm('name', 'Joe')
  cy.enterForm('email', 'foo@bar.com')
  cy.enterForm('confirm', 'foo@bar.com')
  cy.enterForm('address', '1 Pizza st')
  cy.enterForm('postcode', '12345')
  cy.enterForm('phone', '1234567890')
})
it('orders custom pizza', function () {
  cy.enterDeliveryInformation()
  cy.get('button[type="submit"]').should('be.disabled')

  cy.pickToppings('Pepperoni', 'Onion', 'Mozzarella', 'Basil')

  // check the price and order pizza
  cy.contains('.pizza-summary__total-price', 'Total: $12.75')
 
  cy.on('window:alert', cy.stub().as('alert'))

  // now the button should be enabled
  cy.get('button[type="submit"]')
    .should('be.enabled')
    .click()
  cy.get('@alert').should('have.been.calledWithExactly', 'Order placed')
})

Better test

All features can be tested

  • Ordering pizza
  • Edge cases (blank delivery)
  • Requests to the server
  • Internal app state

#FFDC71 

#71FF71

?

Who likes their pizza crust green?

How can we catch things like

#FFDC71 

#71FF71

?

🚫

Visual testing

Automated process of detecting and reviewing visual UI changes.

Gain full confidence in every visual change at every step.

 

Protect against visual regressions and bugs across browsers and responsive widths.

Visual testing

=

Visual diffs

?

Visual testing

Great for web applications, components & component libraries,

end-to-end testing frameworks,

and static sites.

 

Visual testing

  • Catch visual regressions before they reach your users

  • Write smaller, more effective tests

  • Delete/refactor CSS, layouts, and designs without fear

  • Make dependency upgrades safe

  • Test visualizations easily

  • Reach continuous deployment

 

Challenges

🤨 Hard to test complex app states

     🤩 — but not with Cypress!

😳 Performance

😡 Consistent & deterministic rendering

😬 Day-to-day review process & approval workflow.

$ npm install -D @percy/cypress
import '@percy/cypress'
// ...

cypress/support/index.js

cy.percySnapshot('<name>')

1.

2.

3.

@percy/cypress

tip: if you are browsing the slides without demo video,

click down arrow to see the individual steps

it('draws pizza correctly', function () {
  cy.percySnapshot('Empty Pizza')

  cy.enterDeliveryInformation()
  const toppings = ['Pepperoni', 'Chili', 'Onion']
  cy.pickToppings(...toppings)

  // make sure the web app has updated
  cy.contains('.pizza-summary__total-price', 'Total: $12.06')
  cy.percySnapshot(toppings.join(' - '))
})

CI setup

1. Add PERCY_TOKEN to your CI environment variables.

CI setup

# instead of
npx cypress run

# wrap with percy exec:
npx percy exec -- cypress run

1. Add PERCY_TOKEN to your CI environment variables.

2. Wrap your test suite with "percy exec -- ":

Demo! (setup already done)

1. Create a Percy organization & project

 

2. Link GitHub repo to Percy project

 

3. Add percySnapshot and push to "master" branch to set baseline

Visual review workflow

  1. Make pull request with visual changes
  2. Review & approve visual changes
  3. Merge pull request after visual review

tip: if you are browsing the slides without demo video,

click down arrow to see the individual steps

Percy dashboard

How does Percy do cross-browser testing?

Default resolutions

Visual review workflow

<style type="text/css">
  .st0{fill:#FFD8A1;}
  .st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
  .st3{fill:#DFBA86;}
</style>

changes crust color SVG

1. Make pull request with visual changes

Visual review workflow

catch visual changes in the pull request

2. Review & approve visual changes

Visual review workflow

2. Review & approve visual changes

Visual review workflow

New snapshots become baseline images after merging into master branch

3. Merge pull request after visual review

Contents

  • Cypress intro
  • Visual testing intro
  • Cypress + Percy demo
  • How Percy works under the hood
  • Tips and tricks
  • Q & A 

Add your questions during this webinar

How does Percy work?

Integrate

Set up one of Percy's SDKs for web apps, component libraries, end-to-end testing frameworks, and static sites.

Run

Percy renders and compares snapshots across browsers and screens, highlighting relevant visual diffs.

Review

Review and approve visual changes to ensure your UI is always production-ready and keep your team on the same page.

cy.percySnapshot('Empty Pizza')

percy-agent

cypress runner

DOM snapshots + assets

(SHA256 fingerprinted)

Source code integration

Visual rendering & testing infrastructure

cy.percySnapshot();
cy.percySnapshot(snapshotName);
cy.percySnapshot(snapshotName, options);

// examples
cy.percySnapshot();
cy.percySnapshot('Homepage test');
cy.percySnapshot('Homepage responsive test', 
    { widths: [768, 992, 1200] });

"cy.percySnapshot" command

Consistent snapshots

// make sure the web app has updated
cy.contains('.pizza-summary__total-price', 'Total: $12.06')
cy.percySnapshot()

// freezes the system time to Jan 1, 2018
const now = new Date(2018, 1, 1)
cy.clock(now)
... test
cy.percySnapshot()

// use the same data
cy.route('/api/users', 'fixture:users')
... test
cy.percySnapshot()

CSS animation demo

  1. Make a pull request with very long pizza topping drop animation
  2. Percy automatically freezes animation in rendering

branch "long-drop"

pull request #8

Functional tests

Application works correctly

Visual tests

Application looks right

Conclusion

Confidence in every change

More Information

Mike Fotinakis

Co-Founder and CEO

Percy.io

Gleb Bahmutov

VP of Engineering

Cypress.io

Q & A

Visual Testing With Cypress and Percy

By Gleb Bahmutov

Visual Testing With Cypress and Percy

Cypress.io gives you a fast and reliable way to ensure your app is functioning exactly as intended on every commit. Using it with Percy’s visual testing capability provides end-to-end confidence. Join Mike Fotinakis, Co-Founder and CEO of Percy, and Cypress’ VP of Engineering, Gleb Bahmutov, on Wednesday, April 10 at 10 am PDT / 1 pm EDT to learn about the value of using Cypress and Percy, both independently and together, as well as to get an in-depth look at integrating and writing tests for your needs. Video at https://www.youtube.com/watch?v=MXfZeE9RQDw

  • 20,340