Roadmap to Testing a Design System

Marie Drake, Principal Test Automation Engineer

{
	"name": "Marie Drake",
	"originCountry": "🇵🇭",
  	"currentCountry": "🇬🇧",
  	"workRole": "Principal Test Automation Engineer",
  	"otherRoles": [
  		"Cypress Ambassador",
  		"Tech Blogger"
  		"Co-organizer of Cypress UK Community Group"
  		"Mum and Woman in Tech"
  	],
	"interestingFact": "I saw the great Stephen Hawking 
		on a cruise ship"
}

Agenda

1. News UK Tech Reset

2. Product Platforms

3. NewsKit Design System

4. Testing Requirements

5. Testing Issues

6. Testing Strategy and Tools

7. Team feedback and highlight

Reset: Why

Global Strategy

News Corp’s global strategy is to break

down the siloed approach of our companies

to deliver shared services, platforms, tools

and processes to eradicate unnecessary

duplication.

Source: Tech Reset Product Platforms

Delivering Tech expertise outside the core

products creates tension, distraction and a

strain on agreed and existing workloads.

Examples:

• Virgin Radio

• News IQ

• Expert Traveller

Source: Tech Reset Product Platforms

Responding to new brand ideas

‘The Fabulous Pink Incident’

Fabulous changed their brand colour, Tech estimated the change to take 6 months.

There is no single source of truth for any one atom / component / module to overwrite and inherit.

‘Lower priority’ tasks are too prohibitive to be able to justify and deliver.

Source: Tech Reset Product Platforms

Speed to Market

Product Platforms

Objective

To provide a library of shared reusable components that will serve as building blocks from which title teams and brands across News UK and NewsCorp can reuse.

ShareBars across the business..

19 Sharebars = 19 code and design repetitions

NewsKit Design System

Why Design System?

Cost Efficient 💰
Reusable ♻️
Speed to Market 🚀
Scalable 👍🏼
Standard ways of working 👭
Consistent 🔝

NewsKit Components

NewsKit Documentation Site

Testing Strategy

Testing Requirements

Test different components easily
Cross browser visual validation is a must
Write visual tests with less maintenance in mind
Deliver a high performing build pipeline
Build times must be less than 10 minutes
Integrate Design Review earlier on the process
Test for Accessibility on both Component and Site level
Catch functional issues earlier on
Have all the tests written before deploying a feature

Testing Strategy

Unit testing
Snapshot testing
Component testing (NewsKit Components)
- Functional
- Visual
- Accessibility
Front end testing (NewsKit Site)
- Functional
- Visual
- Accessibility
Cross browser testing

Visual Testing is an important testing strategy here at News UK but…

Visual Tests are flaky!

Problem

Functional Testing Drawbacks

checkIfAllLiveblogPostsAreVisible() {
  const failedPosts = [];
  this.requiredLiveblogPosts.forEach(element => {
    browser.scrollElement(element);
    if(!browser.isElementVisible(element)) {
      failedPosts.push(element);
    }
  });
  
  return failedPosts.length === 0 ? true 
  	: Error(`List of elements not visible: ${failedPosts}`);
}

Visual Testing (Manually)

source: http://www.puzzlesandriddles.com/SpotTheDifference14.html

Visual Testing (Automated)

It's not new!
A lot of existing visual tools already (pixel by pixel comparison)

if(isBaselineFound) {
  runSnapshot();
  checkDifferences();
  reportTestStatus();
} else {
  promoteImageAsBaseline();
}

Problem: Flaky tests

Problem: Dynamic data

Problem: Cross browser visual validation

Production Visual Bugs

How can we meet our testing requirements and solve our visual testing issue? 🤔

POC Visual Testing Tools

UI Testing Tools

Testing Tools

Overview of Visual Testing setup

Applitools Setup (Storybook)

module.exports = {
  appName: 'DS Components',
  batchName: 'DS Components',
  storybookUrl: 'http://localhost:6006',
  browser: [
    {deviceName: 'iPhone 5/SE'},
    {width: 1024, height: 768, name: 'firefox'},
    {width: 1024, height: 768, name: 'ie11'},
    {width: 1024, height: 768, name: 'chrome'},
    {width: 1024, height: 768, name: 'safari'},
  ],
  concurrency: 100,
  viewportSize: {width: 1027, height: 768},
  waitBeforeScreenshot: 5000,
};

"dev:storybook": "start-storybook --ci -p 6006 -s ./fonts",
"eyes:storybook": "eyes-storybook --conf applitools.components.config.js",
"test:visual:comps:ci": "start-server-and-test 
	dev:storybook http://localhost:6006 eyes:storybook"

npm i --D @applitools/eyes-storybook

Applitools Setup (Cypress)

module.exports = {
  appName: 'DS Site',
  batchName: 'DS Site',
  browser: [
    {deviceName: 'iPhone 5/SE'},
    {deviceName: 'iPad'},
    {width: 1024, height: 768, name: 'firefox'},
    {width: 2880, height: 1800, name: 'chrome'},
  ],
  waitBeforeScreenshot: 5000,
  concurrency: 20,
};

"serve:docs": "http-server public -p 8081",
"e2e:visual:docs:ci": "start-server-and-test serve:docs http://localhost:8081 
	'TZ=UTC cypress run 
	--config-file cypress/config/cypress.docs.visual.json'"

npm i --D @applitools/eyes-cypress

npx eyes-setup

Writing the visual tests

const routes = {
  articleByline: '/components/article-byline',
  audioPlayer: '/components/audio-player',
  dateLine: '/components/date-line',
  icons: '/foundations/icons',
  radioPlayer: '/components/radio-player',
  shareBar: '/components/share-bar',
  spacing: '/foundations/spacing',
  typography: '/foundations/typography',
  welcome: '/index',
};

Object.entries(routes).forEach(route => {
  const [pageName, path] = route;

  describe(`${pageName} page`, () => {
    it(`should pass visual regression test on ${pageName}`, () => {
      cy.eyesOpen();
      cy.visit(path);
      if (!['spacing', 'welcome'].includes(pageName)) {
        cy.get('[data-testid="sample-code"]')
          .first()
          .scrollIntoView({
            easing: 'linear',
          });
      }
      cy.eyesCheckWindow(`${pageName} page`);
      cy.eyesClose();
    });
  });
});

Visual Test Maintenance

Dynamic Changes

track loading

Overview of Accessibility Testing setup

Component Accessibility Tests

a11yComponentRules.forEach(component => {
  describe(`${component.name} component`, () => {
    it('should pass basic a11y test', () => {
      cy.visit(`?name=${component.name}`);
      cy.injectAxe();

      if (component.disabledRules === undefined) {
        cy.checkA11yWithDefaultRules();
      } else {
        cy.checkA11yWithCustomRule(component.disabledRules);
      }
    });
  });
});

npm i --D cypress-axe


// support/index.js
import 'cypress-axe';

Automated Accessibility Tests

Screen Reader Accessibility Tests

Testing Workflow

Quality is everyone's responsibility

Pull Request Guidelines

Build pipelines (PR Workflow)

Build pipelines (Develop Workflow)

Github Integration

Slack Integration

Applitools Experience

Development Team Feedback

👏🏼 Fulfills our testing requirements

👏🏼 Simplifies our visual testing process

👏🏼 760 component tests less than 5 mins

👏🏼 High performing build pipeline

👏🏼 Test maintenance simpler

👏🏼 Applitools support

Design Team Feedback

👏🏼 Focus on fine visual details

👏🏼 Great level of test coverage

👏🏼 When reviewing visual changes, Applitools dashboard is very intuitive once you grasp the basics.

👏🏼 AI functionality is a time saver

Improvements

🤔 Applitools UI could be more intuitive especially for new users

🤔 Ability to set Baseline images from actual UI design (Sketch/Figma/Abstract plugin?)

🤔 More visual love on Documentation Site (License restrictions)

Accessibility Score

Supporting Times Radio

Use the right tools to support your testing 🙂

Questions?

Marie Drake, Principal Test Automation Engineer

mariedrake.com

@mcruzdrake

/in/marie-drake/

Roadmap to Testing a Design System

By Marie Cruz

Roadmap to Testing a Design System

3,092

Marie Cruz

Marie is a software tester and also a tech blogger at testingwithmarie.com where she talks about testing and test automation in general. Currently, she works as a Developer Advocate for k6.io

Roadmap to Testing a Design System

Agenda

1. News UK Tech Reset

2. Product Platforms

3. NewsKit Design System

4. Testing Requirements

5. Testing Issues

6. Testing Strategy and Tools

7. Team feedback and highlight

Reset: Why

Global Strategy

Responding to new brand ideas

Speed to Market

Product Platforms

Objective

ShareBars across the business..

19 Sharebars = 19 code and design repetitions

NewsKit Design System

NewsKit Design System

Why Design System?

NewsKit Components

NewsKit Documentation Site

Testing Strategy

Testing Requirements

Testing Strategy

Problem

Functional Testing Drawbacks

Visual Testing (Manually)

Visual Testing (Automated)

Problem: Flaky tests

Problem: Dynamic data

Problem: Cross browser visual validation

Production Visual Bugs

How can we meet our testing requirements and solve our visual testing issue? 🤔

POC Visual Testing Tools

UI Testing Tools

Testing Tools

Testing Tools

Overview of Visual Testing setup

Applitools Setup (Storybook)

Applitools Setup (Cypress)

Writing the visual tests

Visual Test Maintenance

Dynamic Changes

Overview of Accessibility Testing setup

Component Accessibility Tests

Automated Accessibility Tests

Screen Reader Accessibility Tests

Testing Workflow

Quality is everyone's responsibility

Pull Request Guidelines

Build pipelines (PR Workflow)

Build pipelines (Develop Workflow)

Github Integration

Slack Integration

Applitools Experience

Development Team Feedback

Design Team Feedback

Improvements

Accessibility Score

Supporting Times Radio

Use the right tools to support your testing 🙂

Questions?

Roadmap to Testing a Design System

More from Marie Cruz