Roadmap to Testing a Design System

Marie Drake, Principal Test Automation Engineer

 

 

{
	"name": "Marie Drake",
	"originCountry": "🇵🇭",
  	"currentCountry": "🇬🇧",
  	"workRole": "Principal Test Automation Engineer",
  	"otherRoles": [
  		"Cypress Ambassador",
  		"Tech Blogger"
  		"Co-organizer of Cypress UK Community Group"
  		"Mum and Woman in Tech"
  	],
	"interestingFact": "I saw the great Stephen Hawking 
		on a cruise ship"
}

Agenda

1. News UK Tech Reset

2. Product Platforms

3. NewsKit Design System

4. Testing Requirements

5. Testing Issues

6. Testing Strategy and Tools

7. Team feedback and highlight

Reset: Why

Global Strategy

News Corp’s global strategy is to break

down the siloed approach of our companies

to deliver shared services, platforms, tools

and processes to eradicate unnecessary

duplication.

Source: Tech Reset Product Platforms

Delivering Tech expertise outside the core

products creates tension, distraction and a

strain on agreed and existing workloads.

 

Examples:

• Virgin Radio

• News IQ

• Expert Traveller





 

Source: Tech Reset Product Platforms

Responding to new brand ideas 

‘The Fabulous Pink Incident’

 

Fabulous changed their brand colour, Tech estimated the change to take 6 months.

 

There is no single source of truth for any one atom / component / module to overwrite and inherit.

 

‘Lower priority’ tasks are too prohibitive to be able to justify and deliver.





 

Source: Tech Reset Product Platforms

Speed to Market

Product Platforms

Objective

To provide a library of shared reusable components that will serve as building blocks from which title teams and brands across News UK and NewsCorp can reuse.

ShareBars across the business..

19 Sharebars = 19 code and design repetitions

NewsKit Design System

NewsKit Design System

Why Design System?

  • Cost Efficient 💰
  • Reusable ♻️
  • Speed to Market 🚀
  • Scalable 👍🏼
  • Standard ways of working 👭
  • Consistent 🔝

NewsKit Components

NewsKit Documentation Site

Testing Strategy

Testing Requirements

  • Test different components easily

  • Cross browser visual validation is a must

  • Write visual tests with less maintenance in mind

  • Deliver a high performing build pipeline

  • Build times must be less than 10 minutes

  • Integrate Design Review earlier on the process

  • Test for Accessibility on both Component and Site level

  • Catch functional issues earlier on

  • Have all the tests written before deploying a feature

Testing Strategy

  • Unit testing 
  • Snapshot testing
  • Component testing (NewsKit Components)
    • Functional
    • Visual
    • Accessibility
  • Front end testing (NewsKit Site)
    • Functional
    • Visual
    • Accessibility
  • Cross browser testing

Visual Testing is an important testing strategy here at News UK but…

 

Visual Tests are flaky!

 

Problem

Functional Testing Drawbacks

checkIfAllLiveblogPostsAreVisible() {
  const failedPosts = [];
  this.requiredLiveblogPosts.forEach(element => {
    browser.scrollElement(element);
    if(!browser.isElementVisible(element)) {
      failedPosts.push(element);
    }
  });
  
  return failedPosts.length === 0 ? true 
  	: Error(`List of elements not visible: ${failedPosts}`);
}

Visual Testing (Manually)

Visual Testing (Automated)

  • It's not new! 
  • A lot of existing visual tools already (pixel by pixel comparison)
if(isBaselineFound) {
  runSnapshot();
  checkDifferences();
  reportTestStatus();
} else {
  promoteImageAsBaseline();
}

Problem: Flaky tests

Problem: Dynamic data

Problem: Cross browser visual validation

Production Visual Bugs

How can we meet our testing requirements and solve our visual testing issue? 🤔

POC Visual Testing Tools

UI Testing Tools

Testing Tools

Testing Tools

Overview of Visual Testing setup

Applitools Setup (Storybook)

module.exports = {
  appName: 'DS Components',
  batchName: 'DS Components',
  storybookUrl: 'http://localhost:6006',
  browser: [
    {deviceName: 'iPhone 5/SE'},
    {width: 1024, height: 768, name: 'firefox'},
    {width: 1024, height: 768, name: 'ie11'},
    {width: 1024, height: 768, name: 'chrome'},
    {width: 1024, height: 768, name: 'safari'},
  ],
  concurrency: 100,
  viewportSize: {width: 1027, height: 768},
  waitBeforeScreenshot: 5000,
};
"dev:storybook": "start-storybook --ci -p 6006 -s ./fonts",
"eyes:storybook": "eyes-storybook --conf applitools.components.config.js",
"test:visual:comps:ci": "start-server-and-test 
	dev:storybook http://localhost:6006 eyes:storybook"
npm i --D @applitools/eyes-storybook

Applitools Setup (Cypress)

module.exports = {
  appName: 'DS Site',
  batchName: 'DS Site',
  browser: [
    {deviceName: 'iPhone 5/SE'},
    {deviceName: 'iPad'},
    {width: 1024, height: 768, name: 'firefox'},
    {width: 2880, height: 1800, name: 'chrome'},
  ],
  waitBeforeScreenshot: 5000,
  concurrency: 20,
};
"serve:docs": "http-server public -p 8081",
"e2e:visual:docs:ci": "start-server-and-test serve:docs http://localhost:8081 
	'TZ=UTC cypress run 
	--config-file cypress/config/cypress.docs.visual.json'"
npm i --D @applitools/eyes-cypress

npx eyes-setup

Writing the visual tests

const routes = {
  articleByline: '/components/article-byline',
  audioPlayer: '/components/audio-player',
  dateLine: '/components/date-line',
  icons: '/foundations/icons',
  radioPlayer: '/components/radio-player',
  shareBar: '/components/share-bar',
  spacing: '/foundations/spacing',
  typography: '/foundations/typography',
  welcome: '/index',
};

Object.entries(routes).forEach(route => {
  const [pageName, path] = route;

  describe(`${pageName} page`, () => {
    it(`should pass visual regression test on ${pageName}`, () => {
      cy.eyesOpen();
      cy.visit(path);
      if (!['spacing', 'welcome'].includes(pageName)) {
        cy.get('[data-testid="sample-code"]')
          .first()
          .scrollIntoView({
            easing: 'linear',
          });
      }
      cy.eyesCheckWindow(`${pageName} page`);
      cy.eyesClose();
    });
  });
});

Visual Test Maintenance

Dynamic Changes

track loading

Overview of Accessibility Testing setup

Component Accessibility Tests

a11yComponentRules.forEach(component => {
  describe(`${component.name} component`, () => {
    it('should pass basic a11y test', () => {
      cy.visit(`?name=${component.name}`);
      cy.injectAxe();

      if (component.disabledRules === undefined) {
        cy.checkA11yWithDefaultRules();
      } else {
        cy.checkA11yWithCustomRule(component.disabledRules);
      }
    });
  });
});
npm i --D cypress-axe


// support/index.js
import 'cypress-axe';

Automated Accessibility Tests

Screen Reader Accessibility Tests

Testing Workflow

Quality is everyone's responsibility

Pull Request Guidelines

Build pipelines (PR Workflow)

Build pipelines (Develop Workflow)

Github Integration

Slack Integration

Applitools Experience

Development Team Feedback

👏🏼 Fulfills our testing requirements

👏🏼 Simplifies our visual testing process

👏🏼 760 component tests less than 5 mins

👏🏼 High performing build pipeline

👏🏼 Test maintenance simpler

👏🏼 Applitools support

Design Team Feedback

👏🏼 Focus on fine visual details

👏🏼 Great level of test coverage

👏🏼 When reviewing visual changes, Applitools dashboard is very intuitive once you grasp the basics.

👏🏼 AI functionality is a time saver

Improvements

🤔 Applitools UI could be more intuitive especially for new users

🤔 Ability to set Baseline images from actual UI design (Sketch/Figma/Abstract plugin?)

🤔 More visual love on Documentation Site (License restrictions)

Accessibility Score

Supporting Times Radio 

Use the right tools to support your testing 🙂

Questions?

Marie Drake, Principal Test Automation Engineer

 

 

Roadmap to Testing a Design System

By Marie Drake

Roadmap to Testing a Design System

  • 209
Loading comments...

More from Marie Drake