E2E testing with Puppeteer

Organised by:

Frontenders Ticino

Hosted by:

by Stefano Magni (@NoriSte)

Front-end Developer

Level: basic
Prerequisites: knowledge of JS Promises and async/await

02/02/2019

Wi-Fi: Tecnopolo Ticino, pwd: SuglioEst5

If you haven't cloned the repo yet:

• clone (Clone it! Don't simply download it!)

https://github.com/NoriSte/feti-workshop-e2e-testing-with-puppeteer

• $ npm install

• $ npm test

If everything works you will see:

 PASS  test/min.e2e.test.js
  Generic check
    ✓ You're ready for the workshop :) (818ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.121s

https://slides.com/noriste/e2e-testing-workshop-feti

Today's slides

First of all:

Why is testing so important?

It's useful to find bugs and corner cases while coding.

To avoid manually testing ourselves the same things over and over.

To avoid any kind of regression while updating our (or others) code.

To confidently release our code without worries.

To constantly check that our products are up and running.

In the end: testing is fundamental to sleep at night...

What kind of tests exist?

With UT you test functions, classes, in general everything you consider a "unit".

You check the behaviour of your units with every kind of input they can receive.

They say little about your code but they are extremely fast, both to be written and to be run.

Unit tests

With Integration tests you test a limited amount of units/components in a controlled environment.

You check how they work together and they allow you to start seeing the bigger picture.

Generally you mock (replace with fake but credible code) almost everything external to what you're focusing on.

Integration tests

They test the visual side of your web-app, they literally make a screenshot of your page and compare every pixel with the previous one.

They are extremely slow and they are almost dedicated to test CSS side of your project.

Regression tests

Funny moment: a GIF I found while studying for this workshop 😁

The goal of the author was to tease the unit testers to introduce them about the need for E2E testing.

... We are going to start in a few minutes, be patient 🙏🏻

End-to-end tests

The testing pyramid

Kent C. Dodds's return of investment testing "pyramid"

Always use the simplest kind of test

Never write an E2E test to test something you can test with an Integration one... And never write an Integration test if you can get the same result with an Unit one.

What is E2E testing

(or UI testing)?

From Wikipedia:

End-to-end testing is a methodology used to test whether the flow of an application is performing as designed from start to finish.

It is useful to test your site/webapp/app in the same way your real users consume it.

The user interactions are automated.

From the Puppeteer.page.click API documentation:

This method fetches an element with selector, scrolls it into view if needed, and then uses page.mouse to click in the center of the element.

The most important thing:

E2E tests are hosted in a real browser.

E2E tests are the only tool that can check if your website really works or not.

What is Puppeteer?

Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol.

Puppeteer lets you control a Chrome instance and fire user-like interactions (click, typing etc.) within the page.

https://github.com/GoogleChrome/puppeteer

It introduces the concept of headless browser that is a full and standard browser that runs without a GUI.

1: Node running the script

2: Chromium launched with Puppeteer

3: Code executed in page

A typical script to launch Puppeteer

Remember some useful characteristics of Puppeteer...

You can install Chrome extensions on launch.

// welcome external devtools
return await puppeteer.launch({
  args: [
    // loading Chrome Vue devtools
    `--load-extension=/.../vue-devtools/shells/chrome/`,
    ]
});

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#working-with-chrome-extensions

You can launch a single and shareable Chrome instance to improve performance.

// if you have previously saved the Chrome endpoint you can connect to it
// instead of creating a new one
browser = await puppeteer.connect({browserWSEndpoint: existingEndPoint});

e.g. https://github.com/smooth-code/jest-puppeteer

Chrome flags (hidden or newer settings) are customisable.

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerdefaultargsoptions

const browser = await puppeteer.launch({
    args: ['--no-sandbox']
});

You can go incognito.

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#browsercreateincognitobrowsercontext

const browser = await puppeteer.launch();
// Create a new incognito browser context.
const context = await browser.createIncognitoBrowserContext();
// Create a new page in a pristine context.
const page = await context.newPage();
// Do stuff
await page.goto('https://example.com');

You could connect to a Chrome of yours (to leverage cookie, history etc.)

const browser = await puppeteer.launch({
    executablePath: '/path/to/Chrome'
});

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerexecutablepath

It allows you to easily understand if there are errors in page.

const browser = await puppeteer.launch({});
const page = await browser.newPage();

// when an error is thrown in Chrome
// it's logged by the node process
page.on('pageerror', e => console.log(e.text))

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#event-pageerror

You can intercept requests.

await page.setRequestInterception(true);

page.on('request', request => {
  // it intercepts every call to register.php and
  // responds with a custom JSON
  if (request.url() === 'register.php') {
    request.respond({
      content: 'application/json',
      body: JSON.stringify({registered: true})
    });
  }
  else {
    // other requests work as usual
    request.continue();
  }
});

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#event-request

You can set cookies.

await page.setCookie({
    name: 'loggedIn',
    value: '1'
});

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetcookiecookies

You can emulate devices.

const device = {
    'userAgent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1',
    'viewport': {
      'width': 375,
      'height': 667,
      'isMobile': true,
      'hasTouch': true,
      'isLandscape': false
    }
  }
await page.emulate(device);

// or you can use one of the presets

const devices = require('puppeteer/DeviceDescriptors');
const iPhone6 = devices['iPhone 6'];
await page.emulate(iPhone6);

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pageemulateoptions

• $, $$, $eval, $$eval: DOM utilities

• addScriptTag, addStyleTag: add resources in page

• page.evaluate: runs a function into the browser

• page.exposeFunction: adds a function

• page.click

• page.hover

And many others

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md

How can we debug a Puppeteer script?

Turn off headless mode

So you can see what the browser is doing.

Directly from

https://github.com/GoogleChrome/puppeteer#debugging-tips

const browser = await puppeteer.launch({
    headless: false
});

Slow down the browser

Not ideal for very long tests but super useful though.

Directly from

https://github.com/GoogleChrome/puppeteer#debugging-tips

const browser = await puppeteer.launch({
  headless: false,
  slowMo: 250 // slow down every action by 250ms
});

Avoid closing the browser

afterAll(async () => {
  // await browser.close();
});

Directly from... me 😊

Jest is a zero-configuration testing framework.

What is Jest?

https://jestjs.io

beforeAll(async () => {
  // code to be run before the whole test suite
});
beforeEach(async () => {
  // code to be run before every test
});
afterAll(async () => {
  // code to be run after the whole test suite
});

describe('Description 1', () => {
  test('Test 1', () => {
      expect(1).toBe(1);
  });
  test('Test 2', () => {
      expect('hello').not.toBe('world');
  });

  describe('Inner description', () => {
    test('Inner test 1', () => {
        expect({foo: 'bar'}).toEqual({foo: 'bar'});
    });
    test('Inner test 2', () => {
        expect(0).toBeFalsy();
    });
  });
});

 PASS  ./test.js
  Description 1
    ✓ Test 1 (5ms)
    ✓ Test 2 (1ms)
    Inner description
      ✓ Inner test 1 (2ms)
      ✓ Inner test 2 (1ms)

Test Suites: 1 passed, 1 total
Tests:       4 passed, 4 total
Snapshots:   0 total
Time:        1.071s
Ran all test suites.

test.js

result of $ jest

It should be obvious but remember: the fact that your suite tests is running in a real and interactive browser... it doesn't mean that you should interact with it!

When you are running the browser in non-headless mode remember to not affect the test with your mouse/keyboard.

Never interact with the test

The button takes the user to another page.

Complete the test that checks it.

git checkout test-1

open ./dist/test-1.html

open ./test/test-1.e2e.test.js

• index.html has a big blue button

• test-1.e2e.test.js misses something (see the comments at line 22)

• launch the test suite with $ npm test

Test 1

Valid for every test: You can't change the HTML files 😇

A rule

 FAIL  test/test-1.e2e.test.js
  That's our first E2E test
    ✕ The button brings the user to the next page (1156ms)

  ● That's our first E2E test › The button brings the user to the next page

    Error

      TimeoutError: Text not found "Hello from FETI"
      waiting for function failed: timeout 500ms exceeded

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        2.223s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

Test 1

$ npm run --> Why it fails?

await expect(page).toMatch('Hello from FETI');

Test 1

It fails because this expectation timed out...

// test/test-1.e2e.test.js

describe(`That's our first E2E test`, () => {
  test(`The button brings the user to the next page`, async () => {
    await page.goto(`file:${path.join(__dirname, './../dist/test-1.html')}`);

    // - run $npm test
    // - the test will fail
    // - use puppeteer to click the button
    // - and then re-run $npm test

    await expect(page).toMatch('Hello from FETI');
  }, 5000);
});

Test 1

p.s. you can't cheat modifying the page itself 😁

git checkout test-1-solution

On the test-1-solution branch you can find a solution of mine.

Solution

describe(`That's our first E2E test`, () => {
  test(`The button brings the user to the next page`, async () => {
    await page.goto(`file:${path.join(__dirname, './../dist/test-1.html')}`);

    // always add a 'data-test' attribute to the elements that will
    // participate to your tests
    await page.click('[data-test="button"]');

    // checking for a specific content is a good way to be 100% sure
    // that the page has been loaded
    await expect(page).toMatch('Hello from FETI');
  }, 5000);
});

Solution

// waiting for your code...

How did you solve it?

Send me the code you found on Stack Overflow 😇 (nori.ste.magni+spam[AT]gmail.com) and we'll discuss it!

<a data-test="button">

<!--
find it with the following selector
[data-test="button"]
-->

Always add a data-test (or data-testid) attribute to the elements that will be referenced by the tests.

Best practice

Why?

Because every attribute has a “role”.

We use .classes for CSS (and they got replaced in case of CSS Modules), #ids for JS, and therefore they can change based on CSS and JS needs…

A dedicated attribute is more resilient to refactoring and leads every (diligent) developer to keep it or, at least, ask himself for the attribute use.

data-test attribute

Ok, so:

• E2E tests are very important...

• the first test seemed really easy to be written…

Amazing, I’ll write ~1000 E2E tests as soon as I back home!!!

Wow...

Let’s speak about the cons!

They’re the longest to be written, not in terms of lines of code but in terms of variables to be considered.

Cons

Why?

Because you aren’t in isolation (like in Unit tests), nor in a super-mocked environment...

You have a real browser on a real network, you’ll face network latencies, temporary downs of the server, service workers, unpredictable AD scripts and banners, (possible) browser drivers inconsistencies…

Cons

Simulating the (exact) user behaviour sometimes could be very tricky (blur, input typos etc.).

Cons

They’re not always parallelisable (in simple cases they are, but they can lead to errors and debugging hell).

Cons

The more complex they are, the more difficult they are to debug.

Cons

If you try to DRY your tests you’re probably adding another (hard to manage) complexity layer.

Cons

They can fail, get used with that, there are too many context variables to be considered.

Cons

They are slow, even with almost instant UI interactions.

Cons

Everyone defines them flaky and brittle...

Cons

and

Keep calm

Code on

git checkout test-2

open ./dist/test-2.html

open ./test-2.e2e.test.js

• now if you launch the test suite (with $ npm test) the test fails, why?

Test 2

// test/test-2.e2e.test.js
beforeAll(async () => {
  browser = await puppeteer.launch({
    headless: true,
    slowMo: 0
  });
  page = await browser.newPage();
});

// ...

describe(`That's our second E2E test`, () => {
  test(`The button brings the user to the next page`, async () => {
    await page.goto(`file:${path.join(__dirname, './../dist/test-2.html')}`);

    // always add a 'data-test' attribute to the elements that will participate to your tests
    await page.click('[data-test="button"]');

    // it fails, why?

    // checking for a specific content is a good way to be 100% sure that the page has been loaded
    await expect(page).toMatch('Hello from FETI');
  }, 5000);
});

Test 2

git checkout test-2-solution

On the test-2-solution branch you can find a solution of mine.

Solution

describe(`That's our second E2E test`, () => {
  beforeAll(async () => {
    await page.goto(`file:${path.join(__dirname, './../dist/test-2.html')}`);

    // don't let the test fail for a silly element like a cookie footer
    // It could be already accepted when you navigate to another page
    if(await page.$('[data-test="cookie-footer-acceptance"]')) {
      try {
        // what happens if it exists but isn't clickable (eg. it's hidden)?
        // A try/catch will manage the case
        await page.click('[data-test="cookie-footer-acceptance"]');
      } catch(e) {
        // the element exists but isn't clickable
      }
    }
  });
});

Solution

// waiting for your code...

How did you solve it?

Send me the code you found on Stack Overflow 😇 (nori.ste.magni+spam[AT]gmail.com) and we'll discuss it!

Never use some "sleep" code, you can’t determine how much a page/script could be waited for (render waitings, network conditions etc.).

Use waiters, promises, framework-specific render callbacks (like Vue.nextTick) but not sleep.

Never "sleep"

git checkout test-3

open ./dist/test-3.html

open ./test/test-3.e2e.test.js

Again! Now the cookie footer disappears with a CSS animation!

Test 3

test(`The button brings the user to the next page`, async () => {
  // always add a 'data-test' attribute to the elements that will participate to your tests
  await page.click('[data-test="button"]');

  // it fails again...

  // check for a specific content is a good way to be 100% sure that the page is been loaded
  await expect(page).toMatch('Hello from FETI');
}, 5000);

Test 3

git checkout test-3-solution

On the test-3-solution branch you can find a solution of mine.

Solution

if(await page.$('[data-test="cookie-footer-acceptance"]')) {
  try {
    await page.click('[data-test="cookie-footer-acceptance"]');
    // you can wait that an element is hidden
    // @see https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagewaitforselectorselector-options
    await page.waitForSelector('#cookie-footer', {
      hidden: true
    });
  } catch(e) {
    // the element exists but maybe it isn't clickable
  }
}

Solution

// waiting for your code...

How did you solve it?

Send me your code (nori.ste.magni+spam[AT]gmail.com) and we'll discuss it!

What should I test?

I suggest you to test only the best cases or the ones that generate value ($$$) for your company.

Testing everything make you lose a lot of time.

Test something that broke in the past or that breaks often.

git checkout test-4

open ./dist/test-4.html

open ./test/test-4.e2e.test.js

The last time with the button, trust me 😊

Now the cookie footer disappears (from a user perspective... not from a CSS one) dispatching an event!

Test 4

test(`The button brings the user to the next page`, async () => {
  // always add a 'data-test' attribute to the elements that will participate to your tests
  await page.click('[data-test="button"]');
  
  // it fails again...
  
  // checking for a specific content is a good way to be 100% sure that the page has been loaded
  await expect(page).toMatch('Hello from FETI');
}, 5000);

Test 4

git checkout test-4-solution

On the test-4-solution branch you can find a solution of mine.

Solution

if(await page.$('[data-test="cookie-footer-acceptance"]')) {
  try {
    await page.click('[data-test="cookie-footer-acceptance"]');

    // @see https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pageevaluatepagefunction-args
    await page.evaluate(() => new Promise(resolve => {
      // the following code will run into the browser page
      window.addEventListener('cookieFooterDidHide', () => resolve());
    }));

  } catch(e) {
    // the element exists but maybe it isn't clickable
  }
}

Solution

// waiting for your code...

How did you solve it?

Send me your code (nori.ste.magni+spam[AT]gmail.com) and we'll discuss it!

With page.evaluate you can execute code into the running browser (like you do when you write code into the DevTools console) and give back the result to your NodeJS script.

Page.evaluate example

const puppeteer = require('puppeteer');

(async (options) => {
  browser = await puppeteer.launch({
    headless: false,
    devtools: true
  });
  page = await browser.newPage();

  console.log('Node logging');
  await page.evaluate(
    () => console.log('Browser logging')
  );
})()

const seven = 7;
const result = await page.evaluate(aNumber => {
  // JQuery is in page? Redux?
  // Use them and give back results to the NodeJS script
  return aNumber * 10;
}, seven);
console.log(result); // 70

Take care of the scope

const seven = 7;
const result = await page.evaluate(() => {
    // error! Because the scope is the Chrome's window!
    // Not the NodeJS script!
    return seven * 10;
});
console.log(result);

To better understand the scope problem: remember that the Node script and the launched Chrome instance are separated

Use a promise for every async code

const seven = 7;
const result = await page.evaluate(aNumber => {
  // you can use a promise for every async stuff
  return new Promise(resolve => {
    setTimeout(() => resolve(aNumber*10), 1000);
  });
}, seven);
console.log(result); // 70

About the test-4 solution

// code refactored without arrow functions
await page.evaluate(function() {
  // a Promise is returned to the page.evaluate
  // so it waits until the promise is fullfilled
  return new Promise(resolve => {
    // now we're in the Chromium instance, we can listen for
    // the event triggered on the window
    window.addEventListener('cookieFooterDidHide', function(){
      // when the event is been triggered we fullfill the promise
      resolve();
    });
  });
});
// the code after the page.evaluate will be run once the event
// in the browser will be triggered

Page.evaluate is often used to retrieve data from the DOM

const SELECTOR = '[href]:not([href=""])';
let link;

link = await page.evaluate((sel) => 
    document.querySelector(sel).getAttribute('href')
  , SELECTOR);

But it isn't the only way to do that!

const SELECTOR = '[href]:not([href=""])';
let link;

link = await page.evaluate((sel) => 
    document.querySelector(sel).getAttribute('href')
  , SELECTOR);

// compare the two following examples

link = await page.$eval(SELECTOR, el => el.getAttribute('href'));

// or

link = await page.$(SELECTOR).getProperty('href').jsonValue();

Let's see some other debugging tips

Capture console output

Directly from

https://github.com/GoogleChrome/puppeteer#debugging-tips

page.on('console', msg => console.log('PAGE LOG:', msg.text()));

Launch Puppeteer with the DevTools already opened

Directly from

https://github.com/GoogleChrome/puppeteer#debugging-tips

const browser = await puppeteer.launch({
  headless: false,
  slowMo: 250, // slow down by 250ms
  devtools: true
});

Add an evaluate statement with debugger inside

Or add debugger to an existing evaluate statement (it works if the Chrome DevTools are opened).

Directly from

https://github.com/GoogleChrome/puppeteer#debugging-tips

await page.evaluate(() => {debugger;});
// The test will now stop executing in the above evaluate statement,
// and chromium will stop in debug mode.

Change default test timeout

Directly from

https://github.com/GoogleChrome/puppeteer#debugging-tips

jest.setTimeout(100000);

Run a single test/suite

describe.only('Temporary run only me please', () => {
    test('The test we want to isolate', () => { ... });
});

describe('A suite that won\'t be launched', () => {
    test('Test 2', () => { ... });
});

describe('Another suite that won\'t be launched', () => {
    test('Test 3', () => { ... });
});

From Jest

https://jestjs.io/docs/en/api#describeonlyname-fn

Skip some tests/suites

describe('Suite 1', () => {
    test('The test we want to isolate', () => { ... });
});

describe.skip('A suite that will be skipped', () => {
    test('Test 2', () => { ... });
});

describe('Suite 3', () => {
    test.skip('A test that will be skipped', () => { ... });
});

From Jest

https://jestjs.io/docs/en/api#describeskipname-fn

console.log (into the Chrome instance) the name of the test, so you always know which test is running if you are watching them.

await page.evaluate(() => console.log('Test name'));

Directly from... me 😊

You can also add a fixed div in page to show the test name at the top of the page, it’s not ideal but it could be very useful in some situations.

Directly from... me 😊

Use the right assertion

https://jestjs.io/docs/en/expect

Directly from... me 😊

Expected: true
Received: false
    3 | describe(`Test`, () => {
    4 |   test(`Leverage assertions`, () => {
  > 5 |     expect(getObj().key === 5).toBe(true);
// Ok, what is the value of getObj().key?

Expected: 5
Received: 4
    3 | describe(`Test`, () => {
    4 |   test(`Leverage assertions`, () => {
  > 5 |     expect(getObj().key).toBe(5);
// ok I'll fix it without relaunching the suite

Run test serially with

Directly from... me 😊

$ npm test --runInBand

because a page executed in the background could have some unexpected behaviors...

Add a Todo and check that it has been showed in page.

git checkout test-5

open ./dist/test-5.../index.html

open ./test/test-5.e2e.test.js

• index.html is the usual Todo List app

• test-5.e2e.test.js is almost empty (see the comments at line 37)

• launch the test suite with $ npm test

Test 5 (step 1)

git checkout test-5-step1-solution

Solution

On the test-5-step1-solution branch you can find a solution of mine.

// fill the input
await page.type(inputSelector, todos[0]);
// click the button
await page.click(buttonSelector);

// run a script in page to get the innerText of the new todo
const innerText = await page.evaluate((selector) => {
  // remember that this function hasn't a scope
  return document.querySelector(selector).innerText;
}, itemSelector);

// and then check it with Jest
expect(innerText.includes(todos[0])).toEqual(true);

Solution

// waiting for your code...

How did you solve it?

Send me your code (nori.ste.magni+spam[AT]gmail.com) and we'll discuss it!

git checkout test-5-2

open ./dist/test-5.../index.html

open ./test/test-5.e2e.test.js

Test 5 (step 2)

• test-5.e2e.test.js, line 63

• add all the todos in the array, remove the first two and then... read the store and check that the last two todos exist in the store itself.

P.s. I exposed a window.vueInstance variable so you can access the store with window.vueInstance.$store.state.todos

git checkout test-5-step2-solution

On the test-5-step2-solution branch you can find a solution of mine.

Solution

const state = await page.evaluate(() =>
window.vueInstance.$store.state.todos);

expect(state.map(item => item.text))
.toEqual(['Pass this test', 'Join the FETI volunteers']);

On the test-5-step2-solution branch you can find a solution of mine.

Solution

This test leaves room for a series of cases that should make you think about how it’s difficult to manage all the cases consistently when E2E testing.

// waiting for your code...

How did you solve it?

Send me your code (nori.ste.magni+spam[AT]gmail.com) and we'll discuss it!

Others best practices

Puppeteer is Chrome/Firefox(at the moment) but if you use Selenium etc. remeber to avoid launching every test on every browser. Choose a reference browser and carefully select the test to be run on the other ones.

Choose where to test

Make a screenshot if a test fails. It can help you avoiding to relaunch the suite with the browser in non-headless mode.

Make screenshots

if (array.length !== 3)
    await page.screenshot({path: 'screenshot.png'})
expect(array.length).toBe(3)

In a page where everything changes soon… try to standardize your testing environment to avoid false negatives.

Attention to the pages that update frequently

Don't think to test every corner case with E2E testing, it's pure madness.

Leave perfectionism at home

An assertion is auto-explicative, an error isn't.

Assert frequently

await page.goto(AUTHENTICATED_ROOT);

expect(page.url()).toEqual(expect.not.stringContaining('/login'));

// you won't have a "element not found"
// if something went wrong with the authentication
await page.click('[data-test="create-new-post"]');

Puppeteer alternatives?

widely used

not so easy to install/setup

it has a WebDriver for every desktop browser (IE too)

Expected Conditions FTW (a sort of page.waitForSelector on steroids)

Selenium WebDriver

amazing "DevTools" UI

clear errors

play/pause functionality

made only for E2E testing, it's not a generic automation browser

Cypress.io

works in real browsers (so you can even test on mobile ones)

supported by BrowserStack and CrossBrowserTesting

who uses it loves it 🙂

TestCafe

Again, first of all: consider what and why you should test on mobile...

You can use the device emulation of Chrome DevTools with Puppeteer

If you really need to test on real devices choose TestCafe

What about mobile browsers?

If you want to see the differences between Puppeteer/TestCafe/Cypress you can take a look at a repository I made to solve an issue on StackOverflow:

https://github.com/NoriSte/stackoverflow-52383438-cypress-issue

Are E2E tests framework-agnostic?

Yes but every framework has (obviously) a lot of libraries to help you write them.

What can I do with Puppeteer as well as test my web apps?

Automate annoying and repetitive browser tasks.

eg. We used it to configure a new website with ISPConfig.

Scrape whatever you want from the browser (eg. data fetch without APIs).

Make self-contained Node.js apps with a UI (thank to Carlo by Google.

Use it as a development tool

If you're developing a "change password" flow, for every change you make to the code you need to manually:

• login

• go to the profile page

• fill the form to change the password

• click logout

• login with the new password

Use it as a development tool

Just to realize that you have a bug... Then fix it and start again...

You could use Puppeteer to bring you directly at the end of the flow.

And once you finished... Add some assertions and your E2E test is ready 😊

Use it while studying

When you're studying a new framework you don't know if you're breaking something you developed just some hours ago... unless you check it manually! Use some tests instead.

Eg. I used them extensively during my first refactors while studying Vue.

Scrape the first 30 results from Google for the given query, have fun 😊

You write the code, I wrote the test to check it, when you're ready launch $ npm test and my test (test/test-6.js) will tell you if everything is fine.

Some tests are initially skipped, enable them one by one.

git checkout test-6

open ./scrape-google-test-6.js

Test 6

git checkout test-6-solution

On the test-6-solution branch you can find a solution of mine.

Solution

// you have two ways for retrieving an element attribute,
// one is through JSHandle functions...
const jsHandle = await els[j].getProperty('href');
const url = await jsHandle.jsonValue();
// ...
// ...
// ... and one with the usual page.evaluate function
const url = await page.evaluate(
  (sel) => document.querySelector(sel).getAttribute('href'),
nextButtonSelector);

Solution

On the test-6-solution branch you can find a solution of mine.

// waiting for your code...

How did you solve it?

Send me your code (nori.ste.magni+spam[AT]gmail.com) and we'll discuss it!

• E2E tests really matter, they don't know anything about your architecture, they test what the end user sees

• they’re quite easy to develop, with a flat learning curve

• write a few amount of tests, don't delegate to E2E testing something you can test with other testing methodologies

Conclusion

• remember to take the tests simple, they could be extremely time-consuming

• never "sleep" the browser

• use "data-test"/"data-testid" attributes

• remember that you can scrape/execute whatever you want on the web with an automated browser

Conclusion

• https://puppeteersandbox.com

Playground

• https://www.blazemeter.com/blog/top-15-ui-test-automation-best-practices-you-should-follow

• https://medium.freecodecamp.org/why-end-to-end-testing-is-important-for-your-team-cb7eb0ec1504

• https://blog.kentcdodds.com/write-tests-not-too-many-mostly-integration-5e8c7fff591c

• https://hackernoon.com/testing-your-frontend-code-part-iii-e2e-testing-e9261b56475

• https://gojko.net/2010/04/13/how-to-implement-ui-testing-without-shooting-yourself-in-the-foot-2/

• https://willowtreeapps.com/ideas/how-to-get-the-most-out-of-ui-testing

• http://www.softwaretestingmagazine.com/knowledge/graphical-user-interface-gui-testing-best-practices/

• https://www.slideshare.net/Codemotion/codemotion-webinar-protractor

• https://frontendmasters.com/courses/testing-javascript/introducing-end-to-end-testing/

• https://medium.com/welldone-software/an-overview-of-javascript-testing-in-2018-f68950900bc3

• https://medium.com/yld-engineering-blog/evaluating-cypress-and-testcafe-for-end-to-end-testing-fcd0303d2103

Sources

• https://egghead.io/courses/end-to-end-testing-with-google-s-puppeteer-and-jest



• https://egghead.io/courses/end-to-end-testing-with-cypress

Egghead courses

• https://testingjavascript.com/

Kent's course (reccomended)

Any feedbacks??

• FETI and Fabio Prada for all the work they do

• Fondazione Agire for the amazing location

• Massimo Foti for inspiring me with his Unit Testing Workshop

• Kent C. Dodds for the amazing “Solidifying what you learn” post

• Luca Previtali and Creeo Studio that allowed me to try this workshop in advance

A special thank to:

Last but not least:

"Share to learn" really works!

Thank you!

Organised by:

Frontenders Ticino

Hosted by:

by Stefano Magni (@NoriSte)

Front-end Developer

The repository with the code and the link to these slides

https://github.com/NoriSte/feti-workshop-e2e-testing-with-puppeteer

02/02/2019

E2E testing workshop for FETI

By Stefano Magni

E2E testing workshop for FETI

In fall 2018 I had a workshop for the FETI (FrontEnders Ticino) community in Chiasso (https://www.meetup.com/it-IT/FrontEnders-Ticino/events/258183736/). The workshop aimed to introduce the attendees to the amazing world of browser automation, mostly for E2E testing but some exercises were about web scraping too.

4,004

Stefano Magni

I'm a passionate Front-end Engineer, a speaker, and an instructor. I love creating high-quality products, testing and automating everything, learning and sharing my knowledge, helping people. I work remotely for Hasura.