Testing

Performance

Web

Why do I talking about Web Performance?

Web

Performance

Product

Why this talk?

...to avoid this

...to prevent this

git blame 🙈

.adCard {
	content-visibility: auto;
	contain-intrinsic-size: 171px;
    ...
}

In Web Performance,

there is no silver bullet

Lab vs RUM

Prevent with RUM & alerts

Implementing a new feature

Definition

If this feature degrades the performance

Implement

Test

Deploy

Web Perf Check

Backlog

🫠

Web Performance Testing

Web Performance Testing | Lighthouse CI

Automate running Lighthouse for every commit, viewing the changes, and preventing regressions


npm i @lhci/cli

Lighthouse CI

Lighthouse CI

...
  "scripts": {
    "start": "serve src",
    "test": "lhci autorun"
  },
...

Lighthouse CI

module.exports = {
  ci: {
    collect: {
      startServerCommand: "npm start",
      numberOfRuns: 1,
      url: [ "http://localhost:3000/" ],
    },
    upload: {
      target: "temporary-public-storage",
    },
  },
};

Lighthouse CI | lighthouserc.js DevBcn

...
    collect: {
      url: [
        "https://www.devbcn.com/",
        "https://www.devbcn.com/schedule",
        "https://www.devbcn.com/talks",
        "https://www.devbcn.com/speakers",
        "https://www.devbcn.com/travel",
        "https://www.devbcn.com/sponsorship",
      ],
    },
...

Lighthouse CI | lighthouserc.js DevBcn

...
    assert: {
      preset: 'lighthouse:recommended',
      assertions: {
        "categories:performance": 
        	["warn", { minScore: 0.8 }],
        "categories.pwa": "off",
      },
    },
...

Lighthouse CI | lighthouserc.js DevBcn

...
  settings: {
    onlyCategories: ["performance"], // only perf
    chromeFlags:
      "--headless --no-sandbox --disk-cache-size=0",
    "throttling-method": "devtools",
    throttling: {
      requestLatencyMs: 70,
      downloadThroughputKbps: 12000,
      cpuSlowdownMultiplier: 1,
     },
  },
...

Lighthouse CI | lighthouserc.js DevBcn

const defaultAssertions = {
  // WARNINGS
  'uses-optimized-images': 'warn',
  'image-alt': 'warn',
  'link-name': 'warn',
  /*
      disable automatic reported metrics from Next
      https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/getting-started.md#add-assertions
    */
  'aria-hidden-focus': 'off',
  'bootup-time': 'off',
  'button-name': 'off',
  canonical: 'off',
  deprecations: 'off',
  'color-content': 'off',
  'color-contrast': 'off',
  'crawlable-anchors': 'off',
  'csp-xss': 'off',
  'dom-size': 'off',
  'efficient-animated-content': 'off',
  'errors-in-console': 'off',
  'first-contentful-paint': 'off',
  'first-meaningful-paint': 'off',
  'font-display': 'off',
  'image-aspect-ratio': 'off',
  'image-size-responsive': 'off',
  interactive: 'off',
  'inspector-issues': 'off',
  'is-crawlable': 'off',
  'legacy-javascript': 'off',
  'main-thread-tasks': 'off',
  'mainthread-work-breakdown': 'off',
  'meta-viewport': 'off',
  'no-document-write': 'off',
  'no-unload-listeners': 'off',
  'non-composited-animations': 'off',
  'offscreen-images': 'off',
  'render-blocking-resources': 'off',
  'robots-txt': 'off',
  'server-response-time': 'off',
  'speed-index': 'off',
  'tap-targets': 'off',
  'third-party-facades': 'off',
  'third-party-summary': 'off',
  'total-byte-weight': 'off',
  'unused-css-rules': 'off',
  'unused-javascript': 'off',
  'unsized-images': 'off',
  'uses-long-cache-ttl': 'off',
  'uses-rel-preload': 'off',
  'uses-responsive-images': 'off',
  'uses-text-compression': 'off',
  'valid-source-maps': 'off',
}

module.exports = {
  ci: {
    collect: {
      numberOfRuns: 3,
      settings: {
        onlyCategories: ['performance'], // enable only performance tests
        chromeFlags: '--disable-dev-shm-usage --headless --no-sandbox --disk-cache-size=0',
        'throttling-method': 'devtools',
        throttling: {
          requestLatencyMs: 70,
          downloadThroughputKbps: 12000,
          cpuSlowdownMultiplier: 1,
        },
      },
    },
    assert: {
      assertMatrix: [
        {
          matchingUrlPattern: 'https://www.devbcn.com/',
          preset: 'lighthouse:no-pwa',
          assertions: {
            'largest-contentful-paint': [
              'warn',
              { maxNumericValue: 3150 },
              'error',
              { maxNumericValue: 3300 },
            ],
            'cumulative-layout-shift': [
              'warn',
              { maxNumericValue: 0.1 },
              'error',
              { maxNumericValue: 0.15 },
            ],
            'total-blocking-time': [
              'warn',
              { maxNumericValue: 5800 },
              'error',
              { maxNumericValue: 6000 },
            ],
            'max-potential-fid': ['warn', { maxNumericValue: 150 }],
            ...defaultAssertions,
          },
        },
        {
          matchingUrlPattern: 'https://www.devbcn.com/talks',
          preset: 'lighthouse:no-pwa',
          assertions: {
            'largest-contentful-paint': [
              'warn',
              { maxNumericValue: 4100 },
              'error',
              { maxNumericValue: 4300 },
            ],
            'cumulative-layout-shift': [
              'warn',
              { maxNumericValue: 0.2 },
              'error',
              { maxNumericValue: 0.38 },
            ],
            'total-blocking-time': [
              'warn',
              { maxNumericValue: 11200 },
              'error',
              { maxNumericValue: 11500 },
            ],
            'max-potential-fid': ['warn', { maxNumericValue: 250 }],
            ...defaultAssertions,
          },
        },
        {
          matchingUrlPattern: 'https://www.devbcn.com/speakers',
          preset: 'lighthouse:no-pwa',
          assertions: {
            'largest-contentful-paint': [
              'warn',
              { maxNumericValue: 4100 },
              'error',
              { maxNumericValue: 4300 },
            ],
            'cumulative-layout-shift': [
              'warn',
              { maxNumericValue: 0.2 },
              'error',
              { maxNumericValue: 0.25 },
            ],
            'total-blocking-time': [
              'warn',
              { maxNumericValue: 8500 },
              'error',
              { maxNumericValue: 8800 },
            ],
            'max-potential-fid': ['warn', { maxNumericValue: 250 }],
            ...defaultAssertions,
          },
        },
      ],
    },
    upload: {
      serverBaseUrl: 'https://localhost:9001',
    },
  },
}

Lighthouse CI | GitHub Action

Lighthouse CI | GitHub Action

name: Lighthouse
on: push
jobs:
  static-dist-dir:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Lighthouse against a static dist dir
        uses: treosh/lighthouse-ci-action@v11
        with:
          # no urls needed, since it uses local folder to scan .html files
          configPath: './lighthouserc.json'

main.yml

{
  "ci": {
    "collect": {
      "staticDistDir": "./dist"
    }
  }
}

lighthouserc.json

Lighthouse CI | GitHub Action

name: Lighthouse
on: push
jobs:
  lighthouse:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Lighthouse on urls and upload data to private lhci server
        uses: treosh/lighthouse-ci-action@v11
        with:
          urls: 'https://example.com/'
          serverBaseUrl: ${{ secrets.LHCI_SERVER_URL }}
          serverToken: ${{ secrets.LHCI_SERVER_TOKEN }}

Upload results to a private LHCI server

Lighthouse CI | GitHub Action

Lighthouse CI | GitHub Action

Lighthouse CI | GitHub Action

Lighthouse CI | lighthouserc.js DevBcn

module.exports = {
...

    assertions: {
      'largest-contentful-paint': [
        'warn',
        { maxNumericValue: 3150 },
        'error',
        { maxNumericValue: 3300 },
      ],
    },
...
}

Lighthouse CI | budget.json

[
  {
    "resourceSizes": [],
    "timings": [
      {
        "metric": "largest-contentful-paint",
        "budget": 2000
      },
      {
        "metric": "max-potential-fid",
        "budget": 80
      },
      {
        "metric": "cumulative-layout-shift",
        "budget": 0.06
      }
    ]
  }
]

Web Performance Testing | Lighthouse CI Server

Lighthouse CI Server

npm install @lhci/server sqlite3
const {createServer} = require('@lhci/server');

console.log('Starting server...');
createServer({
  port: process.env.PORT,
  storage: {
    storageMethod: 'sql',
    sqlDialect: 'sqlite',
    sqlDatabasePath: '/path/to/db.sql',
  },
}).then(({port}) => console.log('LHCI listening on port', port));
npx lhci server --storage.storageMethod=sql\
		--storage.sqlDialect=sqlite\
        --storage.sqlDatabasePath=./db.sql

Why this talk?

...to prevent this

Lighthouse user flows

Lighthouse user flows

async function captureReport() {
  const browser = await puppeteer.launch({headless: false});
  const page = await browser.newPage();

  const testUrl = 'https://web.dev/performance-scoring/';
  const flow = await startFlow(page, {name: 'Cold and warm navigations'});
  await flow.navigate(testUrl, {
    stepName: 'Cold navigation'
  });
  await flow.navigate(testUrl, {
    stepName: 'Warm navigation',
    configContext: {
      settingsOverrides: {disableStorageReset: true},
    },
  });

  await browser.close();

  const report = await flow.generateReport();
  fs.writeFileSync('flow.report.html', report);
  open('flow.report.html', {wait: false});
}

captureReport();

Cypress-web-vitals

describe("Web Vitals", () => {
  it("should pass the audits for a page load", () => {
    cy.vitals({ url: "https://www.google.com/" });
  });

  it("should pass the audits for a customer journey", () => {
    cy.startVitalsCapture({
      url: "https://www.google.com/",
    });

    cy.findByRole("combobox", { name: "Search" }).realClick();
    cy.findByRole("listbox").should("be.visible");

    cy.reportVitals();
  });
});
cy.vitals({ firstInputSelector: "main" });
cy.vitals({ thresholds: { cls: 0.2 } });

"Web Performance is not like doing a diet. It's a lifestyle change"

Danilo Velasquez

Web Performance Testing - DevBcn 2024

By Joan León

Web Performance Testing - DevBcn 2024

That Web Performance is an issue that impacts user experience, SEO positioning and, therefore, visits, and business metrics, is a reality. Knowing the most relevant Web Performance metrics and being able to test them in every pass we make to production will save us a lot of trouble. In this talk we will tell you how to configure and implement an automated system to test Web Performance with Lighthouse CI, and thus avoid degrading the UX of our product.

  • 367