How to test performance

and not die trying

Diego Cardozo - Sr. Web Performance Engineer

NetSuite

Agenda

 

  • What to measure
  • Quick tests
  • Automation
  • Selling performance

What to measure (1)

 

Types of performance testing for web applications (According to Microsoft)

Test type Goal
Performance Determine speed and scalability
Load Determine behavior under normal circumstances
Stress Determine behavior beyond normal circumstances
Capacity How many users or transactions are supported while meeting performance goals

What to measure (2)

 

3 alternatives to set performance goals

  • Performance budget
  • Business-specific measurement
    like Twitter's "time to first tweet"
  • Google's RAIL model

What to measure (3)

 

Response time limits - Ph.D Jakob Nielsenn

Time Perseption Action
0 - 16ms Continuous Animation
0 - 100ms Immediate Response
100ms - 300ms Slight delay
300ms - 1s Natural progression Load
1s+ User loses focus
10s+ Frustration

Quick tests

Automation (1)

First proposal - continuous process

  • Run performance tests along with functional automated tests
  • Works together with performance budget
  • Knowing how performance evolves with time during development is extremely valuable
  • Example: sitespeed.io
  • Keynote

Automation (2)

Second proposal - RUM

  • Performance information is sent directly and passively from real users
  • Ideal for mobile apps
  • The testing team can experiment with RUM tools from day one
  • When the site or app goes live you will already have the necessary know how on interpreting results
  • Example: HP AppPulse (mobile)

Selling performance (1)

Present this ideas to your boss/customer

  • Google: 2% slower = 2% less searching per user
  • Yahoo: 400ms faster = 9% more traffic
  • Amazon: 100ms faster = 1% more revenue
  • Shopzilla: 5s faster = 25% more page views, 7 to 12% more revenue
  • Google uses website speed in search rankings

Selling performance (2)

Generate revenue directly from performance

  • Performance can be sold as a service or added value
  • Generates revenue which will cover for the cost of paid tools
  • If offered but not sold, it will still help you set expectations and prevent performance-related issues
  • Other alternatives such as performance alerts or paid reports can be sold too

Resources

¿Questions?

 

diegocard@gmail.com

Made with Slides.com