How to test performance
and not die trying
Diego Cardozo - Sr. Web Performance Engineer
NetSuite
Agenda
- What to measure
- Quick tests
- Automation
- Selling performance
What to measure (1)
Types of performance testing for web applications (According to Microsoft)
Test type | Goal |
---|---|
Performance | Determine speed and scalability |
Load | Determine behavior under normal circumstances |
Stress | Determine behavior beyond normal circumstances |
Capacity | How many users or transactions are supported while meeting performance goals |
What to measure (2)
3 alternatives to set performance goals
- Performance budget
- Business-specific measurement
like Twitter's "time to first tweet" - Google's RAIL model
What to measure (3)
Response time limits - Ph.D Jakob Nielsenn
Time | Perseption | Action |
---|---|---|
0 - 16ms | Continuous | Animation |
0 - 100ms | Immediate | Response |
100ms - 300ms | Slight delay | |
300ms - 1s | Natural progression | Load |
1s+ | User loses focus | |
10s+ | Frustration |
Quick tests
-
Web page test
- Use ngrok to run local tests
- Utilities and plugins for CI and automation
- Google PageSpeed Insights
- Chrome's dev tools
- Measurements can be saved
- Emulates mobile
Automation (1)
First proposal - continuous process
- Run performance tests along with functional automated tests
- Works together with performance budget
- Knowing how performance evolves with time during development is extremely valuable
- Example: sitespeed.io
- Keynote
Automation (2)
Second proposal - RUM
- Performance information is sent directly and passively from real users
- Ideal for mobile apps
- The testing team can experiment with RUM tools from day one
- When the site or app goes live you will already have the necessary know how on interpreting results
- Example: HP AppPulse (mobile)
Selling performance (1)
Present this ideas to your boss/customer
Selling performance (2)
Generate revenue directly from performance
- Performance can be sold as a service or added value
- Generates revenue which will cover for the cost of paid tools
- If offered but not sold, it will still help you set expectations and prevent performance-related issues
- Other alternatives such as performance alerts or paid reports can be sold too
Resources
- Slides
slides.com/diegocard/testeando-performance - RAIL
www.smashingmagazine.com/2015/10/rail-user-centric-model-performance - Microsoft's performance testing guide for web apps
msdn.microsoft.com/en-us/library/bb924375.aspx
¿Questions?
diegocard@gmail.com
How to test performance and not die trying
By diegocard
How to test performance and not die trying
Presented at Testing Meetup, October 2015
- 1,766