A Tale of Testing the Untestable

Angie Jones

http://angiejones.tech

http://testautomationu.com

@techgirl1908

Senior Developer Advocate, Applitools

Director, Test Automation University

San Francisco, CA, USA

@techgirl1908

I need

a change!

@techgirl1908

QA @ Twitter

testing as a service

@techgirl1908

Twitter Ads

@techgirl1908

Advertiser can create a campaign that targets users who meet certain criteria.

Feature

@techgirl1908

What are we testing?

@techgirl1908

@techgirl1908

@techgirl1908

The test

@techgirl1908

Challenge #1: Company lacks testing culture

@techgirl1908

What about integration?

@techgirl1908

There's a gap

@techgirl1908

Challenge #2:

Company lacks resources for testing

@techgirl1908

Challenge #3: Testing in production

@techgirl1908

Too expensive!

@techgirl1908

Challenge #4:

Targeting

test users

@techgirl1908

User ID Username Gender Interest
1............. ............. F science
2............. ............. M art
3............. ............. N art
4............. ............. U science
5............. ............. M science
6............. ............. F art
7............. ............. F art

@techgirl1908

@techgirl1908

Challenge #5:

Blocker in targeting lists

@techgirl1908

...then how???

@techgirl1908

Challenge #6:

Serving is not guaranteed

@techgirl1908

how does this thing work?

@techgirl1908

Challenge #7:

Going lower in the stack

@techgirl1908

No time!

@techgirl1908

Challenge #8:

Making time for quality

@techgirl1908

No

happy

ending

@techgirl1908

@techgirl1908

The 10 P's of Testability

by Rob Meaney

@techgirl1908

Retrospective

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The people in our team possess the mindset, skillset & knowledge set

to do great testing and are aligned in their pursuit of quality.

@techgirl1908

The people in our team possess the mindset, skillset & knowledge set

to do great testing and are aligned in their pursuit of quality.

@techgirl1908

The people in our team possess the mindset, skillset & knowledge set

to do great testing and are aligned in their pursuit of quality.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The philosophy of our team encourages whole team responsibility for quality and collaboration across team roles, the business and with the customer.

@techgirl1908

The philosophy of our team encourages whole team responsibility for quality and collaboration across team roles, the business and with the customer.

@techgirl1908

The philosophy of our team encourages whole team responsibility for quality and collaboration across team roles, the business and with the customer.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The product is designed to facilitate great exploratory testing and automation at every level of the product.

@techgirl1908

The product is designed to facilitate great exploratory testing and automation at every level of the product.

@techgirl1908

The product is designed to facilitate great exploratory testing and automation at every level of the product.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The process helps the team decompose work into small testable chunks and discourages the accumulation of

testing debt.

@techgirl1908

The process helps the team decompose work into small testable chunks and discourages the accumulation of

testing debt.

@techgirl1908

The process helps the team decompose work into small testable chunks and discourages the accumulation of

testing debt.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The team has a deep understanding of the problem the product solves for their customer and actively identifies and mitigates risk.

@techgirl1908

The team has a deep understanding of the problem the product solves for their customer and actively identifies and mitigates risk.

@techgirl1908

The team has a deep understanding of the problem the product solves for their customer and actively identifies and mitigates risk.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The team is provided the time, resources, space and autonomy to focus & do great testing within the project.

@techgirl1908

The team is provided the time, resources, space and autonomy to focus & do great testing within the project.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The team’s pipeline provides fast, reliable, accessible and comprehensive feedback on every change as it moves towards production.

@techgirl1908

The team’s pipeline provides fast, reliable, accessible and comprehensive feedback on every change as it moves towards production.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

For productivity, the team considers and applies the appropriate blend of testing to facilitate continuous feedback and unearth important problems as quickly as possible.

@techgirl1908

For productivity, the team considers and applies the appropriate blend of testing to facilitate continuous feedback and unearth important problems as quickly as possible.

@techgirl1908

For productivity, the team considers and applies the appropriate blend of testing to facilitate continuous feedback and unearth important problems as quickly as possible.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The team has very few customer impacting production issues but when they do occur the team can very quickly detect, debug and remediate the issue.

@techgirl1908

The team has very few customer impacting production issues but when they do occur the team can very quickly detect, debug and remediate the issue.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

The team proactively seeks to continuously improve their test approach, learn from their mistakes and experiment with new tools and techniques.

@techgirl1908

The team proactively seeks to continuously improve their test approach, learn from their mistakes and experiment with new tools and techniques.

@techgirl1908

The team proactively seeks to continuously improve their test approach, learn from their mistakes and experiment with new tools and techniques.

@techgirl1908

People
Philosophy
Product
Process
Problem
Project
Pipeline
Productivity
Production Issues
Proactivity

@techgirl1908

A Tale of Testing the Untestable

Angie Jones

http://angiejones.tech

http://testautomationu.com

@techgirl1908

Senior Developer Advocate, Applitools

Director, Test Automation University

San Francisco, CA, USA

A Tale of Testing the Untestable

By Angie Jones

A Tale of Testing the Untestable

In this talk, Angie shares her story of trying to test scenarios that were essentially untestable due to various limitations and restrictions. Warning: this story is not filled with warm fuzzies and rainbows. No, it’s filled with challenge after challenge. Angie will discuss the technical challenges, and how coming up with solutions to these were still not enough. She’ll share the details of her harsh realization that the power to test the untestable was not in her hands alone, and that a “whole team” approach was the only way to truly be successful.

  • 3,350