Book Report
How Google Tests Software
Help me test like Google - Life of a TE
by: James Whittaker
Jason Arbon
Jeff Carollo
Agenda
-
Testing Roles at Google
- ACC Planning
- Risk Assessment
Testing Roles at Google
Software Engineer (SWE) = traditional developer role. They follow TDD and write unit tests.
Software Engineer in Test (SET) = a developer role whose focus is on testability and general test infrastructure.
"SETs are partners in the SWE codebase, but are more concerned with increasing quality and test coverage than adding new features or increasing performance. SETs write code that allows SWEs to test their features"
Test Engineer (TE) = Related to the SET role but has a different focus. Many write a lot of code, many write only a little.
"The TE role puts testing on behalf of the user first. TEs organize the overall quality practices, interpret test results, drive test execution, and build end-to-end test automation."
TestIng Workflow
- Test Planning: ACC (Attributes, Components, Capability) & Risk (Frequency of Failure, Impact)
- Test Coverage: Bots find differences between versions
- Bug Evaluation: Human reviews differences to determine bugs vs. features
- Exploratory Testing: Crowd and/or early adopters
-
Bug Filing
-
Triaging and Debugging
-
Deploy New Version & Return to Step 1
ACC - Attributes
1st core value
Adjectives of the System (Qualities and Characteristics)
- Simple
- accurate
- moving (list quickly)
- short
Ask yourself: "What sets this product apart from competitors"
ACC - Components
the first C
Nouns of the System (core pieces)
Often you can just ask developers: "what component are you working on" to pull together a list.
10 is good - 20 is too many
ACC - Capability
Second C
Verbs of the System (general actions performed by the user)
- tie directly back to the attributes
- be testable
- not meant to be atomic actions
Example ACC in GRID
Risk
Testers and stakeholders individually rate items on the ACC and then the answers are compiled to avoid bias in an individual role
Frequency of Failure:RarelySeldomOccasionallyOften
Impact:MinimalSomeConsiderableMaximal
-VS-
Items with Often Frequency of failure and Maximal impact are the highest risk. More time/care is spent creating test cases and testing the high risk items. The lower areas are exploratory tested.
How to Apply to Our Team?
Risk Assessment for Regression
Internal team implementation:
- Identified Areas - Functionality
- Form sent to all stakeholders asking for rating
https://docs.google.com/forms/d/12Sph81IxHJTBgtzw3i2YUoaecB936xVUkkAxD0Pg-jI/viewform?usp=send_form
- Answers compiled into Test Priority Doc
- Priority added to Regression Product Coverage Outline
- Benefits = clearly defined order to test items, surprising outcome
- Drawbacks = time, involvement from other stakeholders, some assumptions
Incorporate ACC for New Fucntionality
We could incorporate ACC documentation into Mind Maps and each charter could tie back
- Benefits: clear documentation, forces tester to know the product, asking questions during planning phase
- Drawbacks: extra time
- Outstanding Questions: Do we have time to complete this in our sprints? Will this help identify bugs? What value is added?
How Google Tests Software
By Melinda Turner
How Google Tests Software
- 311