Book Report

How Google Tests Software
Help me test like Google - Life of a TE

by: James Whittaker
Jason Arbon
Jeff Carollo

Agenda

  • Testing Roles at Google

  • Testing Workflow
  • Test Preparation
      • ACC Planning
      • Risk Assessment
  • Applying ideas to IDS
  • Testing Roles at Google

    Software Engineer (SWE) = traditional developer role.  They follow TDD and write unit tests.
    Software Engineer in Test (SET) = a developer role whose focus is on testability and general test infrastructure.  
    "SETs are partners in the SWE codebase, but are more concerned with increasing quality and test coverage than adding new features or increasing performance.  SETs write code that allows SWEs to test their features" 
    Test Engineer (TE) = Related to the SET role but has a different focus.  Many write a lot of code, many write only a little.
    "The TE role puts testing on behalf of the user first.  TEs organize the overall quality practices, interpret test results, drive test execution, and build end-to-end test automation." 

    TestIng Workflow

    1. Test Planning: ACC (Attributes, Components, Capability) & Risk (Frequency of Failure, Impact)
    2. Test Coverage:  Bots find differences between versions
    3. Bug Evaluation:  Human reviews differences to determine bugs vs. features
    4. Exploratory Testing:  Crowd and/or early adopters
    5. Bug Filing
    6. Triaging and Debugging
    7. Deploy New Version & Return to Step 1

    ACC - Attributes

    1st core value
    Adjectives of the System (Qualities and Characteristics)
    • Simple
    • accurate
    • moving (list quickly)
    • short

    Ask yourself:  "What sets this product apart from competitors"

    ACC - Components

    the first C
    Nouns of the System (core pieces)
    Often you can just ask developers: "what component are you working on" to pull together a list.
    10 is good - 20 is too many

    ACC - Capability

    Second C
    Verbs of the System (general actions performed by the user)
      • tie directly back to the attributes
      • be testable
      • not meant to be atomic actions

    Example ACC in GRID

    Risk 

    Testers and stakeholders individually rate items on the ACC and then the answers are compiled to avoid bias in an individual role
    Frequency of Failure:
    Rarely
    Seldom
    Occasionally
    Often

    Impact:
    Minimal
    Some
    Considerable
    Maximal
    -VS-




    Items with Often Frequency of failure and Maximal impact are the highest risk.  More time/care is spent creating test cases and testing the high risk items.  The lower areas are exploratory tested.

    How to Apply to Our Team?

    Risk Assessment for Regression

    Internal team implementation:

    • Identified Areas - Functionality
    • Form sent to all stakeholders asking for rating 

    https://docs.google.com/forms/d/12Sph81IxHJTBgtzw3i2YUoaecB936xVUkkAxD0Pg-jI/viewform?usp=send_form 

    • Answers compiled into Test Priority Doc
    • Priority added to Regression Product Coverage Outline
    • Benefits = clearly defined order to test items, surprising outcome
    • Drawbacks = time, involvement from other stakeholders, some assumptions 



    Incorporate ACC for New Fucntionality

    We could incorporate ACC documentation into Mind Maps and each charter could tie back

    • Benefits:  clear documentation, forces tester to know the product, asking questions during planning phase
    • Drawbacks:  extra time
    • Outstanding Questions:  Do we have time to complete this in our sprints?  Will this help identify bugs?  What value is added?

    How Google Tests Software

    By Melinda Turner

    How Google Tests Software

    • 311