Firefox OS

Test Strategy

2013 and Beyond

release work

Environment: Devices

• Participate in acceptance criteria for sprint planning
• Identify User stories and Writing test cases in moztrap during sprints
• And automate them in Gaia UI tests (python or javascript)
• Executing mini test runs at the end of each sprint
• Functional testruns during the week of FC (2-3?)
• Branch merge, run a smoke test and
• Execute 3-4 full testruns during the cycle
• 1-2 Localization testruns
• FOTA update testing
• 3rd party apps smoke test (but trying to hand this off)
• Daily Bug Triage work

Platform work

Environment: Emulators

• enable platform tests for B2G (mochitests, layout, reftests, crash, new APIs)
  
• create device API tests for CRB and test apps
• we need a place to converge these App tests so it can be run by automation and community

Automation

Environment: Devices, Desktop Builds


• Gaia UI tests on current production branch (ie. 1.2, 1.3, etc..)
• Gaia UI tests on master/m-c
• MTBF stability testing across multiple devices
• Eideticker performance testing measure FPS startup and pan/zoom
• Endurance testing on core apps
• future: fuzzing on b2g with marionette  (MariFuzz)
• CRB test guidance
  

Miscellaneous

Environment: Various

• Support Flashing tools for nightly and CI machines
• Assisting partners with automation and test case setup
• Support live issues from partners
• test prototypes
  

Community

Environment: Devices, Desktop builds, Apps

• Target device testers
• execute moztrap tests
• download and run test Apps, submit test data
• have meet ups and testdays
• Document!  (irc, blogs, social networking, Planet, QMO)
• Target automation testers
  
• set up test environments 
• Mentor and review contributing testcases
  
• Document!  (github readmes, comments, bugs)
  
• Target app developers
  
• How to properly test apps in HTML5
• How to clone their own test environments we use

January Workweek!