Engineering Productivity Strategy and Roadmap
Automation & Tools
Mandate: anything related to automation or tooling
Engineering Productivity
Mandate: tooling and automation which directly impacts engineering productivity
Automation & Tools
- MozTrap
- Mozmill tests, media tests
- Pulse
- Autophone, MozBench
Engineering Productivity
- No; tool for another team
- No; test cases are best owned by developers
- No? infrastructure
- Maybe - automation yes, but optimally not maintaining hardware pools
What impacts engineering productivity?
Build Speeds
-
local and in automation
- C++ and JavaScript developers
- "Build Faster" project with 8+ resources, 5 from Eng Productivity, goal to dramatically improve build times across the board [in progress]
Version Control
-
Fast clones and fast pushes
-
partially done (bundle clones)
-
narrow and shallow clones [Q3+]
-
-
Support for both hg and git users [in progress]
-
primarily through git support in MozReview, mach commands that interact with version control
-
-
Creation of a unified gecko repo [in progress]
Reviewing Code and Landing Patches
-
MozReview
-
improved UX [in progress]
-
static analysis [in progress]
-
code coverage [Q3+]
-
performance data [Q3+]
-
try runs and automatic landing [deployed, refinements and improvements in progress]
-
Reviewing Code and Landing Patches
-
Autoland
-
land code à la checkin-needed [done]
-
pre-qualify landings with try runs [in progress]
-
handle branch uplifts [Q3+]
-
integration with performance data [TBD]
-
Continuous Integration
- Treeherder: automatic starring [in progress]
- enables a lot of other intelligent automation
- Treeherder: viewing data in more ways [Q2+]
- e.g., viewing test results by test files, instead of by
job
- e.g., viewing test results by test files, instead of by
Continuous Integration
- Perfherder
- Talos test sheriffing integrated with Treeherder [in progress]
- Integration of AWSY, AWFY data [in progress]
Q2 and beyond
Continuous Integration
- E2E times are fast
- greater chunking utilized on non-OSX platforms
- depends on TaskCluster
- automation build times are reduced via artifact builds
- depends on integrating some of the "build faster" work with automation
- SETA is made to operate on a per-directory basis
- depends on automatic starring
- greater chunking utilized on non-OSX platforms
Continuous Integration
- Try runs fail fast
- re-use build artifacts when appropriate for fast builds
- depends on "build faster"
- automatic smart test selection via static and dynamic analysis
- depends on code coverage
- multiple-stage runs with highest value tests run first
- depends on TaskCluster, smarter test selection
- re-use build artifacts when appropriate for fast builds
Continuous Integration
- Code coverage
- C++ and JavaScript
- Make it easy for developers to get code coverage data for their patches
- Generate code coverage data for all in-tree tests
- Integrate with try runs/MozReview so it's easy to see coverage deltas per patch
Debugging Tests
- Make test machine checkouts more useful
- Provide test machines with ready-to-go dev environments
- Allow interaction with remote test machines via local mach commands
- Make checkout of AWS-based testers self-serve
Debugging Tests
- Make it easy to move between local and remote debugging
- Test invocation locally and via try operates via the same set of local mach commands
- Expose TaskCluster's ability to connect to running tasks
- Chunking/job details aren't necessary to reproduce test failures
The Future
- MozReview: inline reviewer edits?
- Try runs: display unique test failures along with the chance that the failure was caused by that change
- Intermittent failures: ability to automatically bisect to find root cause; detection of change of intermittent failure rate?
- Test debugging: integrate debuggers like gdb with try runs
- Remote dev environments: provide fully functional and configured remote dev environments for developing/debugging on various OS's
Open Questions
- Bugzilla: invest in new views, UI?
- On-device automation: keep it small and manage ourselves, or try to scale?
- Performance tests: do current frameworks provide developers with the tools they need to measure all the kinds of performance they care about?
- Sheriffing automation: spend resources trying to automate things like tree closures, backouts, etc?
- Resource allocation: how many resources will be needed for bringing up new platforms, like TaskCluster, Windows in the cloud, etc?
E
By jgriffin
E
- 328