or: How I learned to stop worrying and care about the website
Core UX Team @ Skillshare
Do you care about the website?
Syndicated White Label "CCN" sites
Native iOS/tvOS/Android app
10+ 2nd party API clients (Adobe products)
100+ 3rd party API clients
Deployments as a Service
Moonbeam - PR Review
- Enforces a reviewers file
- Allows teams to define PR acceptance criteria
- Only reviewed PRs can be deployed
- Admins can deploy unreviewed PRs (with a warning)
Moonbeam - Build Gaurdrails
- Breaks deployments into steps: build, stage, prod
- CI enforced at every step
- Checklists enforce manual tests and watching dashboards
- Build reruns CI same (or stronger) checks that Github PR CI
- Stage runs critical e2e tests and asks for manual verification
- Prod asks for manual verification: "mark as good"
Moonbeam - Keep it Green
- PRs merged to master only after "mark as good" in PROD
- Any time before "mark as good", 1 click to rollback builds
- After "mark as good", admin can emergency rollback to any previous "mark as good", and lockdown the build queue
Moonbeam - Make it fast
- Contribute to critical build pipeline projects: npm, webpack, babel, uglify
- Build server caching: npm, uglify, babel
- Drafting! PR A in PROD, PR A + B in stage, PR A + B + C in build
- PR clustering: User A can can take up multiple approved PRs at a time (although this is discouraged)
Moonbeam - Scale it Out
- What started as a napkin drawing is used
by 100+ teams at Adobe to ship code
- Fully dedicated RE team
- Continual improvements to UX, as it's a
critical step of modern development
- Small PRs encouraged. Minimal change that can be considered complete.
- Say something positive on every PR.
- Quick tight rounds of feedback. PRs shouldn't sit for days.
- Ignore style comments at all costs, use robots for that.
- New features are incrementally developed and shipped to PROD behind flags.
- Design and community and team use the product in production environment before real users see it.
- Allows for continuous design review, and enables small PRs, even for new features.
Testing - Unit Tests
- No coverage enforcement yet
- Minimal Mocking: test behavior, not implementation
- Culture of testing
Testing - e2e Tests
- Critical suite runs in Moonbeam on stage
- Full suite runs against prod with alerting out of band
- Webdriver.io on Saucelabs (no Cypress yet)
Testing - QA tests
- "Continuous QA" via rainforest QA
- Mechanical Turk style integration tests
- New Relic (is amazing)
- Sumologic (Splunk) to aggregate and query logs
- Datadog for measuring common ops metrics
- Pagerduty rotations for devs
- alerting from e2e tests, error rate, key metrics
- Maintain Error Zero for it's psychic and cultural effects
- Support team was called "Community"
- Community team helps fix bugs in realtime in our "issues" slack channels. Emphasis on empathy for the user.
- Community participates in sprint planning, and has lots of input into prioritization.
- Team leaders are outspoken advocates for users and for fixing stuff with community.
- Deployments are announced to community team as code is shipped to keep them continuously in the loop.
Support each other
- Devs are humans!
- Blameless postmortems
- No one gets in trouble just for breaking the site
- Team needs to feel unjudged and comfortable deploying
- Yes, it's scary to deploy to a big site! But that fear should be about ruining things for users, not for our jobs or shaming in front of peers.