Where are we?

• Comprehensive test suite
  • Unit and integration tests
• E2E tests which proved to catch real errors
• A good CI infrastructure where we can setup and run pipelines

• Avg. 80 mins to push a commit to dev env
• We wait for 8 hours in a day for the pipeline
• Avg. 28 mins to run all the tests
• In the last 3 months, we waited 3 days for 4 test files to run
• We spend at least 2 hours on UAT to run all the E2E tests in every deployment
• We don't have monitoring and alerting about CI issues and slowdowns

What we need?

&

Track #1

Run all tests regularly

Track #1

Run all tests regularly

• Ensure supertest tests are running in nightly
• Ensure case study tests are running with the proper filters
• Run playwright tests in nightly
• Convert this job to run it regularly (eg. every hour)
• Notify teams when this job breaks

Track #2

Pipeline Monitoring

Track #2

Pipeline Monitoring

• Push Job, Job Section, Test Suite,
  Test File metrics to Datadog
• Notify developer if a pipeline broke on their branch
• Notify developers if main pipeline broke
• Create dashboard for CI metrics in Datadog

Track #3

Better auto resource management for builds

Track #3

Better auto resource management for builds

• Faster autoscaler reaction to peak pipeline usage
• Analyse and improve CPU and memory usage of pipeline pods

Track #4

Speed up tests

Track #4

Speed up tests

Track #4

Speed up tests

Track #5

Speed up CI steps

Track #5

Speed up CI steps

Track #6

Speed up builds

Track #6

Speed up builds

• Try out rspack and/or vite to replace Webpack