The latest updates from

Brian Mann

Gleb Bahmutov

Founder & Creator

VP Engineering

Oct 16, 2018 Webinar

The Agenda

1. Parallelization & Grouping

2. Dashboard paywall

5. Q&A

3. Test retries

4. Cross-browser support

Q & A - use Zoom to ask a question

Cypress test parallelization and grouping

I want my tests to finish quickly

If my CI can allocate 10 machines for me - why can't the tests run 10x faster?

And I want zero config

$ cypress run --record --parallel

Cypress v3.1.0


Machine #1

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

Dashboard

specs

specs


Machine #1

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

a.spec.js

b.spec.js

c.spec.js

...


Machine #1

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

Dashboard

 

 

 

 

 

a.spec.js (2s)

b.spec.js (10s)

c.spec.js (5s)

...

Dashboard

 

 

 

 

 


Machine #1

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

b.spec.js (10s)

c.spec.js (5s)

a.spec.js (2s)

...

Dashboard

 

 

 

 

 


Machine #1

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

🔄 b.spec.js (10s)

c.spec.js (5s)

a.spec.js (2s)

...

b.spec.js

Dashboard

 

 

 

 

 


Machine #1

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

🔄 b.spec.js (10s)

🔄 c.spec.js (5s)

a.spec.js (2s)

...

c.spec.js

b.spec.js

Dashboard

 

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

c.spec.js

🔄 b.spec.js (10s)

🔄 c.spec.js (5s)

a.spec.js (2s)

...

b.spec.js

Dashboard

 

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

 

How parallelization works

pass / fail

🔄 b.spec.js (10s)

🔄 c.spec.js (5s)

a.spec.js (2s)

...

b.spec.js

Dashboard

 

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

 

How parallelization works

🔄 b.spec.js (10s)

c.spec.js (5s)

a.spec.js (2s)

...

b.spec.js

Dashboard

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

 

How parallelization works

🔄 b.spec.js (10s)

c.spec.js (5s)

🔄 a.spec.js (2s)

...

a.spec.js

b.spec.js

Dashboard

 

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

 

How parallelization works

a.spec.js

🔄 b.spec.js (10s)

c.spec.js (5s)

🔄 a.spec.js (2s)

...

b.spec.js

Dashboard

 

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

 

How parallelization works

pass / fail

🔄 b.spec.js (10s)

c.spec.js (5s)

🔄 a.spec.js (2s)

...

b.spec.js

Dashboard

 

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

 

How parallelization works

🔄 b.spec.js (10s)

c.spec.js (5s)

✅ a.spec.js (2s)

...

and the process repeats until all specs have completed ...

b.spec.js

Dashboard

 

 

 

 

 

 

Machine #1

 

$ cypress run --record --parallel
$ cypress run --record --parallel

 

Machine #2

How parallelization works

cypress-example-kitchensink demo

Zero config

2x-electron:
  # tell CircleCI to execute this job on 2 machines simultaneously
  parallelism: 2
  steps:
    - run: npm run e2e:record -- --parallel  
  

Zero config

2x-electron:
  # tell CircleCI to execute this job on 5 machines simultaneously
  parallelism: 5
  steps:
    - run: npm run e2e:record -- --parallel  
  

Zero config

Zero config

Balancing act

Split longest running spec

Balancing act

Split longest running spec

Experiment with number of machines

🤖🤖🤖

🤖🤖🤖

🤖🤖🤖

cypress-dashboard parallelization

Text

cypress-dashboard parallelization

# machines run duration time savings
1 22:50 ~
2 11:47 48%
3 7:51 65%
4 5:56 74%
6 3:50 83%
8 3:00 87%
10 2:19 90%

cypress-dashboard parallelization

10 machines = 10x speed up

# machines run duration time savings
1 22:50 ~
2 11:47 48%
3 7:51 65%
4 5:56 74%
6 3:50 83%
8 3:00 87%
10 2:19 90%

cypress-dashboard parallelization

10 machines = 10x speed up

Grouping

I want to run a few smoke tests then all tests

I want to run tests, deploy, then run tests against the deployed server

I want to run tests using Electron, then again using Chrome / X

$ cypress run --record --group 1x-electron
$ cypress run --record --group 2x-electron --parallel

Read about grouping

The Dashboard Paywall

Going up October 20th 2018

Test runner remains 100% OSS

Upcoming Dashboard features

Detailed information about every test

Historical information about all tests

Upcoming Dashboard features

Detailed information about every test

Historical information about all tests

Flakiness coefficient

Test retries, cross-browser support, Q&A

switching slides to Brian Mann ...

Remember to ask a question, we will answer them at the end of the webinar