Parallel tests

in comparable time

Artur Trzop

X 2014

KRUG

A long time ago

many, many commits ago

Build was green

and fast

Tests were even faster

But winter finally came

Actually it was summer

and build was sl   w

Save time

&

get faster feedback

Parallel specs

splitting your test suite

Test suite example

1_fast_spec.rb
2_fast_spec.rb
3_fast_spec.rb
4_fast_spec.rb

5_slow_spec.rb
6_slow_spec.rb
7_slow_spec.rb

8_super_slow_spec.rb

Balancing

split all test files across N nodes:


Node 1

1_fast_spec.rb
2_fast_spec.rb

Node 2

3_fast_spec.rb
4_fast_spec.rb

Node 3

5_slow_spec.rb
6_slow_spec.rb

Node 4

7_slow_spec.rb

8_super_slow_spec.rb

It ends up like this...

Time for the rescue

let's split tests based on the time execution
with Knapsack gem

Node 1

8_super_slow_spec.rb
 
 
 

Node 2

7_slow_spec.rb
4_fast_spec.rb
1_fast_spec.rb
 

Node 3

6_slow_spec.rb
3_fast_spec.rb

Node 4

5_slow_spec.rb

2_fast_spec.rb


How ab   ut time

How to set up Knapsack?

# Gemfile
gem 'knapsack'

spec_helper.rb

require 'knapsack'
Knapsack::Adapters::RspecAdapter.bind

Rakefile

require 'knapsack'
Knapsack.load_tasks 

Generate time execution report for you spec files

 

run it on CI machine

$ KNAPSACK_GENERATE_REPORT=true \
bundle exec rspec spec

Commit generated report

into your repository

knapsack_report.json

Setup your CI server

$ CI_NODE_TOTAL=2 CI_NODE_INDEX=0 \
bundle exec rake knapsack:rspec

That's it.

What else?

integration with:

 

CircleCi.com

Travis.com (try Matrix Feature)

Semaphoreapp.com

 

Upcoming features

Cucumber support

 

shippable.com CI support

Parallel tests in comparable time

By ArturT

Parallel tests in comparable time

Knapsack splits tests across CI nodes and makes sure that tests will run comparable time on each node.

  • 4,094