Misusing Cucumber

Andrey Mikhaylov (lolmaus)
frontend developer & EmberJS enthusiast
@ Mainmatter
https://lolma.us
https://github.com/lolmaus
https://mainmatter.com

repuprposing a BDD library for conventional acceptance testing

💖 tests
- Give confidence
- Detect regressions
- Enable refactoring
- Have business value
🤮 tests
- Low quality code
- Overly complex code
- Overly complex seeding patterns
I have a dream
of a test suite that would:
- standardize test code
- make tests truly readable
- and understandable
by non-developers - focus on intention, hide implementation details
- speed up writing tests
- make writing tests enjoyable

- Started over 14 years ago as a Ruby library, on top of RSpec.
- Evolved into a full-fledged testing framework.
- Ported to most popular programming languages.
- JavaScript has the official CucumberJS and the unofficial YaddaJS.
Cucumber feature file
tests/acceptance/blog.feature Syntax is called "Gherkin"
@setupAcceptanceTest
@setupMirage
Feature: The blog
Background:
Given there's a user Tom
And I'm authorized as Tom
Scenario: Visiting the blog with posts
Given there are 3 posts in the database
When I visit the blog page
Then I should see 3 posts
And I should not see the empty blog message
Scenario: Visiting an empty blog
Given there are 0 blog posts in the database
When I visit the blog page
Then I should see 0 posts
And I should see the empty blog messageStep implementations
When I visit the blog page
Then I should see 3 postsimport { find, visit } from '@ember/test-helpers';
export default {
'When I visit the blog page' () {
return visit('/blog');
},
'Then I should see 3 posts' () {
const posts = find('.post');
this.assert.equals(posts.length, 3);
},
}import { find, visit } from '@ember/test-helpers';
export default {
'When I visit the blog page' () {
return visit('/blog');
},
'Then I should see (\\d+) posts' (countStr) {
const posts = find('.post');
const countInt = parseInt(countStr, 10);
this.assert.equals(posts.length, countInt);
}
}import { find, visit } from '@ember/test-helpers';
export default {
'When I visit the blog page' () {
return visit('/blog');
},
'Then I should see $number posts' (count) {
const posts = find('.post');
this.assert.equals(posts.length, count);
}
}
tests/acceptance/blog.feature
tests/cucumber/steps.js
Multiline steps
Then the declaration of independence should say:
---
The unanimous Declaration of the thirteen united States of America,
When in the Course of human events, it becomes necessary for one people
to dissolve the political bands which have connected them with another,
and to assume among the powers of the earth, the separate and equal
station to which the Laws of Nature and of Nature's God entitle them,
a decent respect to the opinions of mankind requires that they should
declare the causes which impel them to the separation.
---And there are records of type "accessory" with the following properties:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| id | name | code | sorting | thumbnailImage | priceType | priceValue | accessoryCategoryId | teaser | minTotalWidth | maxTotalWidth | profileIds |
| "tshirts" | "tshirts-name" | "tshsirts" | 1 | "tshirts.jpg" | "constant" | 100 | "2" | "tshirts" | | | ["p1","p2","p3"] |
| "tshirtm" | "tshirtm-name" | "tshsirtm" | 2 | | "constant" | 110 | "2" | "tshirtm" | | | ["p1","p2"] |
| "tshirtl" | "tshirtl-name" | "tshsirtl" | 3 | "tshirtl.jpg" | "constant" | 120 | "2" | "tshirtl" | | | [] |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Test case mattresses

Test case mattresses


Test case mattresses
matrices


Test case matrices


Test case matrices
Scenario: Exceeding min/max value on the Roof-Width-Field, input [User Input], angle: [Angle]
When I visit URL "/configurator/unterkonstruktion"
And I select item "Individual" in the dropdown Preset-Field
And I fill "[Angle]" into the Input of the Angle-Field
And I fill "[User Input]" into the Input of the Roof-Depth-Field
Then the Input in the Roof-Depth-Field should have text "[Actual Value]"
And the Input in the Rafter-Length-Field should have text "[Rafter Length]"
Where:
-------------------------------------------------------
| Angle | User Input | Actual Value | Rafter Length |
| 5 | 999 | 1000 | 1000 |
| 5 | 7001 | 7000 | 7000 |
| 25 | 999 | 1000 | 1097 |
| 25 | 7000 | 6383 | 7000 |
| 25 | 7001 | 6383 | 7000 |
| 15 | 25000 | 6758 | 7000 |
| 25 | 25000 | 6383 | 7000 |
| 35 | 25000 | 5922 | 7000 |
| 45 | 25000 | 5439 | 7000 |
-------------------------------------------------------- Separation of concerns
- Improved readability
- Ease of validation
- ...by all team members
- Code reusability
- High speed of writing tests
- Simple maintenance
- Enforced discipline
- Enforced code coverage
- Uniform code style
- Integration into workflow
Cucumber benefits

sucks
Cucumber issues
-
The library of step implementations keeps growing indefinitely.
Finding implementations for steps is difficult.

Cucumber issues
- The library of step implementations keeps growing indefinitely.
Finding implementations for steps is difficult.
- Every step is a black box.

Scenario: Viewing blog posts
Given there are 3 posts in the database
When I visit the blog
Then the title of the 1st post should be "Hello World"Cucumber issues
- The library of step implementations keeps growing indefinitely.
Finding implementations for steps is difficult.
- Every step is a black box.
- Steps implicitly depend on each other.

When I expand the 2nd post
Then the expanded post should contain a comments sectionCucumber issues
- The library of step implementations keeps growing indefinitely.
Finding implementations for steps is difficult.
- Every step is a black box.
- Steps implicitly depend on each other.
- CSS selectors are counter-productive.

When I click on div p ul li:nth-child(2) aCucumber issues
- The library of step implementations keeps growing indefinitely.
Finding implementations for steps is difficult.
- Every step is a black box.
- Steps implicitly depend on each other.
- CSS selectors are counter-productive.
- Debugging is notoriously difficult.

Cucumber issues
- The library of step implementations keeps growing indefinitely.
Finding implementations for steps is difficult.
- Every step is a black box.
- Steps implicitly depend on each other.
- CSS selectors are counter-productive.
- Debugging is notoriously difficult.
- Nobody cares about BDD anyway.

Solutions
-
Abandon BDD.
Focus on the convenience of acceptance testing
for myself as a developer.

Solutions
-
Step library size: limit to a small amount of tiny, fully reusable steps.
For example:
- click
- fill text into field
- hover the mouse pointer
- read text from an element
- read text from a text field
- select an item from a dropdown list
Solutions
- Step library size: limit to a small amount
of tiny, fully reusable steps.
- Hidden logic: extract "the truth"
from step implementaions into feature files.
Given there are Posts in the database:
-------------------------------------
| title | body | authorId |
| "Hello World" | "..." | "alice" |
| "Lorem Ipsum" | "..." | "bob" |
| "Foo Bar Baz" | "..." | "charlie" |
-------------------------------------
Then the title of the 1st post should be "Hello World" Given there are 3 posts in the database
Then the title of the 1st post should be "Hello World"Before:
Then the 1st product should be selectedAfter:
Then ".product:nth-child(1)" should have HTML class "is-selected"
And I should see ".product:nth-child(1) .comments"Solutions
- Step library size: limit to a small amount
of tiny, fully reusable steps.
- Hidden logic: extract "the truth"
from step implementations into feature files.
- Tangled implementations: steps reference
each other in the feature file,
not inside step implementations.
Then the 4th post should contain a comments section Then the expanded post should contain a comments sectionSolutions
- Step library size: limit to a small amount
of tiny, fully reusable steps.
- Hidden logic: extract "the truth"
from step implementations into feature files.
-
Tangled implementations: steps reference
each other in the feature file,
not inside step implementations.
- CSS selector hell: enforce a restriction to
use only semantic data-test selectors.
Before:
When I click on div p ul li:nth-child(2) aAfter:
When I click on "[data-test-post]:nth-child(2) [data-test-title]"4. Use only semantic
data-test selectors
[data-test-main-menu] data-test-main-menu main-menu Main-Menu4. Use "Labels": a convenient DSL
for data-test selectors
When I click [data-test-post]:nth-child(2) [data-test-expand-button]4. Reverse the order
in compound selectors
When I click the Expand-Button in the 2nd PostBefore:
After:
<div class="container">
<article data-test-post>
</article>
</div>
<div class="container">
<article data-test-post>
</article>
<article data-test-post>
</article>
</div>
<div class="container">
<article data-test-post>
</article>
<article data-test-post>
</article>
</div>:nth-child() does not work as expected
2nd Post
[data-test-post]
:nth-child(2)
1
2
3
4
5
<div class="container">
<article data-test-post>
</article>
</div>
<div class="container">
<article data-test-post>
</article>
<article data-test-post>
</article>
</div>
<div class="container">
<article data-test-post>
</article>
<article data-test-post>
</article>
</div>Use :eq(2) instead of :nth-child(2)
2nd Post
[data-test-post]
:eq(2)
1
2
3
4
5
Solutions
- Step library size: limit to a small amount
of tiny, fully reusable steps.
- Hidden logic: extract "the truth"
from step implementations into feature files.
-
Tangled implementations: steps reference
each other in the feature file,
not inside step implementations.
- CSS selector hell: enforce a restriction to
use only semantic data-test selectors.
- Debugging: imrpove the output of test results.
5. Improving test output
- Step name
- Implementation
- Assertion error
- Arguments
- Stack trace








Thank you!
» ^_^ «


Andrey Mikhaylov
lolma.us
github.com/lolmaus
mainmatter.com

ember i ember-bdd
A drop-in Cucumber solution for your Ember app, powered by YaddaJS
https://github.com/lolmaus/ember-bdd
A secret slide 🤫


Misusing Cucumber (Ember Europe)
By Andrey Mikhaylov (lolmaus)
Misusing Cucumber (Ember Europe)
Using a BDD library for conventional acceptance testing — and benefitting from it
- 688