Stop F5 testing: (ab)using Behat
Miro Svrtan
@msvrtan
web developer
(lazy)
so lazy that I'm using meme's instead of 1000 words :)
Testing
Do you test your code?
Really, you just write code and deploy it?
My excuse #1
Writing tests takes a lot of time and resources, we don't have that luxury.
Manual testing took much more.
My excuse #2
Who is going to pay for that?
My clients are already paying me for a good quality product.
Excuse #2
15.000 vs 100.000 EURs
You had one job
You had one job #2
You had one job #3
You had one job #4
Testing: My experience
- reduced my frustration & stress
- saved time, increased my productivity
- empowered me to try new things
- took extra time in the beginning, payed off quickly
- it's an investment
Automatted testing
"smart people write down, stupid remember"
- username & password must match
- if no username display notice #1
- if no password display notice #2
- if no username & password show notice #3
- if username or password don't match show notice #4
- user must be active to log in
- if user is not activated show notice #5
- if user is disabled show notice #6
- successful login should redirect user back to the page where they clicked login link
Login process
And I didn't even include:
- user can check 'remember me' and will be logged in for 2days
- using 3rd party: facebook, google, github,..
- using 2 factor authentication
- users can use email address to log in
- considering that user might be banned for some time (forums)
- block IP if someone is trying to brute force user/password
- block username for X time if someone is trying to brute force their password from multiple IP's
OK, this login doesn't look so simple any more :)
New feature
Users can use their email address instead of username to log in.
Users with admin privileges need to use separate login form
OR
Better code maintainability
Could you remember all of this rules after 6 months?
Easier knowledge transfer
Are you sure that person working on it is well aware of all the behaviours/rules?
HA! We have 700 pages of documentation!
.. but lets say you had tests too...
instead of verifying old rules manually, machine validates them for you
.. and you just add new rules into the mix
cause i'm lazy
Behat test example
Scenario: Check that login works
Given I am on login page
And I fill in "username" with "joe"
And I fill in "password" with "pass123"
And I press "Login"
Then I should see "Hi Joe!"
And I should be on home page
Why Behat?
- human readable tests
- non-developers can understand even write them
- project managers, clients, QA,...
- no need to learn new DSL
Behat
- BDD (behavior-driven development) framework
- PHP CLI tool
- http://behat.org/
Behat DSL
- uses Gherkin
- built for Cucumber (test tool written in Ruby)
What is Gherkin?
Gherkin is a Business Readable, Domain Specific Language created specifically for behavior descriptions. It gives you the ability to remove logic details from behavior tests.
Gherkin serves as your project’s documentation as well as your project’s automated tests. Behat also has a bonus feature: It talks back to you using real, human language telling you what code you should write.
What is BDD?
It’s the idea that you start by writing human-readable sentences that describe a feature of your application and how it should work, and only then implement this behavior in software.
(Ab)using Behat
Idea is to use human-readable sentences that describe a feature of your application and how it should work to verify that it works as expected after introducing changes.
(Ab)using Behat
- we all have untested legacy code
- easy to setup, use and understand DSL
- possible gateway to becoming a BDD practicioner
Today I'll concentrate on browser testing only
- language agnostic
- doesn't care for language of the web app
- for some advanced/custom stuff you'll need basic PHP knowledge
- framework agnostic
My usage
- I run it on my local dev & on CI server
- I keep features in same repo as code
- you can keep it in separate repo
- all it needs is HTTP access
- integrated into Symfony2
- I have steps that access DB
Mink
- browser controller/emulator for web applications
- http://mink.behat.org/
Mink supports
- GoutteDriver
- Selenium2Driver
- BrowserKitDriver
- ZombieDriver
- SeleniumDriver
- SahiDriver
- WUnitDriver
Goutte & goutte driver
- screen scraping and web crawling library for PHP
- very fast
- doesn't support javascript
- can't produce screenshots
Selenium2 Driver
- uses selenium2 to run browser tests
- almost everything human can do
- runs javascript
- can create screenshots
Selenium2
- uses headless browser
- supports Chrome, Chromium, Firefox ...
- suports defining of screen width/height
Installation
- composer
- to test javascript or use browsers
- selenium
- browsers
composer.json
{
...
"require": {
...
"behat/behat": "~3.0@dev",
"behat/mink-extension": "~2.0@dev",
"behat/mink-browserkit-driver": "~1.2@dev",
"behat/mink-selenium2-driver": "~1.2@dev",
"behat/mink": "~1.6@dev",
"behat/mink-goutte-driver": "~1.1"
},
"config": {
"bin-dir": "bin/"
}
...
}
behat.yml: suites config
suites:
web:
contexts: [ FeatureContext ]
filters:
tags: "~@javascript && ~@wip"
js:
contexts: [ FeatureContext ]
filters:
tags: "@javascript && ~@wip"
wip:
contexts: [ FeatureContext ]
filters:
tags: "@wip"
behat.yml: Mink config
extensions:
Behat\MinkExtension:
base_url: http://demo.sonata-project.org/
sessions:
default:
goutte:
..
javascript:
selenium2: ~
browser_name: firefox
...
Behat DSL readability #1
Scenario: Registration is available from homepage
Given I am on homepage
And I click "Sign Up"
Then I should see "Registration form"
And I should be on registration form
Behat DSL readability #2
Scenario: Registration is available from homepage
Given I am on "/"
And I click "Sign Up"
Then I should see "Registration form"
And I should be on "/registration/"
Check URL redirects
Scenario: Boat page redirects from old to new url using slug
Given I am on "/product/show/product-xx"
Then url is "/product/category1/product-xx-1"
And I should see "Product XX"
Custom step: The url is "%"
/**
* Checks, that URL is equal to specified.
* @Then /^url is "(?P<url>[^"]+)"$/
*/
public function assertUrl($url)
{
$expectedUrl = $this->locatePath($url);
$actualUrl = $this->getSession()->getCurrentUrl();
if ($expectedUrl !== $actualUrl) {
$msg = sprintf('Current url is "%s", but "%s" expected.', $actualUrl, $expectedUrl);
throw new \Exception($msg);
}
}
Another URL change
Scenario: Check boat page redirect from old to new url using boat id
Given I am on "/product/category1/product-xx-1"
Then url is "/product/category1/brand-product-xx-1"
And I should see "Product XX"
Fix old test
Scenario: Boat page redirects from old to new url using slug
Given I am on "/product/show/product-xx"
Then url is "/product/category1/brand-product-xx-1"
And I should see "Product XX"
Redirect URLs with wrong slug
Scenario: Check boat detailview page redirect if slugs don't match
Given I am on "/product/category2/wrong-url-1"
Then url is "/product/category1/brand-product-xx-1"
And I should see "Product XX"
Check 404
Scenario: Getting 404 if no boat found
Given I am on "/product/category3/brand-fake-1000000000"
Then the response status code should be 404
Test search
Scenario: Select category
Given I am on "/search/"
And I select "Category1" from "category"
And I press "Search"
Then I should see "We have found 40 products in Category1."
And url is "/search/?categories=category1"
Test search
Scenario: Search by category & price
Given I am on "/search/"
And I select "Category1" from "category"
And I fill in "price_from" with "25"
And I fill in "price_to" with "35"
And I press "Search"
Then url is "/search/?categories=category1&price_from=25&price_to=35"
And I should see "We have found 4 products."
Test search using javascript
@javascript
Scenario: Filter Example brand on category1
Given I am on "/search/?categories=category1"
And I click "Example"
Then I wait for Ajax request to process
And I should see "We have found 1 product."
And url is "/search/?categories=category1&brand=example"
Registration
Scenario: Check that new users can register
Given there is no user on site with email "email@example.com"
And I am on "/register"
And I fill in "first_name" with "Joe"
And I fill in "last_name" with "Smith"
And I fill in "user_email" with "email@example.com"
And I fill in "user_password" with "pass123"
And I check "terms"
And I press "Register"
Then url is "/"
Multi step scenarios
AngularJS part of the site
@javascript
Scenario: Logged in user buys product using credit card
Given I am on "/some-url-product-1"
And I press "Buy It"
Then I wait for Ajax request to process
And I should be on "/cart/checkout/details"
And I should see "Product XX" on page headline
And I should see "€20"
And I press "Continue to payment plan"
Then I wait for Ajax request to process
....
Multi step scenarios
Scenario: Buy something
...
Then I wait for Ajax request to process
And I should be on "/cart/checkout/payment"
And I should see "Payment details" on page headline
And I should see "€20"
And I fill in "cc_number" with "12345678901234"
And I select "02" from "cc_expiration_month"
And I select "2017" from "cc_expiration_year"
And I fill in "cc_cvx" with "123"
And I press "Complete purchase"
Then I wait for Ajax request to process
....
Multi step scenarios
Scenario: Buy something
...
Then I wait for Ajax request to process
And I should be on "/cart/checkout/thanks"
And I should see "Thank you for buying XX" on page headline
Adding custom step
/**
* @Then I wait for Ajax request to process
*/
public function iWaitForAjaxRequestToProcess()
{
//Wait for 10 seconds or until ajaxProcessFlag is equal to 1.
$this->getSession()->wait(10000, "ajaxProcessFlag==1");
}
Behat feature file
- file defining the feature and it's scenarios
- located in features folder
- files have .feature extension
Feature content
- feature definition
- background tasks (optional)
- scenarios
Feature definition
Feature: User login
In order for users to update content
As a user
I need to log in
Feature: User login
In order for users to buy stuff
As a user
I need to log in
OR
Defining background
Background:
Given I am on "/login"
And I fill in "username" with "admin"
And I fill in "password" with "admin"
And I press "Log In"
Defining scenarios
Scenario: Check that login is accessible from homepage
Given I am on "/"
...
@javascript
Scenario: Check that login works
Given I am on "/login"
...
Scenario: Check that login fails gracefully when wrong password
Given I am on "/login"
...
Running behat
- ./bin/behat
- ./bin/behat features/products/
- ./bin/behat features/products/admin.feature
- ./bin/behat -suite web
Running behat
Ooops ..
Error report in the middle
Error report in the summary
Running it in progress format
My suggestions
- tag features/scenarios
- use fixtures
- setup CI (continuous integration) server
That's it!
Some of the code examples and ideas:
https://github.com/nulldevelopmenthr/standalone-behat
Questions?
Thank you!
twitter: @msvrtan email: miro@mirosvrtan.me
Stop F5 testing: (ab)using Behat
By Miro Svrtan
Stop F5 testing: (ab)using Behat
- 1,714