Stop F5 testing: (ab)using Behat

Miro Svrtan

@msvrtan

web developer

(lazy)

so lazy that I'm using meme's instead of 1000 words :)

Testing

 

Do you test your code?

Really, you just write code and deploy it?

My excuse #1

Writing tests takes a lot of time and resources, we don't have that luxury.

Manual testing took much more.

My excuse #2

Who is going to pay for that?

My clients are already paying me for a good quality product.

Excuse #2

15.000 vs 100.000 EURs

You had one job

You had one job #2

You had one job #3

You had one job #4

Testing: My experience

  • reduced my frustration & stress
  • saved time, increased my productivity
  • empowered me to try new things
  • took extra time in the beginning, payed off quickly
  • it's an investment

Automatted testing

"smart people write down, stupid remember"

  • username & password must match
  • if no username display notice #1
  • if no password display notice #2
  • if no username & password show notice #3
  • if username or password don't match show notice #4
  • user must be active to log in
  • if user is not activated show notice #5
  • if user is disabled show notice #6
  • successful login should redirect user back to the page where they clicked login link

Login process

And I didn't even include:

  • user can check 'remember me' and will be logged in for 2days
  • using 3rd party: facebook, google, github,..
  • using 2 factor authentication
  • users can use email address to log in
  • considering that user might be banned for some time (forums)
  • block IP if someone is trying to brute force user/password
  • block username for X time if someone is trying to brute force their password from multiple IP's

OK, this login doesn't look so simple any more :)

New feature

Users can use their email address instead of username to log in.

Users with admin privileges need to use separate login form

OR

Better code maintainability

Could you remember all of this rules after 6 months?

Easier knowledge transfer

Are you sure that person working on it is well aware of all the behaviours/rules?

HA! We have 700 pages of documentation!

.. but lets say you had tests too...

instead of verifying old rules manually, machine validates them for you

.. and you just add new rules into the mix

cause i'm lazy

Behat test example

  Scenario: Check that login works
    Given I am on login page
    And I fill in "username" with "joe"
    And I fill in "password" with "pass123"
    And I press "Login"
    Then I should see "Hi Joe!"
    And I should be on home page

Why Behat?

  • human readable tests
  • non-developers can understand even write them
    • project managers, clients, QA,...
  • no need to learn new DSL

Behat

  • BDD (behavior-driven development) framework
  • PHP CLI tool
  • http://behat.org/

Behat DSL

  • uses Gherkin
  • built for Cucumber (test tool written in Ruby)

What is Gherkin?

Gherkin is a Business Readable, Domain Specific Language created specifically for behavior descriptions. It gives you the ability to remove logic details from behavior tests.

Gherkin serves as your project’s documentation as well as your project’s automated tests. Behat also has a bonus feature: It talks back to you using real, human language telling you what code you should write.

 

What is BDD?

It’s the idea that you start by writing human-readable sentences that describe a feature of your application and how it should work, and only then implement this behavior in software.

 

(Ab)using Behat

Idea is to use human-readable sentences that describe a feature of your application and how it should work to verify that it works as expected after introducing changes.

 

(Ab)using Behat

  • we all have untested legacy code
  • easy to setup, use and understand DSL
  • possible gateway to becoming a BDD practicioner

Today I'll concentrate on browser testing only

  • language agnostic
    • doesn't care for language of the web app
    • for some advanced/custom stuff you'll need basic PHP knowledge
  • framework agnostic

My usage

  • I run it on my local dev & on CI server
  • I keep features in same repo as code
  • you can keep it in separate repo
    • all it needs is HTTP access
  • integrated into Symfony2
    • I have steps that access DB

Mink

  • browser controller/emulator for web applications
  • http://mink.behat.org/

Mink supports

  • GoutteDriver
  • Selenium2Driver
  • BrowserKitDriver
  • ZombieDriver
  • SeleniumDriver
  • SahiDriver
  • WUnitDriver

Goutte & goutte driver

  • screen scraping and web crawling library for PHP
  • very fast
  • doesn't support javascript
  • can't produce screenshots

Selenium2 Driver

  • uses selenium2 to run browser tests
  • almost everything human can do
  • runs javascript
  • can create screenshots

Selenium2

  • uses headless browser
  • supports Chrome, Chromium, Firefox ...
  • suports defining of screen width/height

Installation

  • composer
  • to test javascript or use browsers
    • selenium
    • browsers

composer.json

{
    ...
    "require": {
        ...
        "behat/behat":                       "~3.0@dev",
        "behat/mink-extension":              "~2.0@dev",
        "behat/mink-browserkit-driver":      "~1.2@dev",
        "behat/mink-selenium2-driver":       "~1.2@dev",
        "behat/mink":                        "~1.6@dev",
        "behat/mink-goutte-driver":          "~1.1"
    },
    "config": {
        "bin-dir": "bin/"
    }
    ...
}

behat.yml: suites config

     suites:
        web:
            contexts: [ FeatureContext ]
            filters:
                tags: "~@javascript && ~@wip"
        js:
            contexts: [ FeatureContext ]
            filters:
                tags: "@javascript && ~@wip"
        wip:
            contexts: [ FeatureContext ]
            filters:
                tags: "@wip"

behat.yml: Mink config

    extensions:
        Behat\MinkExtension:
            base_url: http://demo.sonata-project.org/
            sessions:
                default:
                    goutte:
                        ..
                javascript:
                    selenium2: ~
            browser_name: firefox
            ...

Behat DSL readability #1


  Scenario: Registration is available from homepage
    Given I am on homepage
    And I click "Sign Up"
    Then I should see "Registration form"
    And I should be on registration form

Behat DSL readability #2


  Scenario: Registration is available from homepage
    Given I am on "/"
    And I click "Sign Up"
    Then I should see "Registration form"
    And I should be on "/registration/"

Check URL redirects


  Scenario: Boat page redirects from old to new url using slug
    Given I am on "/product/show/product-xx"
    Then url is "/product/category1/product-xx-1"
    And I should see "Product XX"

Custom step: The url is "%"


    /**
     * Checks, that URL is equal to specified.
     * @Then /^url is "(?P<url>[^"]+)"$/
     */
    public function assertUrl($url)
    {
        $expectedUrl = $this->locatePath($url);

        $actualUrl = $this->getSession()->getCurrentUrl();

        if ($expectedUrl !== $actualUrl) {
            $msg = sprintf('Current url is "%s", but "%s" expected.', $actualUrl, $expectedUrl);
            throw new \Exception($msg);
        }
    }

Another URL change


  Scenario: Check boat page redirect from old to new url using boat id
    Given I am on "/product/category1/product-xx-1"
    Then url is "/product/category1/brand-product-xx-1"
    And I should see "Product XX"

Fix old test


  Scenario: Boat page redirects from old to new url using slug
    Given I am on "/product/show/product-xx"
    Then url is "/product/category1/brand-product-xx-1"
    And I should see "Product XX"

Redirect URLs with wrong slug


  Scenario: Check boat detailview page redirect if slugs don't match
    Given I am on "/product/category2/wrong-url-1"
    Then url is "/product/category1/brand-product-xx-1"
    And I should see "Product XX"

Check 404


  Scenario: Getting 404 if no boat found
    Given I am on "/product/category3/brand-fake-1000000000"
    Then the response status code should be 404

Test search


  Scenario: Select category
    Given I am on "/search/"
    And I select "Category1" from "category"
    And I press "Search"
    Then I should see "We have found 40 products in Category1."
    And url is "/search/?categories=category1"

Test search


  Scenario: Search by category & price
    Given I am on "/search/"
    And I select "Category1" from "category"
    And I fill in "price_from" with "25"
    And I fill in "price_to" with "35"
    And I press "Search"
    Then url is "/search/?categories=category1&price_from=25&price_to=35"
    And I should see "We have found 4 products."

Test search using javascript

 @javascript
 Scenario: Filter Example brand on category1
    Given I am on "/search/?categories=category1"
    And I click "Example"
    Then I wait for Ajax request to process
    And I should see "We have found 1 product."
    And url is "/search/?categories=category1&brand=example"

Registration


  Scenario: Check that new users can register
    Given there is no user on site with email "email@example.com"
    And I am on "/register"
    And I fill in "first_name" with "Joe"
    And I fill in "last_name" with "Smith"
    And I fill in "user_email" with "email@example.com"
    And I fill in "user_password" with "pass123"
    And I check "terms"
    And I press "Register"
    Then url is "/"

Multi step scenarios

AngularJS part of the site

@javascript

Scenario: Logged in user buys product using credit card
    Given I am on "/some-url-product-1"
    And I press "Buy It"
    Then I wait for Ajax request to process
    And I should be on "/cart/checkout/details"
    And I should see "Product XX" on page headline
    And I should see "€20"
    And I press "Continue to payment plan"
    Then I wait for Ajax request to process

    ....

Multi step scenarios

  Scenario: Buy something
    ...
    Then I wait for Ajax request to process
    And I should be on "/cart/checkout/payment"
    And I should see "Payment details" on page headline
    And I should see "€20"
    And I fill in "cc_number" with "12345678901234"
    And I select "02" from "cc_expiration_month"
    And I select "2017" from "cc_expiration_year"
    And I fill in "cc_cvx" with "123"
    And I press "Complete purchase"
    Then I wait for Ajax request to process
    ....

Multi step scenarios

  Scenario: Buy something
    ...
    Then I wait for Ajax request to process
    And I should be on "/cart/checkout/thanks"
    And I should see "Thank you for buying XX" on page headline
 

Adding custom step


    /**
     * @Then I wait for Ajax request to process
     */
    public function iWaitForAjaxRequestToProcess()
    {
        //Wait for 10 seconds or until ajaxProcessFlag is equal to 1.
        $this->getSession()->wait(10000, "ajaxProcessFlag==1");
    }

Behat feature file

  • file defining the feature and it's scenarios
  • located in features folder
  • files have .feature extension

Feature content

  • feature definition
  • background tasks (optional)
  • scenarios

Feature definition

Feature: User login
    In order for users to update content
    As a user
    I need to log in

Feature: User login
    In order for users to buy stuff
    As a user
    I need to log in

OR

Defining background

  Background:
    Given I am on "/login"
    And I fill in "username" with "admin"
    And I fill in "password" with "admin"
    And I press "Log In"

Defining scenarios

  Scenario: Check that login is accessible from homepage

    Given I am on "/"
    ...

  @javascript

  Scenario: Check that login works
    Given I am on "/login"

    ...

  Scenario: Check that login fails gracefully when wrong password
    Given I am on "/login"
    ...

Running behat

  • ./bin/behat
  • ./bin/behat features/products/
  • ./bin/behat features/products/admin.feature
  • ./bin/behat -suite web

Running behat

Ooops ..

Error report in the middle

Error report in the summary

Running it in progress format

My suggestions

  • tag features/scenarios
  • use fixtures
  • setup CI (continuous integration) server

That's it!

 

Some of the code examples and ideas:

https://github.com/nulldevelopmenthr/standalone-behat

 

Questions?

Thank you!

twitter: @msvrtan

email: miro@mirosvrtan.me