Web Site Testing with Selenium

Overview

  • The Why of Testing
  • The What of Testing
  • Intro to Selenium
  • Installing and using Selenium IDE
  • Acceptance testing example

How do you know when something works?

How do you know when something works?

 

 

  • When it does what it's supposed to do
  • When people buy it
  • When you run out of time and money to make it do more

Verification

Knowing that the system does what it's designed to do.

Done with programmatic testing whenever possible.

Validation

Knowing that the system does what people want it to do.

Done with market testing.

It does the thing right.

It does the right thing.

Types of Programmatic Testing

  • Acceptance - running scenarios and verifying observable results from the user’s perspective

  • Load - running many scenarios at once and verifying performance criteria

  • Platform - extended acceptance testing on many platforms (phone, tablet, TV, browsers)

  • Integration - verifying that code systems interact correctly

  • Unit - verifying that code components fulfill their interface contracts 

  • Regression - running the above tests over and over to make sure you didn't break anything

We'll focus on Acceptance testing

 

Knowing that the system does what the user wants it to do

How do you specify what a system is supposed to do?

How do you specify what a system is supposed to do?

 

  • Wireframes and mockups
  • User Stories
  • Scenarios
  • Requirements

Requirements

 

Other criteria the system must attain to be acceptable

  • Performance - how fast? How many users?

  • Reliability - how frequently can it fail? How does it recover?

  • Profitability - what does it cost to operate?

  • Operability - How complicated is it to run?

  • Not so much Functionality any more - those are covered in the user stories 

User Stories

 

How the system provides

relevant value

to 

specific user populations

User Story Format

 

"In order to (goal)

As a (role)

I want to (high-level system feature)"

Features are always expressed as something the customer wants the system to do,
not how the system does it

User Story Examples

 

"In order to choose a kitten

As a adopter

I want to see a list of kittens"

"In order to choose a kitten

As a adopter

I want to see the details of one kitten"

User Story Examples

 

"In order to get rid of a kitten

As a donor

I want to offer the kitten for adoption"

User Story Examples

 

"In order to place as many kittens as possible

As an administrator

I want to understand which kittens get adopted"

Exercise

  • Break into pairs
  • Describe a kitten adoption agency site
  • Identify at least three roles
  • Write at least six user stories 
  • Rank them from most important to least important
  • Be prepared to share

Stories have one or more Scenarios

  • Scenarios are the steps the user takes to obtain value

 

  • Always includes
    • ​Preconditions
    • Steps
    • Observable results
  • Scenarios describe how the system does what the customer wants it to do.

 

Feature: Some terse yet descriptive text
  In order to realize a named business value
  As an explicit system actor
  I want to gain some beneficial outcome which furthers the goal

  Scenario: Some determinable business situation
    Given some precondition
    When some action by the actor
    Then some testable outcome is achieved


  Scenario: A different situation
    ...

Example format

Scenario: User adds item to cart
  Given I'm a logged-in User
  When I go to the Item page
  And I click "Add item to cart"
  Then the number of items in my cart should be 1
  And my subtotal should increment by the item price
  And the warehouse inventory should decrement by 1

These scenarios are written in "Gherkin" - a language that can be interpreted by testing frameworks like Cucumber, Bhat, etc.

Example

Scenario: User adds item to cart
  ...
  And I click "Add item to cart" [1]
  Then the quantity in my cart should be 1 [2]
  ...

These steps must be broken down further into

1. Events that happen to elements on the page and

2. Elementsproperties and contents that can be verified

[1] What element must be clicked?

[2] What innerHTML property should be checked?

Exercise

  • New pairs
  • Write at one scenario for the user story below.
  • If necessary, draw the wireframes
  • Share with the class for critique
  • Rewrite the scenario
  • Share with the class for critique
Feature: put a kitten up for adoption
  In order to get rid of a kitten
  As a donor
  I want to give information to the site about the kitten so the kitten will be accepted for adoption
          
Scenario: Some determinable business situation
  Given some precondition
  When some action by the actor
  Then some testable outcome is achieved

        

Testing tools:
Selenium

 

  • Open source JavaScript-based automation framework. 
  • Uses a real web browser to stimulate and observe a web site
  • Acts just like a person clicking on things, typing, and reading the results.
  • Works on a variety of browsers, including  IE, Firefox, Google Chrome, Safari, and many more. 
  • Easy to use but slow
  • Requires everything in the full stack to work - HTML, CSS, client-side JS, network, server-side code, database, etc.

 

Selenium

Webdriver

  • A set of programming libraries
  • Available in many languages & browsers
  • Allows testers to write sophisticated, data-driven routines that use Selenium to interact with a web site.

Selenium IDE

  • Firefox-only Add-on
  • Records your interactions with a web site
  • Allows editing the recording
  • Organizes tests into test suites
  • Stores tests as text that can be edited and version-controlled
  • Export tests to code that can be run using Webdriver

Selenium Server

 

  • Can be deployed on a grid or the cloud
  • Can execute many tests in parallel
  • Useful for load testing or platform testing

Lab

Install FireFox

  • If it's been awhile, refresh FireFox or just re-install
  • Allow FireFox to update but pay attention
  • Disable your other Add-ons

Install Selenium IDE

  • Go to Selenium HQ
  • Read through all the things...
  • Find the Download page
  • Read through all the things...
  • Find the version number of Selenium IDE
  • Click
  • Allow FireFox to install Add-ons
  • Restart

Find your target system

 

  • http://auraelius.github.io/glossary/
  • Write a simple user story for getting value from a glossary
  • Write a scenario for storing a word and verifying that it is stored correctly

User Story

In order to remember a vocabulary word

 

As a code school student

 

I want to save a word
and its definition

Scenario

  1. Given I am at the glossary page
  2. When I type in the word "foo" in the New word field
  3. And I type the phrase "See bar" in the Definition field
  4. And I click the Add word to dictionary button
  5. Then I should see the word and definition added to the list

Test Creation

 

  • Record the precondition & action steps
    • click
    • clickAndWait
  • Adjust the selectors to be less fragile
  • Manually edit in the verify/assert steps
    • verifyElement
    • verifyText
  • Assert halts on failure
  • Verify logs failure and continues

http://docs.seleniumhq.org/docs/02_selenium_ide.jsp#building-test-cases

Example

Managing test suites

  • Store your tests in your repo
  • projectRoot/test/selenium is a decent place to start
  • test subtree gets more complicated as more types of tests get added

Grow your tests

  • For every scenario
    • "happy path" - what happens if the user does everything right
    • error paths - what happens if the user provides bad inputs
  • Every time there's a bug
    • Reproduce the bug with Selenium IDE
    • Write the verify steps with the way it should be
    • Code until it passes
  • Don't go overboard

Assignment

  • Create Selenium tests for the glossary in your blog
  • Alert Al, who will run the tests
  • Create Selenium tests for the AAA web site
    • Verify that the app initializes correctly with the default list of kittens
    • Verify that you can add a kitten to the list
    • What error conditions can your app handle?
    • How to verify that they actually work?

References

  • http://seleniumhq.org
  • https://seleniumguidebook.com/ has free email lessons
  • http://www.softwaretestinghelp.com/selenium-tutorial-1/ has a long list of online tutorials
  • http://www.meetup.com/pdx-se/ local meetup group
  • Selenium 2 Testing Tools Beginner's Guide., 2012 Packt Publishing, ISBN 978-1-84951-830-7
  • Instant Selenium Testing Tools Starter,  2013, Packt Publishing, ISBN 978-1-78216-514-9 

Next steps

Writing Webdriver in JavaScript

(not covered in this class)

Made with Slides.com