How do you know when something works?

How do you know when something works?

• When it does what it's supposed to do
• When people buy it
• When you run out of time and money to make it do more

Types of Programmatic Testing

• Acceptance - running scenarios and verifying observable results from the user’s perspective

• Load - running many scenarios at once and verifying performance criteria

• Platform - extended acceptance testing on many platforms (phone, tablet, TV, browsers)

• Integration - verifying that code systems interact correctly

• Unit - verifying that code components fulfill their interface contracts 

• Regression - running the above tests over and over to make sure you didn't break anything

We'll focus on Acceptance testing

Knowing that the system does what the user wants it to do

How do you specify what a system is supposed to do?

How do you specify what a system is supposed to do?

• Wireframes and mockups
• User Stories
• Scenarios
• Requirements

Requirements

Other criteria the system must attain to be acceptable

• Performance - how fast? How many users?

• Reliability - how frequently can it fail? How does it recover?

• Profitability - what does it cost to operate?

• Operability - How complicated is it to run?

• Not so much Functionality any more - those are covered in the user stories 

User Stories

How the system provides

relevant value

to 

specific user populations

User Story Format

"In order to (goal)

As a (role)

I want to (high-level system feature)"

Features are always expressed as something the customer wants the system to do,
not how the system does it

User Story Examples

"In order to choose a kitten

As a adopter

I want to see a list of kittens"

"In order to choose a kitten

As a adopter

I want to see the details of one kitten"

User Story Examples

"In order to get rid of a kitten

As a donor

I want to offer the kitten for adoption"

User Story Examples

"In order to place as many kittens as possible

As an administrator

I want to understand which kittens get adopted"

Stories have one or more Scenarios

• Scenarios are the steps the user takes to obtain value

• Always includes
  • ​Preconditions
  • Steps
  • Observable results

• Scenarios describe how the system does what the customer wants it to do.

Feature: Some terse yet descriptive text
  In order to realize a named business value
  As an explicit system actor
  I want to gain some beneficial outcome which furthers the goal

  Scenario: Some determinable business situation
    Given some precondition
    When some action by the actor
    Then some testable outcome is achieved


  Scenario: A different situation
    ...

Example format

Scenario: User adds item to cart
  Given I'm a logged-in User
  When I go to the Item page
  And I click "Add item to cart"
  Then the number of items in my cart should be 1
  And my subtotal should increment by the item price
  And the warehouse inventory should decrement by 1

These scenarios are written in "Gherkin" - a language that can be interpreted by testing frameworks like Cucumber, Bhat, etc.

Example

Scenario: User adds item to cart
  ...
  And I click "Add item to cart" [1]
  Then the quantity in my cart should be 1 [2]
  ...

These steps must be broken down further into

1. Events that happen to elements on the page and

2. Elements, properties and contents that can be verified

[1] What element must be clicked?

[2] What innerHTML property should be checked?

• Open source JavaScript-based automation framework. 
• Uses a real web browser to stimulate and observe a web site
• Acts just like a person clicking on things, typing, and reading the results.
• Works on a variety of browsers, including  IE, Firefox, Google Chrome, Safari, and many more. 
• Easy to use but slow
• Requires everything in the full stack to work - HTML, CSS, client-side JS, network, server-side code, database, etc.

Selenium

Webdriver

• A set of programming libraries
• Available in many languages & browsers
• Allows testers to write sophisticated, data-driven routines that use Selenium to interact with a web site.

Selenium IDE

• Firefox-only Add-on
• Records your interactions with a web site
• Allows editing the recording
• Organizes tests into test suites
• Stores tests as text that can be edited and version-controlled
• Export tests to code that can be run using Webdriver

Selenium Server

• Can be deployed on a grid or the cloud
• Can execute many tests in parallel
• Useful for load testing or platform testing

Find your target system

• http://auraelius.github.io/glossary/
• Write a simple user story for getting value from a glossary
• Write a scenario for storing a word and verifying that it is stored correctly

User Story

In order to remember a vocabulary word

As a code school student

I want to save a word
and its definition

Scenario

1. Given I am at the glossary page
2. When I type in the word "foo" in the New word field
3. And I type the phrase "See bar" in the Definition field
4. And I click the Add word to dictionary button
5. Then I should see the word and definition added to the list