Definitions / Technologies

What is end-to-end testing?

• End-to-end testing is a software testing technique used to check that a running application behaves as expected.

• Generically, the purpose of performing end-to-end testing is to "identify system dependencies and to ensure that data integrity is maintained between various system components"

• This differs from unit testing in that you are testing the whole system instead of small functional units.

• For webdevs, that means doing tasks like loading up a web page and clicking around, interacting with an admin form, or even navigating through a complex workflow.
• Along the way, you test for various app states.

What is phantom.js?

• Phantom.js is a headless WebKit browser instance that you can command using JavaScript.
• It does everything a full-fledged WebKit browser would do to render a web page - HTTP, parsing and executing JavaScripts + CSS, managing the DOM - except rendering the site to your monitor.
• This means any site accessible to the host machine (eg. inside the Turner network, I can hit our dev environments from my laptop using phantom.js) can be hit by the Phantom browser.
• You can even interact with popup windows, perform AJAX and respond to any resulting changes, navigate through a set of links, etc.

What is casper.js?

 

• Casper.js brings a number of nice features and utilities to the otherwise barebones phantom.js:
  • Robust testing framework
  • Ability to chain navigation steps together
  • Ability to capture screenshots of the page (or part of the page, by CSS selector)
  • Ability to fill out, submit forms, and then react to (ie. test) the results
• Casper can also drive slimer.js which is conceptually the same as phantom.js except that it uses Mozilla's Gekko engine instead of Webkit (API might be the same too, not sure)

Is end-to-end testing good enough by itself?

• Probably not. Aim to pair end-to-end tests with unit tests.
• Why?
  • End-to-end tests tend to be slow to run as they execute against the actual application (in our case, a fully-fledged website)
  • They can be fragile and hide bugs that would not commonly manifest on the frontend UI.
  • You can't be expected to write robust end-to-end tests until the project is done or close to being done.
  • A semi-stable version of the app needs to be fully built and deployed to a server before end-to-end tests run.
  • In the case of Casper, the integrity of the tests depend on a solid internet connection, 3rd party services being operational and responsive, etc.

Installation

Installing Casper and Phantom

• Recommended: using NPM (Node Package Manager)

  • $ npm install -g phantomjs

  • $ npm install -g casperjs

  • confirm that everything is working:

  • $ casperjs --version && phantomjs --version

• There are other installation options available if you visit their websites:
  • http://casperjs.org/
  • http://phantomjs.org/
  • https://slimerjs.org/

Casper Basics

Basic Navigation

Before we dive into the testing framework, let's take phantom.js for a spin using casper.js to drive it!

 

We're going to take a look at a simple script that visits www.ncaa.com homepage and then navigates to a scoreboard page.

Remote JS Eval

Casper has the ability to execute arbitrary JS code against the remote page's JS thread. It works like this:

Notice the "return native types" arrow. This allows you to get data back from the remote page and use it for whatever you want in your casper script!

Filling and Submitting Forms

Casper wouldn't be complete if it didn't

have the ability to fill out and submit forms. Of course, you can also

react to whatever the result of that

submission is using the various wait

functions.

 

Let's try logging in as an editor on the

NCAA QA server and then seeing if we

can get to the "create article" page.

Screen Captures

One of the cooler aspects of Casper is that it can take full-page screenshots of whatever the remote page looks like at various moments along the way.

 

You can also use CSS selectors to capture screens of various page regions (this is feature is kind of buggy though depending on how you CSS position the element you want to capture)

Testing with Casper

Software Testing

• Testing with Casper is similar to many other web software testing frameworks.
• First you decide the broad functionality that you would like to test and formulate a checklist of things that must be true in order for that functionality to work to spec.
• You can then translate your checklist steps into assertions using the test framework (don't worry if this doesn't make sense yet)
• The test case is usually a somewhat broad goal, eg. "the login form should accept valid credentials and redirect a successful login to the admin dashboard"
• Your assertions for the test case would be more granular, eg.
  • the username field is present
  • there are 2 fields on the login form
  • pressing submit after completing the form takes you to the dashboard

Casper's Test Framework

• Casper comes with a free-form assert() function (that you probably won't need it much)
• It also has numerous "assert*" functions designed to facilitate with DOM testing specifically.
• Casper's assert functions typically take either a CSS selector to test something about or a function that you provide.
• Anytime you're providing a custom function to assert on, it is expected to return a boolean indicating pass or fail of the assertion.
• Some examples:

  • assertExists / assertDoesntExist

  • assertVisible / assertNotVisible

  • assertElementCount(selector, count)

  • assertTitle(text) / assertTitleMatch(regex)