Who Are We?

We offer trusted digital content, adaptive algorithms, and teaching support tools to make it easier for all teachers to differentiate their instruction so every child performs better.

The team

Mike

Lead developer

Dan

Senior Developer

Mathieu

Senior developer

Philippe

Developer (Main QA)

Challenges & Solutions

An SPA is Disconnected
From it's PHP Backend

• Behat works inside PHP (Process #1)
• An SPA is in a browser (Process #2)
• PHP app (backend) ignores it is in testing mode unless:
  • header
  • query param
  • or other permanent change exists

Problems

• Injecting headers or query params (hard)
• Use a .testing file or a persistent flag somewhere to identify change of environment

Solutions

// If the app_environment is null, use the environment variable
if ($environment == null) {
    if (file_exists(REAL_ROOT_DIR . '/.testing')) {
        $environment = env('LB_TESTING_ENV', 'default,local,testing');
    } else {
        $environment = env('LB_ENV', 'default');
    }
}

• .testing may not be deleted if there is a fatal error
• Easier to activate testing environment:
  • touch .testing

Drawbacks & Good points

• Tracking JS errors, ajax calls and window prompts can be painful

Problems

• Inject a testing library that overrides Alerts/Confirms

Solutions

window.confirm = function (confirmText) {
    var result = lb.testing.confirm.acceptConfirm;
    lb.testing.confirm.active = true;
    lb.testing.confirm.text = confirmText;

    lb.testing.confirm.acceptConfirm = true; //Restore defaults after a call
    return result;
};

window.alert = function (alertText) {
    lb.testing.alert.active = true;
    lb.testing.alert.text = alertText;
    return true;
};

Then I should see "confirm" dialog with message 
    "Are you sure you want to delete the playlist: 'TESTING PLAYLIST'?" 
    and resolve it

• Inject a testing library that monitors javascript errors

Solutions

window.onerror = function(error, url, line) {
    lb.testing.errors.push(error + ". Line: " + line);
};

$numErrors = $this->getSession()->evaluateScript('typeof(lb) != "undefined" 
    ? lb.testing.errors.length : 0');

• Inject a testing library that monitors AJAX calls

Solutions

$(document).on('pjax:start', function () {
    if (lb.testing.pjax.enabled && lb.testing.pjax.activeCount == undefined) {
        lb.testing.pjax.activeCount = 0;
    }
    lb.testing.pjax.activeCount++;
});

$(document).on('pjax:end', function () {
    if (lb.testing.pjax.enabled) {
        lb.testing.pjax.activeCount--;
    }
});

$(document).ajaxSend(function (event, jqXHR, ajaxOptions) {
    lb.testing.ajax.activeCount++;
});

$(document).ajaxComplete(function (event, jqXHR, ajaxOptions) {
    lb.testing.ajax.activeCount--;
    lb.testing.ajax.latestRequest = (new Date()).getTime();
});

And I wait for ajax calls to complete

Challenges & Solutions

Handling PJAX and
Live Dom Changes

• In SPAs, page url and page load events are simulated and don't occur/change when you ask for a new page
• Mink is all about immediate DOM processing and page change handling so most mink steps fail automatically because you don't know when stuff really changes...

Problems

• Implement a completely different set of steps that use javascript to detect DOM changes

Solutions

Then I should be brought to page matching "/search\?community=1"

/**
 * @Then I should be brought to page matching :regex
 * @Then I should be brought to page matching :regex within :timeout seconds
 */
public function shouldBeBroughtToPageRegex($regex, $timeout = self::DEFAULT_TIMEOUT)
{
    $this->getSession()->wait(
        $timeout * 1000,
        "new RegExp('" . $this->locatePath($regex) . "', 'i')
            .test(window.location) == true"
    );
    \Assert\Assertion::regex($this->getSession()->getCurrentUrl(), 
        '|' . $this->locatePath($regex) . '|i');
}

Given I am on "/"
When X item is clicked
Then I should be on "/search?community=1"

• Create several functions
  • shouldEventuallyContain
  • shouldEventuallyDisapear
  • etc

Solutions

$countEqFunc = function () use ($element, $numberOfItems) {
 	if (count($element->findAll('css', '.selectables-container button')) 
            == $numberOfItems) {
        return true;
    }
    return false;
};

if (!$element->waitFor(self::DEFAULT_TIMEOUT, $countEqFunc)) {
    throw new ExpectationException("message");
}

Solutions

public function waitFor($timeout, $callback)
{

    $start = microtime(true);
    $end = $start + $timeout;

    do {
        $result = call_user_func($callback, $this);

        if ($result) {
            break;
        }

        usleep(100000);
    } while (microtime(true) < $end);

    return $result;
}

• A lot of functions to redefine
• Naming convention gave us headache at first and became confusing
• Build slowly and think about naming

Drawbacks

Challenges & Solutions

Readability and

Change Support

• Gherkin is hard to read if you have complex scenarios
• Mink offers steps that read out well in KISS pages but not in complex apps
• Gherkin can easily become tropical jungle of CSS selectors
• Change your app's HTML/Javascript/CSS and you have to rewrite a lot of tests

Problems

• Create more steps that better describe the actions, less CSS, more words (Widget feature traits)

Solutions

When ActionsButton ".actions-widget" is clicked
Then ActionsButton ".actions-widget" should contain "Edit" action

trait ActionsButtonTrait
{

    /**
     * @When ActionsButton :selector is clicked
     */
    public function actionsButtonClick($selector = null);

    /**
     * @Then ActionsButton :selector should contain :text action
     */
    public function actionsButtonAssertHasAction($text, $selector = null);

    /**
     * @Then ActionsButton :selector should not contain :text action
     */
    public function actionsButtonAssertDoesNotHaveAction($text, $selector = null);

• Reuse existing mink functions when possible inside new step

Solutions

And I am logged in as "testing+teacher@learningbird.com"

public function loginAsUser($username)
{
    $this->visit('/signIn');
    $this->fillField('signInEmailAddressField', $username);
    $this->fillField('signInPasswordField', 'testing-password');
    $this->pressButton('signInTrigger');
    $this->userShouldBeLoggedIn();
}

public function userShouldBeLoggedIn()
{
    try {
        $this->elementShouldAppear('css', '#navbar', null, 25);
    } catch (ExpectationException $ex) {
        $this->assertPageNotContainsText('Could not verify your login. Please make sure you entered your details correctly.');
        throw $ex;
    }
}

Challenges & Solutions

No Concept of Context

(Part 2 of readability)

• A step doesn't affect the global scope
• If you want easier to read code you need to keep context

Problems

Example

Then I should see "Create Quiz"

    And BaseList ".quizzes-dashboard .base-list" should 
        contain "1" item

    And BaseList ".quizzes-dashboard .base-list" item 
        ":nth-of-type(1)" data "numberOfAssignments" should contain "0"

    And BaseList ".quizzes-dashboard .base-list" item 
        ":nth-of-type(1)" data "numberOfQuestions" should contain "5"

    And BaseList ".quizzes-dashboard .base-list" item 
        ":nth-of-type(1)" data "commands" should contain a "ActionsButton" widget

    And ActionsButton ".quizzes-dashboard .base-list:nth-of-type(1) .actions-button" 
        should not contain "Edit" action

    And ActionsButton ".quizzes-dashboard .base-list:nth-of-type(1) .actions-button" 
        should not contain "Assign" action

    When ActionsButton ".quizzes-dashboard .base-list:nth-of-type(1) .actions-button" 
        is clicked

• Create steps that preserve context

Solutions

//Save the current element as "BaseList"
return $this->getWidgetElement($listSelector, 'BaseList');

protected function getWidgetElement($selector, $widgetName, $inElement = null)
{
    if ($selector == null) {
        $element = $this->getWidgetContext($widgetName);

        return $element;
    } else {
        $element = $this->getElement('css', $selector, $inElement);
        Assertion::true(
            $element->hasClass($expectedClass = $this->getWidgetClass($widgetName)),
            'Element under "' . $selector . '" found but class "' . $expectedClass . '" not found'
        );
        $this->setWidgetContext($widgetName, $element);

        return $element;
    }
}

Challenges & Solutions

World Setup / Cleanup

is Hard With Behat

• Fact: Tests are always around preparing the world
• Fact: No easy way to reseed or reset database

Facts

• Gerkhin is not geared per se at fixtures or test data
• The behat language features a few constructs that can be used for that but none are perfect for the case:
  • Table Nodes
  • Precondition steps

Problems

• Use TableNodes

Solutions

And teacher "teacher@testing.com" has the following classes:
    | className     | gradeLevel | subject |
    | Math class    | 9          | Math    |
    | Science class | 10         | Science |

/**
 * @Given teacher :teacherEmail has the following classes:
 */
public function createClassesForTeacher(TableNode $classData, string $teacherEmail): array //[classesRecord]
{
    $teacher = $this->getOrCreateTeacher($teacherEmail);
    $account = $teacher->getAccount();
    if ((int)$account->getAccountTypeID() !== accountTypesModel::TEACHER) {
        throw new Exception('referenced teacher must be a rogue teacher');
    }
    $classes = $this->createClassesFromTableNode($classData, $account);

    foreach ($classes as $class) {
        $this->associateTeacherToClass($teacher, $class);
    }

    return $classes;
}

• Creation of pre-condition steps

Solutions

Given a skeleton "teacher" with the email "teacher@testing.com" exists

Given a skeleton "teacher" with the email "teacher2@testing.com" and 
    password "testpwd" exists with 60 licenses

/**
 * @Given a skeleton :userType with the email :emailAccount and password :password exists 
 * with :numLicenses licenses
 */
public function createSkeletonUserAccount($emailAccount, $userType, 
    $password = self::DEFAULT_PASSWORD, $numLicenses = 100)
{
    switch ($userType) {
        case 'teacher':
            $this->createSkeletonTeacher($emailAccount, $password, $numLicenses);
            break;
        case 'student':
            $this->createSkeletonStudent($emailAccount, $password);
            break;
        case 'school_admin':
            $this->createSkeletonSchoolAdmin($emailAccount, $password);
            break;
    }
}

• Gerkhin doesn't feature steps to cleanup or prepare the world
• What it does have is annotation bindings:
  • @beforeSuite
  • @beforeScenario
  • @beforeStep
  • @afterStep
  • ....

Problems

• Using hooks to handle special cases such as seeding/cleaning up

Solutions

/**
 * @BeforeSuite
 *
 * @param \Behat\Testwork\Hook\Scope\BeforeSuiteScope $scope
 */
public static function beforeSuite(\Behat\Testwork\Hook\Scope\BeforeSuiteScope $scope)
{
    $suite = $scope->getSuite();
    file_put_contents(REAL_ROOT_DIR . '/.testing', \Carbon\Carbon::now()->format('Y-m-d G:i:s'));
    if (!file_exists(LOGS_DIR . "/behat-screenshots/{$suite->getName()}/")) {
        mkdir(LOGS_DIR . "/behat-screenshots/{$suite->getName()}/", 0755, true);
    }

    // Kill the screenshots
    /** @var SplFileInfo $file */
    foreach (new DirectoryIterator(LOGS_DIR . "/behat-screenshots/{$suite->getName()}/") as $file) {
        if (substr($file->getFilename(), 0, 1) == '.') {
            continue;
        }
        echo 'Removing ' . realpath($file->getPathName()) . PHP_EOL;
        unlink($file->getPathName());
    }

    self::printSuiteInfo($suite);
}

• Using hooks to handle special cases such as seeding/cleaning up

Solutions

/**
 * @BeforeScenario
 *
 * @param \Behat\Behat\Hook\Scope\BeforeScenarioScope $scope
 */
public function before(\Behat\Behat\Hook\Scope\BeforeScenarioScope $scope)
{
    $this->cleanDatabase();
    $this->resetWidgetContexts();
    $this->getSession()->resizeWindow(1440, 900, 'current');

    #Cookies cause issues when executing/evaluating scripts in 
    #internet explorer
    $this->getSession()->getDriver()->reset();
}

• Using hooks to handle special cases such as seeding/cleaning up

Solutions

/**
 * @AfterStep
 */
public function afterStepTasks(\Behat\Behat\Hook\Scope\AfterStepScope $event)
{
    // Failed step
    if ($event->getTestResult()->getResultCode() == 
        \Behat\Testwork\Tester\Result\TestResult::FAILED) {

        $this->takeScreenshotAfterFailedStep($event->getStep()->getText(), 
            $event->getSuite());

    }

    // Collect and handle javascript errors
    $this->reportJavascriptErrors();
}

• Using hooks to handle special cases such as seeding/cleaning up

Solutions

/**
 * @AfterScenario
 *
 * @param \Behat\Behat\Hook\Scope\AfterScenarioScope $scope
 */
public function after(\Behat\Behat\Hook\Scope\AfterScenarioScope $scope)
{
    // Make sure you can exit without getting asked if you want to leave
    $this->getSession()->executeScript('window.onbeforeunload = null;');
    $this->clearMailFromMemory();
    $this->cleanPlaylists();
}

• Using hooks to handle special cases such as seeding/cleaning up

Solutions

/**
 * @AfterSuite
 *
 * @param \Behat\Testwork\Hook\Scope\AfterSuiteScope $scope
 */
public static function afterSuite(\Behat\Testwork\Hook\Scope\AfterSuiteScope $scope)
{
    unlink(REAL_ROOT_DIR . '/.testing');
}