Testing

Unit testing, E2E testing, Karma, Protractor, Jasmine, Phantom.JS

Samuel Šramko

Testing

  • testing application you are working on is crucial part of development and successful deployment and maintenance
  • TDD - test driven development
    1. create test for the component
    2. test should fail - logic is not implemented yet
    3. implement logic for the component
    4. test should pass
  • most of the companies / teams does not strictly follow TDD principles, but tests should cover at least 60-70% of code

Testing #2

  • there are many types of tests:
    • unit tests - test should cover only 1 concrete method/function of the component, nothing should be executed only the tested function (http call, call to other service -> these service calls should be mocked/stubbed
    • integration tests - these tests will call directly into your code. For example, you can use an integration test to call an Angular service. Typically each test will focus on one function. The test calls the target function with a set of parameters and then checks to make sure the results match expected values.

Testing #3

  • there are many types of tests:
    • end-to-end tests - tests simulating user behaviour on a web page, simulate click on element and what should happen
    • smoke tests - test which verifies that page is working, usually very simple, only displays page and checks wether there are no visible error messages (by sending screenshots to email, etc...)
    • performance tests - performance of app must be measured, e.g. when building dashboard for electric company, each chart must be displayed with minimal latency, etc.

Testing Angular

  • combination of frameworks and technologies to successfully test Angular app
    • Jasmine - test framework used for testing components behaviour, shipped with HTMR test runner which executes tests in a browser
    • Angular testing utilities - create a test environment for the Angular application code under test. Use them to condition and control parts of the application as they interact within the Angular environment
    • Karma - test runner used for running unit and integration test in a browser environment
    • Protractor - used for e2e tests, simulates user behaviour by clicking on elements, invoked events, etc. in a browser

Testing Angular #2

  • tests are written in Jasmine, and run via Karma
  • in Jasmine, test suite is identified via describe() function, which consists of individual tests
  • logic of individual tests is obtained in an it() function
describe('1st tests', () => {
  it('true is true', () => expect(true).toBe(true));
});
  • each test defined in test suite inherits all properties and methods defined in a describe()
  • each component should have 1 test suite, which should be placed in a file called *.spec.[ts|js]
  • tests could be placed directly to the component it tests or, which I prefer for bigger clarity, in a separate folder test[s]

Jasmine

  • Jasmine has a built-in set of matchers, which are used for comparing expected and original values
  • Each matcher implements a boolean comparison between the actual value and the expected value. It is responsible for reporting to Jasmine if the expectation is true or false. Jasmine will then pass or fail the spec.
  • Any matcher can evaluate to a negative assertion by chaining the call to expect with a not before calling the matcher.
it('matchers', () => {
    const a = 12;
    const message = "foo bar baz";
    const b = {
      foo: "foo"
    };

    expect(a.foo).toBeDefined();
    expect(() => throw new TypeError("type error")).toThrowError("type error");
    expect(true).toBe(true);
    expect(true).not.toBe(false);
    expect(a).toEqual(12);
    expect(message).toMatch(/bar/);
    expect(message).toContain("bar");
    expect(b.foo).toBeDefined();
    expect(null).toBeNull();
    expect(3).toBeLessThan(4);
});

Jasmine #2

  • Jasmine has test double functions called spies. A spy can stub any function and tracks calls to it and all arguments. A spy only exists in the describe or it block in which it is defined, and will be removed after each spec. There are special matchers for interacting with spies. 
describe("A spy", () => {
  var foo, bar = null, fetchedBar;

  beforeEach(function() {
    foo = {
      setBar: (value) => bar = value,
      getBar: () => bar
    };

    spyOn(foo, "setBar");
    spyOn(foo, "getBar").and.returnValue(745);

    foo.setBar(123);
    foo.setBar(456, 'another param');
    fetchedBar = foo.getBar();
  });

  it("spy matchers", () => {
    expect(foo.setBar).toHaveBeenCalled();
    expect(foo.setBar).toHaveBeenCalledTimes(2);
    expect(foo.setBar).toHaveBeenCalledWith(456, 'another param');
    expect(foo.setBar).toHaveBeenCalled();
    expect(fetchedBar).toEqual(745);
  });
});

Testing Angular #3

  • to run tests, npm script test should be defined to start karma
  • karma should be configured in a karma.conf.js file
const webpackConfig = require('./webpack.test');

module.exports = function (config) {
    var _config = {
        basePath: '',
        frameworks: ['jasmine'],
        files: [
            {pattern: './config/karma-test-shim.js', watched: false}
        ],
        preprocessors: {
            './config/karma-test-shim.js': ['webpack', 'sourcemap']
        },
        webpack: webpackConfig,
        webpackMiddleware: {
            stats: 'errors-only'
        },
        webpackServer: {
            noInfo: true
        },
        reporters: ['progress'],
        port: 9876,
        colors: true,
        logLevel: config.LOG_INFO,
        autoWatch: false,
        browsers: ['PhantomJS'],
        singleRun: true
    };
    config.set(_config);
};

Testing Angular #4

  • in karma-test-shim.js file, Angular specific test configuration should be set up
  • angular testing utilities from @angular/core/testing module should be added to configuration
  • main testing utility class is TestBed 
Error.stackTraceLimit = Infinity;

require('core-js/es6');
require('core-js/es7/reflect');

require('zone.js/dist/zone');
require('zone.js/dist/long-stack-trace-zone');
require('zone.js/dist/proxy');
require('zone.js/dist/sync-test');
require('zone.js/dist/jasmine-patch');
require('zone.js/dist/async-test');
require('zone.js/dist/fake-async-test');

var appContext = require.context('../src', true, /\.spec\.ts/);

appContext.keys().forEach(appContext);

var testing = require('@angular/core/testing');
var browser = require('@angular/platform-browser-dynamic/testing');

testing.TestBed.initTestEnvironment(
    browser.BrowserDynamicTestingModule, 
    browser.platformBrowserDynamicTesting());

Testing Angular #5

  • TestBed creates an Angular testing module — an @NgModule class — that you configure with the configureTestingModule method to produce the module environment for the class you want to test.
  • In effect, you detach the tested component from its own application module and re-attach it to a dynamically-constructed, Angular test module tailored specifically for this battery of tests.
describe('CustomTextComponent (inline template)', () => {

  let comp:    CustomTextComponent;
  let fixture: ComponentFixture<CustomTextComponent>;

  beforeEach(() => {
    TestBed.configureTestingModule({
      declarations: [ CustomTextComponent ],
    });
    fixture = TestBed.createComponent(CustomTextComponent);
    comp = fixture.componentInstance; // CustomTextComponent test instance

    // write all tests here
    it('should ...', () => {});
  });
});

Testing Angular #6

  • Jasmine runs beforeEach function for every test
  • in order to tell tested component, that some changes were made, call method detectChanges() on a component fixture
it('should display original title', () => {
  fixture.detectChanges();
  expect(comp.title).toBe('Title');
});

it('should display a different test title', () => {
  comp.setTitle('Test Title');
  fixture.detectChanges();
  expect(comp.title).toBe('Test Title');
});
  • to avoid calling detectChanges() every time something in a component changes, we can configure TestBed testing module with auto-detect changes functionality -> only triggers change detection after Promise resolution, DOM event, direct synchronous update is not working and detectChanges() must be called manually
TestBed.configureTestingModule({
  declarations: [ CustomTextComponent ],
  providers: [
    { provide: ComponentFixtureAutoDetect, useValue: true }
  ]
})

Testing Angular #7

  • when a component has dependencies, these have to be provided to a testing module
  • never provide real dependencies to a testing module, use stubbed implementations instead
const colorServiceStub = {
    getColors: {colors: [], chartColors: []}, saveColors: {}
};

const notificationServiceStub = {
    success: {}, error: {}
};

describe('colors component', () => {
    let component: ColorsComponent;
    beforeEach(() => {
        TestBed.configureTestingModule({
            declarations: [ColorsComponent],
            providers: [
                {provide: ColorService, useValue: colorServiceStub},
                {provide: NotificationsService, useValue: notificationServiceStub}]
        });
        const fixture = TestBed.createComponent(ColorsComponent);
        component = fixture.componentInstance;
    });

    it('should change color', () => {
        const color = {value: '#123123'};
        component.colorChanged('#aaaaaa', color);
        expect(color.value).toBe('#aaaaaa');
    });
});

Testing Angular #8

  • when a dependency is needed in a test case, it can be obtained from an injector
    • root injector of a TestBed module
    • injector of the component - safest way, but more verbose
const colorServiceStub = {
    getColors: {colors: [], chartColors: []}, saveColors: {}
};

const notificationServiceStub = {
    success: {}, error: {}
};

describe('colors component', () => {
    let component: ColorsComponent;
    beforeEach(() => {
        TestBed.configureTestingModule({
            declarations: [ColorsComponent],
            providers: [
                {provide: ColorService, useValue: colorServiceStub},
                {provide: NotificationsService, useValue: notificationServiceStub}]
        });
        const fixture = TestBed.createComponent(ColorsComponent);
        const colorService = TestBed.get(ColorService);
        const notificationService = fixture.debugElement.injector.get(NotificationsService);
    });
});

Testing Angular #9

  • testing async methods is done via async() or fakeAsync() testing utility functions, or Jasmine done()
    • async() must be combined with whenStable()
    • fakeAsync() is more simple and looks more "synchronous"
    • done() is passed as a parameter to it() callback and the end of the Promise.then() callback done() is called
it('should show quote after getQuote promise (async)', async(() => {
  fixture.detectChanges();

  fixture.whenStable().then(() => { // wait for async getQuote
    fixture.detectChanges();        // update view with quote
    expect(el.textContent).toBe(testQuote);
  });
}));

it('should show quote after getQuote promise (fakeAsync)', fakeAsync(() => {
  fixture.detectChanges();
  tick();                  // wait for async getQuote
  fixture.detectChanges(); // update view with quote
  expect(el.textContent).toBe(testQuote);
}));

it('should show quote after getQuote promise (done)', done => {
  fixture.detectChanges();
  spy.calls.mostRecent().returnValue.then(() => {
    fixture.detectChanges(); // update view with quote
    expect(el.textContent).toBe(testQuote);
    done();
  });
});

E2E tests

  • used for simulating behaviour of user in our application
    • click on buttons, fill in input fields, navigate across app, etc.
  • testing real use cases, tests for known bugs
  • written in Protractor, run against Selenium server in a chosen browser
    • Protractor is JS CLI utility, test runner similar to Karma
    • Selenium is a portable software-testing framework for web applications. Support for multiple browsers.
  • available via NPM, Selenium WebDriver API consists of Selenium server and webdriver-manager CLI utility, which runs, updates, installs server and browsers locally
    • ​ChromeDriver, FirefoxDriver, InternetExplorerDriver, PhantomJSDriver, SafariDriver, etc.

E2E tests #2

  • test cases have similar definition like those for unit tests in karma
    • filenames should be *.spec.js
    • Protractor configuration is placed in protractor.conf.js
    • Jasmine can be used for matching expectations
    • run via npm script protractor protractor.conf.js 
exports.config = {
  framework: 'jasmine',
  seleniumAddress: 'http://localhost:4444/wd/hub',
  specs: ['spec.js'],
  multiCapabilities: [{
    browserName: 'firefox'
  }, {
    browserName: 'chrome'
  }]
}

describe('Protractor Demo App', () => {
  it('should have a title', () => {

    // browser is a protractor-created global variable
    browser.get('http://juliemr.github.io/protractor-demo/');

    expect(browser.getTitle()).toEqual('Super Calculator');
  });
});

E2E tests #3

  • other global objects are element and by
    • ​element is used for accessing elements in a rendered page
    • by is used for selecting elements by id, css, className, name, tagName or others
  • to send data to input, use element function sendKeys()
  • to emit click on the element, invoke click()
  • to obtain text from element, use getText()
  • to submit form, use submit() function on a located form element
// spec.js
describe('Protractor Demo App', function() {
  it('should add one and two', function() {
    browser.get('http://juliemr.github.io/protractor-demo/');
    element(by.css('first-input')).sendKeys(1);
    element(by.css('second-input')).sendKeys(2);

    element(by.id('gobutton')).click();
  });
});

Smoke tests

  • in smoke tests, usually only screenshots of several pages of the application are made and stored or sent to developers
  • usually are invoked via CI tools like Jenkins or Bamboo
  • one of the the simplest way to capture screenshots of the application is using PhantomJS
    • headless WebKit scriptable with a JavaScript API :-)
    • emulates browser in memory
    • it's possible to open page, invoke JS functions on the page, render page to PNG or even PDF
    • to run PhantomJS script from Node.JS app, PhantomJS for Node module called Phantom must be used -> PhantomJS API is not available in NodeJS by default

Phantom on Node 7+

var phantom = require('phantom'), fs = require('fs'), q = require('q'),
    os = require('os'), argv = require('minimist')(process.argv.slice(2)),
    pageProperties = {width: 1920, height: 1080};

var loginPage, ph;
var hostname = os.hostname();
var baseUrl = 'http://' + hostname + '/#/';

(async () => {
    const ph = await phantom.create(['--web-security=no',
         '--ignore-ssl-errors=yes', '--webdriver-loglevel=NONE']);
    const page = await ph.createPage();
    await page.property('viewportSize', pageProperties);
    const status = await page.open(baseUrl);
    if (status !== 'success') {
        throw new Error('open page failed');
    } 
    await page.evaluate((login, password) => {
          document.getElementById("username").value = login;
          document.getElementById("password").value = password;
          document.getElementsByName("submit")[0].click();
        }, "admin", "admin");
    }
    setTimeout(() => {
        page.render(outputFileName);
        ph.exit(0);
    }, 5000);
})();

Literature