GAIA INTEGRATION TESTS 

using THE MARIONETTE JS RUNNER


Germán Toro del Valle

agenda (for quick access)

  1. Unit tests vs. Integration tests
  2. Why integration tests?
  3. What is Marionette?
  4. The Marionette JS Runner
  5. Running our first integration test
  6. Basic integration test structure
  7. Instantiating the Marionette client
  8. Client: Configuration
  9. Client: Setting the context
  10. Client: Getting information
  11. Client: Navigating
  12. Client: Finding elements

...

agenda (for quick access)

...

  1. Client: Executing scripts
  2. Client: Other
  3. Element: Getting info
  4. Element: Interaction
  5. Element: Finding children
  6. Element: Executing scripts
  7. Element: Other
  8. Client: Helper extension
  9. Client: App extension
  10. Client: Actions extension

...

agenda (for quick access)

...

  1. Client: Multi-Actions extension
  2. Final tricks: Launching apps
  3. Final tricks: Importing scripts
  4. Final tricks: Internationalization
  5. Final tricks: Transitions / Animations
  6. Final tricks: Simulating DOM events
  7. Final tricks: Intermittent failures
  8. References

UNIT vs. integration tests

  1. Checks that the Contacts app navigation library works as expected
  2. Mocks the UI
  3. Dispatches events (animetionend) as the communication means among the involved components

THIS IS NOT A UNIT TEST!!!

why integration tests?


What is Marionette?

THE MARIONETTE JS RUNNER


running our first integration test

  • Pre-requisites: Node.js >= 0.10

git clone https://github.com/mozilla-b2g/gaia.git
make test-integration [VERBOSE=1] [APP=<app-name> | TEST_FILES="<path-to-test-1> <path-to-test-2> ..."]

export VERBOSE=1export APP=communicationsexport TEST_FILES="<path-to-test-1> <path-to-test-2> ..."
$GAIA/bin/gaia-marionette$GAIA/bin/gaia-marionette-parallel

# To avoid the b2g-desktop windows from showing for every test.xvfb-run make test-integration
# To avoid hearing sounds.
PULSE_SERVER=":" make test-integration

PULSE_SERVER=":" xvfb-run make test-integration

# To avoid the b2g-desktop windows from showing for every test and to avoid hearing sounds.PROFILE_FOLDER=profile-test make # generate profile directory in first timemake test-integration-test


BASIC INTEGRATION TEST STRUCTURE

INSTANTIATING THE MARIONETTE CLIENT

client: CONFIGURATION

client: setting the context

client: GETTING INFORMATION

client: NAVIGATING

client: FINDING ELEMENTS

client: EXECUTING SCRIPTS

client: OTHER

Element: Getting info

Element: INTERACTION

Element: FINDING CHILDREN

Element: EXECUTING SCRIPTS

Element: OTHER

CLIENT: HELPER EXTENSION

CLIENT: APP EXTENSION

client: actions extENSION

client: MULTI-actions extENSION

Final tricks: Launching apps

Final tricks: Importing scripts

Final tricks: Internationalization

Final tricks: Transitions / Animations

Final tricks: simulating dom events

  • element.click(), element.tap() sometimes just don't work 

final tricks: INTERMITTENT failures

  • Forcing may runs in a row to detect intermittent failures:
  1. Activate Travis in your forked repository from mozilla-b2g/gaia using https://travis-ci.org/profile
  2. Update .travis.yml according to https://gist.github.com/KevinGrandon/8047042 and add it to your patch
  3. Send a PR to your forked repository master branch
  4. Wait for Travis' veredict

REFERENCES

Gaia Integration Tests using Marionette JS Runner

By Germán Toro del Valle

Gaia Integration Tests using Marionette JS Runner

  • 6,649