Neon Automated Testing Strategy

Key Testing Goals

  • Ensure UI works according to spec
  • Ensure Krypton passes Neon correct data
  • Tests are quick & easy to write
  • Tests execute as quickly as possible

Ensure UI works

According to Spec

  • Only testing the UI / DOM interactions
     
  • Test functionality using different data variances to ensure all use-cases are covered

Requirements

Ensure UI works

According to Spec

  • If an offering has more than one available provider, customer should proceed to Providers view
     
  • However, if an offering has only one provider, customer should by-pass Providers view and proceed directly to Schedule view

Example 1

Ensure UI works

According to Spec

  • Test offering that has 2 or more available providers
     
  • Test offering that only has 1 available provider

Example 1 Tests

Ensure UI works

According to Spec

  • Input field has the following requirements:
    • Required
    • Max length of 35 characters
    • Only contain alphanumeric characters

Example 2

Ensure UI works

According to Spec

  • Try submitting form when field:
    • Is left empty
    • Has less than 35 characters (e.g. 34)
    • Has exactly 35 characters
    • Has more than 35 characters (e.g. 36)
    • Has non-alphanumeric characters
    • Is valid

Example 2 Tests

Ensure UI works

According to Spec

  • We only need to test that all possible interactions are under test for all data variances
     
  • We don't need to test whether Neon was able to talk to Krypton
     
  • Calling Krypton for all of the tests will slow down execution

Considerations

Ensure UI works

According to Spec

  • Data & AJAX responses are mocked, leading to:
    • Faster test writing
    • Faster test execution
    • More maintainable tests

Therefore

Ensure Krypton passes Neon correct data

  • Ensure Neon is receiving the data in a structure it expects to receive
     
  • Only validating schema - values can be anything as long as they're the expected type (caveat: enums must be valid value)

Requirements

{
  type: 'object',
  
  properties: {
    'company_name': { type: 'string' },
    'url_subdomain': { type: ['string', 'null'] },
    'business_description': { type: ['string', 'null'] },
    'address_1': { type: 'string' },
    'address_2': { type: ['string', 'null'] },
    'city': { type: 'string' },
    'state': { type: 'string' },
    'zip_code': { type: 'string' },
    'preferred_color': { type: ['string', 'null'] },
    'font_color': { type: ['string', 'null'] },
    'website_url': { type: ['string', 'null'] },
    'company_email': { type: ['string', 'null'] },
    'phone': { type: ['string', 'null'] },
    'logo_image_url': { type: ['string', 'null'] },
    'business_image_url': { type: ['string', 'null'] },
    'appointments_enabled': { type: 'boolean' }
  }
}

Example JSON Schema

Ensure Krypton passes Neon correct data

  • Only testing AJAX calls
     
  • UI will never be tested during these tests

Considerations

Ensure Krypton passes Neon correct data

The Test Strategy

  • Create an AJAX mock object for each endpoint
     
  • Four different types of tests:
    • Unit Tests
    • Integration Tests
    • UI Tests
    • E2E Tests

The Test Strategy

  • Contains the following properties:
    • Relative AJAX path that's being mocked
    • JSON Schema
    • Function that creates mock data
    • Function that creates mock response
  • Mock data will have logical defaults, with a way to override the values

The Mock Object

The Test Strategy

  • Ensures mock data function returns valid data, based on the schema
     
  • Tests non-UI concerns, such as:
    • Date manipulation functions
    • Currency formatting

Unit Tests

The Test Strategy

  • Ensures that Krypton is returning data in a format that Neon is expecting, based on the JSON schema

Integration Tests

The Test Strategy

  • Mocks out the AJAX calls
     
  • Tests parts of the UI in isolation
     
  • Variances in the data is easily mocked and tested against

UI Tests

The Test Strategy

UI Tests

/* Example mocked AJAX call */
page
  .ajax['/offerings/bookable_with_providers']
  .mock
  .response({
    rowCount: 9,
    uniqueCategoryCount: 3
  })

The Test Strategy

  • Nothing is mocked - seed data is required
     
  • Tests scenarios / flows, such as:
    • Customer can successfully navigate through the appointment booking process

E2E Tests

The Test Strategy

  • By ensuring that both Krypton and the mock data validate against the JSON schema, we have confidence that our tests represent plausible use-cases
     
  • By separating the tests based on what we're testing, we ensure the tests are efficient to both execute and maintain

The Result

Our Current Topology

  • Customer has been using this strategy for the past couple of months, tweaking the process when issues / inefficiencies are discovered
     
  • Merchant has started to use mocked AJAX, but not all test have been written this way and the implementation is slightly different

PaySimple - Neon Automated Testing Strategy

By Ryan Haugh

PaySimple - Neon Automated Testing Strategy

  • 448