Background

Who Am I?

Harper Maddox

CTO, EdgeTheory

Ruby Developer since 2013

Someone who has rarely had time for tests and will only write them when dragged kicking and screaming

Our Stack 📚

Hard Split between backend and frontend

Angular JS

Ruby w/ Sinatra

MySQL

Rackspace Cloud

 Microservices

We use lots of little APIs

UIs talk to them

They talk to each other

They force us to have loose coupling

We can test each one independently

They are easy to refactor

Manual Tests

Smoke Tests

Everyone does these tests

UI

• Does the site come up?
• Do the links work?
• Can I modify the data?
• Are the modifications permanent?

API

• Can I CURL the API?
• Do the routes work (no 404)?
• Can I modify the data?
• Are the modifications permanent?

User Acceptance Tests

Create a test plan

Can users execute all of the test cases?

Bring the JIRA!

Tons of administrative work

🤓🗄️🐞

Automated Tests

How tests work

Test Suite - all the tests for your app

Test Cases - test specific behavior

Assertions - test correctness of specific behavior

You can run your whole test suite, a test file, or a specific test case in a file.

Assertions

Common Assertions in MiniTest

assert        # is it truthy?
assert_nil    # is it nil?
assert_equal  # do the 2 parameters match?
assert_match  # test string against a pattern

Assertions are the backbone of automated testing.

We use them to verify correctness in a test case

It is normal to have multiple assertions in a test case

Assertions

An example using MiniTest

@user       = User.new
@user.name  = 'Bruce Springsteen'
@user.title = 'The Boss'
@user.state = 'New Jersey'
@user.save

assert 'Bruce Springsteen', @user.name  # pass
assert 'The Boss',          @user.title # pass
assert 'Hawaii',            @user.state # FAIL

unit tests cover the trees

integration tests cover the forest

Unit Tests

Useful for libraries

Ideal for isolated code

Great for pure functions

Overkill for business logic

Easier to test your routes than test the details

Use Cases

Character Counter

We write tools for social media management. We have to validate the length of text before our users send/schedule a message.

Url Shortener

We run our own custom url shortener that generates a new hash for each domain. A common function is to convert an integer (database table ID) into a base 62 string and vice-versa.

Ex. Url Shortener

require './test/test_helper'

class BaseConversionEncodingTest < MiniTest::Test
  def setup
    @logger = Logger.new(STDOUT)
  end

  def test_convert_b10_to_b62_zero
    assert_equal('0', Leadify::LinkShortener::Math.to_base(0, base=62))
  end

  def test_convert_b10_to_b62_one_digit
    assert_equal('5', Leadify::LinkShortener::Math.to_base(5, base=62))
  end

  def test_convert_b10_to_b62_two_digit
    assert_equal('A', Leadify::LinkShortener::Math.to_base(36, base=62))
  end

  def test_convert_b10_to_b62_two_digit_b62
    assert_equal('10', Leadify::LinkShortener::Math.to_base(62, base=62))
  end

  # ...
end

Integration Tests

We either test individual API calls or test a series of API calls.

We are no longer testing small, distinct components.

The Big Picture

CRUD

These are the most common routes in a REST API

They heavily use a permanent data store (database)

All of our routes except deletes return JSON data

Rack::Test

It provides the http methods: get, post, put, patch, and delete

And it includes a simple syntax for calling them:

<http method> <route name>, <parameters>, <headers>
# Examples
get  '/messages'
post '/messages', text: "Gotta catch 'em all"
post '/shares', { facebook_post_id: '1337' },
                'rack.session' => { visit_id: 1 }

Rack::Test

def test_post_messages_with_message
    post '/messages', text: '@ash_ketchum_all, check out this pikachu
 I found digging through the dumpster behind the #Pokestop!'
    assert last_response.ok?
    response = Oj.load(last_response.body)
    assert(response.key?('message'))
    message = response['message']
    assert_equal '@ash_ketchum_all, check out this pikachu I found
 digging through the dumpster behind the #Pokestop!', message['text']
end

We test both the HTTP status code that is returned and the JSON output.

Code Coverage

Patterns for writing good test cases that cover all of your codebase

Test Patterns

My test cases fall into four different patterns:

1. Input Validation

2. Expected Output

3. Persistent Data

4. Business Logic

1) Input Validation

Does the route handle non-existent or invalid parameters the way we expect?

When validation fails it should

• stop execution

• return an error code

• sometimes return an error message (E.g., “name is a required field”).

2) Expected Output

Step 1 - Was the http status was success or failure?

If it failed, we can stop there and we have to figure out why it had an error.

On Success

Step 2 - Verify that we saw the expected JSON output

• Is it a properly formatted object?

• Is the data what we expected?

• Is multi-level data properly formed?

• If we have a problem, where specifically was the problem?

3) Persistent Data

When we make a POST, PUT, or DELETE, we expect to modify our data.

We can validate that it returned the correct output, but that doesn’t mean our app permanently stored the data.

We have to check the data store after the API call.

4) Business Logic

It defines how your objects interact with each other.

By definition it is hard to isolate.

But maintanable apps require us to cover our code.

Examples of biz logic

New User Signup

1. Create a login and an account in the same call
2. Create a subscription and connect it to a Stripe subscription
3. Some of our users authorize their Twitter account
4. Signup is complete, user is live

Since we’re relying on a few external dependencies (Stripe and Twitter), they aren't accessible by our API in test mode. Furthermore, would we want to?

So how do we deal with external dependencies?

Mocks

Also known as Stubs 🎫

Stubs

Stubs override methods

Mocks are fake objects

Stripe::Plan.stubs(:create).returns(
      stripe_uuid: 'vindaloo',
      name: 'Red Dwarf',
      subtitle: 'Spaceship',
      statement_descriptor: 'EDGETHEORY SB VIP',
      amount: 9999,
      currency: 'usd',
      interval: 'month',
      interval_count: 1,
      is_active: false
    )

Here's a stub overriding a class method, so we don't have to call the Stripe API:

Mocks

It's easy to stub a class method.

Simulating an instance method requires a bit more finesse.

You’ve got to first create a mock object, then you have to extend that mock object with a stub.

# Mock object needed to stub an instance method
object = mock('object')
Stripe::Plan.stubs(:retrieve).returns(object)
object.stubs(:delete).returns(true)

Why are they useful?

These mocks allowed us to eliminate the unpredictability and slowness of an external api.

We run these two code snippets in the setup block of our test, so that when we create or delete a plan we can get a predictable result.

We always return the same hash when we create a plan and we can cleanly delete one as well.