Software Testing

More than you ever wanted to know

What I Did/Do?

  • 2009 - C++ "Hello, World"
  • 2010 - iOS App
  • 2011 - Game
  • 2012 - iOS App
  • 2013 - Game
  • 2014 - iOS App
  • 2015/6 - Game
  • 2017/8 - iOS App

What is Testing?

Why Would You Test?

Why Do You Test?

  • Avoid surprises
  • Saves time
    • Iteration
    • Documentation
  • Better quality
  • Better architecture

How do you test?

I DON'T

Manual

Unit Testing

Where it all starts,
Where most people stop

Overview

  • Technical spec
  • Simple, isolated, lacks creativity
  • Covers simple class of bugs:
    • Arithmetic
    • NPE
    • Buffer overflow
    • Typos
  • If it's not simple,
    you're doing it wrong!

Example #1

def greet(person: str) -> str:
  return f"Hello, {person}!"
def test_greet_func():
  person = "Elon"
  assert greet(person) == "Hello, Elon!"

Example #2

from datetime import datetime

class TimeProvider:
  @staticmethod
  def current_time():
    return datetime.now()

When to Unit Test?

  • Transform functions and classes
  • Minimum side-effects
  • Minimum dependencies
  • e.g.
    • Parsers
    • RFC-compliance

What Could Go Wrong?

Feature Testing

The absolute minimum,
The most valuable

Overview

  • Business spec
  • Involves several components
  • Still, no external systems
  • e.g.
    • UI Testing
    • API Testing

Example #1

class Counter:
  def __init__(self):
    self.count = 0
  def increment(self):
    self.count += 1
def test_counter():
  counter = Counter()
  # first time
  counter.increment()
  assert counter.count == 1
  # second time
  counter.increment()
  assert counter.count == 2

Realworld Example

account = %Account{}
|> Account.changeset(%{username: "rand 12823"})
|> L3DB.Repo.insert!

result = %Device{}
|> Device.changeset(%{@valid_attrs | account_id: account.id})
|> L3DB.Repo.insert

assert {:ok, device} = result
assert Ecto.UUID.cast(device.uid) == {:ok, device.uid}
assert device.token =~ ~r([\w_-]{28})

Realworld Example

daily_reward = %{
  stack_value: %Cost{coins: 2},
  interval: :timer.hours(1) |> div(:timer.seconds(1)),
  max_stacks: 10}

# 0
date_ref = (now - div(daily_reward.interval, 2))

{timestamp, reward} = DailyReward.compute(daily_reward, date_ref)
assert timestamp == date_ref
assert reward == %{coins: 0}

# 3
date_ref = (now - daily_reward.interval * 3 - 100)

{timestamp, reward} = DailyReward.compute(daily_reward, date_ref)
assert_in_delta now - timestamp, 100, 2
assert reward == %{coins: 3 * daily_reward.stack_value.coins}

# > 10
date_ref = (now - daily_reward.interval * 11)

{timestamp, reward} = DailyReward.compute(daily_reward, date_ref)
assert_in_delta now - timestamp, 0, 2
assert reward == %{coins: daily_reward.max_stacks * daily_reward.stack_value.coins}

What Could Go Wrong?

E2E Testing

Hey now, you're a Rockstar!

Overview

  • Real-world interactions
  • Staging environment
  • Get creative!
  • All systems involved,
    NO EXCEPTION.

Realworld Examples

https://github.com/Mazyod/PhoenixSharp

https://github.com/Mazyod/dama-bots-elixir

Realworld Examples

https://blog.codinghorror.com/working-with-the-chaos-monkey/​

Key Learnings

Test-First Development

Never write tests for the sake of writing tests

Write clear, readable, and flat Tests

As much as possible:
Write less tests that cover more code

As much as possible:
Tests should not affect shipped code

Bonus Topics!

Code Coverage

https://stackoverflow.com/questions/90002

Property-based
Testing

https://www.youtube.com/watch?v=hXnS_Xjwk2Y

Workshop

Q&A

QA

Made with Slides.com