Mutation Testing

History

  • Cowboy Coding
  • Dev Server/SVN
  • ...
  • Mercurial
  • Git
  • Unit tests
  • CI
  • Code Coverage

Unit Tests

  • Pros
    • Cement ideas
    • Find bugs
    • Reduce regressions
  • Cons
    • More code
    • Test only what they test
    • Forces code to be written differently

Continuous Integration (CI)

  • Pros
    • Partially removes manual running of tests
  • Cons
    • Only tests what jobs it runs

Code Coverage

  • Pros
    • Better picture of what the unit tests are actually testing!
  • Cons
    • D: Only lines compiled w/ unit test
    • Can miss some parts of lines
      • Complex if statements
      • Statement chains

Mutation Testing

Test the Tests

  • Competent Programmer
  • Coupling Effect

Common Mutant Operators

if (a && b)
if (a || b)
  • Statement deletion
  • Statement duplication or insertion
  • Replace boolean subexpressions with true and false
  • Replace arithmetic operations with a different one
  • Replace boolean relations with others
  • Replace variables with others from the same scope

Example

private bool notUseful(const uint num1, const uint num2)
{
    if (num1 > 5 || num2 < 10)
        return true;
    else
        return false;
}

unittest
{
    assert(notUseful(6, 2));
    assert(!notUseful(4, 11));
}
//The following should be included!
assert(notUseful(2, 2));

Relevant Links

Made with Slides.com