Mutation Testing
History
- Cowboy Coding
- Dev Server/SVN
- ...
- Mercurial
- Git
- Unit tests
- CI
- Code Coverage
Unit Tests
- Pros
- Cement ideas
- Find bugs
- Reduce regressions
- Cons
- More code
- Test only what they test
- Forces code to be written differently
Continuous Integration (CI)
- Pros
- Partially removes manual running of tests
- Cons
- Only tests what jobs it runs
Code Coverage
- Pros
- Better picture of what the unit tests are actually testing!
- Cons
- D: Only lines compiled w/ unit test
- Can miss some parts of lines
- Complex if statements
- Statement chains
Mutation Testing
Test the Tests
- Competent Programmer
- Coupling Effect
Common Mutant Operators
if (a && b)
if (a || b)
- Statement deletion
- Statement duplication or insertion
- Replace boolean subexpressions with true and false
- Replace arithmetic operations with a different one
- Replace boolean relations with others
- Replace variables with others from the same scope
Example
private bool notUseful(const uint num1, const uint num2)
{
if (num1 > 5 || num2 < 10)
return true;
else
return false;
}
unittest
{
assert(notUseful(6, 2));
assert(!notUseful(4, 11));
}
//The following should be included!
assert(notUseful(2, 2));
Relevant Links
Mutation Testing
By Jonathan Crapuchettes
Mutation Testing
- 688