Unit Tests

The Good, the Bad and the Ugly

Michał Urbanek

What is it not about?

• Integration testing
• UI testing 
• Acceptance testing
• Smoke and sanity testing
• Security testing
• Stress testing
• ...

What is it about then?

Good Unit Tests

• trustworthy
• maintainable
• readable

Trustworthy

• test the right thing
• easy to run
• run them often
• make them fast
• failure == problem

Obvious, right?

• Always see the failing test first
• Keep your tests independent from each other
• Enforce test isolation - no flickering tests
• Clean the environment before tests, not afterwards ​
• DRY = reuse test code

Piece of cake?

Parallel test execution

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
      <forkCount>2</forkCount>
      <reuseForks>true</reuseForks>
    </configuration>
  </plugin>

Naming

public void testCreateUser()

public void testCreateUser2()

Think about the scenario

@Test
public void shouldReturnFalseIfTransactionIsPending() {

    transaction.setState(PayoutTransactionState.PENDING);
    assertThat(transaction.isPaid()).isFalse();
}

@Test
public void shouldNotConsiderPendingTransactionAsPaid() {

    transaction.setState(PayoutTransactionState.PENDING);
    assertThat(transaction.isPaid()).isFalse();
}

VS

Structure

public void shouldCreateUserStoreUserInDatastoreAndPersistAllFields() {

    // given
    User user = new User(id)
    
    // when
    user.save()
    
    // then
    assertThat(User.get(id)).isEqualTo(user)

}

KISS

• No logic in tests! Even the simplest will be evil! 
• test logic == test bugs
• No ifs, switches or loops
• Only
  • create/configure
  • act
  • assert

What to test?

• Test everything that can possibly break! 
• Write tests which make you confident that your system works 
• Don't test only happy paths
• When your code evolves, take care that your
  tests also evolve 

• Single Responsibility Principle
  "Each test method should verify just one scenario"
   
• Test only public methods
  "Separate the business scenario from low-level details"
   
• Or even better
  "Test Behaviour Not Methods!"

What to test?

• Verify expected result and state
  "Focus on results and not on solutions"
   
• Avoid multiple assertions
  "Each test should have one and only one reason to fail"

What to assert?

@Test
public void shouldAddTimeZoneToModelAndView() {
    //given
    Context context = mock(Context.class);
    ModelAndView modelAndView = mock(ModelAndView.class);
    given(context.getTimezone()).willReturn("timezone X");

    //when
    new UserDataInterceptor(context).postHandle(modelAndView);

    //then
    verify(modelAndView).addObject("timezone", "timezone X");
}

Verify expected result

The rules are there

to be broken

@Test
public void shouldRecognizeItsDistrict() {
    ..
    //when
    City city = new City(district, NUMBER_OF_PEOPLE);
    //then
    assertThat(city.isLocatedIn(district)).isTrue();
}
@Test
public void shouldRecognizeDifferentDistrict() {
    ..
    //when
    City city = new City(district, NUMBER_OF_PEOPLE);
    //then
    assertThat(city.isLocatedIn(anotherDistrict)).isFalse();
}

Assertions

• standard JUnit
• Hamcrest
• FEST assertions

AssertJ

// unique entry point to get access to all assertThat methods and utility methods
import static org.assertj.core.api.Assertions.*;

• rich and easy to use
• meaningful error messages
• customizable
• assertions for guava/joda
• support for java8

AssertJ

// common assertions
assertThat(frodo.getName()).isEqualTo("Frodo");

assertThat(frodo).isNotEqualTo(sauron)
                 .isIn(fellowshipOfTheRing);

assertThat(sauron).isNotIn(fellowshipOfTheRing);

// String specific assertions
assertThat(frodo.getName()).startsWith("Fro")
                           .endsWith("do")
                           .isEqualToIgnoringCase("frodo");

AssertJ

// collection specific assertions
assertThat(fellowshipOfTheRing).hasSize(9)
                               .contains(frodo, sam)
                               .doesNotContain(sauron);

// extract properties
assertThat(fellowshipOfTheRing).extracting("name").contains("Boromir", "Gandalf", "Frodo")
                                                  .doesNotContain("Sauron", "Elrond");

assertThat(fellowshipOfTheRing).extracting("name", "age", "race.name")
                               .contains(tuple("Boromir", 37, "Man"),
                                         tuple("Sam", 38, "Hobbit"),
                                         tuple("Legolas", 1000, "Elf"));

// map specific assertions
assertThat(ringBearers).hasSize(4)
                       .contains(entry(oneRing, frodo), entry(nenya, galadriel))
                       .containsEntry(narya, gandalf)
                       .doesNotContainEntry(oneRing, aragorn);

AssertJ

// WesterosHouse class has a method: public String sayTheWords()
List<WesterosHouse> greatHouses = new ArrayList<WesterosHouse>();
greatHouses.add(new WesterosHouse("Stark", "Winter is Comming"));
greatHouses.add(new WesterosHouse("Lannister", "Hear Me Roar!"));
greatHouses.add(new WesterosHouse("Greyjoy", "We Do Not Sow"));

// extract results of method call on iterable
assertThat(greatHouses).extractingResultOf("sayTheWords")
                       .contains("Winter is Comming", "We Do Not Sow", "Hear Me Roar")
                       .doesNotContain("Lannisters always pay their debts");

AssertJ

// filter group of object before asserting
assertThat(filter(fellowshipOfTheRing).with("race", HOBBIT).get())
          .containsOnly(sam, frodo, pippin, merry);

// nested property are supported
assertThat(filter(fellowshipOfTheRing).with("race.name").equalsTo("Man").get())
          .containsOnly(aragorn, boromir);

// you can chain multiple filter criteria
assertThat(filter(fellowshipOfTheRing).with("race").equalsTo(MAN)
                                      .and("name").notEqualsTo("Boromir").get())
                                      .contains(aragorn);

AssertJ

@Test
public void host_dinner_party_where_nobody_dies() {
   Mansion mansion = new Mansion();
   mansion.hostPotentiallyMurderousDinnerParty();
   assertThat(mansion.guests()).as("Living Guests").isEqualTo(7);
   assertThat(mansion.kitchen()).as("Kitchen").isEqualTo("clean");
   assertThat(mansion.library()).as("Library").isEqualTo("clean");
   assertThat(mansion.revolverAmmo()).as("Revolver Ammo").isEqualTo(6);
   assertThat(mansion.candlestick()).as("Candlestick").isEqualTo("pristine");
   assertThat(mansion.colonel()).as("Colonel").isEqualTo("well kempt");
   assertThat(mansion.professor()).as("Professor").isEqualTo("well kempt");
}  

AssertJ

• Using custom comparison strategy
• Using String assertions on the content of a file
• Reflection based assertions
• ... and much more

Readable assertions

@Test
public void testChargeInRetryingState() throws Exception {
    // given
    TxDTO request = createTxDTO(RequestType.CHARGE);
    AndroidTx androidTx = ...
    Processor processor = ...
    ... much more complex set-up code here

    // when
    final TxDTO txDTO = processor.processRequest(request);

    // then
    assertEquals(txDTO.getResultCode(), ResultCode.SUCCESS);
    final List<AndroidTxStep> steps = new ArrayList<AndroidTxStep>(
            androidTx.getTxSteps());
    final AndroidTxStep lastStep = steps.get(steps.size() - 1);
    assertEquals(lastStep.getTxState(),
        AndroidTxState.CHARGE_PENDING);
    assertEquals(lastStep.getMessage(), ClientMessage.SUCCESS);
    ... some more assertions here
}

Readable assertions

@Test
public void invalidTxShouldBeCanceled() {
 ... some complex test here

 // then
 String fileContent = FileUtils.getContentOfFile("response.csv");
 assertTrue(fileContent.contains("CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));
}

@Test
public void invalidTxShouldBeCanceled() {
  ... some complex test here

  // then
  String fileContent = FileUtils.getContentOfFile("response.csv");
  assertThat(fileContent).hasTransaction("123cancel").withResultCode(SUCCESS);
}

VS

Not enough tests

public class FizzBuzzTest {
    @Test
    public void testMultipleOfThreeAndFivePrintsFizzBuzz() {
        assertEquals("FizzBuzz", FizzBuzz.getResult(15));
    } 
    @Test
    public void testMultipleOfThreeOnlyPrintsFizz() {
        assertEquals("Fizz", FizzBuzz.getResult(93));
    }
    @Test
    public void testMultipleOfFiveOnlyPrintsBuzz() {
        assertEquals("Buzz", FizzBuzz.getResult(10));
    }
    @Test
    public void testInputOfEightPrintsTheNumber() {
        assertEquals("8", FizzBuzz.getResult(8));
    }
}

JUnitParams

@RunWith(JUnitParamsRunner.class)
public class FizzBuzzJUnitTest {
    @Test
    @Parameters(value = {"15", "30", "75"})
    public void testMultipleOfThreeAndFivePrintsFizzBuzz(int multipleOf3And5) {
        assertEquals("FizzBuzz", FizzBuzz.getResult(multipleOf3And5));
    }
    @Test
    @Parameters(value = {"9", "36", "81"})
    public void testMultipleOfThreeOnlyPrintsFizz(int multipleOf3) {
        assertEquals("Fizz", FizzBuzz.getResult(multipleOf3));
    }
    @Test
    @Parameters(value = {"10", "55", "100"})
    public void testMultipleOfFiveOnlyPrintsBuzz(int multipleOf5) {
        assertEquals("Buzz", FizzBuzz.getResult(multipleOf5));
    }
    @Test
    @Parameters(value = {"2", "16", "23", "47", "52", "56", "67", "68", "98"})
    public void testInputOfEightPrintsTheNumber(int expectedNumber) {
        assertEquals("" + expectedNumber, FizzBuzz.getResult(expectedNumber));
    } 
}

JUnitParams

@RunWith(JUnitParamsRunner.class)
public class TokenServiceTest {
    @Test
    @Parameters
    public void testTokenGeneration(User user, boolean expectedResult) {
        assertThat(tokenService.hasToken(user)).isEqualTo(expectedResult);
    }
    public Object[] parametersForTestTokenGeneration() {
        return $($(new User("John", FACEBOOK, "johnsFBId"), true),
                 $(new User("Jeremy", FACEBOOK, "jeremysFBId"), true),
                 $(new User("Eric", GOOGLE_PLUS, "+eric"), true),
                 $(new User("Bill", GOOGLE_PLUS, "+bill"), true),
                 $(new User("Donald"), false),
                 $(new User("Jerry"), false));
    }
}

Trust nobody

public class UserServiceTest {
    @Test
    public void shouldAddUser() {
        User user = new User();
        userService.save(user);
        assertEquals(dao.getNbOfUsers(), 1);
    }
}

Name variables

public static Object[][] userPermissions() {
    return new Object[][]{
        {"user_1", READ},
        {"user_2", READ},
        {"user_2", WRITE},
        {"user_3", READ},
        {"user_3", WRITE},
        {"user_3", DELETE}
    };
}

Name variables

@Test
public void shouldRetrieveOnlyValidTransaction() {
    // given
    Transaction transaction1 = new Transaction(5);
    Transaction transaction2 = new Transaction(-1);
    persist(transaction1, transaction2);

    // when
    List<Transaction> transactions = transactionService.getValidTransactions();

    // then
    assertThat(transactions).isNotNull().containsExactly(transaction1);
}

Expect exception anywhere

@Test(expected=IllegalArgumentException.class)
public void testNegative() {
    MyList<Integer> list = new MyList<Integer>();
    list.add(1);
    list.add(2);
    list.add(3);
    list.add(4);
    list.add(-1);
}

catch-exception

@Test
public void shouldThrowExceptionWhenTryingToAddNegativeValueToList() {
    // given
    MyList<Integer> list = new MyList<Integer>();
    list.add(1);
    list.add(2);
    list.add(3);
    list.add(4);

    // when
    catchException(list).add(-1);

    // then
    assertThat(caughtException())
            .isExactlyInstanceOf(IllegalArgumentException.class)
            .hasMessageContaining("negative value");
}

Creation of objects

@Before
public void initialize() {
    User user = new User("email@example.com", "Example", "Example", "qwerty",
        "Europe/Ljubljana", UserState.NOT_VERIFIED, new Address());
    ... 
}
@Test
public void shouldCommitTransaction() {
    User user = new User("firstName", "lastName", "password",
        "email@example.com", "qwerty", UserState.ACTIVE, new Address());
    user.setRegistrationDate(oneDayAgo.toDate());
    user.setAccessCode("Access Code");
    ...
}
@Test
public void shouldGetUserByCompanyData() {
    User user = new User("email", "FirstName", "LastName", "Password",
        "Europe/Ljubljana", UserState.ACTIVE, address);
    user.setRegistrationDate(new Date());
    user.setCompany(company);
    user.setAccessCode("Access Code");
    ...
}

MockServer server = new MockServer(responseMap, true,
        new URL(SERVER_ROOT).getPort(), false);

Creation of objects

Test Doubles

(Stubs, Fakes, Mocks, Spies ...)

• isolate the code under test
• speed up test execution
• make execution deterministic
• simulate difficult/exceptional conditions
• observe interactions that is invisible to your code

//Let's import Mockito statically so that the code looks clearer
import static org.mockito.Mockito.*;
 
//mock creation
List mockedList = mock(List.class);

//stubbing
when(mockedList.get(0)).thenReturn("first");
 
//using mock object
String firstElement = mockedList.get(0);
 
//verification
verify(mockedList).get(0);

// argument matchers
verify(mock).someMethod(anyInt(), anyString(), eq("third argument"));

// verify number of interactions
verify(mockedList, times(2)).add("twice");
verify(mockedList, never()).add("never happened");
verify(mockedList, atLeastOnce()).add("three times");
verify(mockedList, atLeast(2)).add("five times");
verify(mockedList, atMost(5)).add("three times");

// stub method to throw an exception
doThrow(new RuntimeException()).when(mockedList).clear();

// verify in order
List singleMock = mock(List.class);
InOrder inOrder = inOrder(singleMock);
inOrder.verify(singleMock).add("was added first");
inOrder.verify(singleMock).add("was added second");

// verify no interaction
verifyZeroInteractions(mockedObject);

java.lang.NullPointerException
    at PlantWaterer.generateNPE(PlantWaterer.java:24)
    at DefaultValuesTest.shouldReturnNicerErrorMessageOnNPE(DefaultValuesTest.java:64)

import static org.mockito.BDDMockito.*;

Seller seller = mock(Seller.class);
Shop shop = new Shop(seller);
 
public void shouldBuyBread() throws Exception {
  //given
  given(seller.askForBread()).willReturn(new Bread());

  //when
  Goods goods = shop.buyBread();

  //then
  assertThat(goods.getGoods()).contains(new Bread());
}

Mocks Are Good 

@Test
public void shouldGetTrafficTrend() {
    //given
    TrafficTrendProvider trafficTrendProvider
            = mock(TrafficTrendProvider.class);
    Report report = new Report(null, "", 1, 2, 3,
            BigDecimal.ONE, BigDecimal.ONE, 1);
    TrafficTrend trafficTrend = new TrafficTrend(report, report,
            new Date(), new Date(), new Date(), new Date());
    given(trafficTrendProvider.getTrafficTrend()).willReturn(trafficTrend);
    TrafficService service = new TrafficService(trafficTrendProvider);

    //when
    TrafficTrend result = service.getTrafficTrend();

    //then
    assertThat(result).isEqualTo(trafficTrend);
}

Know your tools

public class SimpleTest extends JerseyTest {

    @Path("hello")
    public static class HelloResource {
        @GET
        public String getHello() {
            return "Hello World!";
        }
    }
 
    @Override
    protected Application configure() {
        return new ResourceConfig(HelloResource.class);
    }
 
    @Test
    public void test() {
        final String hello = target("hello").request().get(String.class);
        assertEquals("Hello World!", hello);
    }
}

WireMock

private void mockJerseyClient() {
    Client client = mock(Client.class);
    WebResource webResource = mock(WebResource.class);
    WebResource.Builder builder = mock(WebResource.Builder.class);
 
    ClientResponse clientResponse = mock(ClientResponse.class);
    when(builder.get(ClientResponse.class)).thenReturn(clientResponse);
    when(clientResponse.getEntity(String.class)).thenReturn("true");
    when(webResource.accept(anyString())).thenReturn(builder);
    when(client.resource(anyString())).thenReturn(webResource);
}

@Test
public void testWithAtomicPseudoClosure() {
    with().pollInterval(ONE_HUNDRED_MILLISECONDS)
        .and().with().pollDelay(20, MILLISECONDS)
        .await()
        .atMost(10, SECONDS)
        .untilCall(to(userRepository).size(), equalTo(3));
}

private AtomicInteger atomic = new AtomicInteger(0);

@Test
public void testWithAtomicNumber() {
    await().untilAtomic(atomic, equalTo(1));
}

Awaitility

Doing things the right way takes more time than doing them any old way.

But this extra effort usually pays off in the long term. 

Things to Remember

Hard to write a test?

Maybe the production code is of low quality?

...or maybe you should consider writing tests before production code?

Things to Remember

Things to Remember

Be pragmatic, and let experience be your guide.

It doesn’t matter if someone advises you to employ some technique or other.

If it doesn’t work for you, simply don’t do it!    

References

• http://joel-costigliola.github.io/assertj/
• http://code.google.com/p/catch-exception/ 
• https://code.google.com/p/junitparams/ 
• http://mockito.org
• https://code.google.com/p/awaitility/
• http://wiremock.org

• Lasse Koskela "Effective Unit Testing"
• Tomasz Kaczanowski "Practical Unit Testing"

• Tomasz Kaczanowski "Bad Tests, Good Tests"