Testing

What we will talk about

Why Tests?

Testing is not easy, it requires time

Why Tests?

Unit vs Integration Testing

• Unit Testing

Test an isolated component: A class, or a method.

• Integration Testing

Test a component in its environment: test how it interacts for real.

Problems

• What to use?
• What to test?
• How to be efficient?
• When should I run Tests?

Android Test Framework

Start with basics.

Android Test Framework

How it works

How to use it

Create a test project

android create test-project -m <path-app> -n <project-name> -p <test-path>

Run tests

adb shell am instrument -w <package>/<runner-class>
or, with maven
mvn android:instrument

Android Test Framework

Let's analyse the classes provided!

TestsCases

There are two families of Test Cases

• InstrumentationTestCase

TestCases for test Activities and Account Sync

Provides Context, and Instrumentation

• AndroidTestCase

TestCases for test Providers, Services, Loaders and Applications

TestCases

 

 

 

TestCases

InstrumentationTestCases

syncProvider(Uri uri, String accountName, String authority)


cancelSyncsandDisableAutoSync()

• ActivityUnitTestCase - Isolated test

setApplication(Application application)
startActivity(Intent intent, Bundle savedInstanceState, Object lastNonConfigurationInstance)
setActivityContext(Context activityContext)
Inner Mock Activity: class MockParent extends Activity

• ActivityInstrumentationTestCase2 - Functional test

getActivity()
setActivityIntent(Intent i)

TestCases

InstrumentationTestCases

Functional Test of an activity, but launched one time for all tests methods in the TestCase.

(ActivityUnitTestCase and ActivityInstrumentationTestCase launch activity each time setUp() is called)

TestCases

AndroidTestCases

• ApplicationTestCase - Launch and terminate Application

createApplication()
terminateApplication()
getSystemContext()

• LoaderTestCase - get data synchronously from Loader

getLoaderResultSynchronously(final Loader<T> loader)

• ServiceTestCase - Start, bind, stop services

startService(Intent intent)
bindService(Intent intent)
shutdownService()

TestCases

AndroidTestCases

• ProviderTestCases2 

Test Isolated Provider, using a MockContext, and a MockContentResolver.

Data is not saved at the same place as usual!

It can also be used from other TestCases

newResolverWithContentProviderFromSql(Context targetContext, String filenamePrefix, Class<T> providerClass, String authority, String databaseName, int databaseVersion, String sql)

Mocks

However, most of them are not really useful: they are only classes skeleton, that throw UnsupportedOperationException...


...Excepted MockContentResolver

Mocks

MockContentResolver

A ContentResolver that allows you to plug yourself your providers!

addProvider(String name, ContentProvider provider)
notifyChange(Uri uri, ContentObserver observer, boolean syncToNetwork)

In case of ContentProviderTestCase2, only one provider is plugged.

Mocks

There are two classes that takes benefits of ContextWrapper to mock Contexts

    A contextWrapper is a Context, that delegate all method execution to another Context. Subclass it for... Mock Context!

Mocks

Prevent tests from talking to the device:

checkUriPermission() return permission Granted
getFilesDir() return File("/dev/null")
getSystemService() return null

getAndClearBroadcastIntents()

Mocks

Performs Database and File operations with a renamed database/filename!

providerWithRenamedContext(Class<T> contentProvider, Context c, String filePrefix)
makeExistingFilesAndDbsAccessible()

Asserts

ViewAsserts

Assert View and ViewGroup Objects

Test:

• Presence of a view in screen
• Location of a view in screen
• Alignement
• ViewGroup contains view or not

assertGroupContains(ViewGroup parent, View child)
assertLeftAligned(View first, View second)
assertHasScreenCoordinates(View origin, View view, int x, int y)
assertOnScreen(View origin, View view)

Asserts

MoreAsserts

Assert Java objects

Test:

• Inheritance
• Equality
• Regex validation
• Order of Collection content
• Collection content

assertEmpty(Map<?,?> map)
assertContentsInOrder(Iterable<?> actual, Object... expected)
assertMatchesRegex(String expectedRegex, String actual)
assertEquals(int[] expected, int[] actual)
assertAssignableFrom(Class<?> expected, Object actual)

Utils

TouchUtils

Set of static methods to generate Touch Events

• Apply in InstrumentationTestCases
• Manipulate views
• Different types: drag, tap, long-click, scroll
• Locate a view

scrollToTop(ActivityInstrumentationTestCase test, ViewGroup v)
dragViewToBottom(ActivityInstrumentationTestCase test, View v)
tapView(InstrumentationTestCase test, View v)
longClickView(ActivityInstrumentationTestCase test, View v)
getStartLocation(View v, int gravity, int[] xy)

Annotations

@SmallTest, @MediumTest, @LargeTest, @Smoke

• One for suppress a test

@Suppress

• One for define tolerance

 @FlakyTest(tolerance = 2)

• And one that force test to execute on UI Thread

 @UiThreadTest

Annotations

In sources, there are other classes, not part of sdk for now.

Some of these looks promising!

 @TimedTest, @BandwidthTest, @RepetitiveTest

Android Test Framework

Now we can use it!

TestCases in Action

 public class TestCarProvider extends ProviderTestCase2<CarProvider>{    
    public CarProviderTest() {
        super(CarProvider.class, "com.roly.carseller.carprovider");
    }

    public void testInsert() {
        //Given
        CarProvider provider = getProvider();
        Car car = new Car("BMW");
        provider.insert(car);        //When
        Car fromDB = provider.query(car.getId());
        //Then
        assertEquals(car.getBrand(),fromDB.getBrand());   }}

¡This is not real code!

TestCases in Action

public class TestServerResponseParsing extends AndroidTestCase{    ServerResponseParser parser;   @Override
   protected void setUp() throws Exception {
      super.setUp();
      parser = ((ClientApplication) getContext().getApplicationContext()).getParser();
      parser.prepare();
   }
   public void testErrorResponse(){
      ServerResult result = parser.parse(JSON_ERROR_EXAMPLE);
      assertEquals(result.getCode(), ServerResult.ERROR);
   }
   @Override
   protected void tearDown() throws Exception {
      super.tearDown();
      parser.clearAllTasks();
   }

TestCases in Action

public class MainActivityTest extends ActivityInstrumentationTestCase2<MainActivity>{

   public MainActivityTest() {
      super(MainActivity.class);
   }

   public void testIntentValueUpdateTextView() {
      Intent intent = new Intent();
      intent.putExtra("userId",45);
      setActivityIntent(intent);
      getInstrumentation().callActivityOnCreate(getActivity(), null);
      TextView view = (TextView) getActivity().findViewById(R.id.userIdText);
      assertNotNull(view.getText());
   }
}

Classify

@SmallTest
public void testActivityGetValueFromIntent()

@MediumTest
public class TestServer extends ServiceTestCase<RestService>
@LargeTest
public void testSyncAllContacts()

 adb shell am instrsument -w -e size small ... mvn android:instrument -Dandroid.test.testSize=small  ...

Organize

Make your own TestSuite

• In Manifest
  

 <instrumentation android:name="com.roly.mycoolapp.test.CoolAppInstrumentation" android:targetPackage="com.roly.mycoolapp"/>

• In a subclass of InstrumentationTestRunner

public class CoolAppInstumentation extends InstrumentationTestRunner{
   @Override
   public TestSuite getAllTests() {
      TestSuite testSuite = new TestSuite();
      testSuite.addTest(new TestSuiteBuilder(CoolAppInstrumentation.class)
      .includePackages("com.roly.mycoolapp.test.sync")
      .build());
      return testSuite;
   }
}

Organize

Predicate<TestMethod> smallTests = new Predicate<TestMethod>(){
   @Override
   public boolean apply(TestMethod testMethod) {
      return testMethod.getAnnotation(SmallTest.class) != null;   }}

new TestSuiteBuilder(MyInstrumentation.class).include(...)
   .addRequirements(smallTests)
   .build();

...

Simplify

class MyTest extends AndroidTestCase {
   @Override @Suppress
   public void testAndroidTestCaseSetupProperly()

...
class MyServiceTest extends ServiceTestCase {

   @Override @Suppress
   public void testServiceTestCaseSetUpProperly()

...

Keep only what you care about

Take control

Contexts are great objects to convey Mocks

In all AndroidTestCases:
public void testMeImFamous(){
   setContext(myMockContext); ...
}
In ContentProvider constructor:
public ContentProvider(Context context, String readPermission, String writePermission, PathPermission[] pathPermissions)

And everywhere you pass a Context.

Take Control

mResolver = new MockContentResolver();
MockContext mockContext = new MockContext() {
   @Override
   public ContentResolver getContentResolver() {
      return mResolver;
   }
};mProvider = new MyProvider();
mProvider.attachInfo(mockContext, null);
mResolver.addProvider("com.roly.myapp.coolnessprovider", mProvider);

Isolate

Try to isolate as much as possible your tests

Use RenamingDelegatingContext, and IsolatedContext

mContextWrapper = new RenamingDelegatingContext(getContext(), "test-");
mContext = new IsolatedContext(mResolver, mContextWrapper);

It will prevent you from:

• Erase file
• Modify database
• Add crap account on your phone

Tips for Testing

You'll be stronger

Async becomes Sync

When you write tests, you don't want to run async code!

You can force you test to wait if you get any callback or event

Use Java's CountDownLatch for that.

final CountDownLatch latch = new CountDownLatch(1);
final BroadcastReceiver syncReceiver = new BroadcastReceiver() {
   @Override
   public void onReceive(Context context, Intent intent) {
      latch.countDown();
   }
}
try {
   latch.await(60, TimeUnit.SECONDS);
} catch (InterruptedException e) {
   fail();
}

Reflection

In tests parts, you can enjoy Reflection API
It can facilitate your life!

//Get private field
Field privateField = myObject.getClass.getDeclaredField("mParser");
//Set it accessible
privateField.setAccessible(true);
//Inject a value
privateField.set(myObject, new Parser());
//Then invoke a method
parseMethod.invoke(myObject, "stuff to parse");

Builders

Make builders so that you can construct Objects easily

public class ContactBuilder {
   private String firstName;

   public ContactBuilder firstName(String firstName) {
      this.firstName = firstName;
      return this;
   }

   public Contact build() {
      Contact contact = new Contact();
      contact.setFirstname(firstname);
      return contact;   }
}

new ContactBuilder().firstName("tata").build(); 
new ContactBuilder().firstName("toto").build();

Random

Randomize your test data is a great thing. (imho)

BUT Don't do it yourself

Suggestions

• RandomStringUtils (Apache commons lang)
  
• Random (Java.utils)
• DataFactory 
• Great library that generate random consistent data
• Example: getName() can return "Peter"

Make Utils

Make high level static methods that wrap code reusable.

createUser()
login()
deleteUser()
forceDeleteUser()

Facilitate test writing to other developers is important.

Initialize/Revert

Note that the class TestSetup from jUnit3 is not part of the framework

Libraries

Robotium

public void testDisplayBlackBox() {
   //Enter 10 in first edit-field
   solo.enterText(0,"10");
   //Enter 20 in second edit-field
   solo.enterText(1,"20");
   //Click on Multiply button
   solo.clickOnButton("Multiply");
   //Verify that resultant of 10 x 20
   assertTrue(solo.searchText("200"));
}

Robolectric

No need to deploy on device = So much time saved!

@RunWith(HttpTest.RobolectricTestRunner.class)
public class HttpTest {
   @Before
   public void setup() {
      Robolectric.setDefaultHttpResponse(200, "OK");
   }
   @Test 
   public void testGet_shouldApplyCorrectHeaders(){
      HashMap<String, String> headers=new HashMap<String, String>();
      headers.put("foo", "bar");
      http.get("www.example.com", headers, null, null);
      HttpRequest sent = Robolectric.getSentHttpRequest(0);
      assertThat(sent.getHeaders("foo")[0].getValue(), equalTo("bar"));
   }
}

FestAndroid

FEST Assertions for Android

assertEquals(View.GONE, view.getVisibility());
⇩
assertThat(view).isGone();

assertThat(layout).isVisible()

.isVertical()

.hasChildCount(4)

.hasShowDividers(SHOW_DIVIDERS_MIDDLE);

And it's also extensible, you can had your own assertions

Mockito

A powerful Mock Framework

mock(Class) create a mock instance of your class
verify(myMock).someMethod() verify that a method has been called
when(myMock.getUser()).thenReturn(new User()) specify what to return when a method is called

Context context = Mockito.mock(Context.class);
Mockito.when(context.getContentResolver())
       .thenReturn(Mockito.mock(ContentResolver.class));

//Do something
Mockito.verify(context).getContentResolver();

Spoon

Great tool to automate test on multi-devices

Generate a cool web interface to get tests result

Industrialization

Automate all these tests!

Coverage

For test Coverage, Framework suggest to use Emma

adb shell am instrsument -w -e emma true ...
mvn android:instrument -Dandroid.test.testCoverage=true  ...

Continuous Integration

On Jenkins, you can find an android emulator plugin.
It provides all you need to automate tests.

Additionnally, if you want to get the state 'unstable', you need to add a property with maven.

mvn android:instrument -Dmaven.test.failure.ignore=true  ...