Don't Trust, Test!

Android Testing





@rolios

o.gonthier@gmail.com

About Me


Gonthier Olivier

Freelance Developer/Trainer in Paris

Contributor infoq FR 





Work at Orange Vallee on libon

Testing


What we will talk about

Why Tests?


Testing is not easy, it requires time

BUT

You need stability in your apps

AND
tests are there to detect regressions

SO

You need to practice it.

Why Tests?



The red guy write tests and try to maintain them.

The black guys will do everything to break his tests.

Unit vs Integration Testing


  • Unit Testing

Test an isolated component: A class, or a method.


  • Integration Testing

Test a component in its environment: test how it interacts for real.

Problems



  • What to use?
  • What to test?
  • How to be efficient?
  • When should I run Tests?

Android Test Framework


Start with basics.

Android Test Framework



Part of the SDK
Based on jUnit3 (...)
Instrument Android Components

How it works


How to use it


Create a test project

android create test-project -m <path-app> -n <project-name> -p <test-path>

Run tests

adb shell am instrument -w <package>/<runner-class>
or, with maven
mvn android:instrument

Android Test Framework


Let's analyse the classes provided!

TestsCases


There are two families of Test Cases


  • InstrumentationTestCase
TestCases for test Activities and Account Sync

Provides Context, and Instrumentation


  • AndroidTestCase
TestCases for test Providers, Services, Loaders and Applications
Provides Context, and permissions asserts

TestCases

 
 
 
 

TestCases

InstrumentationTestCases

  • SyncBaseInstrumentation - Start/cancel Sync
syncProvider(Uri uri, String accountName, String authority)
cancelSyncsandDisableAutoSync()

  • ActivityUnitTestCase - Isolated test
setApplication(Application application)
startActivity(Intent intent, Bundle savedInstanceStateObject lastNonConfigurationInstance)
setActivityContext(Context activityContext)
Inner Mock Activity: class MockParent extends Activity
  • ActivityInstrumentationTestCase2 - Functional test
getActivity()
setActivityIntent(Intent i)

TestCases

InstrumentationTestCases

  • SingleLaunchActivity
Functional Test of an activity, but launched one time for all tests methods in the TestCase.

(ActivityUnitTestCase and ActivityInstrumentationTestCase launch activity each time setUp() is called)

TestCases

AndroidTestCases

  • ApplicationTestCase - Launch and terminate Application
createApplication()
terminateApplication()
getSystemContext()

  • LoaderTestCase - get data synchronously from Loader
getLoaderResultSynchronously(final Loader<T> loader)

  • ServiceTestCase - Start, bind, stop services
startService(Intent intent)
bindService(Intent intent)
shutdownService()

TestCases

AndroidTestCases

  • ProviderTestCases2 
Test Isolated Provider, using a MockContext, and a MockContentResolver.
Data is not saved at the same place as usual!


It can also be used from other TestCases

newResolverWithContentProviderFromSql(Context targetContext, String filenamePrefix, Class<T> providerClass, String authority, String databaseName, int databaseVersion, String sql)

Mocks

We can find various mocks
  • MockApplication
  • MockContentProvider
  • MockContentResolver
  • MockContext
  • MockCursor
  • MockDialogInterface
  • MockPackageManager
  • MockResources
However, most of them are not really useful: they are only classes skeleton, that throw UnsupportedOperationException...
...Excepted MockContentResolver

Mocks

MockContentResolver


A ContentResolver that allows you to plug yourself your providers!

addProvider(String name, ContentProvider provider)
notifyChange(Uri uri, ContentObserver observer, boolean syncToNetwork)

In case of ContentProviderTestCase2, only one provider is plugged.

Mocks


There are two classes that takes benefits of ContextWrapper to mock Contexts


    A contextWrapper is a Context, that delegate all method execution to another Context. Subclass it for... Mock Context!

Mocks


  • IsolatedContext

Prevent tests from talking to the device:
  • checkUriPermission() return permission Granted
  • getFilesDir() return File("/dev/null")
  • getSystemService() return null

  • The class keep a list of broadcast Intents we tried to send.

    And it provides also a MockAccountManager!

    getAndClearBroadcastIntents()

    Mocks


    • RenamingDelegatingContext

    Performs Database and File operations with a renamed database/filename!


    providerWithRenamedContext(Class<T> contentProvider, Context c, String filePrefix)
    makeExistingFilesAndDbsAccessible()

    Asserts

    ViewAsserts


    Assert View and ViewGroup Objects

    Test:

    • Presence of a view in screen
    • Location of a view in screen
    • Alignement
    • ViewGroup contains view or not
    assertGroupContains(ViewGroup parent, View child)
    assertLeftAligned(View first, View second)
    assertHasScreenCoordinates(View origin, View view, int x, int y)
    assertOnScreen(View origin, View view)

    Asserts

    MoreAsserts

    Assert Java objects

    Test:

    • Inheritance
    • Equality
    • Regex validation
    • Order of Collection content
    • Collection content
    assertEmpty(Map<?,?> map)
    assertContentsInOrder(Iterable<?> actual, Object... expected)
    assertMatchesRegex(String expectedRegex, String actual)
    assertEquals(int[] expected, int[] actual)
    assertAssignableFrom(Class<?> expected, Object actual)

    Utils

    TouchUtils


    Set of static methods to generate Touch Events

    • Apply in InstrumentationTestCases
    • Manipulate views
    • Different types: drag, tap, long-click, scroll
    • Locate a view
    scrollToTop(ActivityInstrumentationTestCase test, ViewGroup v)
    dragViewToBottom(ActivityInstrumentationTestCase test, View v)
    tapView(InstrumentationTestCase test, View v)
    longClickView(ActivityInstrumentationTestCase test, View v)
    getStartLocation(View v, int gravity, int[] xy)

    Annotations


    • Some allow us to classify tests

    @SmallTest, @MediumTest, @LargeTest, @Smoke
    • One for suppress a test
    @Suppress
    • One for define tolerance
     @FlakyTest(tolerance = 2)
    • And one that force test to execute on UI Thread
     @UiThreadTest

    Annotations


    In sources, there are other classes, not part of sdk for now.

    Some of these looks promising!

     @TimedTest, @BandwidthTest, @RepetitiveTest

    Android Test Framework


    Now we can use it!

    TestCases in Action

     public class TestCarProvider extends ProviderTestCase2<CarProvider>{    
        public CarProviderTest() {
            super(CarProvider.class, "com.roly.carseller.carprovider");
        }
    
        public void testInsert() {
            //Given
            CarProvider provider = getProvider();
            Car car = new Car("BMW");
            provider.insert(car);        //When
            Car fromDB = provider.query(car.getId());
    //Then assertEquals(car.getBrand(),fromDB.getBrand()); }}
    ¡This is not real code!

    TestCases in Action

    public class TestServerResponseParsing extends AndroidTestCase{    ServerResponseParser parser;   @Override
       protected void setUp() throws Exception {
          super.setUp();
          parser = ((ClientApplication) getContext().getApplicationContext()).getParser();
          parser.prepare();
       }
       public void testErrorResponse(){
          ServerResult result = parser.parse(JSON_ERROR_EXAMPLE);
          assertEquals(result.getCode(), ServerResult.ERROR);
       }
       @Override
       protected void tearDown() throws Exception {
          super.tearDown();
          parser.clearAllTasks();
       }

    TestCases in Action

    public class MainActivityTest extends ActivityInstrumentationTestCase2<MainActivity>{
    
       public MainActivityTest() {
          super(MainActivity.class);
       }
    
       public void testIntentValueUpdateTextView() {
          Intent intent = new Intent();
          intent.putExtra("userId",45);
          setActivityIntent(intent);
          getInstrumentation().callActivityOnCreate(getActivity(), null);
          TextView view = (TextView) getActivity().findViewById(R.id.userIdText);
          assertNotNull(view.getText());
       }
    }

    Classify



    @SmallTest
    public void testActivityGetValueFromIntent()
    
    @MediumTest public class TestServer extends ServiceTestCase<RestService>
    @LargeTest public void testSyncAllContacts()
     adb shell am instrsument -w -e size small ... mvn android:instrument -Dandroid.test.testSize=small  ...

    Organize

    Make your own TestSuite

    • In Manifest
     <instrumentation android:name="com.roly.mycoolapp.test.CoolAppInstrumentation" android:targetPackage="com.roly.mycoolapp"/>

    • In a subclass of InstrumentationTestRunner
    public class CoolAppInstumentation extends InstrumentationTestRunner{
       @Override
       public TestSuite getAllTests() {
          TestSuite testSuite = new TestSuite();
          testSuite.addTest(new TestSuiteBuilder(CoolAppInstrumentation.class)
          .includePackages("com.roly.mycoolapp.test.sync")
          .build());
          return testSuite;
       }
    }
    

    Organize


    So you want to make a TestSuite based on Test size?
    You have to define "predicates".
    Predicate<TestMethod> smallTests = new Predicate<TestMethod>(){
       @Override
       public boolean apply(TestMethod testMethod) {
          return testMethod.getAnnotation(SmallTest.class) != null;   }}
    
    new TestSuiteBuilder(MyInstrumentation.class).include(...)
       .addRequirements(smallTests)
       .build();
    
    ...

    Simplify


    class MyTest extends AndroidTestCase {
       @Override @Suppress
       public void testAndroidTestCaseSetupProperly()
    
    ...
    class MyServiceTest extends ServiceTestCase { @Override @Suppress public void testServiceTestCaseSetUpProperly()
    ...

    Keep only what you care about

    Take control


    Where can we introduce Mocks?

    Contexts are great objects to convey Mocks
    In all AndroidTestCases:
    public void testMeImFamous(){
       setContext(myMockContext); ...
    }
    In ContentProvider constructor: public ContentProvider(Context context, String readPermission, String writePermission, PathPermission[] pathPermissions) And everywhere you pass a Context.

    Take Control

    And then subclass your Context to deliver other Mocks

    mResolver = new MockContentResolver();
    MockContext mockContext = new MockContext() {
       @Override
       public ContentResolver getContentResolver() {
          return mResolver;
       }
    };mProvider = new MyProvider();
    mProvider.attachInfo(mockContext, null);
    mResolver.addProvider("com.roly.myapp.coolnessprovider"mProvider);

    Isolate

    Try to isolate as much as possible your tests

    Use RenamingDelegatingContext, and IsolatedContext

    mContextWrapper = new RenamingDelegatingContext(getContext(), "test-");
    mContext = new IsolatedContext(mResolver, mContextWrapper);
    


    It will prevent you from:

    • Erase file
    • Modify database
    • Add crap account on your phone

    Tips for Testing


    You'll be stronger

    Async becomes Sync

    When you write tests, you don't want to run async code!

    You can force you test to wait if you get any callback or event

    Use Java's CountDownLatch for that.

    final CountDownLatch latch = new CountDownLatch(1);
    final BroadcastReceiver syncReceiver = new BroadcastReceiver() {
       @Override
       public void onReceive(Context context, Intent intent) {
          latch.countDown();
       }
    }
    try {
       latch.await(60, TimeUnit.SECONDS);
    } catch (InterruptedException e) {
       fail();
    }

    Reflection

    In tests parts, you can enjoy Reflection API
    It can facilitate your life!

    //Get private field
    Field privateField = myObject.getClass.getDeclaredField("mParser");
    //Set it accessible
    privateField.setAccessible(true);
    //Inject a value
    privateField.set(myObject, new Parser());
    //Then invoke a method
    parseMethod.invoke(myObject, "stuff to parse");

    Builders

    Make builders so that you can construct Objects easily

    public class ContactBuilder {
       private String firstName;
    
       public ContactBuilder firstName(String firstName) {
          this.firstName = firstName;
          return this;
       }
    
       public Contact build() {
          Contact contact = new Contact();
          contact.setFirstname(firstname);
          return contact;   }
    }
    
    new ContactBuilder().firstName("tata").build(); 
    new ContactBuilder().firstName("toto").build();

    Random

    Randomize your test data is a great thing. (imho)


    BUT Don't do it yourself

    Suggestions

    • RandomStringUtils (Apache commons lang)
    • Random (Java.utils)
    • DataFactory 
      • Great library that generate random consistent data
      • Example: getName() can return "Peter"

    Make Utils


    Make high level static methods that wrap code reusable.

    createUser()
    login()
    deleteUser()
    forceDeleteUser()


    Facilitate test writing to other developers is important.

    Initialize/Revert

    The Framework uses jUnit3, so @BeforeClass  and  @AfterClass  don't exist.

    There are (crap) solutions to get around this
    • Use constructor
    • Use static block
    • boolean set to true when first setUp
    • Count visited tests

    Note that the class TestSetup from jUnit3 is not part of the framework

    Libraries


    As usual, many community projects!

    Robotium


    • Most famous library for testing
    • Extension of the base framework
    • Facilitate a lot manipulation of UI
    public void testDisplayBlackBox() {
    //Enter 10 in first edit-field solo.enterText(0,"10"); //Enter 20 in second edit-field solo.enterText(1,"20"); //Click on Multiply button solo.clickOnButton("Multiply"); //Verify that resultant of 10 x 20 assertTrue(solo.searchText("200")); }

    Robolectric

    It allows to run tests on an usual jvm
    It uses jUnit4!
    No need to deploy on device = So much time saved!
    @RunWith(HttpTest.RobolectricTestRunner.class)
    public class HttpTest {
       @Before
       public void setup() {
          Robolectric.setDefaultHttpResponse(200, "OK");
       }
       @Test 
       public void testGet_shouldApplyCorrectHeaders(){
          HashMap<String, String> headers=new HashMap<String, String>();
          headers.put("foo", "bar");
          http.get("www.example.com", headers, null, null);
          HttpRequest sent = Robolectric.getSentHttpRequest(0);
          assertThat(sent.getHeaders("foo")[0].getValue(), equalTo("bar"));
       }
    }

    FestAndroid


    FEST Assertions for Android

    assertEquals(View.GONE, view.getVisibility());
    assertThat(view).isGone();
    assertThat(layout).isVisible()
    .isVertical()
    .hasChildCount(4)
    .hasShowDividers(SHOW_DIVIDERS_MIDDLE);

    And it's also extensible, you can had your own assertions

    Mockito


    A powerful Mock Framework

    • mock(Class) create a mock instance of your class
    • verify(myMock).someMethod() verify that a method has been called
    • when(myMock.getUser()).thenReturn(new User()) specify what to return when a method is called
    Context context = Mockito.mock(Context.class);
    Mockito.when(context.getContentResolver())
           .thenReturn(Mockito.mock(ContentResolver.class));
    
    //Do something
    Mockito.verify(context).getContentResolver();
    

    Spoon

    Great tool to automate test on multi-devices

    Generate a cool web interface to get tests result


    Industrialization




    Automate all these tests!

    Coverage



    For test Coverage, Framework suggest to use Emma


    adb shell am instrsument -w -e emma true ...
    mvn android:instrument -Dandroid.test.testCoverage=true  ...

    Continuous Integration


    On Jenkins, you can find an android emulator plugin.
    It provides all you need to automate tests.


    Additionnally, if you want to get the state 'unstable', you need to add a property with maven.

    mvn android:instrument -Dmaven.test.failure.ignore=true  ...

    Further more

    There are so much things that we haven't talked about!

    • Selenium
    • Mock environment
    • Monkey
    • Cucumber?
    • ...

    End

    -

    Thanks a lot!

    Any Questions?



    @rolios

    o.gonthier@gmail.com

    Android Testing

    By Gonthier Olivier

    Android Testing

    Do not Trust, Test!

    • 9,743