Android on mobile phone

An Android developer's guide to mobile testing and tools

The testing experience in Android has never been easy. Mobile testing by itself is a challenge, but Android has been particularly difficult in the past because of poor tooling options and sparse documentation. Now that's changing.

As an experienced Android developer, I want to give fellow coders a clearer picture of the testing space and how it applies to the Android ecosystem. Today there's a strong Android toolset to perform time-tested techniques in software testing. Whether you're a beginner or a veteran, there's some useful information here for you.

Here are the basic concepts of app testing and why the anti-testers out there are often making a bad decision when they don't take testing seriously. Also, the following tools are key for setting up a solid testing process for Android applications.

Mobile Testing: Build your mobile QA skills

Testing—why bother?

Before you start learning about any topic, you should rightly ask, "Why should I do this?" For a long time in my initial development career, I was not certain about why I should spend time in writing automated tests rather than doing the tests manually. 

I wasn't uninformed. I knew the so-called theoretical reasons, but I was not clear on the actual implications of writing or not writing tests.

Building confidence in your code

There were several instances where I developed a particular feature and shared the build with the QA (quality assurance) team only to find that the build failed in the first round of testing.

This would never have happened if I had decided to write tests while developing the app—a time-tested technique known as test-driven development (TDD). Identifying success and failure cases and then writing tests for those cases will ensure that you have confidence in the code you write and deliver.

Getting rid of regression bugs

Suppose you have been working hard on a new feature and have released it to production, but within a day of release, you find out that some old feature has broken. You are absolutely clueless about what went wrong, and you're likely to waste a lot of time identifying these "regression" bugs and fixing them.

This situation would never have happened if you had written tests for your old features. With those tests, you could work on your new feature while running tests on that old feature to make sure that you didn't break it in the process. If you did write some code that broke the old feature, you would instantly catch the bug the next time you ran your regression test suite.

Writing minimal production code

When you write tests for your code, you will eventually end up writing a lot less production code than you would have written without tests. Now your main goal is only to write code to make a particular test pass.

Suppose you are following TDD and you have written a test. Now your main target will be to write only the required code to make that particular test pass—no extra or spurious code.

In the long run, you will save a lot of time, while releasing robust features faster than before.

Legacy code is any code that does not have tests.

The testing pyramid

Now let’s take some time to understand the testing pyramid. This will essentially give you a very clear idea of the various types of tests you need to write and how they differ from one another.

 

Unit tests

Unit tests will be the biggest chunk of tests you write in your app—around 70% of your total tests. They are small, highly focused tests that can run right on your local development machine, and since they run directly on your local environment, they are very fast.

You should write these types of tests for small units of business and application logic. Each test should be small, targeted, and only test one particular scenario, nothing more. You are likely to have hundreds (or even thousands) of these tests, and you'll run them very often while developing your app.

The only downsides of these tests are that they don’t run on a real device and their fidelity for real-world situations with real data can be low. You will be using various mocking techniques to emulate various states and behavior, but you might not always be sure that they will work on an actual device.

Integration tests

You should try to run these tests on a real device and not a local machine. They are broadly meant to test that the integration between different modules or components is working. Generally, these tests are medium-sized and should comprise around 20% of the total tests written for your app.

In Android, a simple example of an integration test would be to check to see whether clicking on a button for a particular Activity takes you to a different Activity. Because these tests run on a real device or emulator, they tend to be much slower than unit tests, and you would typically want to run them a lot less often.

End-to-end tests

You are going to write only a few of these tests for your app. Around 10% of your total tests should be end-to-end tests. As the name suggests, you will use these kinds of tests to check a user flow completely, from beginning to end.

A multistep user sign-up flow is an example use case where you'd want to write an end-to-end test. As with integration tests, these tests should run on a real device or emulator.  That means they'll be slower, so you'll want to run them less frequently.

The fidelity of these tests is much higher because they simulate a real-world use case on real hardware (if you're using a real device for the test). But one caveat is that it can be difficult to debug what went wrong if the test fails, since the test has to check so many moving parts.

Testing types in Android

Now let’s try to understand the types of tests you would generally write on Android. They're not very different from what was discussed in the last section, but in Android, we tend to classify tests on the basis of whether they are running on a real device or an emulator.

Local unit tests

You can run these tests locally on your development machine because there's not a huge difference between doing that and testing on mobile devices (since these tests check functionality, not performance). These tests reside under the path module/src/test/java and do not have access to any Android framework APIs.

Suppose you have a method named validateEmailAddress(String) and you want to get this method’s functionality tested. You would want to write very small and focused tests for each scenario—for example, when an email passed to this method is null, empty, or not valid.

For unit testing in Java, the most common tool is JUnit. You can also use Robolectric, another popular testing framework that allows you to unit test Android dependencies.

Instrumented tests

These tests belong to the integration and end-to-end categories that I previously discussed. They can be found in the module/src/androidTest/java path.

Notice the addition of the “android” prefix before “Test” in the path name, which signifies that these tests should run on an Android device or an emulator and that they will have access to Android dependencies. These are known as “source sets,” in case you've heard that term and didn't know what it meant.

Instrumented tests are fairly slow to execute because a new test APK is created every time you test. The APK runs along with your original app under test. Both these apps run on the same process, which enables the test app to access methods and fields on the actual app and automate user interactions as well.

You can use instrumented tests to automate clicking on any button, typing some text, swiping some views, scrolling through lists, or performing various actions in your app that only an actual user can perform manually.

You might wonder why these are called “instrumentation” tests. It's because they use the Android Instrumentation API via InstrumentationRegistry to control Android components and their lifecycle manually instead of letting the system control them.

The most popular framework my team uses to write instrumentation tests is Espresso. But you can also use UIAutomator to write tests that span multiple apps at the same time, which Espresso is not capable of.

Busting the myth

Many developers are driven by the myth that testing is highly time-consuming, can increase your development time, and can significantly delay the release of new features. Sounds scary, but none of that is true.

Testing might take extra time initially, but in the long run it will give you a lot back. When I first started writing tests, I was slow simply because I didn’t know the right tools to use or how to use them properly. I was spending lots of time figuring out how to write tests rather than actually writing them.

But after a while, things started falling into place and I was writing tests much faster. I followed the TDD approach, writing tests before writing any code. That helped me write concise, laser-targeted code (which is more maintainable)—just enough to pass the tests and not waste any time writing extra code.

The tests I wrote helped me catch bugs as I was developing the app.

The sooner you catch the bugs, the easier and less time-consuming it is to fix them.

What’s in your testing toolbox?

Assuming I've convinced you that testing is important to the success and sanity of your development process, you now need to know the different tools that are available to you to make testing possible.

It's important to be familiar with the tools and frameworks available to you, understand when you should use them, and, most importantly, know how you can use them. You will find testing difficult only if you don’t know about all the tools available or if you don't don't know how to use them.

I recommend you spend a good amount of time learning about and playing with each tool so that you know them front to back. As you get more familiar with these tools, you'll be able to write better tests faster.

JUnit

JUnit, the popular unit-testing framework for Java, provides simple and convenient APIs to write tests and perform common testing operations such as setup, teardown, and assertions with ease.

This is the first testing tool you should learn for Android. Whatever tests you're writing (unit tests, integration tests, etc.), you can use the JUnit framework as the base. You should spend some time learning about the testing lifecycle and the various annotations, such as @Test, @Before, @After, and @Rule.

You should always remember that each JUnit test is isolated and should not be dependent on other tests in any way. You should also disregard the order in which these tests run. That's not really important.

Tip: Here is an excellent tutorial to help you get started with JUnit.

Mockito

The core idea of unit tests is to test a unit of code in isolation from any of its dependencies. When you are unit testing a piece of code, you really don’t care if its dependencies work properly or not, since you will need to test them separately anyway.

But sometimes you can't test code without data from another part of the application. Therefore, to make unit testing possible, you need to "mock" out the dependencies. This is where Mockito comes into play. This amazing framework provides a very simple and fluent API to help you create mock objects with ease. You can stub the state of a particular method or create mock objects to verify interactions. (You should understand the difference between mocks and stubs, since developers often confuse the two.)

You should definitely check out the mock()when(), and thenReturn() APIs to stub method states, and also the verify() API to check mock interactions. You should also consider checking out the ArgumentCaptor API, which can help you capture arguments passed to the mocks so you can act on them later.

Tip: Check out this tutorial to get started with using Mockito in your tests.

Espresso

Espresso is an excellent framework developed by Google to help you easily and quickly automate UI tests that simulate real user behavior. These tests will run on a real device or emulator.

The Espresso APIs are very simple to understand and use. You can find any View using the onView() method. Once you have the view, you can either perform() any action (such as click(), swipe(), etc.) on it and check() that certain properties of the view match the expected results.

The great thing about Espresso is that it can automatically handle UI synchronization, so you can get rid of those smelly Thread.sleep() methods. The APIs are also quite extensible, and they give you rich error information on failures.

Tip: Examples can always help you learn faster. Here is an excellent repository created by Chiu-Ki Chan to help you write better Espresso tests. Also, consider trying out the Espresso Test Recorder.

Robolectric

Espresso tests are great, but the only problem is that they are slow because they cannot run on the local JVM—they need to run on a real device or emulator. This makes testing Android APIs less convenient than running JUnit tests.

You should run your tests as frequently as possible so that you can catch bugs early in your development cycle. But if your tests take a lot of time to run, you will not want to run them as often, which sometimes defeats the purpose of testing early and often.

Robolectric makes it possible to run Android tests right on your local machine by providing a dummy implementation of the Android SDK using shadow classes.

But there are some downsides to using Robolectric. It is always one step behind the latest Android SDK available. And though Robolectric usually works well, when it doesn’t, you might spend days or even weeks fixing the issue. So you might want to use it only when you really need to.

Tip: Here is a great tutorial to help you get started with Robolectric.

UIAutomator

Espresso is really good at what it does, but it has limitations. You will not be able to write UI tests that involve interactions across multiple apps or the system UI with Espresso.

There could be several scenarios where you need to write UI tests that can span multiple apps. For example, you might have an app in which users can tap a button to open the image viewer, pick a particular image, and then return to the original app to display it. Espresso simply won't work (without making some sort of compromise) if you need to test these kinds of scenarios.

UIAutomator does have the ability to interact with views that are beyond your app’s boundary. You won't always need to use this framework, but it's useful to pull it out when building apps that interact a lot with third-party apps and system UI components.

AssertJ

Tests can generally be broken down into three parts: arrange, act, and assert. You can check to see if a test is passing or failing only based on the assertions you make at the end of it, which means assertions are one of the most important parts of any test.

JUnit provides APIs for you to write assertions, but those assertions are not very readable or pleasant to use. AssertJ is an awesome framework that can help you simplify your assertion statements and make them more readable. Its fluent interface can make it super easy for you to write assertions, and the IDE's code-completion feature can help you even more.

It is an optional framework, but I highly recommend you give it a try. Once you have used it for a few of your tests, you will never want to go back to the old style of writing assertions.

Tip: Read this useful tutorial to learn how you can make the most of AssertJ. Also look at AssertJ Android, a set of assertions to make Android-based assertions even more simple.

Hamcrest

Matchers are something you will use a lot while writing tests. Even if you are writing a local JUnit test or an Espresso test, there is no way you can get away from using matchers.

Hamcrest is a matcher framework that provides readable matchers to match specified rules for your unit tests. The framework packs in a lot of predefined matchers that you can use to write fluent assertions in your tests.

Not only that, but you can always create your own custom matchers to match special rules or any custom object, if you need to. You should definitely block some time to play with the available matchers in Hamcrest and see how you can make the best use of them in your tests.

Go forth and test

You should now be clear about why you need to do testing. The Android testing ecosystem should also feel a little less mysterious to you now that you know the different types of tests you can write and the various frameworks and tools available to write those tests. Follow the experts in the Android community to keep up with changing testing trends for Android. And be sure to learn the fundamentals of mobile testing in the Mobile Testing track on TechBeacon Learn.

Mobile Testing: Build your mobile QA skills
Topics: Quality