Android fragmentation makes app testing like trying to put together a puzzle with too many pieces. To make mobile testing manageable, you need to take some of these pieces off the table. Here's how:
- Identify the minimum set of devices you must support
- Use responsive design to limit testing to a representative subset of devices
- Don't overdo your test automation
- Use cloud services to minimize the number of devices you need to test in-house
Let's take a closer look at the Android testing problem and how to solve it.
Fragmentation makes app testing harder
The 2015 OpenSignal report, "Android Fragmentation Visualized," shows massive Android fragmentation with over 24,000 distinct Android devices from more than 1,200 manufacturers. The devices differ not only in screen size and resolution but also in hardware, such as Bluetooth, Wi-Fi, near field communication (NFC), camera, and GPS. There are also different versions of the Android operating system, including original design manufacturer (ODM) versions with different implementations for common actions (such as saving a file), and applications, such as the picture gallery and music player.
This massive matrix of variety makes it hard to anticipate how a mobile app will behave on any particular device. One good example is battery usage. Depending on how your app uses the device's CPU and RAM, it may be fine on one device, but a battery hog on another. And if your app is a battery hog, it's likely to be uninstalled quicker than you can say "update."
A subtler example is social media sign-on SDKs like Facebook or LinkedIn. These may also behave differently on different devices, and if your users can't sign in quickly and easily, they will uninstall.
Performance is another critical factor. Is your app zippy on high-end devices with the latest CPU but sluggish on other devices? What will your target users do if your app isn't responsive enough?
For a QA tester, this presents a dizzying challenge when trying to certify an app for release on Google Play. Happily, there are several steps you can take to reduce the number of devices you need to test on to something manageable.
1. Identify the minimum set of devices you must support
The simplest way to cut your testing workload is to reduce the number of devices you support. While you may lose some users, you'll save on development, sales, marketing, and technical support costs, and focus your expenditure on your primary markets.
Know who your users are
We all know that you can't please everyone, but you should certainly choose whom to please by supporting their device.
Define your primary target markets, determine the leading brands in those markets, and define the main personas that characterize the majority of your users. Sales executives might prefer tablets, while teen gamers are more drawn to large-screen cell phones.
Once you've established the core set of devices to support and you start getting good feedback on those, you can look at additional ones in the second and third target layers as your resources permit.
Define your product availability matrix (PAM)
If you're only supporting a certain set of devices, you need to specify which ones in terms of Android versions, screen resolutions, Bluetooth versions, and any other parameter that's relevant to your application. Start off with a minimal range of devices that you support—think of it as a minimum viable product (MVP) for your product availability matrix (PAM).
Thoroughly test your app and make sure it works well on representative devices in your MVP. You can widen the range as you receive and address feedback from your users, letting your app stabilize and mature.
2. Use responsive design
Solving the mobile testing puzzle of Android fragmentation actually begins long before the app reaches QA, with responsive design. While there are several factors involved in responsive design, the most prominent one is screen resolution. Start with the resolution that's most representative of your target market (will most of your users be on a phone or a tablet?) and make sure your app works perfectly on that one. Then, if necessary, you can make adjustments to make it work on other typically representative resolutions, called breakpoints.
Image Source: Sonata Michalcova
While these are only a few of the resolutions available out there in the massively fragmented landscape, if your app looks good on them, it's likely to look good on all the resolutions in between.
3. Automate with discretion
Automation testing is a critical part of any modern QA process. It's a worthwhile investment in resources, but it's expensive to implement and maintain, so you can't automate everything. And while automation testing has come a long way in the realm of desktop PCs, mobile automation testing has yet to mature to the same degree.
Therefore, you need to exercise discretion when deciding which user flows to test with automation, and on which devices. Invest more in the main devices that surface from the PAM you defined, but also be mindful of additional manual testing you'll need to do.
4. Use the cloud
One of the reasons for massive Android fragmentation is the dizzying pace at which new devices are introduced to the market. New devices that fall within your specified PAM may not always be readily available for your QA teams to test on. Once your app does stabilize, and you increase your range of supported devices, getting your hands on enough devices for testing may become a problem.
This is where cloud testing services can help. These services give you access to the latest devices released, as well as older devices, so that you can easily cover the PAM you defined for your target market. In most cases, you'll even be able to run your automation scripts in the cloud, making these services very cost-effective.
Take away the puzzle pieces
Android fragmentation makes mobile app testing a truly puzzling challenge. But there are ways to reduce the number of pieces and make it more manageable. With responsive design, you can limit testing to a representative subset of devices, and define your PAM and personas to identify the most important device categories. If you exercise discretion when laying down your automation strategy and take advantage of the efficiency of cloud testing services, you may find it's less daunting than you thought.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
