If your mobile app doesn't work—or works badly—you're losing money and damaging your reputation. That's why application monitoring is a must-have for nearly every organization. But simply deploying a monitoring system isn't enough; you must use the right tools to monitor the right functions.
To deliver the most effective monitoring, I recommend combining synthetic monitoring (testing with simulated transactions) with real devices (testing actual phones). This gives you the full picture of how apps interact with back-end infrastructure and how they perform on user devices.
Fundamental rules for synthetic application monitoring
Synthetic monitoring provides a way to look deeply into application performance, allowing app developers to identify problems at the source and resolve them. I recommend following two rules for synthetic monitoring:
1. Establish a baseline
One of the key benefits of synthetic monitoring is the ability to place probes around the globe. This lets you test the user experience anywhere in the world on demand.
However, this data is useful only if you can distinguish your app performance from network performance. Without this knowledge, you won't be able to fully diagnose any performance problems you encounter.
The solution here is to place one synthetic monitoring probe as close to the data center as possible. This eliminates (or at least minimizes) noise in the data coming from network issues, giving you a baseline that you can compare other results against.
2. Get full coverage
You also need to cover enough operations to get a full picture of the application's availability and performance. I suggest starting with three basic operations:
- Access the home page to test the web server
- Perform a log-in to ensure that the database server is running properly
- Complete a calculation or other operation to validate application server function
The more realistically you can mimic real user behavior, the better—but the key thing is to make sure you test the full stack.
How to get the most from real devices
I also advise application owners to test these three operations on real devices. By measuring the performance of actual devices, you can greatly add to the insights gained by synthetic monitoring.
To accurately assess mobile applications, the monitoring system must interact with the various elements of a mobile device, such as operating system functions, the touch screen, and the camera. In this endeavor, simplicity is everything. With so many devices and OSs (as well as different versions of OSs) available to users, real device monitoring can quickly become a complex undertaking.
Rather than trying to test everything, it's a good idea to determine which devices are most popular among your users and choose a representative subset of them for your test devices. I also recommend monitoring iOS users separately from Android users to detect problems specific to either user base.
Monitoring must also keep tabs on how an application works with multiple service providers. Users sometimes have different experiences with different providers, even if they're running the same devices and OSs. Problems at any step can degrade performance. The goal is for users to have the same experience no matter which device, OS, or carrier they use.
Another important practice is to place the devices where your end users are. It doesn't make sense to put your mobile device in your lab if your users are located worldwide. You want to make sure the monitoring sees the same experience that users are getting in the real world.
How setting thresholds improves monitoring
In measuring performance and availability, it's important to set effective thresholds. Each action that an app performs should have a specific time associated with it. Developers must understand how fast users expect response times to be. In many cases, if users have to wait more than two or three seconds for a page to load or an application to respond, they may become frustrated. Availability is even more critical. When users can't get your application to work, they'll find one that does.
Effective device monitoring tools can be set up to alert administrators if thresholds for performance or availability aren't being met. These alerts can be executed in different ways. The monitoring system can automatically submit a report to a ticketing system, or it can send an email or a text message to an application owner or a development manager.
An application monitoring script should set parameters for the data that'll be collected. This allows you to use the same script to monitor updated versions of the application simply by changing specific parameters within the script. For example, when a new version of an application is loaded into production, changing parameters will allow the monitoring script to continue functioning if something such as a button or the name of a server has been changed.
It's also a good idea to hang on to previous versions of your scripts. This can prove valuable when problems arise with a new script. Having older versions to fall back on can help get your app running again.
The best of both worlds
Combining synthetic monitoring with real devices is a great way to get a detailed view of your app's performance. Of course, this isn't the only monitoring strategy you can use. Other approaches, such as real user monitoring, can deliver additional insights. The important thing is to make sure you think through your test strategy to make sure you have a realistic picture of actual users' experiences.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
