When I think about application performance management (APM) and the tools used in the various process associated with it, I’m often reminded of the “observer effect”—that to observe something is to change its behavior, just by watching. Especially watching people. When we narrow our perspective to how people engage with mobile computing, this effect is even more exaggerated.
That's because mobile customers know they’re being observed, and that's why they're so willing to make their happiness, or unhappiness, very obvious to us observers via their app ratings and comments in app stores and blogs. They know that for any given app, its marketing, development, and operations teams are paying close attention to their reviews.
I’d like to briefly discuss some of the challenges that we mobile app providers, as well as our customers, face in getting to a better user experience. Then I want to suggest a few ways that mobile app providers can greatly improve their offerings.
How patient is your mobile customer? The move to intimacy
Because online customers know they’re being tracked, they expect to find what they’re looking for quickly or to be able to get answers to a chat message almost immediately. If not, they’ll search elsewhere for another option. And because they’re anonymous, they have zero problems complaining. The online shopping experience removes the social filter, which in human space is still relevant. Complaining in person can be hard for polite people, who don’t want to sound obnoxious. But not online. Online, we’ve created the most impatient and often most vocal customer on the planet.
This new focus is the “digital user experience”—it’s not just APM anymore.
When people have a bad experience online and on your mobile site, whatever it is, they don’t care. It doesn’t matter if the problem is being caused by a third party, or even if the problem is being caused by their own device. The expectation falls squarely on you, because you're the company they are dealing with.
Tools that monitor availability and performance have been around at least ten years. But what we need today are tools that increase customer intimacy—that is, tools that give us the information to understand what customers are trying to accomplish, what they’re able to do with mobile apps, and whether the experience is something that delighted them. Companies that focus on these metrics—companies that get, and invest in, this portion of their business—see explosive growth.
Organizations frequently have—and spend—the right amount of money, but they’re not focused on what will improve digital user experience.
Hewlett Packard Enterprise calls this new focus the “digital user experience”—it’s not just APM anymore. That’s because digital performance monitoring is not just about apps, but about the full range of digital experience as well. The goal is to transform the digital user experience by focusing on the fringe data that allows businesses more insight into how their customers move, feel, and transact business. And there are several reasons why we’re just not there yet.
Barriers to improving the digital user experience
The mobile market is huge, but it can be hard to get businesses to invest toward success in this space. For example, a CTO or dev lead for a large toy manufacturer is certainly interested in improving customers' mobile user experience. It’s even likely that they’re getting some top-down pressure to improve customers’ digital user experience. However, if you ask what tools they’re using, the most likely response is “none.” Or they’ll say they have some dev tools and use some analytics, but that’s all they really need.
Well, it’s not. They need tools to monitor the website, tools to monitor internal applications. The $3 million they spend per year on synthetic monitoring is simply not going to cut it. In other words, organizations frequently have—and spend—the right amount of money, but they’re generally not focused on what will improve their customers' digital user experience.
6 mobile performance imperatives
If your organization supports mobile app development and delivery, here are six things you can work on to improve your app’s success rate.
1. Get past cowboy behavior in mobile testing and delivery
Hewlett Packard Enterprise recently surveyed roughly 35 developer organizations. Among the things we asked was, “How do you monitor mobile performance before you release an app into the wild?” Respondents overwhelmingly told us things such as: “Oh, we check the app on a really old iPhone, then we check it on one of the latest versions; if it works on both, then we’re good to go.” Or, “We don’t really do formal testing, but we have this intern who spends three to four hours per week playing with new applications, and if she doesn’t complain, we put it out there.”
For now, the testing of mobile apps is woefully inadequate, and the monitoring of users is erratic or non-existent.
Only one team that we surveyed indicated that it understood how to use DevOps with continuous mobile delivery, how to monitor applications, and how to release them.
We know that the cowboy behavior going on in today’s mobile development world will someday come to an end. But for now, the testing of mobile apps is woefully inadequate, and the monitoring of users is erratic or non-existent. Simply put, companies that are not focusing on mobile app performance are missing out on explosive growth.
2. Understand the differences between web and mobile apps
Today, most companies don’t create their mobile apps from scratch. They use platforms that convert their web technology into mobile technology. And while that process creates poorer-performing apps, they get the job done in many people's opinions.
But here’s the problem. Mobile apps created in this way are like doggy bags after a restaurant meal, leftovers that are packaged up in whatever way, and no one seems to care how things taste when you get home.
Sometimes I hear executives say, “Look, we’ve had good results with our web app. We need to convert that app for use by our growing mobile customer base.” Or, “Well, I don’t write code, so what do you expect me to do?” Okay. Never mind what I expect. Here’s what I think consumers expect from businesses that provide their mobile apps.
- First, we’ve turned a major corner with mobile devices. Although desktops are still creating the lion's share of content on the web, users prefer to consume that content using mobile applications.
- Second, this trend is only going to grow, because there will always be more consumers than creators.
3. Apply development-stage testing artifacts in production
Mobile APM experience isn’t bound by the data from synthetic monitoring, real user monitoring (RUM), or even diagnostic tools. All of these have a role, and in some cases one tool works better than another. But if I’m an app developer, I want to use tools that will give me the most metrics possible, including both development and production metrics.
Mobile apps created from web apps are like doggy bags after a restaurant meal, leftovers that are packaged up in whatever way, and no one seems to care how things taste when you get home.
For example, with synthetic monitoring, I can test all my applications before I go out into the real world. I can ensure they’re able to support 1,000 users, 100,000 users, maybe 1 million users before my app crashes. And here’s what’s important: When I’m done doing the load testing of my apps, I want to be able to use those same scripts in production—to make sure users in China and in London and in San Francisco are getting into my apps the way I expect them to.
But we’re simply not seeing that conversion from the app testing environment into the mobile production space, which true APM requires. APM lives in the production world. And there’s a really clear reason for the divide between the development and production worlds—part of it is that these teams don’t have the same bosses, the same budget, or the same initiative, and in some cases they aren’t even the same company. So it’s hard to expect these teams to work together once the project or product is shipped to the production team.
Unfortunately, DevOps is a unicorn few teams have ever seen. Mobile development organizations have much to gain from DevOps practices, even if they start simply by sharing a few resources, such as test scripts in the example above, or sharing feedback from the production team about bugs discovered so that future releases can address those issues.
4. Focus on the end-to-end process, where the details of UX live
A small startup I know in Europe allows customers to take pictures on their phones and have those pictures mailed to them; after 10 of these, the customers have to watch a brief video, then they can get 10 more. During an initial trial period, the company managed to attract only 100 users to sign up. So it downloaded a mobile APM tool to analyze where it might improve things and make the software more attractive. That APM tool showed the startup a major gap in its UI, one it hadn’t been testing for. It fixed those user issues in the next release, and before the trial expired—before the 30 days were up—it went from 100 customers to 100,000. All because it was able to measure, focus, and improve on the UI. The result was explosive growth.
DevOps is a unicorn few teams have ever seen.
This is what’s possible, and it may be that small startups such as this are the only ones that are architecting their dev and production environments to handle end-to-end quality for their users. At least, when you take a look at more established companies trying to move into the mobile space, you find, ironically, that they just don’t have their hands on all the pieces of the end-to-end process.
Maybe the future is to hold development teams accountable for what they’re releasing, based on feedback from APM processes. But right now, most things are unmonitored. And if they are monitored, the teams are only looking at the physical crash. Furthermore, because they don’t have enough metrics to reproduce the issue that caused the crash, they just chalk it up to a device issue. That's because we are in a bring-your-own-device (BYOD) economy, requiring only easy answers, which means teams that should be focusing on UX are allowed to look the other way.
5. Use sentiment analysis beyond mere app performance
I think the next big wave is going to be around sentiment analysis. Some companies are already working on this. I have a friend who recently moved to the US, and we went—well, let’s say to a major showroom-based department store—to buy furniture for his new apartment. He needed to have all this delivered and was told “No problem; within three days.” Three days came and went. On the fourth day he called, the store opened a ticket, went through the whole process, and again promised to deliver. But it didn’t.
Because the teams don’t have enough metrics to reproduce the issue that caused the crash, they just chalk it up to a device issue.
After the seventh call, he went onto Facebook and wrote five sentences about his experience. Eight minutes later the head of marketing for the furniture company called to say, “I heard you had a bad experience, and I want to resolve it.” Eight minutes!
Organizations with a large social presence and reputation are using tools to monitor posts such as the one my friend created, and they’re just as concerned about how people feel about their app—when people speak out, they’re listening. They’re using tools to monitor videos and tweets; each word and phrase that comes up is evaluated and given a point structure, and based on how this scoring goes, that email is marked as either a positive user experience or a poor one. It’s transforming text, voice, and video information into a number that can be evaluated.
6. Use the land-and-expand approach
In my experience, most production teams are not using the tools currently available for APM, even though these tools aren’t hard to set up or costly to implement.
What I encourage teams to do is adopt a “land and expand” approach to a mobile initiative. That means that, if you don’t have the full budget today, you spend what you can now, and grow as you see the value. Powerful tools start at only a few hundred dollars a month, and the value you’ll get will far outweighs the costs. Take a little of the budget being spent in the dev and testing stages of app delivery and invest in a few monitoring tools that offer metrics for how your customers are actually using the product.
Measure to improve
After all, you can improve only what you measure. The next wave of tools will allow you to know not just how your app is performing, but also how people feel about your app, how people are talking about it, how social media is responding to it. The tools take all those outside metrics—including maybe your current stock price—and give you a single pane to see the picture of sentiment in a meaningful and contextual way.
Tools that can provide this sort of insight will remove the subjective nature of how people feel about the apps we create and allow us to measure, focus, and improve them over time. That’s the future of APM, or what we’re calling the “digital user experience.”
Image credit: Flickr
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
