Want to improve the quality of your apps? Consider making changes to your processes and metrics. This might include a change of technology, but not necessarily. What you do before your next release depends on the needs of your organization.
The goal of a winning organization shouldn’t be just to change the quality of an app, but to improve the overall customer experience. Digital user experience (UX) takes into account application performance, crashes or uptime. In a recent Dimensional Research User Experience survey of 522 IT professionals responsible for user experience, 75% cited “user complaint” as the primary way they learn about a problem.
There are better ways to manage digital UX than treating your customers like guinea pigs. Here's how.
How do your peers understand digital UX?
Most respondents to this HPE-sponsored survey, defined digital UX as having to do with application performance, crashes or uptime. But some defined it as ease of use and navigation, user productivity, user behavior, and flows; while still others said it had to do with battery or cellular data consumption, or UI design appeal. The consensus is that there’s no clear consensus: The UX can cover almost all aspects of a customer’s engagement with a digital product.
Your users expect any engagement with your applications to be valuable, elegant, and useful. User experience is something that can be emotional, driven by how users think, perceive, and feel. Most importantly, users expect mobile apps to work seamlessly, regardless of their choice of technology, location, or context.
5 areas for digital UX improvement
If you want to improve how you currently manage the digital UX, here are several things you should consider.
Understand your users. Really.
This may sound obvious, but you need to gather hard data about your different user types and how they’ll be engaging with your app. This data should be constantly re-validated, because usage patterns change often.
Focus on the following factors:
- Geo-location
- Version adoption
- Average usage time
- Operating system
- Device type
- Connection type
- Browser type
Measure what matters
When you truly understand your users and what’s important to them, you’ll know exactly which variables of the application UX to measure. You need to measure what matters from the user’s perspective—things like performance, stability, errors, battery usage, network consumption, and more.
Here are some of those related metrics:
- UI response time by location
- Search and display specific user visits
- Holistic UX score
- Performance
- Stability
- Resource usage
- E2E operational data
Monitor the user experience over time
What’s needed is a lifecycle approach to monitoring the UX, including a process for resolving issues you identify. The term “monitor” should graduate to “manage,” since the full process should be continuous and proactive for all key user interactions, devices, and so on, and should trigger alerts to DevOps teams when problems occur.
Ultimately, DevOps should implement a single measurement, to be determined by the organization, that encapsulates the overall user experience so that you can track it over time, and address issues continuously.
Experience is not a snapshot in time, but an evolving phenomenon. Key metrics include:
- Common user flows in real time (click-stream data)
- Emulation of common user actions & devices
- Most used actions
- App usage patterns
- Transaction history
- Reports of trends over time
- Alerts
Correlate digital UX to backend services
Watch transactions from end-to-end. DevOps teams should be able to trace transactions from the point a user clicks, touches, or swipes; and follow the transaction all the way to the back-end services.
What does this require?
- Trace transactions end-to-end
- Correlate user actions to issues with back-end services
- Store big data to establish trends over time
- Submit issues to defect management tools
- Agile closed-loop process for fixes
Improve integration in your UX technology stack
You know the parable of the blind men and the elephant, in which one man feels only the trunk, the other just a leg, another only the tail, and so on. The entire beast is never fully understood. This is also a parable about poor integration.
A similar fragmentation of processes and technologies underlies the confusion and mystery of the UX beast. An integrated set of UX-related technologies can help by:
- Capturing both performance behaviors across the application infrastructure
- Capturing behaviors of the users interacting with digital services
- Supporting the full spectrum of UX and customer experience management stakeholders, including operations, development, IT service management teams, IT executives, and business stakeholders.
5 key findings from the digital UX survey
In addition to the general areas for improvement described above, findings from the Dimensional Research User Experience survey reveal a number of specific challenges in managing the digital user experience. In some cases, the behavioral changes required are easy to envision. In others, IT organizations will need to bring unique talent and creativity to make improvements in digital UX. Here are the five key findings:
Metrics and scores ultimately need to be vetted and correlated against feedback from users.
Survey data indicates that defining and measuring the user experience is rather complicated. No single metric defines the experience. In fact, those surveyed are currently selecting numerous metrics to represent application performance, availability, infrastructure reliability,and several other metrics. As the graphic below shows, there are numerous metrics you can use to help define the user experience.
Most companies gather metrics to know whether a server is up or a service is running, but these provide little indication of the actual user experience. Some companies use an equation to combine numerous metrics into a single UX score. However, metrics and scores ultimately need to be vetted and correlated against real feedback from users.
The dangers of user abandonment are real
A previous Dimensional Research project found that 80% of users would only try a problematic app up to three times, and 49% of users expected a mobile app to respond in 2 seconds or less. Yet participants in this research, those who are directly responsible for the user experience, reported that only 9% of companies deliver with excellence in this category.
What's more, 39% confessed that their applications offer a substandard or poor user experience. This research illustrates the need for development organizations to move to an improved development and release process. Although DevOps is being heavily adopted, the research data indicates that organizations are still a long way from consistently delivering applications that users enjoy using.
UX designers have low real-time visibility
Just 47% of those who design the digital UX actually have real-time visibility into their products in use. Given that, you have to wonder about the remaining 53%. Part of any improvement process is the ability to gather and inject feedback into the beginning of the process. However, less than half of participants (47%) who design and build the UI and define the user experience have access to the user UX metrics needed as part of a feedback loop. Optimization requires that you have benchmark or threshold metrics to define the user experience so that all teams that design, build, test, deploy, and operate, including teams outside of IT can agree on what a positive user experience should be. The cliché, “you get what you measure,” has never been more true.
Releasing apps that fail to meet UX objectives is risky
Participants offered disturbing insights as to why users encounter problematic applications. While 60% say they hold an app until it meets all user experience release criteria, the other 40% admit that they release applications that contain known user experience issues.
Actual user experience may be even worse when unknown bugs and defects are factored in, compounding the problem. Many organizations still follow the traditional practice of releasing buggy apps so long as the app offers some new, competitive feature. They believe that it's worth the risk of frustrating customers. But given the rate at which users abandon problematic applications, this type of quick release is a serious business gamble.
Don’t rely on user complaints to learn about app issues
Perhaps one of the most shocking findings of this research is that customers are being used as guinea pigs. Nearly three quarters (72%) of companies find out about user experience only after customers complain. This is akin to finding out if something is hot by burning yourself. This approach means that your customers are already affected, have already had a bad experience, and then have to waste additional time to complain.
Only 26% of survey respondents said their teams proactively examine the user experience after the app is released to production. But the metrics that define a positive UX need to be monitored proactively to predict issues before they happen, and to provide the opportunity to resolve them before users report a negative experience. Your customers will turn to a competitor quickly if they feel neglected.
Get your UX act together
With many development teams increasing app delivery frequency toa daily or weekly cadence, the risk of poor user experience is growing. To mitigate this risk, define your UX attributes and metrics, and ensure that your teams are able to measure them.
The entire application lifecycle process must enforce measurement at each phase, and grant authority to halt development when user experience thresholds cannot be achieved. And once you release an app, proactively track user experience metrics in real time, and make that information available to all stakeholders to achieve a constant state of improvement.
Do this right, and your development team will have happy users, and capture market—and revenue—from competitors who continue to abuse their customers.
Image credit: Flickr
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
