You are here

Testing fail: How performance engineering can help dev avoid disaster

public://pictures/Todd-DeCapua-CEO-DMC.png
Todd DeCapua, Technology leader, speaker & author, CSC

[ Learn how to apply DevOps principles to succeed with your SAP modernization in TechBeacon's new guide. Plus: Get the SAP HANA migration white paper. ]

Performance testing isn't enough

Software, hardware, and the needs of application users have all changed radically in recent years, so why are the best practices many developers use to ensure software quality seemingly frozen in time? The world has evolved toward performance engineering, but too many developers still rely on performance testing alone. This can lead to disaster.

The initial failures of the Healthcare.gov website revealed how fragile underlying systems and integrated dependencies can be. Simple performance testing isn't enough. If you don't develop and test using a performance engineering approach, the results can be both costly and ineffective.

What went wrong with Healthcare.gov? A report in Forbes cited these eight reasons for the site's massive failure:

  1. Unrealistic requirements
  2. Technical complexity
  3. Integration responsibility
  4. Fragmented authority
  5. Loose metrics
  6. Inadequate testing
  7. Aggressive schedules
  8. Administrative blindness

President Obama, the CEO in this scenario, received widespread criticism over the troubled launch, which should have been a high point for his presidency. Instead, the site's poor performance tainted the public's perception of the program. When you embarrass your boss, you don't always get a second chance. In the case of Healthcare.gov, the Obama administration had to bring in new blood.

So, how do failures like this happen?

When developers and testers were working in a mainframe or client-server environment, the traditional performance testing practices were good enough. As the evolution of technology accelerated, however, teams have had to work with a mix of on-premises, third-party, and other cloud-based services, and components over which they often have little or no control. Meanwhile, users increasingly expect quick access anywhere, anytime, and on any device.

[ Is it time to rethink your release management strategy? Learn why Adaptive Release Governance is essential to DevOps success (Gartner). ]

4 key areas of focus for performance engineers

Performance engineering practices help developers and testers solve these problems and mitigate risks by focusing on high performance and delivering valuable capabilities to the business.

The key is to start by focusing on four key areas:

  1. Building in continuous business feedback and improvement. You accomplish this by integrating a continuous feedback and improvement loop into the process right from the beginning.
  2. Developing a simple and lightweight process that enables automated, built-in performance. In this way, the application, system, and infrastructure are optimized throughout the process.
  3. Optimizing applications for business needs.
  4. Focusing on quality.

How do you apply these four practices in the real world?

Your team can head off unrealistic requirements by asking for and receiving feedback and improvement recommendations. To avoid technical complexity, your team must share a common goal to quickly define, overcome, and verify that all systems are engineered with resiliency and optimized for business and customer needs. Integration responsibility must be built into all environments, along with end-to-end automated performance validation. This should even include simulations for services and components that are not yet available.

The issue of fragmented authority won't come up if you create a collaborative and interactive team, and the problem of loose metrics can be avoided by using metrics that provide stakeholders the information they need to make informed business decisions. Inadequate testing will never be an issue if you build in automated testing, including functional testing for:

  • Performance
  • Security
  • Usability
  • Disaster recovery
  • Capacity planning

Overly aggressive schedules are unlikely to occur if you provide automated quality results reports that highlight risks and offer optimization recommendations to support informed decision making. Finally, to prevent administrative blindness, focus on business outcomes, communicate with all stakeholders throughout the process, and build in accountability and responsibility for delivery.

It's your responsibility to ensure that your organization is moving from antiquated methodologies based on performance testing only to more comprehensive performance engineering practices. After all, no one wants to be the next Healthcare.gov.

*Image source: Flickr

[ Get Report: Buyer’s Guide to Software Test Automation Tools ]