Having originated from Unicorns like Amazon and Etsy in recent years, DevOps continues to rise in prominence across corporate IT, and for the last five years, Puppet and DORA have surveyed thousands of IT professionals to determine what impact DevOps is having on IT organizations and overall business performance.
The answer across the board has been that DevOps has had a significant and sustained impact—and this impact is becoming more tangible every year. The 2016 State of DevOps Report looked at some of the biggest questions affecting DevOps today, including issues surrounding the ROI of DevOps, the integration of security and quality into the development process, and whether DevOps is actually driving employee satisfaction in the workplace.
As it turns out, the news on all fronts has been incredibly positive. Here are five key highlights from the 2016 State of DevOps Report.
IT's high performers are in a class of their own
If one thing is clear from this study, it’s that the best IT organizations—“high performers” in the parlance of the report—are wildly outperforming their peers. The numbers are now staggering. High performers (which, it follows, are stronger adherents to agile and DevOps practices) have significantly better metrics across the board when compared to lower performers, including:
- 200 times more frequent deployments
- 24 times faster to recover from a failure (MTTR)
- 3 times lower change failure rate
- 2,555 times shorter change leadtime
Not only are these numbers incredibly impressive, but, by and large, high performers are “pulling away from the pack” when you look at the data over the last three years. In 2015, high performers were completing about 200 deployments per year on average. Today that number has skyrocketed to an average of 1,460 deployments, reflecting a move from about one deployment per day to upwards of six. Lead time for changes has improved from a matter of days to minutes. Key metrics for low performers, on the other hand, have not changed significantly in the last three years.
The DevOps mantra of continuous improvement is both exciting and real, pushing companies to be their best and leaving behind those who do not improve. Clearly, what was state of the art three years ago is just not good enough for today's business environment.
DevOps has a positive, and now measurable, ROI
One of the central questions surrounding DevOps since its inception has been whether it has a positive ROI (since IT has historically been considered a cost center). A related question is whether that ROI can be accurately measured.
For the first time, the 2016 State of DevOps Report has attempted to quantify the ROI related to DevOps practices. Those findings primarily surround the issue of unplanned work and rework, with the study noting that high-performing organizations spend 50 percent less time on these tasks than lower-performing teams. Instead, they are able to devote that time to new work, which is directly associated with profitable business.
It isn’t easy to generate a universal ROI figure for DevOps because organizations differ in size widely and their cost of downtime will vary based on that size and other variables. Through some fancy math, the study was able to show that the higher performing (and faster) your IT operation is, the less downtime will cost you on a per-deployment basis. In financial terms, the cost per instance of downtime for high performers was just $37,500, while the cost for low and medium performers runs well into the millions of dollars.
This makes sense logically. Faster deployments mean that each of those deployments—successful or not—is less expensive. The study contrasts this with the heavy costs associated with an instance of downtime in bigger deployments.
When you deploy infrequently, you release much larger, more complex bundles of code into your production environment, making integration and support of that new code challenging and making it very difficult to identify the cause of failure.
As a caveat, the report notes that decreasing downtime is not necessarily the same thing as increasing uptime and that employee time saved in this fashion must be funneled into productive activities in order to effectively increase ROI.
Security is becoming more integrated with development
At last developers are starting to acknowledge that fixing security problems after a product is released is costly, difficult, and dangerous. Integrating security during development makes sense, and the integrative aspects of DevOps are finally making that a reality.
High-performing teams are putting this policy into practice in earnest: Security objectives are being integrated into overall business objectives, which is resulting in a reduction of 50 percent in the amount of time that has to be spent on security remediation. Security reviews and security testing are now becoming a key and expected part of the delivery lifecycle.
'Quality first' means less fixing down the line
Similarly, a focus on building quality into the development process from the start is really taking hold. In opposition to the traditional waterfall development process of design-develop-test-release, testing is shifting from a massive batch step performed at the end of the cycle to a collection of smaller steps interspersed throughout the development process.
The study notes that it is difficult to measure software quality in numerical terms, but, as with the ROI calculations discussed above, the study uses the metric of unplanned work as a proxy for this. Again, lower performers are spending 50 percent more time on unplanned work or rework—and are spending barely half the time that high performers are on planned work or new work.
High-performing organizations spend 50 percent less time on unplanned work and rework, and as a result, they are able to spend 66 percent more time on new work, such as new features or code. They are able to do this because they build quality into each stage of the development process through the use of continuous delivery practices, instead of retrofitting it at the end of a development cycle.
The bottom line: Integrating quality practices earlier in the development process is enabling developers to spend considerably less time fixing problems instead of developing new projects.
DevOps makes employees happier (and companies stronger)
One of the more interesting questions answered in the study is whether adopting DevOps principles makes employees more satisfied at work. The answer is yes: Employees at high-performing organizations were 2.2 times more likely to say their company was a great place to work and 1.8 times more likely to recommend their specific team to a friend; both figures were arrived at via use of the Net Promoter Score methodology.
Why does this matter? Because workers who are engaged with their jobs grow revenues more than those who aren’t—2.5 times more, in fact—and the stocks of publicly traded companies with more engaged workers consistently outperform the stocks of companies whose workers aren’t as engaged.
Employee engagement is not just a feel-good metric—it drives business outcomes.
In other words: DevOps equals happiness, which in turn equals profits. Now it’s science!
Image source: Flickr
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
