A deeper look at the Puppet Labs 2015 State of DevOps Report, including expert insight from DevOps leaders and an analysis of the top 10 things you should know from this year's report.

2015 State of DevOps Report: Top 10 takeaways and insights

Get 40-Page ReportNOW AVAILABLE: State of Performance Engineering 2015-16

Puppet Labs recently released its fourth State of DevOps report. The Puppet Labs 2015 State of DevOps Report highlights trends in the DevOps revolution and includes a variety of valuable insights that organizations considering DevOps, or those that have already embraced it, can learn from. There are also a few things the report doesn't talk about—and those provide perhaps even more valuable insight.

Download the latest State of Devops report by Puppet Labs.The annual report from Puppet Labs has already been established as a barometer for the DevOps movement. This year's survey includes responses from 4,976 individuals scattered across the globe. One interesting note about the demographics of the survey is that 55 percent of the respondents are in companies of fewer than 500 people, yet 48 percent of the respondents have an infrastructure of 500 or more servers. It's a testament to what can be achieved with cloud computing and DevOps for small organizations to be able to deploy and manage such vast infrastructures.

I reviewed the Puppet Labs 2015 State of DevOps Report myself and spoke with a number of DevOps leaders to get additional insight. "This report is a good indicator as to where we are with DevOps and as you can see, we are still explaining it after four years, which has to tell you something," says TK Keanini, CTO of threat protection firm Lancope. "DevOps to those who are in it is critical and for those new to it, they are sometimes hunting for a reason, which is awkward. The kind of awkwardness you have with a mask and snorkel when there is no water around you—you can find a use case I'm sure, but it is going to be weird at best."

To the extent Keanini's assessment is accurate, this report provides some valuable information to help those organizations figure out their use cases and embrace DevOps effectively. Here is a breakdown of the 10 most important takeaways.

1. DevOps accelerates deployment

There's a long-standing mantra in IT that you can have speed or reliability but not both. The findings of this report disagree. High-performing organizations deploy code 30 times more frequently than their peers. The survey also found that the lead time—measured as the time required for changes to go from "code committed" to successfully running code in production—is 200 times faster for these high-performing organizations.

2. DevOps prevents failures and streamlines recovery

That was the faster part, but what about the reliability? According to the report, high-performing organizations also achieve higher levels of stability through DevOps practices. Researchers looked at the mean time to recover (MTTR)—the time required to restore service when an outage or incident occurs—and found that these organizations have 60 times fewer failures in the first place and recover 168 times faster when an incident does occur.

The report quotes Jez Miller of Heartland Payment Systems: "The number of issues we had from production emergencies that were triggered by an ops change essentially went to zero. Because we were able to roll changes out in an automated fashion, and then test those changes in the various environments, by the time code got to production, it had been through three other environments—dev, integration, customer test—before it got to production."

3. Deliver sustainable value

Continuous deployment, which consists of continuous delivery, continuous integration, and continuous testing, combined with the lean management principles of DevOps culture, enables better software to be delivered more consistently. The automated nature of DevOps tools improves organizational performance because it streamlines routine operations so developers and IT personnel can focus on more important tasks.

Gene Kim—a DevOps researcher and coauthor of The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win—has worked closely with researchers on the report for the last three years. One of his favorite findings was about how DevOps increases developer productivity. "This year's report is just awesome—I loved that high performers are getting better all the time. They're just as agile and productive, but compared with last year, they have 20 times higher change success rates and four times lower mean time to repair."

Kim continues, "I also loved that we are crystallizing the capabilities required for high performance: Jez Humble and David Farley described in their book Continuous Delivery about how DevOps requires code and infrastructure in version control, comprehensive and reliable testing, and the test and deployment automation—we found that these were prerequisites of high performance. One of the results is that high performers can scale developer productivity linearly with the number of developers."

4. High performance is possible with any app

One of the prevailing beliefs (or myths) about DevOps is that it's only valuable for greenfield projects and can't be adapted effectively to legacy or brownfield applications. The 2015 report states, "Our research shows it doesn't matter how old or new your systems are; high performance can be achieved if the application is architected for testability and deployability. So if you think you can't implement DevOps practices because your app runs on a mainframe, think again."

5. IT managers are crucial

There are tools and principles that make up DevOps, but many will argue that the most important element of DevOps is the culture. Organizations and IT managers have to be able to define and communicate their vision and provide the policies and tools necessary to facilitate it. Those who are too lazy or ill-equipped to do so end up falling back on some sort of cookie-cutter cultural framework like ITIL or COBIT. The report found that high-performing organizations have IT managers who are able to build teams that cooperate effectively and aren't afraid to take risks.

6. DevOps can prevent tech burnout

IT burnout is often more a reflection of the culture and environment than of the individual it affects. The State of DevOps report found that high-performing DevOps organizations also have lower incidences of employee burnout. This seems to be a reflection of the fact that DevOps culture fosters a more respectful, supportive, collaborative work experience that gives individuals a greater sense of purpose and value.

7. Minimize pain of deployment

It should go without saying that continuous delivery practices and automated testing and deployment tools streamline deployment and minimize the headaches involved. Researchers found that organizations that have successfully embraced DevOps have much less deployment pain and avoid common problems like failures that are difficult to diagnose or fix, unnecessary manual effort required to deploy, or inefficient deployments resulting from too many hand offs between teams.

8. Strive for gender diversity

Despite persistent efforts toward more gender diversity and evidence demonstrating that teams involving more women have higher group intelligence and better performance, this report shows that DevOps has a long way to go in this respect. A third of the respondents indicated that there are no women on their teams, and 56 percent reported working on teams that are less than 10 percent female.

The report declares, "We can do better. It's up to all of us to prioritize diversity and promote inclusive environments. It's good for your team and it's good for the business."

9. The elephant in the room: security

The report has a lot of data and valuable metrics, but it avoids talking about the proverbial elephant in the room: security.

I asked Ben Bernstein, cofounder and CEO of Twistlock, for his thoughts on the report. He explains, "Obviously my thoughts drifted to the landscape I know well: security and containers. Both of them challenge Puppet's DevOps model to some extent. Both of them are very obviously missing in the report, which is essentially a praise to how DevOps is great."

Bernstein told me that as the report evolves and matures, he would expect Puppet to expand on it and explore other areas of concern or interest as well. "I feel it's almost like Samsung would now release a report about how smartphones turn people 50 times more productive than if they were using their legacy phones, rather than address more interesting topics about smartphones."

10. What's missing?

Bernstein's analysis raises the question of what else is missing. One of the immutable laws of statistics is that the numbers say whatever the researcher wants them to say. The results are often more a function of the questions that were asked, while the real value lies somewhere in the questions that weren't asked.

The Puppet Labs 2015 State of DevOps Report is a great tool for measuring the evolution and effectiveness of DevOps over time, and it does seem to suggest that DevOps is maturing quickly and becoming part of mainstream IT culture. There's always another side to the coin, though, so make sure you also consider the pitfalls that might be associated with DevOps.

Get 40-Page ReportNOW AVAILABLE: State of Performance Engineering 2015-16
Topics: DevOps