You are here

You are here

5 DevOps infrastructure challenges—and how to overcome them

Rob Lemos Writer and analyst

As DevOps continues to be adopted by companies, many of them are finding that a variety of roadblocks—including infrastructure—are hindering their progress.

Nearly three-quarters of development teams spend a significant amount of time—anywhere from 10% to 50% of their workweek—updating and upgrading software, according to a study by Atlassian, a provider of DevOps tools and services. On average, companies struggle with visibility into the status of their applications, requiring an average of 3.3 tools to determine status.

The result is that companies, though they want to move fast, are getting bogged down. They already have the technical debt of legacy technology and are finding themselves adopting new technologies that need to be understood, configured, and optimized, says Ian Buchanan, solutions engineer at Atlassian.

"All of that technology and infrastructure are a kind of weight, a technical debt, and we should not overlook that people have to pay that down."
Ian Buchanan

This technical debt is a significant blocker for many companies, preventing them from creating more mature and efficient development and deployment cycles. Using a measure of software delivery and operational (SDO) performance, the DevOps Research and Assessment (DORA) group found that about 48% of DevOps-focused companies are classified as high performers, while 37% are medium performers and 15% are low performers, according to the 2018 State of DevOps Report.

While low-performing companies initially make significant gains by automating development, medium-performing companies are slowed down by adding additional infrastructure to speed testing and deployment, preventing them from crossing the threshold into the high-performance category.

The difference between doing DevOps right and doing it wrong is stark. Elite performers—a subset, about 7%, of high performers—are able to deploy code more then 2,500 times faster from the time they commit, compared to the low performers, and have a change failure rate that is seven times lower, according to the DORA group, which is now part of Google.

There is no easy path to the high-performing category. It takes time and effort, say experts. Here are five areas your company should focus on.

1. Emphasize culture

Repeatedly, DevOps experts emphasize that having the right culture is key to making DevOps work. While deploying and optimizing infrastructure can slow down development and operations, not having the right mindset among your teams can cause a project to crash, says Jayne Groll, CEO of the DevOps Institute.

"You need a culture of innovation, a culture of disruption, and, most importantly, a culture of continuous learning," she said. "None of the major case studies in DevOps success find that a company succeeds because they have a great pipeline—it's because they did something that was very, very people-focused. They kept the humanity in DevOps."

CloudBees, a provider of continuous delivery services, agrees. A company with the right culture can overcome any and all infrastructure and skill issues, says Brian Dawson, DevOps evangelist with the company.

"Skills and training bleed into culture. Knowing how to implement configuration and code really doesn't matter until you are in the mindset that is required to work cross-functionally, collaboratively, and in a way where you are likely to be experimenting, failing, and then trying again."
Brian Dawson

[ Also see: Infrastructure as code: The engine at the heart of DevOps ] 

2. Soft skills are as important as technical

In its latest report, the DevOps Institute found that companies hiring for DevOps positions prioritized soft skills—such as collaboration, problem-solving ability, and interpersonal skills—as often as they prioritized technical skills.

"Scale requires skills, and that skill requires people. There really was kind of a clear recognition that the higher-level skills were the problem-solving skills and the critical thinking skills."
Jayne Groll

While having employees who are willing to collaborate, learn, and jump into any problem is important, companies also emphasized automation skills, especially information-technology operations and infrastructure knowledge on the technical side, and software development, source control, and analysis on the process side.

Yet most companies will not often find an exact match for their requirements, so finding a strong cultural and interpersonal match is often better than an exact technical match.

3. Automation is key (of course)

The key to DevOps is automation, and the most important skills and infrastructure are often focused on eking out more efficiency from the software development and deployment cycle. This is supported by the latest studies as well: Elite and high performers do less manual work, according to the DORA study.

Infrastructure for automation will need constant improvements and tweaking of policy to eliminate blockers. There are three attributes of good automation, says CloudBees' Dawson.

First, companies need the ability to access their process quickly. Second, they need reproducibility between what a real production environment looks like versus the pre-production environment.

"How much am I able to ensure that my environment stays the same and doesn't drift too much, and if I'm creating separate instances of it, that they are not different from each other?"
—Brian Dawson

Finally, the process needs to be resilient to failure, he says.

[ Also see: How to manage full-spectrum DevOps that spans multiple clouds ]

4. Manage change with faster decisions

The process and infrastructure for making change to an application have also become a major source of inefficiency for companies, says the DevOps Institute's Groll.

"We talk about continuous deployment and continuous delivery, but most organizations want change announcements two weeks in advance. So there is a disconnect there between what we say we want and the reality."
—Jayne Groll

Typically, companies classify changes in one of three categories: Standard changes, normal changes, and emergency fixes. Normal is the typical process for making changes, while standard changes are ones that can be made without pre-approval or, at least, expedited. With automation in DevOps, however, far more changes should be able to be made quicker, says Atlassian's Buchanan.

"Standard is the new normal. With all of this automation going on, the standard changes—pre-approved, well-understood, low-risk, well-specified—can be made because we know how those processes work and we don't have to worry that there is a lot of risk."
—Ian Buchanan

5. Plan to scale up

As the DORA study showed, there is a hurdle for most companies when they begin to scale up their DevOps infrastructure from the low-hanging fruit of quick automation to the more difficult-to-achieve infrastructure projects. Often, when people want to stand up DevOps in their organization, they create another functional team, says CloudBees' Brian Dawson.

"You are in danger of creating another silo. I don't think the sheer number of choices in terms of infrastructure technology is the problem. I think it is how people make the choices."
—Brian Dawson

Rather, companies need to adopt infrastructure that allows them to quickly scale up their operations. Organizations that significantly used cloud infrastructure, for example, were 23 times more likely to be high performers in the DORA study.

How fast do companies need to move? One restaurant chain created an application for online ordering late last year, and within six months, orders through the application accounted for 15% of the companies revenues, says Zane Lackey, chief technology officer and co-founder of Signal Science, an application security provider.

In the end, DevOps infrastructure needs to serve the company, he says.

"You can't have a technology for only one style of how a customer deploys applications. Companies want to lift and shift some applications to the cloud, others they want on-prem, and everything in between. You need to meet them anywhere where they are deploying their apps."
Zane Lackey

Keep learning

Read more articles about: App Dev & TestingDevOps