Big foam finger at Red Sox game

IT Ops analytics: IT gets its game on with better business alignment

Innovation is the operative word in the IT operations analytics space. It's a field defined by capabilities including using more advanced heuristics, leveraging more dynamic and relevant data feeds, and emphasizing outcome-driven operations in the service of more sophisticated use cases. It can help you position the business to be a more collaborative partner with IT in the digital enterprise.

Digital transformation is the biggest driver of the changes underway in the IT operations analytics market, said Sapan Shah, team lead of ICT research at MarketsandMarkets. It projects that the IT Ops analytics market will grow from$2.8 billion last year to almost $20 billion by 2022, a compound annual growth rate of 38.7%

The digital organization requires connecting the dots end-to-end, tying the customers and their experience—a slow shopping cart during an e-commerce transaction, for instance—all the way back to the behavior of the underlying IT infrastructure. So, connecting event-based information across the enterprise and providing different dashboard views into it, depending on individuals' roles or technical expertise levels, becomes an important part of IT operations management.

"We want a more coordinated effort to tie operational visibility to business visibility," said Jason Bloomberg, president of Intellyx, an industry analysis firm.

It's really about having vendors provide a unified abstraction layer across what is still a very complex IT story—across multiple environments—to manage the entire end-to-end digital business, in a way that IT and line-of-business executives can work together, Bloomberg said.

"That raises the bar on IT Ops analytics and that's now driving the innovation in the space—not just speeds and feeds but business value."
Jason Bloomberg

Root cause analysis: A real-world demonstration of IT Ops analytics [video]

The link between IT Ops analytics and business

Josh Simon, vice president of cloud services and R&D at Atlantic.Net, shared an example of how robust IT Ops analytics can help solve business challenges. The company provides on-demand cloud hosting to tens of thousands of customers and needs to predict usage and capacity patterns in a dynamic environment. Its goals: Optimize CPU, RAM, disk space and network capacity for customer workloads on its customer-facing cloud as well as internal production and development environments.

"The analytics we generate come from hundreds of thousands of data points collected every minute," he said. Its IT Ops analytics system predicts usage by constantly updating and analyzing its data sets. That information feeds into an automation engine to optimize system resources for strong cloud performance management. "You need a robust analytics platform to help you accomplish the necessary efficiencies and gains to keep things humming along," he said.

Another win for the business: This information has proved invaluable for the company's marketing initiatives, Simon said.

"We can now scale our marketing campaigns in concert with our predictive engine to ensure resources are used optimally."
Josh Simon

Data, data from everywhere

IT Ops analytics is big data analytics which, when applied correctly, makes good business sense. "When we can collect richer information from [more] components—not just the points you know about—now you have a representative set of data that you can begin to apply analytics on that give you insights into the unknowns, the things occurring in your environment that you aren't aware of," said Gary Brandt, product manager of OpsBridge Analytics at Micro Focus.

"There is tremendous value hidden in the data generated by our IT assets [so] that with the advent of analytics we can see things (like anomalies, patterns, and relationships) that we would not otherwise see.”
Gary Brandt

That opens the door to triggering actions to proactively address problems or change situations before they can affect customers, revenue and the business, he added. "The traditional tools of monitoring IT are being displaced by next-generation IT analytics technologies," added Swati Dhiman, research analyst at MarketsandMarkets.

Third-party advanced IT analytics tools, said EMA vice president Dennis Drogseth, offer the best route to dynamic, relevant data feeds for informing business impact, performance and capacity management, change impact awareness and change optimization, as well as for Agile/DevOps and integrated security scenarios.

“What's new is the growth in broader data acquisition, in bigger data pots. Most of the vendors really have big data stores—Cassandra probably being the most prominent—and greater sophistication in use case awareness, in directing machine learning at use case requirements and in many cases improved time to value," Drogseth said. "That’s one difference I see with advanced IT analytics vs. the traditional use of big data for business."

Increasingly crystallized use cases, such as the convergence of IT operations and security analytics, are leading more vendors to build turnkey offerings or defined solution packs. These take advantage of the underlying analytics, but also bundle and deliver their capabilities in a seamless way for a more intuitive experience for IT operations staff, Brandt said.

Analytics matures: Bring on heuristics

IT Ops analytics systems' heuristics strength is growing too, with almost all players in the market advancing the state-of-the-art. Machine learning, anomaly detection, and predictive trending are among the most common, in addition to advanced correlation, if/then change/performance insights (particularly valuable for assessing cloud migrations), and natural language search data capabilities and/or sentiment analytics. The latter can filter to IT through integration with IT service management teams and workflows where they normally surface.

"Due to technologies like machine learning and artificial intelligence, IT Ops analytics is really moving to advanced analytics."
Swati Dhiman

Keep in mind, though, that having more heuristics capabilities doesn't necessarily mean a particular ITOA offering is right for you. “If/then analytics, though valuable, may be less central to what you need," he said. Then, too, some of the heuristics present in a product may not yet be fully realized. "Generally you could say the richer the choice of heuristics the better, but you really need to see how they are being applied, their usability, visualizations and integrations," he cautioned.

Bloomberg also said that the complexity of environments can create challenges. In traditional enterprise hybrid environments with more than one cloud that use SaaS apps, predicting when problems may occur, and preventing them, can be just as hard as it has always been 

And largely, in today’s complex infrastructures, recommendations for automated recovery from failure "can’t replace seasoned personnel who understand the environment to make the appropriate choices. You need to understand the broader context of what goes on in an environment, which is why it’s difficult for technology to consistently automate these kinds of processes."

But keep an eye on things. Brandt, for example, sees IT Ops analytics vendors investing more in natural language processing and natural language search to make it easier to ask questions about the data, as well as for the analytics to infer what other contextual data might be related to the query. The idea is to come up with more complete answers to issues even in complex IT environments.

Deep analytics algorithms ultimately will reach the point where they can actually learn from themselves to create new algorithms that draw upon past experiences—and the associated successful or unsuccessful outcomes—to provide IT Ops with better remediation recommendations.

"It will be an augmented way of helping the user become more efficient," Brandt said, and the next stage will be full automation, assuming that IT is willing to relinquish final control in at least some situations. For instance, tech staffers may initially be comfortable with IT Ops analytics tools performing automatic password resets, but less so with the tools automatically rolling back a database in response to a predicted failure.

"It’s a maturity thing, and over time as accuracy improves and efficiencies are gained and the business benefits, the acceptance level will grow."
—Brandt

When IT Ops has a clear business impact

In its buyers' guide, Enterprise Management Associates (EMA) assessed the advanced IT analytics market, which it defines as including tools centered but not necessarily limited to IT operations stakeholders. It listed "business impact" among its adoption scenarios. This requires linking IT application service performance and business performance by using capabilities such as integrated data and analytics, and a common dashboard with visualizations that support business metrics such as revenue and business process optimization.

EMA's Drogseth said this is helping to pull out customer disaffection with respect to transactional performance. IT Ops analytics relates that back to IT and business stakeholders when something critically and persistently impacts customer behavior. 

Root cause analysis: A real-world demonstration of IT Ops analytics [video]
Topics: IT Ops