Collective Intelligence: Where the Promise of AIOps Is Realized

The true promise of AIOps is realized when it yields collective intelligence that spans traditional silos. This blog post examines the power of the collective intelligence AIOps can deliver, and it details the three key pillars upon which successful AIOps implementations are built.

Introduction: The Common Misperceptions Surrounding AIOps

Artificial intelligence (AI) has seen major advancements, across an increasingly broad set of areas, in a short time. Scan recent news and you can see how AI is being employed in everything from art, to agriculture, to anti-doping efforts—and those are just a few examples starting with “A’s.”

Clearly, the opportunities are vast, but when executives think about AI in their enterprise, it often seems to be within the context of a data scientist off in some remote lab, crunching numbers and algorithms. Any intelligence gathered gets thrown back to business leadership, who may or may not act on the information.

However, when you look at what leading enterprises are doing in this area, these perceptions completely miss the mark. The reality is that AI is delivering significant value today, and the surface is just starting to be scratched in terms of what’s possible.

People, Process, and Technology: Before and After AIOps

To fully appreciate the power of this concept of AIOps, it’s important to start by looking at the history of intelligence within traditional IT organizations. The reality is that IT and operations data sets were siloed, as were people, processes, and technologies. Here’s a high-level picture of each:

• People: Historically, people gathered in disparate, isolated groups, with teams focused on networks, applications, storage, and so on.
• Processes: Processes were also siloed in nature. When issues arose, for example, processes in place revolved around troubleshooting and remediating the specific technologies in a given administrator’s purview. Across silos, the process, if you could call it that, was to have massive, all-hands-on-deck calls in which different teams shared what they were seeing. From a development standpoint, waterfall-based approaches ruled, where one disparate team, say development, handed a product off to a QA team for testing.
• Technology: Here, too, the technologies employed were narrowly focused on a specific domain. Tools, data sets, and data types were distinct. Network engineers worked with time-series data, application administrators looked at topology-focused views, and so on.

There were a number of causes for the continued existence of these silos, but one of the reasons they were most persistent is because of data. In too many organizations, establishing unified, cross-domain intelligence simply didn’t happen.

The real promise of AIOps is realized when it yields collective intelligence that spans traditional silos. Contrast the siloed approaches outlined above with an organization that’s been harnessing the advantages of AIOps:

• People: Roles are filled by generalists. For example, a site reliability engineer will have expertise and insights into a site’s entire computing stack, including coding, applications, servers, and networks.
• Processes: Agile, DevOps, and continuous delivery pipelines are established, and AIOps-fueled visibility is key to optimizing these approaches. Through this visibility, teams can collaborate seamlessly, iterate continuously, and speed innovation.
• Technology: Instead of isolated tools for specific domains, all teams can gain access to unified, service-centric visibility that spans the entire ecosystem.

Keys to Building an Effective AIOps Foundation

As they set out to establish an AIOps implementation, enterprise IT teams have a number of choices, including whether to leverage commercial offerings or build their own capabilities using open-source technologies. No matter which approach is employed, there are three key pillars upon which a successful AIOps implementation is based:

• Establish a unified, comprehensive data lake. It’s essential to establish a data lake that ingests and stores a wide range of data sets and data types, including topological data, alarm metrics, log files, configuration management databases, and more. These different, disparate data sets need to be normalized and correlated.
• Leverage the right algorithms. Teams don’t need to reinvent the wheel. The reality is that the algorithms required for AIOps have existed for some time. The key is knowing which algorithm to use at which time.
• Ask the right questions. Throughout the process, it’s critical to have the right questions in mind and ensure that AIOps delivers the intelligence needed to answer the questions that matter. What’s the optimal mix of cloud and on-premises resources? What workloads should get migrated to a cloud environment? How do issues get identified and preempted before users ever experience a hiccup? AIOps implementations can yield powerful insights into these areas and many more.

Collective Intelligence: The True Payoff of AIOps

The full benefit of AIOps is realized when it delivers truly collective intelligence. This collective intelligence yields invaluable advantages as organizations look to break through legacy silos, fueling true, efficient, and meaningful collaboration. In this way, AIOps delivers invaluable insights that fuel optimized operations and service levels. If you think about the insect world, and a swarm of bees or a colony of ants, you can get an analogy of the power of collective intelligence. Consider how massive numbers of individual insects can be so connected, so efficient, and so focused on a singular goal. Apply that to the realm of IT operations, and you get a picture of what’s possible.