December 19, 2020
Introduction
Forrester, the independent market research firm, recently helped unveil the concept of value-based DevOps. Value-based DevOps is centered on the idea that companies not only want to develop software with greater speed and efficiency, but to ensure that the software developed delivers more value to customers and the business
That sounds obvious, but value is seldom part of the discussion in DevOps. Forrester, in partnership with Broadcom, is seeking to put that right. This infographic shows the data underpinning Forrester analysts’ thinking. The data is based on a global survey that identifies priorities, challenges, and outcomes around three major value-based DevOps characteristics: transparency, efficiency, and value.
Most organizations implementing DevOps today focus on efficiency, velocity, quality, and, to some extent, visibility and transparency. This is exemplified by DORA metrics, which are considered industry standard for DevOps. In essence, these are IT and technically focused metrics. Improving these metrics has been shown to improve business performance. However, these metrics do not actually capture and track delivery and realization of actual business value or business outcomes.
To realize true business benefit from DevOps, we therefore need to evolve our DevOps approaches from tracking technical metrics to focusing on value metrics.
In this blog, we will discuss how we evolve DevOps platforms, practices, and solutions to support this value-based approach.
DevOps Platforms: An Evolution Roadmap
Based on our customer engagements, we see organizations following the evolutionary path described in the figure below.
The stages of this evolution are described below.
Stage 1: DevOps Tool Chains
In this stage, customers are mostly focused on efficiency. From a technology perspective, this manifests itself in the form of tool chains. Tool chains typically rely on custom, point-to-point integrations between tools across the DevOps lifecycle, from planning to operations.
In this stage of maturity, the focus is on automating the flow of physical artifacts (such as code, builds, binaries, data, configurations, packages, and other deployable or monitorable items) through the DevOps pipeline. An example of a typical DevOps toolchain is shown in the figure below:
Image Courtesy: https://infocus.delltechnologies.com/bart_driscoll/common-devops-tool-chains-pitfalls
The key metrics that are typically tracked during this stage are relevant to speed and efficiency, for example:
- Build frequency, times, and failure rate
- Deployment frequency/velocity and granularity
- Lead time for change (from commit to production)
- % of pipeline tasks automated
- Change failure rate
- Build rollback rate
- Quality metrics at the physical artifact level (such as build failure rate, code coverage, defects/LOC, and released defects/build)
This stage is most useful for DevOps practitioners like developers, deployment engineers, and operations engineers.
At this stage of maturity, most DevOps toolchains are built using tools like Jenkins. While this is a popular approach, it has its drawbacks (see our webinar on this subject here). Following are some of the challenges associated with these toolchains:
- Lack of built-in, end-to-end process orchestration, which typically has to be scripted.
- Due to a large and expanding number of scripts, these toolchains can be difficult to maintain and scale. Writing “code” to create and deploy the pipeline (in addition to actual application code) may be onerous for many organizations challenged with limited development talent.
- Lack of support for cross-lifecycle data collection, visibility, and analytics. Add-on tools like Hygieia can help address this gap.
- Typically, such toolchains lack built-in security, governance, and auditability. Therefore, all of these capabilities have to be scripted into the toolchain.
Stage 2: DevOps Delivery Platforms
In this stage, customers are mostly focused on the delivery of the logical payload of the DevOps pipeline. In this context, payload refers to application features, stories, and other backlog items. In addition, DevOps delivery platforms also address many of drawbacks in the toolchain approach discussed in the previous section. These platforms provide easier, low-code or low-maintenance integration of tools or underlying toolchains. They also provide orchestration of processes across the pipeline.
Such platforms also typically provide built-in support for DevOps lifecycle data collection and organizational metrics, as well as support for security, governance, and auditability. Some of the key differences between DevOps toolchains and DevOps delivery platforms are summarized in the figure below.
DevOps delivery platforms shift the focus to such personas as product managers, product owners, and release managers. These team members are more interested in tracking the progression of logical application components (such as new features or stories or other backlog items) through the pipeline.
This implies that such platforms will always have strong integration with the requirements and release planning/definition tools in which the logical payload is defined or managed. Note that, for DevOps delivery platforms, integration with backlog management systems is not just at the team level—it also requires integrations at the program and enterprise levels. This enables teams to track logical payloads at different levels of abstraction, from epics down to stories, across multiple pipelines and release trains.
The delivery platform implicitly ties logical components to the underlying physical components of the application, such as builds.
The key metrics that are typically tracked during this stage (in addition to those in stage 1) are relevant to visibility and orchestration, for example:
- Feature delivery velocity and burn-down rate
- Delivery predictability (for example, features planned versus delivered on cost in a program increment)
- Release cycle time (from release planning to production)
- % of pipeline processes automated
- Feature/release risk
- Feature/release rollback rate
- Quality metrics at the feature level (such as functional coverage, released defects/feature, and so on)
- GRC metrics (such as compliance and audit velocity)
DevOps delivery platforms are typically set up and managed by a centralized team. Also, these central teams are typically responsible for technology, governance, and data collection standards and for developing a “reference platform.” However, these platforms are typically instantiated by individual application delivery teams for their own pipelines. (See figure below.)
The model provides application teams with some tool-chain level flexibility at the local level, while enforcing certain standards at the global level. Through data collection, global teams can enforce governance across local pipelines. (See figure below.)
Stage 3: DevOps Value Stream Management (VSM) Platforms
In this stage, teams are primarily focused on the delivery of value and value stream management in the DevOps pipeline. Essentially, this stage raises the level of abstraction and tracking in the DevOps pipeline to the value of the logical components, and managing the flow of this value.
DevOps VSM platforms shift the focus to personas such as product managers, line-of-business leaders, and others who are more interested in tracking the progression of value through the pipeline. This includes the elements of value associated with the logical components that we discussed in stage 2. This implies that such platforms will always have strong integration with value definition and tracking tools, which is where the value associated with the logical payload is defined and managed. The delivery platform implicitly ties the value of logical components to the underlying DevOps events, such as releases. These events are then tied to the physical components of the application, such as builds.
Due to the dependencies between value delivered, and the underlying logical and physical components they are tied to, DevOps VSM platforms are typically layered on top of DevOps delivery platforms—in fact they can connect to a number of underlying platforms used by different teams.
Note that DevOps VSM platforms are not generic value stream management systems. DevOps VSM platforms are specialized for handling the value streams associated with the DevOps lifecycle—from value planning to value delivery, and potentially value realization. There is a distinction between value delivery and realization:
- Value delivery is a hypothesis that attempts to measure the value that will be released to production.
- Value realization is the actual value realized from the system operating in production.
Most DevOps VSM platforms focus on value delivery. While this is helpful, the real benefit from a business perspective lies in tracking value realization. This gets us to the real level of BizOps (see the next section on BizOps).
Like all general VSM platforms, DevOps VSM platforms help to identify bottlenecks in the flow of value and recommend process improvements. Unlike in stage 1 and 2, where bottlenecks are often simply defined in terms of cycle time, in stage 3, such bottlenecks are prioritized based on the value impediment rate, often across multiple application pipelines. In other words, impediments are prioritized based on the value of underlying logical components affected.
The key metrics that are typically measured during this stage (in addition to those in stage 2) are relevant to flow of value. Following are several examples:
- Value flow rate, which tracks the velocity of value delivery
- Value delivery cycle time, tracking lead time from value planning to delivery or realization
- Value delivery attrition (such as delivered value/planned value)
- Value debt (a measure of the gap between value hypothesis and the value delivered)
- Value impediment rate, tracking process efficiency (such as process time/lead time)
Going Beyond DevOps VSM Platforms: BizOps Platforms
As discussed in the previous section, being able to track business value realization is the ultimate aim of value planning and investments. This helps to extend the realm of DevOps into the business, something we call BizOps. (See the “What is BizOps?” page to learn more.)
BizOps includes three key capabilities relative to value management (see figure below):
- Value planning. This is where value is defined. This is typically done in value estimation and planning tools.
- Value delivery. This is where planned value is instantiated and delivered to production. This is typically done as part of DevOps delivery and VSM platforms discussed above.
- Value realization. This is where actual business value is delivered to production and is realized by customers. This is typically done using value tracking tools—such as monetization, consumption, and customer experience platforms—that are integrated with IT operations systems.
BizOps platforms span and integrate all three capabilities, enabling teams to correlate planned value with the value delivered and realized. These platforms provide a new approach to data-driven decision making that connects business operations and technology functions together to drive business outcomes. By successfully employing BizOps, organizations can build a new level of transparency, traction, and trust that drives collaboration across departments. Through BizOps, enterprises can apply maximum resources and efforts to achieving vital business outcomes, such as boosting business growth, enhancing the customer experience, and increasing profitability.
To realize the benefits of BizOps, teams need a new AI-driven approach that augments and even automates decision making within a BizOps framework. In subsequent blogs, we will discuss more about the architecture of BizOps platforms and the key processes and metrics they support. Until then, we hope your DevOps initiatives are adding value.