BizOps Virtual Summit Videos

    Automated Decision Making: Keys to Boosting Accuracy and Outcomes

    Thought Leader Tom Davenport examines the process of automated decision making within the context of BizOps.

    I first started writing about automated decision making back in 2013. Frankly, not many people paid much attention back then. However, given the pace of change and the continued advancement in technologies, that’s changed a lot in recent years. Particularly with increased interest in artificial intelligence (AI), there’s a lot of focus on how to harness automation in different areas, and decision making is a key part of that. In this post, I examine the process of decision making within the context of BizOps, examining how to look at decision making, apply new AI tools to automate decisions, and improve decision making processes and outcomes. 

    Decision Making Ripe for Improvement

    Human decision making—there’s plenty of evidence that we as humans don’t do a particularly good job of it. Following are a few of the key reasons for this:

    • Decisions aren’t driven by data. There’s been a fair amount of research into the following effect. Managers will say, “Give me data, give me analysis, I want to make a good decision.” But the reality is that, even after a lot of data and analytics are offered, they don’t actually use this data very often. Ultimately, they fall back on their intuition, experience, and biases. Humans prefer to make decisions based on intuition and experience. That’s because this tends to be easier and it doesn’t require you to bring a lot of data together. But it turns out intuition and experience are not very useful as a guide to decisions, at least not unless you make the same decisions on a repeated basis.
    • Decisions are shaped by biases. There’s a well-known field called behavioral economics, which is basically the study of how cognitive biases shape the way we make decisions, particularly economic ones. Doing a Google search on cognitive biases, you’ll find very lengthy lists of biases. The numbers are evolving, but last time I checked there were 124 different kinds of biases that have been shown to affect our decision making.
    • Decision processes and outcomes are often bad. We have a lot of good information about what makes a good decision-making process: involve multiple people, look at it from multiple framing perspectives, and don’t get into a group-think mindset, to name a few.  There’s no shortage of cases where those principles are violated. Decisions often take too long, and get revisited over and over. Sometimes decisions involve too many people, other times too few. Plus, it’s important to recognize that even with good decision processes, the outcomes can still be bad.

    Challenges in Automating Decisions

    Business and technology environments have become so complex, dynamic, and data-rich that humans can’t even really make sense of many key decisions. This is why there’s an increasing move toward decision automation: Many leaders understand that they can make decisions much better if they program analytics and algorithms into the decision process, and enable the human to step back a little. However, in the move toward automated decision making, many encounter obstacles along the way. Following are a few of the challenges that have stifled improved, automated decision making:

    • Resistance from leadership. Particularly when it comes to infrequent, strategic decisions, many senior executives will think, “That’s why I get paid the big bucks. I’m good at those kinds of decisions.” Almost by definition, if it’s a strategic decision, that means a senior manager needs to make it. Executives feel making these decisions is their right, and may not be very enthused about automating them. Forward-looking leaders within the organization will need to say to these executives, “We want you to help us frame the decision and structure, but we want to make decision making as data driven as possible.”
    • Unclear, informal processes. Deciders don’t always know how they decide. Often, leaders may think they’re using data, but they often will fall back on their biases. When I ask executives about the most important decisions they’ve made in recent years, it becomes clear that there aren’t formal processes, and no analysis of the efficacy of the processes in place. How have processes changed? Have they gotten better? Most teams can’t answer these questions with any certainty. 
    • Lack of algorithm transparency. Machine learning models may not be transparent. That means in some industries, such as health care and financial services, teams may not be able to use more complex models. That’s because regulations may prevent using algorithms that don’t enable decision makers to explain how a decision was made. The good thing about rules-based models is they can support these kinds of explanations. The problem is that these models don’t scale very well, which is why many move to machine-learning-oriented approaches.

    Three Levels of Decision Automation

    Historically, we have had quite weak ties between data, information, and knowledge that can be assembled to support a decision. To improve decision making, we have to pull information and decision-making processes closer together. Developing analytics is often the first step, then automation follows. Increasingly, AI and machine learning will play pivotal roles in these advancements.

    Broadcom commissioned a survey of more than 200 business executives with Harvard Business Review Analytics Services. (The findings are available in a report entitled “BizOps: Connecting IT to Business Outcomes.”1 ) The survey found that 87% agreed that human workers using AI to support business decisions helps improve decision making. I think we’ll be seeing more of that in the near future. 

    As teams assess their strategies for improving and automating decision making, it’s helpful to consider the various levels of automation, which are examined below. 

    1. No Automation—“Free Range” Decision Making

    Effectively, this is the tier that most decision-making processes have fallen into. I call it “free range” decision making because we leave virtually everything up to the decision maker, including how to use information, what tools to use, and how to frame the decision in the first place. This is certainly the most common approach, and it’s the easiest from a technology perspective. Effectively, you just tell a manager, “Here are your data warehouse, Excel spreadsheets, and data lake. Maybe some visual analytics to use. Go have fun.” That’s the extent of the technology team’s involvement.

    In the free-range approach, the information and the decision are far apart; it’s up to the decision maker to pull them together, and that doesn’t always happen. That’s a risky approach because many people don’t actually end up using information enough, or in an effective enough way, or even use information at all in many cases. 

    This approach is easy, and provides a lot of autonomy, which decision makers like. This approach also accommodates greater complexity. In establishing automated decision making, it can often be hard to anticipate every different contingency and decision process. With the free-range approach, you can pull in data and tools you need, when you need them. The downside is that, because they’re not very reliant upon automation, these approaches don’t provide as much accuracy as more automated approaches. 

    At the free-range level, very common, very inexpensive tools are used, such as Excel and Tableau. However, it is largely up to the individual to do the integration work required. Historically, a large percentage of decisions have been handled via this free-range approach. This includes higher level, more strategic decisions, such as determining whether to move forward with an acquisition or new product development. 

    2. Semi-Structured Automation Approach 

    In this level, teams tend to have a technology platform that helps lead users through the decision process, pulling the information in that’s needed, when it’s needed. The platform may make a suggestion to the decision maker, but the final decision is always left to the human. 

    Semi-structured approaches may feature a specialized display or dashboard for a specific type of decision process. In this scenario, platforms can also feature scorecards, which can be useful in narrowing down the information involved.

    A semi-structured approach could be used in the medical profession, for example. Based on analysis of the patient, diagnosis, and the disease, the platform could make a recommendation in terms of the drug to prescribe. The doctor may go along with the recommendation or may make an alternative decision. Other typical examples of semi-structured approaches include insurance underwriting and business loan approvals. 

    3. Highly Automated Decision Making

    In this category, decisions are made with little, if any, human intervention. To institute this level of automation, you need some data in order to establish the models, algorithms, and rules that would guide an automated process. At this level, teams rely on some combination of machine learning, rules engines, and workflows. 

    Rules engines are not based on today’s AI technology, but my research indicates about half of large US companies still have some rules engines in place. Today, typical examples of semi-structured automation include pricing, credit authorization, fraud detection, and IT remediation.

    ES_2020_Bizops.com_Blogs_Automated-Decision Making-Keys-to-Boosting-Accuracy-and-Outcomes_936X536

    Image 1: Automated decision making can be broken out into three levels; with each level having a distinct set of use cases and tools.

    Key Considerations

    To start, it is important to be very clear on what decision is being made. If not, you won’t know what information you’ll need to bring to the table. 

    Ultimately, in plotting your automated decision-making strategy, you have to look at each specific decision and determine which approach is best. In general, you do get better decision accuracy and outcomes as you move up on the automation pyramid. 

    However, there are some downsides as you move up the pyramid as well. It’s harder to build automated decision processes—that’s why they are at the top of the pyramid; they are inevitably less common. In the highly automated approach, information and the decision must be perfectly intertwined. This takes a lot of effort and tight process and system integration. These typically require greater sophistication on the part of data scientists and programmers. 

    New Frontiers in Decision Making

    The Starting Point: Frequent and Tactical Decisions

    Historically, much of the focus on automation has been centered on decisions that were frequent and relatively tactical. Pricing is a great example of how automated decision making can work, and generate a lot of revenues. Through internal and external intelligence, teams can easily see what people are paying for products, and start to optimize and automate pricing strategies to maximize both sales and revenues. 

    Initially, staff may complain because they’ve had pricing autonomy in the past. But people who are in charge of ensuring profitability usually see that automated pricing decisions are a lot better. Some opt to allow overrides, but senior executives soon see that those overrides cost the business money, and they stop enabling them. 

    Also, digital marketing is another good example. When it comes to which ad to put on which consumer’s computer, much has become automated. Now, analysis of users’ cookies to identify interests, analysis of ad placement and pricing options, and delivery of targeted advertising can all happen within a few hundred milliseconds. It would be impossible to do this without automation. 

    Other examples include software, where automation is determining whether to release a new version to production based on what’s happening in development and testing. Also, automation is used in IT, where systems automatically restore the network when an issue arises.

    In many companies, these types of repetitive efforts are well on their way to being mastered, though teams can continue to make ongoing refinements. 

    The Next Level: Infrequent, Strategic Decisions

    A.G. Lafley, former CEO of Proctor and Gamble, said during his lengthy tenures as CEO, he only had a few of what he referred to as “big swing” decisions, such as whether to enter a new product category, develop a new product, or acquire a competitor. 

    Moving forward, these infrequent, strategic decisions are going to become increasingly automated, or at least put into a semi-automated workflow. In the realm of IT, these more strategic decisions can include architectural choices. For example, architects may employ automation to determine whether and when to shift computing to the cloud. Over time, they can make more frequent decisions about where to get computing, particularly as cloud services become more commoditized. For a particular workload, architects can ascertain whether it should be run on premises or in the cloud, and, if the cloud, moved to the cloud service provider that is offering the best pricing. 

    New Frontiers: External Data

    Historically, teams and initiatives have largely been focused on internal data. When it comes to internal data, we’re all familiar with enterprise resource planning (ERP) platforms, customer relationship management (CRM) systems, and IT monitoring and management tools, which generate a lot of metrics for how servers and networks are performing. We can also acquire a lot of data from e-commerce systems, particularly around what people are buying. Through most of the history of IT, teams have largely had internal data to work with. 

    It’s great to know what’s happening inside the organization, but what about outside? We may know about customers who buy from us based on data in the CRM, but what about potential customers who haven’t purchased from us yet? 

    The late, great Peter Drucker used to say he didn’t focus much on technology, because information systems of the day tended to be based on internal data. To highlight the contrast, he mentioned examples like Alfred Sloan, the former chairman of General Motors. Sloan used to spend his vacations visiting car dealerships (not how I’d want to spend my vacations), looking to find out what customers were really thinking about GM cars. That’s because they were limited to internal data. 

    Now, if Alfred Sloan were still around, he could actually spend his vacations on the beach, and use social media to see what people were saying about GM cars. He could also look at all the data being generated by connected cars, including how people are driving, where they’re going, and more. 

    All sorts of external data sources have emerged that we can take advantage of, and start to use in decisions that are less frequent and more strategic. Now, through external sources, we can gather data on a range of areas:

    • Prospective customers. You can get data about what consumers think, where they’re buying, and what they’re buying within your categories. Through connected car data, an automobile manufacturer’s teams can see how driving trends have changed, and they can look at search traffic to see whether buyers in a specific segment are starting to research new purchases.
    • Competitors. You can find out about competitors and what products they’re producing. You can get vast amounts of information off job boards, for example, including what types of positions competitors are trying to fill, what kinds of skills they are looking for, and so on. You can also look to determine what kinds of patents they have been filing.
    • Suppliers and markets. The pandemic has disrupted, and put a lot of pressure on, the supply chains of many businesses. More organizations are now trying to move up the supply chain, and determine how much risk the business faces in terms of disruptions. Through external sources, you can find out not only about your suppliers but those businesses who supply your suppliers. You can get information about the broader environment, including business, regulatory, and social factors.

    Conclusion

    It’s an exciting time for improving decision making. Technology is improving at a rapid rate, to the point where technology is really no longer a barrier to effective decision-making processes in many cases. Today, many organizations across a range of industries are successfully leveraging automated decision making. Still, there are a lot of cases in which our decision processes can use a lot of help. Improving them will be an exciting thing to do, and BizOps has a lot of potential in this area, offering a way to start connecting IT and business groups and supporting data-driven decision making. 

    For More Information

    I was able to join a number of industry experts and practitioners at the BizOps Virtual Summit event, where I gave an in-depth presentation on key considerations for establishing automated decision making. To learn more, be sure to visit the BizOps Virtual Summit resource center page. At this page, you can access my complete presentation as well as those of a range of other speakers.

     
     

    1. Harvard Business Review Analytic Services, in association with Broadcom, “BizOps: Connecting IT to Business Outcomes,” June 2020, URL/Link: https://learn.broadcom.com/hubfs/BizOps.Com/Resources/HBR%20BizOps%20Connecting%20IT%20to%20Business%20Outcomes%20Final.pdf