“Big Data does offer promise, but it demands context, interpretation and relationships among data points,” CIMS Academic Fellow Mariann Jelinek wrote in our Sept/Oct 2014 issue (1). Now, two years later, Prof. Jelinek observes that while Big Data Analytics has been gaining ground with business decisions where data is more qualitative, overall progress has been slower.
“Virtually every industry might use Big Data,” she says, “because new links and interactions create potential for new business models, new product possibilities, and even new industries. But first, the links and implications and interactions must be understood, the possibilities envisioned and options weighed. These complex decisions with their traps far surpass unaided human decision-making.”
Her article below explains how the CIMS decision process addresses the decision traps that lurk in strategic choices. It describes how CIMS uses the Comparion software package to deploy the Analytical Hierarchy Process. AHP “bookends” the Big Data decision cycle: at the front end, it helps to generate informed consensus among the decision team for the core question and its parameters; at the back end, it displays the logic of the discussion to facilitate choice among a weighted set of options.
“These procedures create a vastly more knowledgeable client team; informed, evidence-based discussion that fuels better choices among options—and a huge advantage for strategic decisions to come,” Prof. Jelinek reports.
Strategic Decisions: Messy, Uncertain, Just Plain Hard
Strategic decisions are hard because they are messy and uncertain, and because they typically address circumstances and consequences that are novel, unfamiliar and possibly contrary to “business as usual.” The characteristics that make strategic decisions hard have been extensively studied, revealing the decision traps that decision makers face.
These traps are hardwired into the human brain—everybody is potentially vulnerable to them. AHP offers a mathematically and logically rigorous tool to overcome and mitigate these hazards (2). Here is how:
The CIMS 8-step decision process draws on decision science research to identify the stages any complex decision must go through. Each step is important, but each is also vulnerable to particular hazards that can derail an effective decision (see illustration, next page).
Very briefly, decision traps are natural cognitive processes that often degrade good decisions. Heuristics and biases are mental shortcuts, “hardwired” into human cognition and used by experts and novices alike as simplifying strategies—but they degrade decision quality. Availability and representativeness heuristics lead us to notice what we expect to see, and analyze complex problems in terms of what is familiar to us whether or not those assumptions are valid.
Also, the first bit of information we see, or our first tentative choice, tends to anchor our expectations; adjustment from this anchor may well be insufficient.
The CIMS decision process deploys additional outside expertise, abundant, real-time evidence, and, via AHP, rigorously comparison of decision factors to mitigate these hazards. Thus, the first step is to identify the right question and frame it properly, perhaps incorporating new ideas and approaches.
Without proper framing, there is simply no way to assess whether information is relevant or not. Novel information or perspectives can initially appear irrelevant or “wrong” in the context of prior assumptions or certainties. Thus, strategic decisions concerning innovation require a means of recognizing what may be newly relevant, or what may undermine prior assumptions.
Similarly, the information must also be sufficient, reliable and timely. The Internet provides a wealth of information, but not all sources are trustworthy. Decision team discussions drive these crucial assessment tasks.
Finally, at the back end of the process, choosing among alternatives is best directed by real evidence and the ability to see which factors influenced the choice. In short, there is an enormous amount to keep track of, and great need for a rigorous, systematic approach that nevertheless allows for intuition, experience, as well as novel ideas and possibilities.
The Analytical Hierarchy Process
The Analytical Hierarchy Process is a formal, logical system to impose discipline on thinking through messy problems. AHP helps decision makers identify relevant factors and capture tradeoffs, rank-ordering elements of the process, and using mathematical algorithms to array the results of facilitated discussion.
CIMS uses a facilitated discussion to break down a question—e.g., which strategic partner to choose—into factors that drive the decision, like specific technical expertise, manufacturing capacity, geographic proximity or presence in a desired market, for example. Systematic, pair-wise comparisons eventually produce a weighted decision model reflecting the opinions of discussion participants.
This preliminary model is the team’s initial best estimate of the strategic arena. Preferences, experience and both quantitative data and qualitative judgments can readily be included in this discussion, and thus in the resulting array of factors deemed to affect the final decision.
AHP is well documented, well researched and has been widely used in corporate settings for about 25 years (3,4). We use the Comparion software package to capture and display discussion results. The software does not “make” the decision; rather, it captures and displays the elements that decision-makers feel are relevant, facilitates their systematic assessment, and then make visible the impact of the various factors on the final decision.
The approach lends itself well to the adjustment and reconsideration of factors’ impact and indeed to testing choices among options through discussion in a live sensitivity analysis.
Book-ending Strategic Decisions
Nowhere is the need for collaborative discussion more critical than when innovating. As innovation and new products and processes become more complex, the elements relevant to a given decision demand in-depth expertise across multiple fields.
Discipline in the decision process comes both from facilitated discussion, to capture different perspectives and opinions, and from subsequent forced-choice, pairwise comparisons among elements that produces the eventual consensus decision model.
This preliminary consensus, the result of the front-end discussion, reflects the decision-makers’ best estimate of what the strategic arena will be like, and also identifies what might change participants’ conclusions. This best estimate can then be tested against real-world data and revised if necessary.
Where the data disconfirms the decision-makers’ best estimate model, that is critical information for strategic decisions and indicates the need to reconsider decision factors. For example, where new technology offers a dramatically shifted cost structure, models assuming old pricing acceptability are called into question. Where alternative distribution channels enable rapid entry by competitors— online sales through the Amazon platform, for example—older business models face challenges.
Front-End Benefits: The major front-end benefits of AHP address early decision traps by expanding decision-makers’ shared understanding of the strategic arena, resulting from facilitated discussion and AHP’s forced-choice consensus. Understanding the factors important to the decision team’s own experts, plus outside experts with additional, informed perspectives, results in every participant being better informed on initial “best thinking” about the decision.
This discussion addresses the framing and prior assumptions in early stages of the illustrated process. This starting point offers crucial preparation for assessing information found in the search that might disconfirm assumptions, offer new opportunities, or identify new threats. The discussion loosens fixed ideas, compares framing assumptions and increases readiness to understand the information that Big Data Analytics will provide.
This “best preliminary thinking” can also point to previously unconsidered information sources or potentially relevant factors, as well as preparing decision-makers to recognize anomalous or disconfirming evidence. In short, it releases decision traps by sharpening the decision team’s ability to understand the information that is gathered.
Back-End Benefits: Once information has been gathered, and assessed—for quantity, relevance and whether it corroborates or disconfirms the team’s “best estimate” model of the strategic environment—AHP facilitates the final decision process by keeping track of the details and helping to highlight how factors affect the outcome.
If the team has a high degree of confidence that they have the information they need but are uncomfortable with the outcome initially recommended by the model, Comparion software facilitates reconsideration of factors and shows precisely why a given option is weighted more positively than another.
A live reweighting option readily demonstrates the impact of changing various factors, and thus helps to assess new technology potentials or price points or market opportunities (for example). Decision-makers can easily see how the many factors they identified previously shape the desired options, and see the effects of biases, assumptions or heuristics. Because the software keeps track, human cognitive limits and hardwired biases are mitigated.
Weighted options and evidence-based tradeoffs link the decision to real-world information and implications for action. A shift in product capability (for example, a diagnostic test that efficiently identifies a wider range of diseases, or distinguishes effectively between drug-resistant and garden variety TB) can raise the value of a hitherto ignored technology, and direct research targets to develop it.
New information about the increasing incidence of dengue fever in the U.S. (both Hawaii and Florida have had indigenously-acquired cases in recent years) can favor development of a vaccine for dengue, which had not previously been considered important.
Pairwise comparisons of the factors, and the live reconsideration that Comparion software supports, discipline the final choice of options. In light of the substantial amount of information gathered—typically in the millions of web pages—AHP’s forced-choice comparisons highlight logical and evidence-based decisions in place of gut feel or intuition.
Because the factors affecting a strategic decision may be numerous, initially unfamiliar, and may hinge on expertise at first unknown to decision-makers, the software’s ability to keep track of factors and highlight their impact is a crucial support for complex decision-making.
Decision Support, Not Automatic Decision-Making:
Strategic decisions are ultimately the responsibility of the decision team, and Comparion software does not make the decisions for them. Instead, the software provides clear and understandable analysis of abundant, relevant and timely information produced by computerized search.
That is to say, Comparion properly disciplines Big Data so that decision makers can understand it and use it to make better, more informed decisions based on the best real-world information available. As I wrote in 2014, the CIMS Directed Big Data Analytics process is far from an “easy button” but clearly more powerful than gut feel, biased decision-making or inadequately sourced strategies.
- Mariann Jelinek; “No Easy Button: The Real Story of Big Data Analytics,” CIMS Innovation Management Report, Sept./Oct. 2014, pp. 7-11.
- Mariann Jelinek, Steve Barr, Paul Mugge, and Richard Kouri; “The Big Data Lever for Strategic Alliances,” Ch. 4 in Open Innovation Through Strategic Alliances, ed. Refik Culpan; Palgrave-MacMillan, 2015.
- Saaty, Thomas L.; Decision Making for Leaders: the Analytic Hierarchy Process for Decisions in a Complex World. Pittsburgh, PA: RWS Publications, 2012. Third Edition, Fifth Printing.
- See also http://expertchoice.com/; and a plethora of YouTube videos on AHP.
Mariann Jelinek, Ph.D., The Richard C. Kraemer Professor of Strategy, Emerita, College of William and Mary, and CIMS Academic Fellow email@example.com