*The authors thank Christoph Zott and two anonymous reviewers for insightful comments on drafts of this article.
Address correspondence to: Michael E. Solt, Lucas Graduate School of Business, San Jose State University, San Jose, CA 5192-0162. Tel.: (408) 924-3422. Fax: (408) 924-3426. E-mail: firstname.lastname@example.org.
New ventures often do not correctly foresee real market opportunities or the best way to address them. How to cope with unforeseen, unpredictable factors, also referred to as unknown unknowns, is critical for new ventures. Findings in the fields of innovation and project management have shown that dealing with the unpredictable requires management approaches different from those used for classical plan-and-achieve-the-target projects. Management approaches for novel initiatives include a combination of trial-and-error learning (i.e., flexible redefinition of the new venture business model as new information emerges) and selectionism (i.e., running multiple parallel trials and choosing the best performing approach ex post). The management approach must be chosen when the venture is set up. This requires a venture management team to diagnose at the outset whether unknown unknowns are present (or possible), although unknown unknowns cannot be identified initially by definition because they emerge over time. Anecdotal testimony from experienced venture managers and project managers suggests they have a feeling for where their knowledge is limited. However, such a claim is controversial. Some researchers think the concept of diagnosing unforeseeable influence factors is an oxymoron. Thus, the research question in this article is this: How can unforeseeable influence factors in a new venture be diagnosed at the outset? Research to date has insufficiently addressed the a priori identification of the type of uncertainty faced by a new venture. Based on models from decision theory, this article suggests dividing the overall problem of structuring the venture into subproblems for which the management team can identify knowledge gaps. Using a case study, the article describes how knowledge gaps were identified for the subareas of a new venture in a real situation and how this diagnosis was used to correctly identify the areas where unknown unknowns lurk. These areas were managed in a different way (i.e., with learning and experimentation) than the other subproblems (i.e., with targets and deadlines). As a result, the venture could successfully respond to unforeseeable events. The results of this study suggest that a decomposition of the overall venture management problem into subproblems is feasible and natural to managers, that a qualitative assessment of knowledge gaps and vulnerability to unknown unknowns is possible, and that a structured, process-like approach can be used to identify subproblems, to determine their uncertainty profiles, and to update the uncertainty profiles. These results are immediately useful to venture management and venture capitalists in setting up the venture's structure for effective response to uncertainty. The results advance research about uncertainty management by offering a systematic set of questions for the diagnosis of unknown unknowns before they can be formally described. The usefulness of this process can be tested further in more formal empirical research.
New ventures often do not correctly foresee real market opportunities or the best ways of addressing them, so they are forced to adapt and modify their approach over time. This is illustrated by a classic quote (Drucker, 1985, p. 189):
When a new venture does succeed, more often than not it is in a market other than the one it was originally intended to serve, with products and services not quite those with which it had set out, bought in large part by customers it did not even think of when it started, and used for a host of purposes besides the ones for which the products were first designed.
This challenge is often reflected in the contracts between venture capital (VC) investors and entrepreneurs: Under high uncertainty, the VC installs control mechanisms that allow for redefinition of the venture's actions in response to unexpectedly emerging events (e.g., Kaplan and Strömberg, 2003, 2004). This is also discussed in the business press (Brokaw, 1991, p. 54) and for ventures within large corporations (e.g., Van de Ven et al., 1999). Adapting to the unpredictable is difficult—many new ventures fail. This is one reason why many VC firms retracted after the 2001 burst of the dot-com bubble and started investing in later-stage, lower-risk ventures (Pricewaterhouse Coopers, 2005).
How to cope with unforeseen, unpredictable factors, also referred to as unknown unknowns or unk unks, is critical for new ventures, for which organizations are set up and investments made before the effects of unforeseen influences are revealed. If founders and managers could anticipate unknown unknowns before they could be identified, they could prepare to respond effectively to whatever was ahead. This article offers a process that founders and managers can apply to diagnose the presence of unknown unknowns before they emerge. The central feature of this process involves decomposing the new venture's aggregate, complex situation into subproblems with different types of uncertainty that can be managed with adapted approaches. As a result, the likelihood of new venture success can be improved.
Why Is This Research Important?
Anecdotal testimony from experienced venture managers and project managers suggests that they have a feeling for where their knowledge is limited. However, such a claim is controversial. Some researchers think the concept of diagnosing unforeseeable influence factors is an oxymoron. Thus, the research question in this article is, How can unforeseeable influence factors in a new venture be diagnosed at the outset? Research to date has insufficiently addressed the a priori identification of the type of uncertainty faced by a new venture. Application of previous theory requires risk identification, and unknown unknowns are too rampant at a new venture's beginning for risk identification to be sufficient.
Findings in the fields of technology management, innovation, and project management in the last decade show that dealing with the unpredictable requires management approaches and supporting systems that differ from those used for classical plan-and-achieve-the-target projects (Mullins and Sutherland, 1998; Thieme et al., 2003). Management approaches for novel initiatives include a combination of trial-and-error learning (i.e., flexible redefinition of the venture business model as new information emerges) (e.g., Lynn, Morone, and Paulson, 1996) and selectionism (i.e., running multiple parallel trials and choosing the best performing one ex post (e.g., De Meyer, Loch, and Pich, 2002; Pich, Loch, and De Meyer, 2002). Management approaches for a new venture must be chosen at the outset. Therefore, choosing approaches that can deal with unforeseen influences requires that the potential for unknown unknowns be diagnosed at the beginning, before they emerge.
Information gap decision theory (Ben-Haim, 2001; Regev, Shtub, and Ben-Haim, 2001) and discovery-driven planning (McGrath, 1995; McGrath and MacMillan, 2000) also aim at taming unforeseen influences. As discussed in the following section, they are not as suitable for new venture situations as the process presented in this article. Keizer, Halman, and Song (2002) showed how risk-diagnosing methodology (RDM) can help large, established firms identify and manage the risk of new product innovation, and they detailed a process that managers can use in this regard. The present article complements Keizer, Halman, and Song (2002) by also developing a process for managerial use, but whereas they concentrated on diagnosing foreseeable uncertainty, the focus here is on what cannot be foreseen. The process offered here is one a new venture team can readily understand and directly apply. Thus, the present article develops a model that builds from current research on unknown unknowns and risk management and that is practical and useful in new venture settings.
The next section discusses the nature of uncertainty, the existing theory of unforeseeable uncertainty, and previously identified methods for managing it. The following section develops the approach for diagnosing unknown unknowns by decomposing the new venture's situation into subproblems. A case study of a start-up, Escend Technologies, explores whether an initial diagnosis of unk unks is possible under plausible circumstances. In the case study, the presence (although not the identity) of unknown unknowns was successfully predicted by diagnosing gaps in the team's knowledge about certain subproblem areas. Based on this case study, a process is outlined for systematically identifying the vulnerability to unforeseeable uncertainty.
An Overview of Previous Research on Uncertainty
Uncertainty is a fundamental concept that has a rich history of examination in economics and management, and its major characteristics are reviewed next. The simplest form of uncertainty is risk, or the possibility of several outcomes for a situation, each with a probability of occurrence that can be measured (from experience or experiments). For example, in roulette, black or red may come up, but which one is not known. Yet, the probability of 25/51 can be assigned to each (1/51 being the probability of a zero).
Knight (1921) pointed out that often probabilities are not known. As Keynes (1937, p. 211) put it, “There is no scientific basis on which to form any calculable probability whatever. We simply do not know.” This more challenging situation is referred to as Knightian uncertainty and sometimes as ambiguity (i.e., the absence of a probability distribution).
Methods have been developed to deal with ambiguity. Savage (1954) introduced the concept of subjective probability and showed that a mathematical treatment is still possible when people guess their own probabilities. Ambiguity also can be represented as a probability distribution over a multitude of possible probability distributions (e.g., Camerer and Weber, 1992), making possible the mathematical treatment of this extended concept of uncertainty. The discipline of project risk management has developed principles of risk identification, risk prioritization and risk management (i.e., preventive, mitigating, and contingent action), and risk incentives (Amit, Brander, and Zott, 1998) which can deal with ambiguity as long as all important factors (although not their values) and ranges of outcomes are known (e.g., Chapman and Ward, 1997; Smith and Merritt, 2002).
However, these concepts do not fully capture the unforeseeable uncertainty faced by a novel venture. They assume the relevant variables and influence factors and their possible outcomes and causal connections are known; only the probabilities are unknown. In a novel venture, management often knows much less and is plagued by “the inability to recognize and articulate variables and their functional relationships” (Schrader, Riggs, and Smith, 1993, p. 73). They actually called their concept ambiguity in contradiction to the use of the term in economics and decision theory. Important parameters and possible outcomes are not known; some things out there are not on the horizon at all. Economists called this difficult state of affairs unawareness or unforeseen contingencies (Kreps, 1992; Modica and Rustichini, 1994); scholars in public policy referred to wicked problems (as opposed to tame problems that researchers know how to analyze) (see Rittel and Webber, 1973), and engineering and project management professionals used the term unknown unknowns (expanding the known unknowns of Knightian uncertainty) or unk unks (Wideman, 1992) a folklore term that has been used for decades in aerospace, electrical machinery, and nuclear power project management.
When a venture develops a new technology or tackles a new market, unknown unknowns are rampant. For example, Sun Microsystem's Java was conceived as a remote control containing an operating system for household devices, but it ended up being a programming language for the World Wide Web—which did not exist when the project began (see Bank, 1995).
Responding to Unknown Unknowns
Existing models from project management can help a new venture respond to unknown unknowns. A project can be defined as a unique interrelated set of tasks with a beginning, an end, and a well-defined outcome (PMI Standards Committee, 1996). A venture is like a project that starts with an entrepreneur's idea, obtains funding, follows agreed on milestones, and ends when a merger, acquisition, initial public offering (IPO), or a transition to a self-financing and profitable ongoing concern has been achieved—or when the business closes.
Consider a model of decision making in a project or a venture with unforeseeable uncertainty (based on Pich, Loch, and De Meyer, 2002; Sommer and Loch, 2004). The model conceptualizes a venture as an outcome, represented by a payoff function Π=Π(ω, A) (e.g., think of Π as the IPO valuation). The project payoff depends on the state of the world ω∈Ω and a chosen set of actions A (which represents what the team and the investors do over the course of the venture).
Ω denotes the set of all possible states of the world relevant to the outcome of a project, with ω=(w1, …, wN) as a generic element. Each parameter wi may take any value from its domain Di. One ω represents one combination of realizations of all parameters. A state of the world may include management team capabilities, resource costs, competitor moves, market demographics, emergence of other technologies, technology difficulty, regulatory changes, and myriad additional influences.
The established discipline of project risk management gives management teams tools to choose a best course of action A*, maximizing the expected payoff E[Π(ω, A)] (or some other risk-adjusted measure). Hedging, buffers, or contingency plans are known tools that maximize the expected payoff in the face of uncertainty. These methods are powerful when all important elements of the state vector ω have been identified and their ranges are known.
Decision theory is concerned with possibly imperfectly observed Markov processes and Bayesian updating (e.g., Lovejoy, 1991; Marschak and Radner, 1972). States of the world, conceptualized events, and probabilities can be mathematically represented with probability spaces. Mathematical entities are not described in full in this article because the purpose is to delineate what diagnosing unk unks entails—not to perform mathematical operations.
However, the presence of unforeseeable uncertainty means an entire set of influences is unidentified: The management team knows only of the existence of the first n influences ωknown=(w1,…, wn). Thus, performance is also conceptualized in a smaller number of dimensions Πknown, a function of fewer variables.
The team is unaware of the unk unks, or unforeseen dimensions, from n+1 to N and therefore is not aware of additional actions that would be available if the team knew of the additional influence dimensions. For these unk unks, the team proceeds under implicit and possibly wrong default assumptions as if they were set to fixed values, which, except by chance, will differ from true values that the project will later encounter. Thus, the unforeseen dimensions are taken as parameters, as given, without being recognized as such.
Circored, a breakthrough facility to convert ore into iron (a key material in steel), highlights the effects of unforeseen influences. Built in Trinidad between 1997 and 1999, it was a joint venture between the U.S. company Cleveland Cliffs and the German company Lurgi Metallurgie (Loch and Terwiesch, 2002). Careful risk management was conducted, but because the facility used radically new technology the required scale-up of the chemical process from lab tests to an industrial facility created unforeseeable uncertainty: Material dust cycles occurred where they should not have; material stuck to pipe walls; material flowed differently than expected. The facility was forced into a two-year trial-and-error learning process to stabilize performance.
Selectionism and Learning
Unforeseeable uncertainty requires methods that go beyond risk management (Williams, 1999). The model previously outlined yields two fundamental management approaches, or combinations thereof, that cover the range of possible responses (Leonard-Barton, 1995; Pich, Loch, and De Meyer, 2002). The first is selectionism, or parallel trials of multiple approaches, observing what works and what does not (without necessarily having a full explanation why) and choosing the best approach ex post. For example, pharmaceutical companies use this approach when investing in back-up molecules for the same target indication to provide insurance if the lead molecule fails. This approach is also possible on a smaller scale within a venture, for example, by trying several configurations, user interfaces, or marketing messages to make sure one of them works successfully (e.g., Beinhocker, 1999; McGrath, 2001; Sobek, Ward, and Liker, 1999).
A key issue for a new venture management team is how to choose between these two management approaches. Of course, costs are important. Sometimes pursuing several solutions in parallel is simply not affordable, or the time delay associated with experimenting and learning is unacceptable. Most often, however, cost differences are either not clear or they are dwarfed by the differences in the value potential that the two approaches offer. Previous work suggests the answer depends on two additional factors: whether significant unforeseeable uncertainty is present and the level of complexity of the initiative. Complexity in a system refers to so many system elements interacting such that the behavior of the whole cannot be predicted from the behavior of the parts (Simon, 1969, p. 195). In a complex venture, events are hard to predict or correctly interpret even in hindsight, because so many influences are conflated that causality is unclear (Kauffman, 1993; Pich, Loch, and De Meyer, 2002; Rivkin, 2000). Complex systems exhibit unexpected behavior that requires vigilance, iteration, and flexibility (Weick and Sutcliffe, 2001). A new venture's complexity can cause unknown unknowns even when no component or influence factor by itself behaves unpredictably.
As an example of a start-up that faces both complexity and unforeseeable uncertainty (Sommer, Loch, and Dong, 2006), consider a company that produces novel chemical products for industrial sewage purification. Product efficacy is greatly influenced by its composition, temperature, humidity, time, container, sense of responsibility of employees, subtle processing differences across producing labs, inconsistencies in the customers' plants and products, process, and technology. All these factors interact in ways that are too complex to be understood. Worse, this list is not complete; not all factors are even known. Therefore, product quality and efficacy are not stable. The company responds to this combination of unforeseeable uncertainty and complexity by pursuing selectionism: It has signed several technology cooperation contracts with institutes and laboratories and routinely tests multiple formulations before launching one, but there are still product failures.
Figure 1 summarizes the circumstances under which each management approach is suggested to have the higher value potential. The decision theory is developed in Sommer and Loch (2004), and empirical support for the theory is offered in Sommer, Loch, and Dong (2006). When both unforeseeable uncertainty and complexity of the venture are low, neither selectionism nor learning is necessary, and neither should be used because each is expensive. Instead, a classical planning approach with risk management is sufficient to achieve a high-performance outcome for the venture.
Complexity pushes for selectionist trials: If the problem is so complex that one cannot plan for a best solution, sending parallel attacks (and at some point settling on the most promising one) is the best course of action, as long as there is little vulnerability to unknown unknowns. If major unforeseeable uncertainty is present but complexity is not high, trial-and-error learning is most appropriate, because it allows the team to adjust to the shifting circumstances and to achieve high performance even under a substantially changed context.
The most difficult situation is when high levels of unforeseeable uncertainty and complexity combine. Learning is no longer the best answer, because even after substantial redefinition a single trial runs the risk of getting stuck on a local optimum, a result that cannot be improved by making further moderate changes but is in a generally inferior region of the search space. Selectionism, on the other hand, is useless if the best parallel trial must be chosen before the unforeseeable uncertainty is resolved and before previously unforeseen influence factors have emerged. Selectionism is the best solution if the parallel trials can all be kept alive until information about the initially unforeseeable factors is revealed (e.g., customer needs or behavior in an unknown market is determined by market testing of fully functioning prototypes, including product mix and promotional mix).
Whether all unknown unknowns have emerged cannot be ascertained until well after the novel product becomes established in the market. However, as the venture gains a deeper and broader understanding of the business, managers can still conclude that unk unks loom less. These predictions have been empirically supported in a study of 65 start-up companies in Shanghai (Loch et al., 2006; Sommer, Loch, and Dong, 2006).
When a management approach has been chosen for the various parts of the venture—planning, selectionism, or learning—management systems must be put in place to achieve effective execution. Appropriate management systems include how to plan, what to monitor, and how to evaluate team members (Figure 2). It is based on Loch, De Meyer, and Pich (2006), in which detailed configurations of management systems adapted to the management of unknown unknowns are proposed. Management systems must differ depending on the chosen management approach (planning, selectionism, or learning). For example, in a part of the venture with low vulnerability to unknown unknowns, a detailed plan can be developed, targets can be set and enforced, progress monitored, and fulfillment evaluated. However, in a learning subproject, the plan and targets may fundamentally change, so only iteration loops can be planned up to the resolution of a major question, and then new and original planning must be undertaken (not only execution and refinement). Thus, only the resolution of the iterations can be monitored (questions answered), and employees cannot be evaluated on success alone (because it is not fully under their control), but the process and effort must also be taken into account.
In a selectionist part of the venture, in which multiple solution candidates are pursued in parallel, information sharing must be encouraged, and criteria must be set for when a trial is considered out of the race (and discontinued) or declared the winner. All participating parties must have a stake in the total outcome or they will not collaborate.
Diagnosing Unknown Unknowns
The previous section summarized the existing knowledge developed about management responses to the presence of unknown unknowns. However, the theory summarized and the management approaches identified require a venture management team to be able to recognize at the outset whether unknown unknowns are present (or possible), although unknown unknowns cannot be identified initially by definition; they emerge over time. Whether the presence of unforeseeable influence factors can be diagnosed at the outset is precisely this article's research question. If the management team diagnoses the potential presence of unknown unknowns, it needs to consider the use of selectionism or learning.
Previous Approaches to Diagnosing Unknown Unknowns
Two proposals have been made in previous work about how to diagnose unk unks. First, information gap decision theory proposes to address severe uncertainty when no probability distributions, or even ranges, for certain important influence variables are known (e.g., the number of clients or future market prices (see Ben-Haim, 2001; Regev, Shtub, and Ben-Haim, 2001). Information gap theory develops a mathematical method of dealing with the situation where the variation of the parameter around its believed value is unbounded.
This corresponds to the management team knowing the elements of the state of the world ω, but not possessing probability distributions, or even ranges, for certain parameters wi. Similar to ambiguity as discussed previously, the variables influencing the venture are known although their ranges and distributions are not. Though useful, this is not sufficient to diagnose the presence of true unforeseeable uncertainty.
Second, discovery-driven planning proposes to explicitly acknowledge that unknown unknowns exist and to uncover them with four analyses (McGrath, 1995; McGrath and MacMillan, 2000): (1) A reverse income statement calculates what market share and revenues must be achieved to reach a given return target; (2) a pro forma operations specification shows key steps for producing desired output and asks whether these steps can be performed with normal process capabilities (or whether heroic feats are required for successful execution); (3) an assumptions checklist compares the plan with experiences in similar situations or with expert advice (e.g., “We assume the average selling price to be around $1.60—is that justified?”) (McGrath, 1995, p. 51); and (4) milestone planning anticipates points at which risks can be eliminated so that the next investment round is justified.
Steps 1 and 2 propose to carefully examine the planned effect of the action set A* on the payoff function Π for inconsistencies: Is this plan economically realistic, or have parameters been overlooked that are required for the causal mapping to be true? Step 3 corresponds to questioning parameters in the performance function Π and the causal effects of the actions built into its structure or whether upon further probing the parameters can be influenced. Step 4 recommends explicitly learning about and eliminating risks as a condition for continuation. This approach is consistent with a body of work that explains how testing hypotheses and examining unexplainable outcomes that contradict initial assumptions can build knowledge and can reduce unforeseeable uncertainty over the course of a venture (e.g., Thomke, 2003).
Discovery-driven planning is useful and influential. The present approach is complementary: What if the lack of knowledge is so severe that management cannot calculate backward from the desired end result or check the consistency of assumptions? Suppose the desired return cannot be translated into a market-share goal or operational milestones because the market is too new and emerging? The approach offered as follows overcomes such limitations.
This article proposes something very basic: Make explicit that the lack of knowledge is so severe, and ask what part of the venture is in danger of being affected—for example, technology, the market, regulations. The case example of Escend Technologies establishes that guiding the choice of management approaches is plausible with these basic questions.
Diagnosing Unknown Unknowns by Decomposing the Problem
The previous discussion suggests that a new venture's performance function Π typically is complex, but the various decision areas in a venture are at least partially decomposable. This article suggests dividing the overall problem of structuring the venture into subproblems of highly interdependent influences within the subproblem, but with fewer interactions across subproblems. In other words, overall performance can be written as a function of K subproblems, Π=f(Π1(S1),…,ΠK(SK)). Each subproblem depends on some subset Si of the entire state of the world variables ω, not on all variables in ω. For example, subproblems relate to different modules of the product or service to be developed, the customer approach, financing structure, and so on. If the overall problem is perfectly decomposable, the sets Si do not intersect, and each subproblem can be solved in isolation. Though perfect decomposability is rare, subproblems are often identifiable that have few interactions with other subproblems (variables in the intersections of the Si).
Decomposability is a key principle of system design in engineering (see, e.g., Suh, 1990). It has also been a topic in organizational theory for 40 years, from principles of specialization and integration to evidence, that organizational structure is closely related to the architecture of the system developed (Sosa, Eppinger, and Rowles, 2004). In a risk-management framework, Keizer, Halman, and Song (2002) decomposed the overall problem of new product innovation into technology, market, finance, and operations domains.
The present article opens the possibility of estimating the potential for unknown unknowns qualitatively without requiring as much information as information gap theory or discovery- driven planning: At the subproblem level, managers can estimate how much knowledge and experience they have, how many of the known influence variables have been encountered before, how many are new, and how many tests have been performed. In other words, a nominal scale of knowledge gaps can often be estimated, which is analogous to the nominal scale used in information gap theory (where it is applied to a specific parameter). This allows the danger of unforeseeable uncertainty to be identified at a broader level than checking the consistency of the team's causal model or probing whether specific parameters are in fact variables. The price of this broader approach is that individual unk unks will not be identified; only their possible presence will be indicated. Still, this can be sufficient to decide whether planning, selectionism, or learning should be used in the new venture.
Summarizing the Unforeseeable Uncertainty Model
Pich, Loch, and De Meyer (2002) proposed a theoretical model that articulated what unforeseeable uncertainty is and derived selectionism and learning as the two fundamental responses to the presence of unk unks. The model described here is an abbreviation of the models proposed by Pich, Loch, and De Meyer (2002) and Sommer and Loch (2004) that are used to motivate the current study. The focus of the current article is on an empirical example of how to diagnose unknown unknowns, which is not addressed in the previous articles.
The mathematical expression provided previously characterizes the overall problem that the new venture faces at the outset and shows where complexity and unforeseeable uncertainty arise. One point the model makes clear is that the n+1 to N unforeseen dimensions ultimately will affect new venture performance, whether or not the team diagnoses them. Unforeseen dimensions are assumed to be taken as given parameters although the team, if aware, might take different actions. Our experience suggests that such situations regularly face venture capitalists who typically sit on six or seven new venture boards of directors while additionally investing in four or five other new ventures. Chief executive officer (CEO) reports are reviewed, suggestions are offered, and decisions and plans are made, but often the CEO's model is accepted as valid and complete when it should be viewed as set of assumptions (The third coauthor's experience suggests that such situations occur with regular frequency.). Although the new venture team and investors might think they are purposefully managing risk, reliance on an incomplete model or an inappropriate management approach can lead to an inferior local optimum—or to failure.
If complexity and unforeseen influences make tackling the overall new venture model difficult, then decomposing the overall problem into subproblems that are addressed individually or in parallel can help improve the new venture's success. Though this is the main extension here of the theory in Pich, Loch, and De Meyer (2002) and Sommer and Loch (2004), this article also offers the team a managerial process for diagnosing unforeseeable uncertainty that is applied to the subproblems.
The process delineated here identifies subproblems relating to customer need, industry readiness, product functionality, cash usage, and several issues involving the venture team. For each subproblem, the appropriate management approach is determined (i.e., risk management or unk unk), and then the subproblem is solved. The process proves to be beneficial in the Escend Technologies case study. Thus, this article starts in theory and ends in practice.
Unforeseen influences undoubtedly affect new project or new venture success. Keizer, Halman, and Song (2002) described how Unilever spent more than $450 million developing and marketing a new laundry detergent using new technology that could damage clothes (which was vigorously pointed out by competitor Procter & Gamble Co.). Although Unilever changed the formula, the new product was unsuccessful. Despite Unilever's experience in consumer products, unforeseen influences can have a huge impact on its new product innovation.
New ventures in consumer products can also face unforeseen uncertainty, as Webvan exemplifies. Based on a simple model, in the late 1990s Webvan set out to change how consumers bought groceries by using the Internet to order home delivery of their selections. Webvan's business model was designed to generate profits by creating operating efficiencies from its website and system of computerized warehouses. However, Webvan miscalculated customer acceptance (which never materialized), the amount of capital needed to build a nationwide warehouse system, and the cost of acquiring, maintaining, and staffing a fleet of vans. Webvan burned through $50 million of VC funding, and although it had a successful IPO with $8 billion in market value, Webvan filed for bankruptcy in July 2001.
These two examples indicate how innovation based on models that do not consider the effects of unforeseen influences will find success elusive. The case study discussed next shows how diagnosing unforeseeable uncertainty was successful for a high-tech company on the brink of failure. Although this case illustrates the usefulness of the approach, achieving general applicability will require further research involving a larger set of firms.
Case Study: Diagnosing the Unforeseeable at Escend Technologies
The theory and evidence described previously does not illuminate whether this reasoning about the diagnosis of unknown unknowns applies in reality, so the approach was applied in an exploratory case study with a Silicon Valley start-up company, Escend Technologies. The case study was prompted by a conversation between Escend's CEO, Elaine Bailey, and her coauthors, two academics with whom she regularly exchanged ideas. Bailey was a general partner of a Silicon Valley VC firm and had just been named CEO to turn around the struggling company. She asked, “What should I do? Some of the problems and urgent moves are crystal clear. But I have no idea where the real fundamental problem lies. Perhaps it's just execution, but perhaps it's something to do with the industry. It's all opaque, like trying to see through a rock.” This situation presented an opportunity to explore the validity of diagnosing unknown unknowns.
A case study was the appropriate research tool, as available theory was insufficient for making predictions about effort required and the effect of diagnosing unforeseeable uncertainty. Bailey and her coauthors met regularly, looked at status reports and company internal and external documents, and made detailed notes of the decision situation, following a participant-observer approach (Yin, 1994, pp. 13, 79).
Escend Technologies was founded in 1999 to enable semiconductor and electronic component manufacturers to connect and collaborate with their extended sales force, the manufacturers' representatives (reps) who sell their components to electronics original equipment manufacturers (OEMs). OEMs sell products to the end consumer. OEMs often perform marketing and system design but outsource component design and manufacturing (Figure 3).
During the 1990s component manufacturers, OEMs, distributors, and contract manufacturers in consumer electronics were becoming increasingly fragmented and disconnected due to outsourcing and globalization. There was no common customer record that tracked new products through the design, prototype, and manufacturing phases across the multiple companies involved.
In response to inconsistent and changing customer need statements, Escend's business model changed several times. The founding team in 1999 was aware that Escend was attempting to exploit a complex industry opportunity, but it conceptualized the opportunity as collaboration among industry players who would want to be part of Escend's business-to-business (B2B) community. This conceptualization was evolving. Figure 4 displays three successive descriptions of Escend by previous CEOs, and it is not an illustration of sloppy market research but rather of the fundamentally unmapped terrain that Escend faced. By mid 2003, Escend was floundering, having burned through $16 million in venture funding and asking for $6 million more to continue operations.
Escend's customer base consisted of 4 manufacturers and 20 rep firms. Sales stagnated a year earlier, and the cash burn rate grew to $650,000 per month. The CEO had no convincing plans for improvement, only vague promises. Each month, the CEO reported progress on the product and pipeline and maintained it was only a matter of time before the company would be on its feet.
When the initial reaction to the $6 million request was not positive, the board (composed of the key investors) thought the message, not the business or management team, was the reason. They rewrote the executive summary and VC presentation and made additional contacts with potential investors but without success. For the company to survive, major changes had to be made, but they were not obvious. In July 2003, Bailey took over as interim CEO to assess the company and to recommend next steps. Bailey's experience running her own rep firm in the 1980s made her the logical choice and meant that she knew the right questions to ask—or that she would discover what they were.
Bailey had three weeks to decide whether to recommend that her VC firm participate in another round of financing. Deciding yes would give Escend a chance to fix its problems, to become successful, and to provide a positive return on her firm's investment. A no decision would end Escend's existence when the remaining cash was spent as well as lose the current investment.
Diagnosing the Unforeseeable
Bailey interviewed each of the 30 employees and the 4 existing customers. Unsatisfied with what she learned, she examined the company's expense reports and telephone records. She soon realized she had inherited a mess. The previous CEO had invited buddies into the management team who drew large salaries but had little interest in growing the business. The application (product) had significant usability and reporting deficits (e.g., in specifying custom reports). The architecture made the application extremely rigid, causing even small changes to generate up to 200 hours of quality assurance (QA) due to software changes cascading to other parts of the code because of many complex interfaces. The Oregon-based design team was naïve and uncooperative. Telephone logs revealed the sales staff spent little time talking to customers and prospects. The $15 million pipeline was grossly inflated, and the actual figure was $256,000. Morale hit rock bottom.
To diagnose the unk unks, she examined each part of the business and identified open issues. In other words, she identified subproblems of the turnaround and evaluated how much the management team knew about them. The subproblems quickly became clear. Some, such as reducing the burn rate and head count, were obvious for an experienced venture capitalist. She realized the fundamental problem area was not within the company; it was a question of, Is this a viable business? Delineating the problem areas, after some consultation and iteration, led to Table 1. It is typical that the areas of highest uncertainties are the ones identified here. However, this is by no means universal and needs to be ascertained case by case.
Table 1. Problem Areas with Uncertainty Profiles
Problem Area (Subproblem)
1. Customer Need (External)
Would customers buy Escend's products? Why? What is the customers' pain?
High potential for unk unks
2. Industry Readiness (External)
No successful collaboration software play; minimal competition, slow adoption
High potential for unk unks
3. Product Functionality (External)
Holes in functionality, too rigid, based on once-per-day batch mode when customers wanted real-time product
Foreseeable, possible unk unk
4. Cash Burn Rate (Internal)
5. Executive Management Team (Internal)
Complacent and dishonest, risk of lawsuit
6. Sales Team (Internal)
Not sufficiently active; inflating results
7. Head Count (Internal)
30 employees (70 at high point, including 50 software developers), lack of performance
8. Geographically Dispersed Operations (Internal)
Sales offices in Alaska, California, Massachusetts, New Hampshire, and Texas; development team in Maine, Nevada, Pennsylvania, and Oregon
9. Development Team (External and Internal)
Either outmoded or inexperienced, uncooperative
10. Support Team (External and Internal)
Strength of the company
Not a problem
Problem area 10 was not a problem at all; rather, the customer support team was a key asset. Problem areas 4 through 9 were relatively straightforward. They contained risks, but what had to be done was quite clear. Problem area 3 was more difficult: How to plug the product's functionality gaps seemed like an area of unknown unknowns. However, a proposed fix was already on the table that would temporarily meet the market requirement for speed and flexibility while the application was completely reworked.
Fundamental knowledge gaps lay in areas 1 and 2. The technology enabling Escend (XML, Rosettanet) had not existed before 2000. XML made collaboration possible from one database to another, and Escend aimed at connecting the global demand creation activities with supply chain management. A few small competitors offered pieces of the solution, but analysts had not yet defined the problems that existed on the demand side of the semiconductor and electronics manufacturing industry, nor were they following the segment. If the market existed, it was still early enough to be a significant player, but because customer needs were undefined, the willingness of the channel players to collaborate on opportunities via common software was uncertain. Therefore, the required functionality was unknown. Customers could articulate their problems but not their needs, and different players would name mutually incompatible benefits because no one understood where the product would ultimately create the most value or who would pay for the product—which is typical in new markets (O'Connor and Veryzer, 2001).
Identifying significant subproblems can be done systematically following a natural structure of the overall problem, but there is no scientific process. Analysis has to be complemented by experience and experimentation. No process can guarantee that all relevant subproblems and potential unk unk areas are identified. Instead, it can greatly improve the management team's odds by searching systematically.
Following the Diagnosis to Let the Unknown Unknowns Emerge
The unk unk problem areas (1 through 3) required Bailey to assess the viability of the business opportunity and to determine if the company was funded and managed correctly: Could the business provide an acceptable venture capital return? This involved an open-ended search with an unknown result and required switching gears as compared with the execution mode that characterized the other problems (and most of what a VC partner usually did). This worked because Bailey was the person in charge. When she became CEO, Escend's morale was so low that no amount of cajoling was effective. After laying off most of the employees, Bailey and the five remaining employees made decisions by consensus (via daily and weekly meetings), creating a cohesive team that accepted both responsibility and accountability.
In parallel to the areas for which a planning approach was used, the knowledge gap around customer needs and the readiness of the industry became a learning project. Bailey and her team reflected and gathered information from multiple parties about these subproblems without knowing what to expect. They might find nothing of significance or something that might fundamentally change the business model—or even a reason to shut Escend down.
During her first three weeks as CEO, Bailey interviewed enterprise firms, end customers, analysts, consultants, VCs who did not invest (not believing in either the management or the business model), managers of different collaboration start-up companies, and academics. She searched and probed for reasons why collaboration solutions had not succeeded, for the needs of the enterprise customers, and for the problems that the complex industry structure really posed.
In face-to-face meetings, people provided useful nonverbal cues—information that could never be obtained by only attending board of directors meetings. Slowly, information emerged but kept changing as the process evolved. Figure 5 exemplifies how Bailey and her team diagnosed the unk unks, erasing and recovering the whiteboard daily.
After three weeks of intense probing, Bailey concluded that (1) demand existed for Escend's product; (2) the potential market was large enough to possibly generate a sufficient return on investment; (3) competitors were far enough behind that the opportunity had not yet been tapped; and (d) Escend represented one of the last remaining large enterprise software plays remaining for VC investors. She recommended that her firm invest additional funds, and in August 2003, Escend closed a $7 million round from two VC firms.
Conclusion of the Case Study
This section summarizes how Bailey managed Escend after the unk unk areas were diagnosed after following a probe-and-learn approach and demonstrates that the initial diagnosis was useful and identified the correct areas of vulnerability.
Tackling the Planning Subprojects with Foreseeable Uncertainty
Managing foreseeable risks represents standard fare for a competent operational VC who knows how to plan and manage resources. In addition to being foreseeable, these risks also have consequences that are primarily internal to the firm, and the typical VC risk management process is sufficient. Bailey threw her experience and forcefulness behind solving the foreseeable risk problem areas (4 through 9). For each, she set targets, timelines, and progress metrics, following time-proven project management principles. She executed this part of the turnaround working part time, delegating and monitoring.
Her first task was to right-size the company. Bailey moved immediately to isolate and remove the top-management team after gathering evidence of negligence (avoiding severance payments). Bailey kept the business development vice president, two customer service managers who were holding the operations together, the human resources manager who helped with the downsizing, and the chief financial officer. During the interviews, every employee told the same story: Each was not at fault but did not know any solutions, and the market, economic times, or 9/11/01 was to blame. Each person's anger showed through, and each broke ranks, revealing to Bailey their information about product problems and top-management integrity issues. On the product development front, she sidelined the Oregon team and tasked them with maintaining the existing application. Fixing the burn rate was a methodical, hunt-search-and-kill process that resulted in slashing all budgets, consolidating operations/eliminating locations, and letting go 25 nonperformers out of 30 employees, securing signed releases in exchange for two weeks' severance pay.
She hired two senior software architects and a database architect to design a solution to product problems. After several analysis and brainstorming sessions, a powerful and elegant solution emerged: Escend would develop a web shell around the functional kernel, which facilitated ease of use and allowed adding functionality (e.g., real-time customized reports) without having to change the base code. In September 2003, the shell could be tested with the first enterprise firms, and they liked it. Escend had solved another hurdle—customer retention.
A Probe-and-Learn Approach to Respond to Unforeseeable Uncertainty
For problem areas 1 through 3, Bailey adopted a learning approach. It proved correct that unknown unknowns resided only in those three areas. Over the next 12 months, two influences that changed Escend's strategy emerged that could not have been anticipated before.
First, Bailey learned the electronic components market was quickly becoming global. For Escend to add value, the technical vertical it served (industry jargon for the end-user industry application) had to be a complex product that was global in nature. This was confirmed in August 2004 by a report from industry experts that Escend commissioned for $40,000. Bailey concluded, “Experts are good at messaging what you already know, but not at what you don't know.” She convened the board together to brainstorm about unk unks for other vertical markets. The global requirement implied a global platform and design-win tracking for component manufacturers bidding to get their components into end products, but developing this platform—another product redesign—would consume precious funds and resources. Thinking potential competitors also would have to redesign their products, Bailey decided to bite the bullet, hopeful that it would put Escend in the lead. The redesign incorporated multiple languages, multiple currencies, and multiple access points per customer but indicated that Escend's target customers and growth strategy had changed.
Second, any firm in the industry network (Figure 3) had limited visibility of the entire network. Component manufacturers, through their rep firms, could not track either sales or the process and, thus, could not be sure that they would receive full and timely payment. OEM buyers cared about the design win but not about tracking it. Wanting to buy at the lowest possible price but having to allow for the profit margin of the contract manufacturers and design-win firms, OEM buyers often used different manufacturers who offered lower costs after the design was won. As a result, component manufacturers were changing the way they sold products, shifting from reps to distributors and taking back some of the activities reps had performed. Therefore, Escend needed to build distribution functionality into the product, and, in October 2003, it produced a prototype that offered shipping and debiting, samples management, and pricing and quoting functionality. Coding would take another 12 months, and the plan was to go live in January 2005.
In late October 2003, Bailey's search for information uncovered another start-up (funded by a competitor VC) whose collaboration software product covered the demand cycle of the industry. The two had the potential of forming a perfect match if their products could be made to work together. Although Bailey was confident the two software products could be made interoperable (this was a problem area with foreseeable uncertainty), the merger fell through because investors of the competing start-up had recently recovered part of their investment and wanted out.
The flexible way of proceeding, including repeated unplanned product changes and three major strategy changes (counting the aborted merger), was very stressful but possible because Bailey, combining the roles of chairman, CEO, and partner of a major investor, was leading it. Whereas Bailey had authority, access to investors, and prior operational experience, the new Escend management team implemented the process. She assembled a new team in late summer 2003 but made further changes—for example, replacing the vice president of sales again in June 2004. Over time, the team became fully engaged in the learning process. Unk unks became a commonly used term reflecting the new mindset Bailey instilled in the company.
Escend's business model slowly crystallized. Figure 6 is from an Escend white paper and stands in sharp contrast to the complex Figure 3. Tracing the flow of the design-win process is easy in Figure 6, and the demand cycle and demand fulfillment (i.e., supply) cycle is also clearly delineated. Over time, the unknown unknowns have emerged and are now known: The industry structure looks understandable, and the effect of actions taken can be traced. Through learning, unforeseeable uncertainty has been transformed into risk.
In fall 2004, Escend seemed to have turned the corner and was becoming an excellent bet for investors. At the end of October, the company obtained another $3 million from existing investors. Daiwa Securities translated the product into Japanese and went into Japan in December 2004 targeting 12 Japanese companies. Customer interest took off, and by the end of June 2005, the sales pipeline had grown to more than 50 companies with cash flow breakeven by the end of the year. Bailey, signaling her confidence, replaced herself as CEO. Escend is still operating in 2007 as this article goes to publication.
Proposing a Process of Diagnosing Unknown Unknowns
Discussion: Identification of Patterns
This case study illustrates that it is indeed possible to diagnose the presence of unknown unknowns as previously proposed: The overall venture management problem was divided into subproblems, and a qualitative assessment of knowledge gaps and vulnerability to unforeseeable uncertainty was made. Although generalizing from a single case study is difficult, it demonstrates that the observation of patterns in light of the theory yields a diagnosis process, the validity of which can be tested in future studies.
Bailey and her team found it natural to divide the venture into subproblems with different types of uncertainty. Though decomposition is widely used in systems engineering and organizational design, its usefulness needed to be verified in this case. The case also shows that a qualitative assessment of general knowledge gaps per problem area is possible; indeed, after the management team had digested the concepts, it made the assessment of knowledge gaps part of its annual reviews. Unknown unknowns were uncovered in the vulnerable problem areas and not in the others.
After being diagnosed, the areas vulnerable to unknown unknowns were attacked with management approaches adapted to the type of uncertainty. As the venture progressed, new information emerged, and the problem pieces and approaches were updated and modified. This process was iterative and gradual, tracking the evolution of the industry (Figures 3 and 6).
How uncertainty changes as information emerges can be represented by uncertainty profiles (De Meyer, Loch, and Pich, 2002). Figure 7 depicts graphically where management sees the greatest amount of uncertainty, whether variations (deviations from planned schedules and budgets), foreseeable uncertainty (where contingent planning and risk management methods apply), or unforeseeable uncertainty (the potential for unk unks). Uncertainty profiles can be drawn for each major problem area of the start-up company with Figure 7 representing how uncertainty evolves over the typical stages of development.
Though planning was a critical foundation in all stages, chaos reigned when Bailey took over. Without planning, a team has no baseline from which to judge deviations; this is strongly demonstrated for major engineering projects in Miller and Lessard (2000). Escend had to completely abandon previous assumptions and to redefine its business model. The uncertainty profile shifted toward foreseen uncertainties when customer needs became clearer and customers reacted positively to targeted product changes. Later, when a clear model was to be executed and Bailey was planning to step down, standard project management methods were sufficient. This evolution of Escend is, in fact, similar to the development stages of typical ventures (seed, early, expansion, and later stages).
A Process for Diagnosing Unknown Unknowns
The manner in which Bailey diagnosed unforeseeable uncertainty, explored and managed the problem areas in parallel using differing approaches, and altered the management style as the uncertainty evolved follows a systematic pattern that is outlined in the diagnosis process in Figure 8. This process uses systematic problem-solving tools, namely categorizing uncertainty by problem area (Table 1), asking probing questions (Figure 5), and evolving the uncertainty profiles (Figure 7). The process of asking probing questions is consistent with empirical observations of strategic alliances. Alliances also represent projects (usually, alliances end when the learning or growth goals have been achieved) that have to navigate unexpected events as new territory is entered. Less structured mutual discovery processes have to stand alongside approaches for well understood parts of the alliance (see Doz, 1996). The term tools refers to structured approaches to solving problems that with suitable modification may be transferable to other ventures or situations.
Bailey identified the problem structure (i.e., What must be accomplished, and what causal connections are understood?) in her initial examination of Escend. Next she broke the overall problem (i.e., How can the return on investment in Escend be maximized?) into subproblems, such as software functionality, management and sales force problems. For each subproblem, she questioned what she knew, estimating knowledge gaps and, thus, the potential for unknown unknowns to emerge. Then she managed the pieces in parallel, using different management approaches according to the uncertainty category: iteration and learning for the pieces that were threatened by unforeseeable uncertainty, and more structured approaches for the pieces with foreseeable uncertainty. Finally, she updated the uncertainty categorization as new information emerged, changes were carried out, and the correct business model crystallized.
The concept of process demands caution. First, although the term process expresses that a systematic search for unk unks is indeed possible, this always requires experience, flexibility, and adaptation to unforeseen events. This can never be reduced to a mechanical and delegated process. Second, this process of diagnosing and managing unforeseeable uncertainty is not free. The VC firm of which Bailey was partner invested a large amount of resources, including making one of its partners CEO. Bailey, in turn, invested a large amount of Escend's resources in the form of management time, budgets for travel and for external investigative reports, and social capital by visiting customers and picking their brains.
Major Research Results
The Escend Technologies case study illustrates how a new venture leader and management team can undertake the diagnosis of unknown unknowns and how the process used was essential in keeping Escend from failing. In particular, the case study suggests that a decomposition of the overall venture management problem into subproblems is feasible and natural to managers, that a qualitative assessment of knowledge gaps and vulnerability to unknown unknowns is possible at the broader subarea level rather than checking individual influences, and that a structured process can be used to identify subproblems, to determine their uncertainty profiles, and to update these profiles.
Theoretical and Managerial Implications
These results are immediately useful to venture management and venture capitalists in setting up the venture's structure for effective response to uncertainty. Building on theoretical models that include unforeseen influences, the results advance research about uncertainty management by offering a systematic set of questions for the diagnosis of unknown unknowns before they can be formally described. The usefulness of this process or set of questions can now be tested in more formal empirical research.
Limitations and Future Research Directions
A key limitation of this study lies in the single case study demonstration of the theoretical arguments. The results cannot claim to be generally applicable. However, the usefulness of the conceptual steps in the process can be empirically tested in research on a larger number of ventures that are vulnerable to unknown unknowns. The components of the diagnosis process in Figure 8 are general enough for wider testing. Future research can examine and perhaps quantify the increased likelihood of survival offered by the process developed here. In addition, general methods for subproblem decomposition can also be analyzed. These avenues of research would sharpen the model and increase its applicability.
A management team that masters diagnosing unknown unknowns and flexibly works through them will still suffer failures (that is the nature of business), but such a team is poised to extract more value from its venture and investments. One cannot control fate, but one can load the dice.
Dr. Christoph H. Loch is the GlaxoSmithKline Chaired Professor of Corporate Innovation and Dean of the Ph.D. Progam at INSEAD. His research revolves around the execution of product innovation. He is also interested in the emotional aspects of motivation and performance in organizations (such as research and development [R&D] organizations). He is a department editor of Management Science and Production and Operations Management and a senior editor of Manufacturing & Service Operations Management, and he serves on the editorial board of the Journal of Engineering and Technology Management. His work has appeared in many leading academic and practice-oriented journals. He has coauthored three books: one on management quality in manufacturing, one on portfolio management in R&D, and one on project management in highly uncertain projects. He consults European corporations on strategy and innovation and serves on the supervisory board of an educational software start-up company. He holds a Ph.D. in business from the Graduate School of Business at Stanford University, an M.B.A. from the University of Tennessee in Knoxville, and a Diplom-Wirtschaftsingenieur degree from the Darmstadt Institute of Technology in Germany. Prior to joining INSEAD, he worked as a consultant for McKinsey & Company in their San Francisco and Munich offices.
Dr. Michael E. Solt is associate dean and professor of finance at the Lucas Graduate School of Business at San Jose State University (SJSU). His work has appeared in many respected academic journals, and he has published articles on corporate exposure to foreign exchange risk, the weekend effect in stock prices, and the effect of inflation on stock prices. His current research interests include entrepreneurship and venture capital development in China. He holds D.B.A. and M.B.A. degrees from Indiana University and an undergraduate degree in finance from The Ohio State University. He serves on the board of directors of the SJSU Research Foundation and Yakima Filters, and he has taught in China, France, and Singapore.
Elaine M. Bailey is the managing partner of The Bailey Group, a Mountain View, California, consulting firm focusing on helping early-stage start-up companies achieve success. She is a serial entrepreneur and venture capitalist with 20 years of experience as a start-up and turnaround chief executive officer, including PaperWise Document Imaging (acquired by Bell+Howell), Applied Medical Informatics (acquired by Merck & Co.), Escend Technologies, and ViewCentral. She was the general manager of Fogdog Sports and Cornerstone Imaging. Her success in starting fast-growth technology companies earned her the positions of vice president of the Michigan Strategic Fund and General Partner at Novus Ventures, both early-stage venture capital firms. She has been the small business advisor to the governor of the State of Michigan and served as a member of the board of directors for the University of Michigan's Entrepreneurial Program and Eastern Michigan University's School of Business. She is a frequent guest lecturer on entrepreneurship and venture capital at Stanford University, the Haas School of Business, University of California–Berkeley, and the Lucas Graduate School of Business at San Jose State University.