SEARCH

SEARCH BY CITATION

Keywords:

  • Aleatory;
  • ambiguity;
  • Deepwater Horizon;
  • epistemic;
  • Fukushima;
  • perfect storms;
  • probability black swans;
  • risk analysis;
  • risk management;
  • uncertainties

Abstract

  1. Top of page
  2. Abstract
  3. 1. “THE PERFECT STORM” AND “THE BLACK SWAN”
  4. 2. DETECTING PERFECT STORMS AND BLACK SWANS IN ENGINEERING RISK ANALYSES
  5. 3. MONITORING NEAR-MISSES AND PRECURSORS
  6. 4. CONCLUSION
  7. REFERENCES

Two images, “black swans” and “perfect storms,” have struck the public's imagination and are used—at times indiscriminately—to describe the unthinkable or the extremely unlikely. These metaphors have been used as excuses to wait for an accident to happen before taking risk management measures, both in industry and government. These two images represent two distinct types of uncertainties (epistemic and aleatory). Existing statistics are often insufficient to support risk management because the sample may be too small and the system may have changed. Rationality as defined by the von Neumann axioms leads to a combination of both types of uncertainties into a single probability measure—Bayesian probability—and accounts only for risk aversion. Yet, the decisionmaker may also want to be ambiguity averse. This article presents an engineering risk analysis perspective on the problem, using all available information in support of proactive risk management decisions and considering both types of uncertainty. These measures involve monitoring of signals, precursors, and near-misses, as well as reinforcement of the system and a thoughtful response strategy. It also involves careful examination of organizational factors such as the incentive system, which shape human performance and affect the risk of errors. In all cases, including rare events, risk quantification does not allow “prediction” of accidents and catastrophes. Instead, it is meant to support effective risk management rather than simply reacting to the latest events and headlines.


1. “THE PERFECT STORM” AND “THE BLACK SWAN”

  1. Top of page
  2. Abstract
  3. 1. “THE PERFECT STORM” AND “THE BLACK SWAN”
  4. 2. DETECTING PERFECT STORMS AND BLACK SWANS IN ENGINEERING RISK ANALYSES
  5. 3. MONITORING NEAR-MISSES AND PRECURSORS
  6. 4. CONCLUSION
  7. REFERENCES

1.1. Uncertainties and Excuses

1.1.1. Excuses

Some industries dealing with dangerous products and processes—such as the oil industry—seem to wait for an accident to take risk management measures, and their regulators appear to have the same attitude.(1) They tend to ignore near-misses as if “an inch was as good as a mile” when they should (and could) have done better. The excuse is often that these events are so rare as to be unimaginable, or that data on early natural disasters (e.g., more than 1,000 years old) should not be accounted for.

1.1.2. The Truly Unknown Versus Rare Conjunctions of Known Events that Were Not Envisioned

Indeed, rare events present major risk management challenges in engineering, medicine, geophysics, and many other fields, including finance. Two images, “black swans” and “perfect storms,” have struck the public's imagination and are used—at times indiscriminately—to describe the unthinkable or the extremely unlikely. Unfortunately, they are also stated as reasons for lack of proactive risk management. The question is whether and how the risk of rare events involving these two kinds of uncertainties can be addressed before the fact, based on signals and existing knowledge.

1.1.3. The Stories of Black Swans and Perfect Storms

The Black Swan is a 2007 book by Nassim Taleb focused on extremely rare events that have never been encountered before (to the best of the observer's knowledge) and in principle, cannot be anticipated.(2) He likens those situations to the discovery of black swans by Dutch sailors in Australia in the 17th century, when all swans known to the Europeans were white. He calls “black swans” events that he considers outliers, of extreme consequences, and for which, in his opinion, people find explanations after the fact, but cannot anticipate, much less assess, the risks that they represent. He observes that these risks are often arbitrarily characterized by Gaussian curves. He divides the world of hazard realizations into two regions: “mediocristan,” the middle of such distributions, and “extremistan,” their tails. He concludes that these attempts at descriptions of such uncertainties are hopeless. He bases this mental model on his own story as a trader who faced unexpected financial downturns and crises, which he believes could not have been imagined based on recent experience. One of his points is that quantitative assessments of financial risks, often based on statistics, are useless because they are likely to miss rare events that are not part of the database. He also (rightly) points to the frequent misuse of Gaussian curves that miss many risk characteristics including asymmetries in loss distributions and “fat tails”(3) representing low-probability, high-consequence outcomes.1

The Perfect Storm was a 1997 book by Sebastian Junger(4) later made into a movie by Wolfgang Petersen in 2000. The story is that of a devastating storm in the northern Atlantic that caught some boats by surprise and killed 12 people in October 1991. It was the result of a conjunction of a storm that started over the United States, a cold front coming from the North, and the tail of a tropical storm coming from the South. All three types were known before and occur regularly, but their conjunction is very rare. A fishing boat, whose crew had decided to take the risk of facing the storm, did not anticipate its strength, was caught in a huge wave, capsized, and sank. No one on board survived.

1.1.4. Rare Events of Different Natures, and Different Types of Uncertainties

Both books describe rare events, but of different nature, and represent uncertainties of two different types. “Perfect storms” involve mostly aleatory uncertainties (randomness) in conjunctions of rare but known events. “Black swans” represent the ultimate epistemic uncertainty or lack of fundamental knowledge2 (see, e.g., Refs. 6–8), where not only the distribution of a parameter is unknown, but in the extreme, the very existence of the phenomenon itself.3

In reality, most scenarios involve both types of uncertainties. A good illustration of their combination is described by the story of the thumbtack. A more complex example is that of seismic hazard analysis, where the parameters are uncertain,(13) but given their probability distributions, there remains the corresponding randomness of seismic loads and structural capacities.(14) In the extreme case of true “black swans” and “perfect storms,” different approaches to risk analysis and management need to be used, and decisionmakers may need different types of information (in the case of seismic risk, about earthquakes and structures). The questions are thus: To what extent does risk management depend on the nature of the uncertainties involved and what have we learned about rare events in engineering and their probabilistic risk analysis (PRA) that can be transferred to other fields given these two extreme types of uncertainties?

1.2. Structuring the Analysis

1.2.1. Systematic Identification of Scenarios, Signals, and Warnings

These fundamental problems are squarely addressed in engineering risk analysis through systematic identification of essential functions (e.g., water storage and pumping in a cooling system), and systematic structuring of scenarios, including rare ones, as collectively exhaustive and mutually exclusive conjunctions of events.(15) Obviously, the truly unimaginable cannot be envisioned upfront, but signals (for instance, medical alerts that a new virus has appeared, or new intelligence information) can be observed, suddenly or gradually.4 Reasoned imagination is thus an important part of risk assessment because it implies, first, anticipating by systematic analysis scenarios that have not happened yet, and second, recognizing and communicating these unusual signals.

1.2.2. How to Think About These Two Kinds of Rare Events?

A rare conjunction of known phenomena is an alignment of factors whose possibilities are known separately but whose simultaneity seems extremely unlikely.5 When a combination of weather factors happens at sea, it can result in wave heights and wind forces rarely experienced that some boats cannot survive.6 The probability of each of these factors can be assessed separately or together because they have occurred before in various combinations, and their mechanisms are sufficiently understood. If the events are independent, the probability of a conjunction is simply the product of the marginal probabilities of each event. If they are dependent, the probability of their conjunctions may be severely underestimated by this product.

1.2.3. The Fukushima Tsunami and Nuclear Accident: The Risk of No Risk Analysis

One example of such a conjunction is the magnitude 9 earthquake in Japan in March 2011, where the seismic energy was released in a known subduction zone and caused in the Northeast part of Japan, a tsunami that reached 14 m. Such a combination of events had not occurred in recent times, but had been recorded at least twice in history(17) in that area, in the 9th and 17th centuries.7 The magnitude of the Sanriku earthquake of 869 was estimated at 8.6 and the subsequent tsunami penetrated the land by at least 4 km. The magnitude of the Sanriku earthquake of 1611 was estimated at 8.1 and it caused a tsunami that reached a maximum estimated height of about 20 m. These records apparently were not accounted for in the design of the Fukushima Daiichi nuclear reactors, which depended on sea water for their cooling and were designed for a maximum wave height of 5.7 m.8 Among other accident initiators, the control room and the generators were flooded and some intake pipes were clogged by debris. This load combination was not unimaginable even if at the time of the plant construction, formal PRA methods had not been fully developed. The probability of the load combination and the subsequent risks of a nuclear reactor's failure could have been estimated based on the existing record.(19,20) Instead, “a large number of numerical calculations [were] carried out under various conditions within a reasonable range”(21) and design decisions were made based on recent earthquakes (such as the Chile earthquake and tsunami of 1960). Yet, in 2006, the Japanese authorities estimated that the probability that a tsunami in the Fukushima area could be more than 6 m high was less than 10−2 in the next 50 years.(18) Key issues, of course, were whether this last estimate of the failure risk included all available information (apparently not), was well understood, and judged acceptable, a decision that was made by industry, regulators, and elected officials.

1.2.4. Labeling Risks After the Fact

Among these different metaphors and the mixed uncertainties that they evoke, deciding, for instance, whether the financial crisis that started in 2008 was a “black swan” or a “perfect storm” depends on one's perspective and is not very important after the fact. It may only be an excuse for failing to detect precursors and warning signals. When looking forward at financial regulations, the question is whether to be proactive and reduce the risk at some cost or, instead, reap the immediate benefits—with minor caveats—and rely on risk management after the fact to limit the damage. In reality, such market failures can be described as “brewing bubbles,” whose bursting was likely but the timing was unclear9 (see, e.g., Ref. 22). They also reflect, as discussed further, the risk attitudes of the managers of financial institutions.

1.2.5. Risk Quantification Involving Rare Events Can Seldom be Based on Statistics Alone

The characterization of risks involving low probabilities and large consequences has been the object of a vast field of literature on both statistical and Bayesian definitions of probability.(23-25) Classical statisticians have captured rare events in risk analysis in several ways. In financial markets, they seek correlations between the market at large and the performance of specified assets.(26) In other cases, they use extreme value distributions (such as Ref. 27).10 But statistics alone (and frequencies) can characterize only randomness.(29-31) They are helpful when a phenomenon is relatively stable, the sample size sufficient, and dependencies well understood. But they fail to represent epistemic uncertainties when new or poorly known factors are at play (new economic structure, climatic conditions, or technologies). One then needs Bayesian probability(32-34) to quantify and combine aleatory and epistemic uncertainties.11

1.3. Engineering Risk Analysis in a Nutshell

1.3.1. The Power of Systems Analysis and Probability

Engineering risk management requires an indepth analysis of the system, its functions, and the probabilities of its failure modes. The engineering PRA method was designed to address cases in which failure statistics at the global level were not sufficient to assess the failure risks, including conjunctions of unlikely and often dependent events.(35-40) The scenarios considered involve both epistemic and aleatory uncertainties. The structure of the model is generally based on event trees and fault trees (or as an equivalent, influence diagrams) and an assessment of the outcomes of the different scenarios. Therefore, a full PRA often requires several submodels, and a global assembly model was proposed earlier by Garrick.(41) The data are based on all existing information, in situ data, surrogate data, test data, engineering models, and expert opinions. This type of analysis is especially critical in the development of new systems—nuclear reactors, satellites, oil rigs, etc.—which involve prospective benefits, costs, and risk. The results can be represented by a single risk curve, which is the probability of exceeding annually different levels of losses (i.e., the complementary cumulative distribution function of the outcomes).

1.3.2. Dependencies Are Key Risk Factors: The Role of External Events and Human Errors

A critical feature of the probability of a scenario is the level of dependence among the factors involved.(42) Understanding these dependencies is critical in the analysis of the risks of “perfect storms.” In the failure of engineered systems, external events and human errors that can affect several subsystems (e.g., the failures of several aircraft engines that were maintained at the same airport and were due to a common error) are major sources of failure dependencies. In the same way, the root causes of the tsunami damage to the Fukushima nuclear reactors in 2011 involved design criteria that ignored part of recorded history. Obviously, as mentioned earlier, large tsunamis in that area are not unimaginable, any more than large earthquakes in California.

1.3.3. Human Errors Are Seldom “Black Swans”

Most accidents are rooted in errors, often several of them in the same chain of events, and these behaviors, in turn, are often influenced by the structure, procedures, and culture of the organization.(43) After the fact, the unpredictability of human errors is often invoked. Yet, mistakes and destructive behaviors can hardly be considered unimaginable events causing accidents that are impossible to account for in a risk analysis. Whether they are explicitly included in a risk analysis often depends on the level of detail of scenarios description.12

In fact, human behaviors and errors can and should be included in a system PRA. They are often basic causes of accidents and result from operators’ level of alertness and competence, and each error can make the next one more likely. They are generally rooted in management decisions such as hiring and training practices, work schedule, and most importantly, perhaps, incentives. Including those factors in a risk analysis model requires linking the performance of the physical system to agents’ behaviors, negative or positive,13 rational or not, and these behaviors, in turn, to management factors in order to provide a more complete assessment of the failure risks.14 Examples of such models include the SAM model (“System-Action-Management”(46)) and the WPAM model (“Work Process Analysis”(47)). This analytical structure, applies directly to failures of engineered systems and of medical procedures,(48) but can also be useful to address some of the problems of the financial industry. Many of these behaviors (e.g., in trading) are the direct results of incentives that can be anticipated given the rewards, yet are sometimes dismissed as “black swans” after the fact.

An example of surprises that should not be are cases in which managers set schedule or budget constraints that are too tight and may cause their agents to cut corners in ways that they would probably disapprove if they were aware of it,15 thus increasing significantly the failure probability of the system that they are developing or building.(49,50) An accident of that kind may well be seen as unthinkable because no one has seriously considered ahead of time the link between incentives, constraints, behaviors (some with strange effects), and risks.

1.3.4. Different Dynamics of Risk Perception and Management

By nature, the dynamics of “perfect storms” and “black swans” are thus distinctly different and so are, in general, the corresponding risk management approaches. Anticipating “perfect storms” requires a long-term observation of the records of threat scenario components and a careful evaluation of their marginal and conditional probabilities. The possibilities of combinations can then be taken into account in the design of systems and risk management strategies can be implemented before the fact. By contrast, “black swans” may be unknown a priori, but signals can emerge and have to be properly observed and interpreted to permit a fast reaction. The risk assessment thus involves reasoned imagination and updating of probabilities based on degrees of belief. One example is the 9/11 attacks on the United States, which after the fact were characterized by some as unpredictable,(51) whereas Zelikow as director of the 9/11 Commission, called the misreading of precursors to these events “failure of imagination.”(52) Indeed, similar attempts at attacks using airplanes had occurred before.16 Risk analysis and management, in such cases, involve updating the chances of a failure, an accident, or an attack based on diverse observations, reinforcing and protecting the system, and minimizing damage if the event occurs.

In both cases, the first way to reduce risks—and especially poorly known ones—is the systematic observation and recording of near-misses and precursors.(53-55) Careful observation and informed interpretation of signals are critical to the response to a new virus, a new disease like AIDS, or the emergence of a new terrorist or criminal group and, of course, timing is critical. But monitoring and setting up an effective warning system depends on costs and values. When the signal is imperfect, deciding when to issue an alert involves managing a tradeoff between false positives and false negatives. Therefore, the question is: When to respond to the warning and at what level of risk given the quality of the signal, the lead time, and the consequences of an event?(56) These tradeoffs at the margin thus involve the decisionmaker's utility and willingness to take risks.17

1.3.5. Managing Different Types of Uncertainties: Risk Aversion and Ambiguity Aversion

In all cases (from “black swans” to “perfect storms”), risk management involves by definition decisions under uncertainty that can be addressed by decision analysis.(58,59) The objective of a quantitative PRA is to support these decisions, including reinforcement of the system in a cost-effective way. An engineering system fails when the load exceeds its capacity and at least one of its critical functions is compromised. Single points of failure are not a problem in themselves if the component is robust enough. Otherwise, redundancies can be part of the solution if they provide the capacity required. They may be of little worth, however, if their failures are highly dependent.(60) Furthermore, technical solutions alone may not be sufficient. As a general rule, a global risk management strategy also has to involve organizational factors, including appropriate incentives, information, and training. This is true, for example, in the oil industry, where delays in drilling and operations can cost hundreds of millions per day, but shortcuts can be catastrophic. In most cases, choices depend—often implicitly—on attitudes and preferences of both managers (“principals”) and operators (“agents”), regarding both randomness and epistemic uncertainties.

The problem at the decision stage is how to deal with different types of uncertainties. The classic rationality assumptions require that decisionmakers be indifferent to ambiguity and the nature of uncertainties.(5,32,61) This assumption, however, does not hold for many deciders, who, everything else being equal, prefer to face a lottery that is “firmer” (less epistemic uncertainties), and who are very explicit about it. This attitude reflects ambiguity aversion and represents, somehow, a second-degree reaction to uncertainty.(62,63)

“Black swans,” since they are initially unknown, are not even on the radar screen of risk managers. But as soon as they begin to be perceived, they lead to decision problems that are dominated by epistemic uncertainty, and unless surprises accumulate, gradually involve more randomness. Decisionmakers may choose to treat the situation according to the classic rationality axioms based on “mean” (average) probabilities (which, of course, is not to say the expected value of the outcomes). Alternatively, they may want to account explicitly for epistemic uncertainties. Given a mix of uncertainties, the attitude of the decisionmaker toward risk and ambiguity can be described by more complex preference function(s), for example, two separate utility functions.(6,8,64) The decision problem, at any given time, can then be treated according to the true values of individuals, organizations, and nations.18

1.3.6. Risk Analysis as an Alternative to the “Stuff-Happens” Philosophy (or Words to that Effect)

Risk analysis is thus an alternative to the “stuff-happens” philosophy—ignoring signals or deciding that accidents are “normal” events or are too unlikely to be accounted for. To allow for explicit treatment of both types of uncertainties, one challenge is to represent accurately, in the risk results, the existing information about epistemic and aleatory uncertainties, especially those that result from disagreements among experts.(67) When appropriate, the results can be displayed as a family of risk curves instead of the classic single one showing mean probabilities.(48) Each curve represents a fractile of the probability distribution of the chances of exceeding a given level of loss shown on the x-axis.19 In this way, for instance, the analyst can convey to the decisionmaker the effect of the fundamental uncertainties about models and parameter values.20 This analysis is required for some PRAs (e.g., for nuclear reactors). The computations often have to rely on simulation. One can display, separately if chosen, the failure probabilities of different components.(68)

2. DETECTING PERFECT STORMS AND BLACK SWANS IN ENGINEERING RISK ANALYSES

  1. Top of page
  2. Abstract
  3. 1. “THE PERFECT STORM” AND “THE BLACK SWAN”
  4. 2. DETECTING PERFECT STORMS AND BLACK SWANS IN ENGINEERING RISK ANALYSES
  5. 3. MONITORING NEAR-MISSES AND PRECURSORS
  6. 4. CONCLUSION
  7. REFERENCES

2.1. Engineering Cases

2.1.1. Risks of Failure of Nuclear Reactors

The most classic example of PRA is that of nuclear reactors, whose theory was developed in a landmark study(69) at a time when the risk could only be assessed based on the identification of events combinations that had not happened before but could be anticipated, based in part on the experience of nuclear-powered ships of the Navy that had been developed in the 1950s. The result was thus a mix of aleatory and epistemic uncertainties based on scenarios of “perfect storms.”21 Since then, many applications and advancements of that study have been performed, secondary levels of uncertainty have been displayed and the framework extended to many different fields.

2.1.2. Failure Risk of a U.S. Space Shuttle Due to a Failure of its Thermal Protection System

When the shuttle Columbia exploded in 2003, the size of the gap in the heat shield caused by a hit by a fragment of the external tank insulation and the extent of the damage that it caused were a surprise to the agency. It should not have been. In a 1990 study, the risk of losing a U.S. space shuttle mission due to a failure of the black tiles of the orbiters had been estimated based on a systems analysis, including dependencies among the different subsystems’ performances in failure scenarios(70-72) and the effect of organizational factors on the risk of losing of a mission dues to the tiles.22 This type of heat shield failure had never happened at the time of the study, and for lack of statistical data, the risk was considered tolerable. Yet, the Columbia accident was one of the scenarios explicitly described in this 1990 study. Therefore, one cannot argue that the rationalization of the Columbia disaster happened after the fact. The possibility was identified and the risk was assessed—but, of course, the event was not “predicted”—well before the accident occurred.

2.2. A Medical Case: Patient Risks in Anesthesia

2.2.1. Human Errors and Their Effects Are Not Unpredictable

More complex perhaps are rare scenarios that are based in large part on inexcusable behaviors, some of which are sometimes dismissed after the fact as unimaginable. Consider the case of patient risks in anesthesia. In the Western world, and when performed in large modern hospitals by trained anesthesiologists, the probability of an anesthesia accident is in the order of 1/10,000. These accidents are often rooted in human errors. A dynamic risk analysis was performed, accounting for rare combinations of events and scenarios involving accumulations of mistakes that can cause the start of an accident chain, and failure to diagnose and respond in time.(73-75) The study involved a mix of randomness and epistemic uncertainties that could only be quantified using Bayesian probability to set priorities among risk mitigation policies. The data were based both on statistics gathered in an Australian database and on interviews of a wide spectrum of experts.

2.2.2. The Results Did Not Reflect the Original Concerns

This work was originally motivated by fears of substance abuse among practitioners—residents using anesthetics for recreational purposes, or alcoholism among older anesthesiologists. As it turned out, the problems were much closer to home and the most effective measures appeared to be better monitoring of residents and regular retraining and recertification. There was nothing unimaginable there, but mostly predictable, dependent mistakes that could be addressed by improving the management of the hospital system.

2.3. Bankruptcies in the Property and Casualty Insurance Industry

2.3.1. A Systems Approach to a Financial Problem

“Perfect storms” in the financial world are often not detected by global statistics because they are rare, and in an ever-changing world, may never have occurred quite in the same way. Therefore, existing statistics may have lost their relevance. A quantitative analysis of failure probability of the engineering type was funded by the insurance industry to try to anticipate potentially disastrous conjunctions. The focus was on the failure probability of property and casualty insurance companies given their age and size.(76,77) There were some statistics about bankruptcy in the preceding 20 years, but there was also a feeling in the industry that the economic, legal, and physical environments were changing and that these statistics did not account for the dangers ahead. Data included observations, but also the opinion of about 20 recently retired CEOs of the industry. The simulation of the possible failure scenarios was based on four (dependent) stochastic processes:

  • • 
    The occurrences of external events such as hurricanes in Florida;
  • • 
    The variation of the size of jury awards, which considerably increased in the United States;
  • • 
    The cycles of hard and soft markets in the insurance industry;
  • • 
    The returns on the insurance industry's investments, depending on the economy.
2.3.2. Effects of a Company's Age and Size on the Failure Risk

The results linked the size and the age of a company to its probability of failure in a given timeframe (e.g., 10 years). They also showed that the pricing of premiums was a critical factor in the companies’ performance, and that yielding to the pressures of softening markets was a dangerous thing to do.23 Yet, it may require a certain strength and size to resist these cyclical market forces. The results were then compared to the statistics provided by rating agencies. It was found that, in fact, they were rather similar for the past few years, which was not surprising given that the conditions that were modeled as a system included past circumstances. But for the years ahead the results showed that performances could diverge a great deal, precisely because the model accounted—among other scenarios—for “perfect storms” that could well happen (and in fact did). Again, as usual, there is no claim here that failures were “predicted” but that the results covered a much wider range of scenarios combining dependent events than statistics provided.

3. MONITORING NEAR-MISSES AND PRECURSORS

  1. Top of page
  2. Abstract
  3. 1. “THE PERFECT STORM” AND “THE BLACK SWAN”
  4. 2. DETECTING PERFECT STORMS AND BLACK SWANS IN ENGINEERING RISK ANALYSES
  5. 3. MONITORING NEAR-MISSES AND PRECURSORS
  6. 4. CONCLUSION
  7. REFERENCES

Successful risk management often relies on quick reactions to signals, improbable as they may look, either by acting immediately or gathering further data as quickly as possible. This is true in the case of anesthesia accidents, oil drilling blowups, space systems mishaps, or losses to banks attributable to a single trader's activities. Sometimes, however, the information is not communicated or believed at the decision level.24

New viruses and diseases are perhaps the most striking illustration of a new hazard whose consequences depend directly on the observation and reaction time of the health system across the world. The AIDS epidemic was detected in the United States by the Center for Disease Control in 1981, and given that it had probably been spreading for decades, the response was relatively slow.(78) By contrast, in 2009, the reaction to the H1N1 flu virus was quick and the epidemic was controlled early.

In a different world, an intelligence signal can come as a bolt out of the blue and ignored as too improbable to be actionable. History abounds with such examples. The Federal Bureau of Investigation (FBI) in 2000–2001 did not believe, above a certain level in the hierarchy, that unlikely (and dangerous) pilots were taking flying lessons on large aircraft, and the U.S. intelligence community missed the attacks of 9/11. Another (but very different) example of denial was that of Stalin in 1941, ignoring intelligence warnings of an impending German invasion of the USSR, when all intelligence converged toward that conclusion. In the best case, when such a signal appears, further information needs to be gathered to confirm or deny what looks unlikely. It is a case in which a Bayesian analysis can be useful to update the probability of what seemed to emerge from the original message.(79,80) The organizational structure and culture then determine the effectiveness of communication and reactions.25 The problem, of course, is to manage a tradeoff between the credibility of the signal (and the severity of the potential event that it reveals) and the risk of a false alert.

4. CONCLUSION

  1. Top of page
  2. Abstract
  3. 1. “THE PERFECT STORM” AND “THE BLACK SWAN”
  4. 2. DETECTING PERFECT STORMS AND BLACK SWANS IN ENGINEERING RISK ANALYSES
  5. 3. MONITORING NEAR-MISSES AND PRECURSORS
  6. 4. CONCLUSION
  7. REFERENCES

Whether a rare event is a “black swan” or a “perfect storm” is often in the eyes of the beholder and may not matter that much in practice. Problems arise when these terms are used as an excuse for failure to act proactively. As stated by Augustine (“Law” XLV(81)): “One should expect that the expected can be prevented, but the unexpected should have been expected.” Clearly, one cannot assess the risks of events that have really never been seen before and are truly unimaginable. In reality, there are often precursors to such events. The best approach in that case is thus a mix of alertness, quick detection, and early response. By contrast, rare combinations of known events that can place heavy loads on human or technical systems can be anticipated and their probabilities assessed based on a systematic risk analysis anchored in history and fundamental knowledge. Risk management procedures can then be designed to face these events, within limits of risk tolerance and resource constraints. In any case, “it was a ‘black swan’ or “a ‘perfect storm’” is not an excuse to wait until a disaster happens to take safety measures and issue regulations against a predictable situation.

Footnotes
  • 1

    Normal (Gaussian) distributions are generally insufficient to describe such uncertainties because they assume symmetry and do not generally represent the “fat tails” of loss distributions. Gaussians are sometimes used to add a “fudge factor” to a quantity instead of representing the actual state of information about it, without fundamental reason for the distributions’ shape and symmetry.

  • 2

    The distinction between the two types of uncertainties can be illustrated in its simplest form by the case of a thumbtack.(5) Before it is thoroughly tested, there is an uncertainty about the probability that it will fall in the next throw on its head or on its side (epistemic uncertainty). Even if one knows that probability with certainty, aleatory uncertainty remains in the outcome of the next throw, which still involves the randomness of the process.

  • 3

    Other approaches to the treatment of uncertainties have been proposed; for example, Shafer(9) relies on belief functions that include a range of ignorance and Smets et al.10) propose in addition the use of fuzzy sets; in his seminal work on uncertainty and information, Klir(11) identifies four levels needed in a theory for dealing with uncertainties (from mathematical formulation to the calculus of measurement of uncertainties); Ayyub(12) addresses the question of portfolio optimization through risk decomposition and simulation.

  • 4

    One key problem, of course, is to ensure that these signals are detected, interpreted, and communicated in time to allow for a timely response, for example, the flight training of terrorists prior to the 9/11 attacks.

  • 5

    Reason,(16) for example, describes such a rare combination of events by the image of parallel slices of Swiss cheese, in which the holes must be perfectly aligned for a failure to occur.

  • 6

    It should be noted here that the “perfection” of the storms is in the eyes of the beholder. For each “perfect” storm one can often imagine a worse one. There may also be some devastating “imperfect storms” that do not combine worst-case values for all the factors involved, but are catastrophic nonetheless. The key here is that these factors are not anticipated because their conjunctions seem too rare to care about.

  • 7

    Note that, of course, one cannot claim to have “predicted” this recent earthquake on the sole observation of a two-event record. One could, however, use that information along with the complete known record in a more complex model of probabilistic assessment of the seismic hazard.

  • 8

    It seems that according to Japanese sources, events that occurred more than 1,000 years ago were ignored even if they were known. In the case of Fukushima, deterministic evaluation techniques(18) and truncation of the historical database led to a serious underestimation of the design criterion for tsunamis (5.7 m).

  • 9

    The same is true of the eruption of the Icelandic volcano, which paralyzed the air traffic over the Atlantic and western Europe for a while in 2010. Another one is the failure of the BP Deepwater Horizon platform, which was preceded by several near-misses that were apparently ignored both by the industry and the regulators, simply because an accident had not happened.

  • 10

    See Benjamin and Cornell(28) for extreme value distributions.

  • 11

    Bayesian probability represents a degree of belief based on all available information. It can be used, for example, to compute the probability of natural phenomena such as earthquakes in specific areas where they have been rare but the seismic mechanisms are relatively known and can be further explored. But again, that information does not allow “predicting” whether an earthquake will occur next week or next year any more than statistics do.

  • 12

    In one instance, an operator in the Browns Ferry nuclear reactor caused a fire by trying to locate a leak in a pipe using a candle.(44) Rather than identifying such specific scenarios, the PRA method aggregates them, so that their probabilities can be assessed at a more global level. An analysis of fire risks in nuclear power plants was structured to include technical failures as well as human errors without specifically including all the detailed ways in which such problems can occur.(45)

  • 13

    Positive actions are those in which the actor performs beyond expectations and may save the day by unusually courageous, competent, or effective behavior. An example of these is the decision of a U.S. Airways pilot in 2010 to land an aircraft on the Hudson River instead of attempting an improbable landing at a more distant airport.

  • 14

    Note that in some (stable) cases, the probability and the effects of human behaviors are accounted for in the failure statistics and don't need to be addressed separately.

  • 15

    This was the case, for example, of a construction project in the Central Valley of California in which, under time constraints, the workers did not properly reinforce shear walls, causing structural failures of the houses in the next earthquake.

  • 16

    Some “black swans” are arguably imaginable because similar events point to their possibility. An example is the 9/11 attacks on the World Trade Center in New York, which had not happened before exactly in that form, but a group of terrorists seemed to have a similar plan in 1994 when they took over, in Algiers, an Air France aircraft bound to Paris, were stopped on their way, and killed in Marseille.

  • 17

    One difficulty in the face of very rare but imaginable events is the potential for social amplification of risk (the flip side of ignoring what is too unlikely), and creating public anxieties that may not be justified.(57)

  • 18

    The Europeans, for instance, have in some cases adopted a “precautionary principle” to address the emergence of what they consider the equivalent of black swans, banning the activity or the product until enough information has been gathered, regardless of the potential benefits.(65,66) In recent years, the benefits of that approach have been seriously questioned.

  • 19

    A vertical line cutting the fractile curves at the level of loss x provides a discretization of the probability distribution of exceeding the chosen loss level.

  • 20

    This kind of information is important in two cases at least: if the decisionmaker is ambiguity averse, and if the “experiment” is repeated; for example, several systems of the same kind are in operation or a single system is in operation and having computed the chances of failure in one time unit, one wants to know the probability that it fails in n time units {and E(Pn) ≠ (E[P])n)}.

  • 21

    An example of a “black swan” was perhaps that of the hydrogen bubble that developed at Three Mile Island but whose possible effects (none in reality) could be assessed at the time based on the plant PRA.

  • 22

    For example, shortcuts in maintenance were shown to be linked to time pressures. Also, the dispersion of space centers and the rivalries among them sometimes turned into a competition between their respective contractors, some holding some critical data and delaying their communications, and others doing the maintenance job under added time pressure.

  • 23

    Note that the framework of that study did not involve fraud and other criminal activity, but could have if needed.

  • 24

    Indeed, “black swans” are often in the eyes of the beholders. Such birds, in fact, were probably part of the heritage of natives of Australia before the Dutch discovered them.

  • 25

    In the intelligence world, this involves the flow of information across agencies and the relations between the analysts and the decisionmaker(s)

REFERENCES

  1. Top of page
  2. Abstract
  3. 1. “THE PERFECT STORM” AND “THE BLACK SWAN”
  4. 2. DETECTING PERFECT STORMS AND BLACK SWANS IN ENGINEERING RISK ANALYSES
  5. 3. MONITORING NEAR-MISSES AND PRECURSORS
  6. 4. CONCLUSION
  7. REFERENCES
  • 1
    National Academy of Engineering. Macondo Well-Deepwater Horizon Blowout: Lessons for Improving Offshore Drilling Safety. Washington , DC : National Academy Press, 2011.
  • 2
    Taleb NN. The Black Swan: The Impact of the Highly Improbable. New York : Random House, 2007.
  • 3
    Koosky C, Cooke R. Adapting to Extreme Events: Managing Fat Tails. Washington, DC : Resources for the Future, 2010.
  • 4
    Junger S. The Perfect Storm. New York : W.W. Norton & Company, 2000.
  • 5
    Howard R, Matheson JE (eds). Readings on the Principles and Applications of Decision Analysis. Menlo Park , CA : Strategic Decisions Group, 1984.
  • 6
    Fischbeck P. Epistemic Uncertainty in Rational Decisions. Doctoral Thesis. Department of Industrial Engineering and Engineering Management, Stanford University, CA , 1990.
  • 7
    Apostolakis GE. The concept of probability in safety assessments of technological systems. Science, 1990; 250: 13591364.
  • 8
    Paté-Cornell ME, Davis DB. A challenge to the compound lottery axiom: A two-stage normative structure and comparison to other theories. Theory and Decision, 1994; 37(3):267309.
  • 9
    Shafer GA. Mathematical Theory of Evidence. Princeton : Princeton University Press, 1976.
  • 10
    Smets P, Dubois D, Prade H, Mamdani A. Non Standard Logics for Automated Reasoning. London : Academic Press, 1988.
  • 11
    Klir G. Uncertainty and Information: Foundations of Generalized Information Theory. Hoboken : John Wiley, 2005.
  • 12
    Ayyub BM. Risk Analysis in Engineering and Economics. London : Chapman and Hall/CRC Press, 1993.
  • 13
    Cornell CA. Engineering seismic risk analysis. Bulletin of the Seismological Society of America, 1968; 58(5): 15831606.
  • 14
    McGuire RK, Cornell CA, Toro GR. The case for using mean seismic hazard. Earthquake Spectra, 2005; 21(3): 879886.
  • 15
    Kaplan S, Garrick BJ. On the quantitative definition of risk. Risk Analysis, 1980; 1(1): 1127.
  • 16
    Reason J. Human Error. Camridge , UK : Cambridge University Press, 1990.
  • 17
    National Oceanic and Atmospheric Administration (NOAA), National Geophysical Data Center. Significant Earthquake Database. Washington, DC , 2011. Available at: http://www.ngdc.noaa.gov/hazard.
  • 18
    Institute of Nuclear Power Operations (INPO). Special Report on the Nuclear Accident at the Fukushima Daiichi Nuclear Power Station. INPO 11-005, Atlanta , GA , 2011.
  • 19
    Cornell CA, Newmark N. On the seismic reliability of nuclear reactors. Proceedings of the ANS Meeting. Los Angeles , CA , 1978.
  • 20
    Ellingwood B, MacGregor JG, Galambos TV, Cornell CA. Probability based load criteria: Load factors and load combinations. ASCE Journal of the Structural Division, 1982; 108(5): 978997.
  • 21
    Japanese Society of Civil Engineers (JSCE), Tsunami Assessment Subcommittee, Nuclear Engineering Committee. Tsunami Assessment Method for Nuclear Power Plants in Japan. Tokyo , Japan , 2002.
  • 22
    Paulson H. On the Brink. London : Headline, 2010.
  • 23
    Bier VM, Haimes YY, Lambert JH, Matalas NC, Zimmerman R. A survey of approaches for assessing and managing the risk of extremes. Risk Analysis, 1999; 19(1):8394.
  • 24
    Barker K, Haimes YY. Assessing uncertainty in extreme events: Applications to risk-based decision making in interdependent infrastructure sectors. Reliability Engineering and System Safety, 2009; 94: 819829.
  • 25
    Cooke RM, Nieber D. Heavy-Tailed Distributions: Data, Diagnostics, and New Developments. Resources for the Future Discussion Paper 11–19, Washington, DC , 2011.
  • 26
    Black F, Michael CJ, Scholes M. The capital asset pricing model: Some empirical tests. Pp 79121 in Jensen M (ed). Studies in the Theory of Capital Markets. New York : Praeger, 1972.
  • 27
    Gumbel EJ. Statistics of Extremes. New York : Columbia University Press, 1958.
  • 28
    Benjamin J, Cornell CA. Probability Statistics and Decisions for Civil Engineers. New York : McGraw Hill, 1970.
  • 29
    Fisher RA. On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society, 1922; 222: 309368.
  • 30
    von Mises R. Probability, Statistics and Truth (2nd revised English edition, 1981). New York : Dover, 1939.
  • 31
    Cramér H. Mathematical Methods of Statistics. Princeton Landmarks in Mathematics. Princeton : Princeton University Press, 1946.
  • 32
    Savage L. Foundations of Statistics. New York : Wiley, 1954.
  • 33
    De Finetti B. Theory of Probability. New York : Wiley, 1970.
  • 34
    Winkler RL. Introduction to Bayesian Inference and Decision. New York : Holt Rinehart and Winston, 1972.
  • 35
    Henley EJ, Kumamoto H. Probabilistic Risk Assessment, New York : IEEE Press, 1992.
  • 36
    Bedford T, Cooke R. Probabilistic Risk Analysis. Cambridge : Cambridge University Press, 2001.
  • 37
    Paté-Cornell ME. Finding and fixing systems weaknesses: Probabilistic methods and applications of engineering risk analysis. Risk Analysis, 2002; 22(2): 319334.
  • 38
    Paté-Cornell ME. Rationality and imagination: The engineering risk analysis method and some applications. In Edward, Miles E and von Winterfeldt (eds). Advances in Decision Analysis. Cambridge : Cambridge University Press, 2007.
  • 39
    Paté-Cornell ME. Probabilistic risk assessment. In Cochran JJ (ed). The Wiley Encyclopedia of Operations Research and Management Science. New York : Wiley, 2010a.
  • 40
    Haimes YY. Risk Modeling, Assessment, and Management, 3rd ed. New York : Wiley, 2008.
  • 41
    Garrick BJ. Recent case studies and advancements in probabilistic risk assessment. Risk Analysis, 1984; 4(4):267279.
  • 42
    Mosleh A. Dependent failure analysis. Reliability Engineering & System Safety, 1991; 34(3):243248.
  • 43
    Paté-Cornell ME. Organizational aspects of engineering system safety: The case of offshore platforms. Science, 1990; 250: 12101217.
  • 44
    US Nuclear Regulatory Commission. The Browns Ferry Nuclear Plant Fire of 1975 and the History of NRC Fire Regulations (NUREG/BR-0361). Washington , DC , 2009.
  • 45
    Kazarians M, Siu NO, Apostolakis G. Fire risk analysis for nuclear power plants: Methodological developments and applications. Risk Analysis, 1985; 5(1): 1151.
  • 46
    Murphy DM, Paté-Cornell ME. The SAM framework: A systems analysis approach to modeling the effects of management on human behavior in risk analysis. Risk Analysis, 1996; 16(4): 501515.
  • 47
    Davoudian KJ, Wu S, Apostolakis G. The work process analysis model (WPAM). Reliability Engineering and System Safety, 1994; 45:107125.
  • 48
    Paté-Cornell ME. Uncertainties in risk analysis: Six levels of treatment. Reliability Engineering and System Safety, 1996; 54: 95111.
  • 49
    Garber RG, Paté-Cornell ME, Managing shortcuts in engineering systems and their safety effects: A management science perspective. Proceedings of PSAM8, New Orleans , 2006.
  • 50
    Garber RG. Corner Cutting in Complex Engineering Systems: A Game-Theoretic and Probabilistic Modeling Approach, Doctoral Thesis, Department of Management Science and Engineering, Stanford University, CA , 2007.
  • 51
    Rice C. National Security Adviser Condoleezza Rice talks with Margaret Warner about Iraq, Online NewsHour, September 25, 2002.
  • 52
    National Commission on Terrorist Attacks Upon the United States. The 9/11 Commission Report. Washington, DC: Government Printing Office, 2004.
  • 53
    Bier VM, Mosleh A. The analysis of accident precursors and near misses: Implications for risk assessment and risk management. Reliability Engineering & System Safety, 1990; 27(1): 91101.
  • 54
    National Academy of Engineering. Accident precursor analysis and management. Proceedings of the National Academy of Engineering Workshop on Precursors, National Academy Press, Washington, DC , 2004.
  • 55
    Paté-Cornell ME. Accident precursors. In Cochran JJ (ed). The Wiley Encyclopedia of Operations Research and Management Science, New York : Wiley, 2010.
  • 56
    Paté-Cornell ME. Warning systems in risk management. Risk Analysis, 1986; 6(2): 223234.
  • 57
    Kasperson RE, Renn O, Slovic P, Brown HS, Emel J. The social amplification of risk: A conceptual framework. Risk Analysis, 1988; 8(2):177187.
  • 58
    Raiffa H. Decision Analysis. Reading , MA : Addison-Wesley, 1968.
  • 59
    Paté-Cornell ME, Dillon RL, Guikema SD. On the limitations of redundancies in the improvement of system reliability. Risk Analysis, 2004; 24(6): 14231436.
  • 60
    Howard R. The foundations of decision analysis. IEEE Transactions on Systems, Science and Cybernetics, 1968; 4(3): 211219.
  • 61
    Von Neumann J, Morgenstern O. Theory of Games and Economic Behavior. Princeton : Princeton University Press, 1947.
  • 62
    Ellsberg D. Risk, ambiguity, and the savage axioms. Quarterly Journal of Economics, 1961; 75(4): 643669.
  • 63
    Fishburn P. The axioms and algebra of ambiguity. Theory and Decision, 1993; 34(2):119137.
  • 64
    Davis D. The Effect of Ambiguity on Lottery Preferences, Doctoral Thesis, Department of Industrial Engineering and Engineering Management, Stanford University, CA , 1990.
  • 65
    European Commission. Communication from the Commission on the Precautionary Principle. Brussels , Belgium , 2000.
  • 66
    Andorno R. The precautionary principle: A new legal standard for a technological age. Journal of International Biotechnology Law, 2004; 1:111.
  • 67
    Morgan MG, Henrion M. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. New York : Cambridge University Press, 1990.
  • 68
    Helton JC, Davis FJ, Johnson JD. Characterization of stochastic uncertainty in the 1996 performance assessment for the Waste Isolation Pilot Plant. Reliability Engineering and System Safety, 2000; 69:167189.
  • 69
    US Nuclear Regulatory Commission. WASH-1400, Reactor Safety Study, Washington, DC , 1975.
  • 70
    Paté-Cornell ME, Fischbeck PS. Safety of the Thermal Protection System of the STS Orbiter: Quantitative Analysis and Organizational Factors, Phase 1: The Probabilistic Risk Analysis Model and Preliminary Observations. Research Report to NASA, Kennedy Space Center, 1990.
  • 71
    Paté-Cornell ME, Fischbeck PS. Probabilistic risk analysis and risk-based priority scale for the tiles of the space shuttle. Reliability Engineering and System Safety, 1993; 40(3): 221238.
  • 72
    Paté-Cornell ME, Fischbeck PS. PRA as a management tool: Organizational factors and risk-based priorities for the maintenance of the tiles of the space shuttle orbiter. Reliability Engineering and System Safety, 1993; 40(3):239257.
  • 73
    Paté-Cornell ME, Murphy DM, Lakats LM, Gaba DM. Patient risk in anesthesia: Probabilistic risk analysis, management effects and improvements. Annals of Operations Research, 1995; 67:211233.
  • 74
    Paté-Cornell ME, Lakats LM, Murphy DM, Gaba DM. Anesthesia patient risk: A quantitative approach to organizational factors and risk management options. Risk Analysis, 1997; 17(4):511523.
  • 75
    Paté-Cornell ME. Medical application of engineering risk analysis and anesthesia patient risk illustration. American Journal of Therapeutics, 1999; 6(5): 245255.
  • 76
    Deleris LA. Firm Failure Risk: Analysis of the Property-Liability Insurance Industry, Doctoral Thesis, Department of Management Science and Engineering, Stanford University, CA , 2006.
  • 77
    Deleris LA, Paté-Cornell ME. Failure risks in the insurance industry: A quantitative systems analysis. Risk Management and Insurance Review, 2009; 12(2): 199212.
  • 78
    Regents of the University of California. The AIDS Epidemic in San Francisco: The Medical Response, 1981–1984, Volume IV, an Oral History Conducted in 1993–1994, 1997.
  • 79
    Paté-Cornell ME. Fusion of intelligence information: A Bayesian approach. Risk Analysis, 2002; 22(3):445454. Erratum, 23(2): 423.
  • 80
    McLaughlin J, Paté-Cornell ME. A Bayesian approach to Iraq's nuclear program intelligence analysis: A hypothetical illustration. Proceedings of IA 2005, the International Conference on Intelligence Analysis Methods and Tools, MITRE Corp. Washington, DC , 2005.
  • 81
    Augustine NR. Augustine Laws. American Institute of Aeronautics and Astronautics, Washington, DC , 1996.