Advances: Engineering Risk Analysis 16

Page 1 of 40

Ch 16 060502 V04

The Engineering Risk Analysis Method and Some Applications M. Elisabeth Paté-Cornell

ABSTRACT Engineering risk analysis methods, based on systems analysis and probability, are generally designed for cases in which sufficient failure statistics are unavailable. These methods can be applied not only to engineered systems that fail (e.g., new spacecraft or medical devices), but also to systems characterized by performance scenarios including malfunctions or threats. I describe some of the challenges in the use of risk analysis tools, mainly in problem formulation, when technical, human and organizational factors need to be integrated. This discussion is illustrated by four cases: ship grounding due to loss of propulsion, space shuttle loss caused by tile failure, patient risks in anesthesia, and the risks of terrorist attacks on the US. I show how the analytical challenges can be met by the choice of modeling tools and the search for relevant information, including not only statistics but also a deep understanding of how the system works and can fail, and how failures can be anticipated and prevented. This type of analysis requires both imagination and a logical, rational approach. It is key to pro-active risk management and effective ranking of risk reduction measures when statistical data are not directly available and resources are limited.

Advances: Engineering Risk Analysis

Page 2 of 40

Ch 16 060502 V04

CONTENTS Engineering Risk Analysis Method: Imagination and Rationality Pro-Active Risk Management Early Technology Assessment and Anticipation of “Perfect Storms” Remembering the Past While Looking Ahead A Brief Overview of the Method and Formulation Challenges The Challenge of Structuring the Model Dynamic Analysis Imagination and Rationality Incomplete Evidence Base Data The Tool Kit Extension of RA to Include Human and Management Factors: The SAM Model Example 1. Ship Grounding Risk: Influence Diagram and SAM Model Representation The Grounding of Oil Tankers or Other Cargo Ships Problem Formulation Based on a SAM-Type Influence Diagram The Overall Risk Analysis Model Example 2. A Two-Dimensional Risk Analysis Model: The Heat Shield of the Space Shuttle Orbiters Example 3. A Dynamic Analysis of Accident Sequences: Anesthesia Patient Risk Example 4. Probabilistic Analysis of Threats of Terrorist Attacks Conclusions

Advances: Engineering Risk Analysis

Page 3 of 40

Ch 16 060502 V04

Engineering Risk Analysis Method: Imagination and Rationality

Risk analysis for well known, well documented and steady-state systems (or stable phenomena) can be performed by methods of statistical analysis of available data. These include, for example, maximum likelihood estimations, and analyses of variance and correlations. More generally, these methods require a projection in the future of risk estimates based on a sufficient sample, of preferably independent, identically distributed data, and other experiences from the past. However, when designing or operating a new type of engineered system, one can seldom rely on such a body of evidence, even though there may exist relevant data regarding parts of the system or the problem. The same is true in all new situations in which the risk can only be evaluated from a rigorous and systematic analysis of possible scenarios, and from dependencies among events in a scenario. For instance, assessing the risk of a terrorist attack on the US requires “imagination” as emphasized in the 9/11 Commission Report (NCTA, 2004). To do so, one has to rely first on a system’s analysis, and second, on Bayesian probability and statistics (e.g., Savage, 1954; Press, 1989). The engineering method of “Probabilistic Risk Analysis” (PRA or here, simply RA), which was designed in the nuclear power industry among other fields (USNRC, 1975; Henley and Kumamoto, 1992; Bedford and Cooke, 2001), was presented in the previous chapter 1 . In what follows, I describe some specific features and applications of the engineering risk analysis method with the objective of finding and fixing system weaknesses, whether technical or organizational 2 (Paté-Cornell, 2000, 2002a). The focus is mostly on the formulation phase of a risk analysis, which can present major challenges. I describe and illustrate four specific problems and possible solutions: the explicit inclusion of human and management factors in the assessment of technical failure risks using influence diagrams 3 , with as an

Advances: Engineering Risk Analysis

Page 4 of 40

Ch 16 060502 V04

example, the case of ship grounding due to loss of propulsion; the characterization of the dynamics of accident sequences illustrated by a model of analysis of patient risk in anesthesia; the treatment of spatially-distributed risks with a model of the risk of an accident caused by a failure of the tiles of the NASA space shuttle; and the challenge of structuring the modeling of a type of threat that is new –at least on the scale that has been recently observed– illustrated by the risks of different types of terrorist attacks on the US.

Pro-Active Risk Management

Early Technology Assessment And Anticipation Of “Perfect Storms” The risk analysis (RA) method used in engineering is based both on systems analysis and probability and allows computation of the risk of system failure under normal or abnormal operating circumstances 4 . More importantly, it permits addressing and computing the risk of “perfect storms”, i.e., rare conjunctions of events, some of which may not have happened yet even though some of their elements may have been observed in the past. These events can affect, for instance, the performance of a new space system faced with a combination of challenges (e.g., a long voyage, cosmic rays, planetary storms, etc.). The same method can be used to perform early technology assessment, which is especially critical in the development of systems such as medical devices, which are expensive to engineer and less likely than not to pass the statistical tests required by the USFDA before approval (Pietzsch et al., 2004). In that context, RA can thus be used to anticipate the effectiveness and the safety of a new medical device when the practitioners may not be accustomed to it, when there may be some design problems, and/or

Advances: Engineering Risk Analysis

Page 5 of 40

Ch 16 060502 V04

when the patients happen to be particularly vulnerable (e.g., premature babies). In a different setting, one can also use this type of analysis to assess the risks of combined factors on a firm’s bottom line, for example, a competitor’s move, a labor strike that affects its supply chain, and/or a dip in demand caused by a major political event. In that perspective, RA can be applied, for instance to the quantification of the risks of bankruptcy in the insurance industry when a company is faced with a decline in market returns, repeated catastrophes, and prosecution of its executives for professional misconduct (Paté-Cornell and Deleris, 2005). Also, as described further, the same RA method can be used to support the choice of counter-terrorism measures, given the limited information provided by the intelligence community, in the face of everchanging situations (Paté-Cornell, 2002b).

Remembering the Past While Looking Ahead Anticipating rare failures, as well as shedding light on mundane but unrecognized problems, can provide effective support for risk management. But there is a clear difference between probabilistic risk analysis and expected-utility decision analysis (e.g., Raiffa, 1968), in which the decision makers are known at the onset of the exercise (Paté-Cornell, 2006). The risk analysis question is often: what are the risks (as assessed by an analyst and a group of experts), and how can the results be formulated to best represent uncertainties and be useful to the eventual decision maker(s)? The key issue, in all cases, is to anticipate problems that may or may not have occurred before, and to recognize existing ones in order to devise pro-active risk management strategies. The engineering risk analysis method permits ranking risk management options and setting priorities in the use of resources. The quantification of each part of the problem by probability

Advances: Engineering Risk Analysis

Page 6 of 40

Ch 16 060502 V04

and consequence estimates allows their combination in a structured way, using both Bayes’ theorem (to compute the probability of various scenarios) and the total probability theorem (to compute the overall probability of total or partial failures). Effective risk management options can then be formulated. They include for instance, adding redundancies, but also, the observation of precursors, i.e., signals and near-misses, which permit anticipating future problems and implementing pro-active measures (Phimister et al., 2004).

A Brief Overview of the Method and Formulation Challenges

The Challenge of Structuring the Model The first step in a risk analysis is to structure the future possible events into classes of scenarios 5 as a set of mutually exclusive and collectively exhaustive elements, discrete or continuous. Each of these scenarios is a conjunction of events leading to a particular outcome. The choice of the model structure, level of detail, and depth of analysis is critical: as one adds more details to a scenario description (A and B and C etc.), its probability decreases. In the limit, the exact realization of a scenario in a continuous space would have a zero probability, making the exercise useless. Therefore, one needs first to formulate the model at a level of detail that is manageable, yet sufficient to identify and characterize the most important risk reduction options. This level of detail may vary from one subsystem to the next. Second, one needs to compute the probability of the outcomes that can result from each class of scenarios, adjusting the level of detail, as shown later, to reflect the value of the information of the corresponding variables as support for risk management decisions. Finally, one needs to quantify the outcomes of these scenarios and to aggregate the results, sometimes as a probability distribution for a single

Advances: Engineering Risk Analysis

Page 7 of 40

Ch 16 060502 V04

attribute (e.g., money), displayed as a single risk curve (e.g., the complementary cumulative distribution of annual amounts of potential damage); or as the joint distribution of several attributes of the outcome space 6 (e.g., human casualties and financial losses). To represent the fundamental uncertainties about the phenomenon of interest, one can display a family of risk curves, which represent a discretization of the distribution of the probability (or future frequency) of exceeding given levels of losses per time unit (Helton, 1994; Paté-Cornell, 1996, 1999b). One can thus represent accident scenarios in various ways. The first is simply accident sequences, starting with initiating events followed by a set of intermediate events leading to an outcome described either by a single measure (e.g., monetary) or by a multi-attribute vector. The distribution of these outcomes allows representation of the risk at various levels of failure severity. Another analytical structure is to identify “failure modes” or min-cut sets, i.e., the conjunctions (without notion of sequencing) of events that lead to system failure described as a Boolean variable (USNRC, 1975). These failure modes account for the structure of the system, e.g., the fact that the failure of a redundant subsystem requires failure of all its components. To model the risk using the accident sequence approach, note p(X) the probability of an event per time unit (or operation), p(X|Y) the conditional probability of X given Y, p(X,Y) the joint probability of X and Y, IEi the possible initiating events of accident sequences indexed in i, and F the (total 7 ) technical failure of the system. In its simplest form, one can represent the result of the PRA model as the probability p(F) of a system failure per time unit or operation as: p(F) = Σi(p(IEi) x p(F|IEi)

(1)

Advances: Engineering Risk Analysis

Page 8 of 40

Ch 16 060502 V04

where p(F|IEi) can be computed as a function of the conditional probabilities of the (intermediate) events that follow IEi and lead to F. The accident sequences can be systematically represented, for instance through event trees and influence diagrams. Alternatively, one can start from the system’s failure modes. Noting Mj these conjunctions of events (min-cut sets), one can write the probability of failure p(F) using the total probability theorem as: p(F) = Σj p(Mj) – Σj Σk p(Mj, Mk) + p (three failure modes at a time) – etc.

(2)

External events that can affect all failure modes (e.g., earthquakes) or the probabilities of specific events in an accident sequence can be introduced in the analysis at that stage. The method is to condition the terms of the equation(s) on the occurrence (or not) of the common cause of failure and its severity level. The choice of one form or another (sequences vs. failure modes) depends on the structure of the available information. In the ship grounding risk analysis model and the risk analysis of a shuttle accident presented later, the accident-sequence structure was chosen because it was the easiest way to think systematically through a collectively exhaustive and mutually exclusive set of failure scenarios. However, faced with a complex system, best described by its functions and by a functional diagram, focusing on the failure modes might be an easier choice.

Dynamic Analysis The robustness of a system as well as the challenges to which it is subjected may change over time. A structure fails when the loads exceed its capacity. On the one hand, one may want to account for the long-term pattern of occurrences of the loads (e.g., earthquakes), as well as the short-term dynamics of the different ways in which such events can unfold, for example, the

Advances: Engineering Risk Analysis

Page 9 of 40

Ch 16 060502 V04

time-dependent characteristics of the pre-shocks, main shock and aftershocks of an earthquake that can hit a structure. On the other hand the system’s capacity may vary as well. It can deteriorate independently from the loads (e.g., by corrosion), or it can decrease because of the fatigue caused by repeated load cycles (e.g., the effect of the waves on a structure at sea). Accounting for variations of loads and capacities requires a knowledge base that may come from different domains, e.g., from geophysics to structural engineering in the case of seismic risks. Another form of dynamic analysis may be required to analyze the evolution of accident sequences in which the consequences depend on the time elapsed between the initiating event and the conclusion of an incident. This is the case of an analysis of risks of fires in oil refineries (Paté-Cornell,1985) as well as that of patient risks in anesthesia described further. In both cases, stochastic processes were used to describe the evolution of the system over time, which is needed when the timing of human intervention is essential to effective risk management.

Imagination and Rationality This RA method has been developed in details in the past for specific cases such as electrical circuits, civil engineering systems, nuclear reactors, aircraft, and space systems. But in its principles, RA as shown further, has applications to many other problems for which one needs to “imagine” systematically, beyond a simple, arbitrary “what-if” exercise, the potential failures in absence of directly relevant experience. In these cases, the choice of evidence is critical because available information may be incomplete and imperfect, yet essential to support a rational decision that needs to be made, before the occurrence of an event such as a specified type of terrorist attack or before a medical device is used in a real setting.

Advances: Engineering Risk Analysis

Page 10 of 40

Ch 16 060502 V04

Imagination and rationality are thus two main bases of the PRA method. Risk analysis is meant to support risk management decisions, assuming a rational decision maker or a homogenous group of them 8 . Rationality is defined here by the von Neumann axioms of decision making (von Neumann and Morgenstern, 1947), and by the definition of probability that they imply 9 . This Bayesian definition differ from the classical frequentist approach in that it relies on a degree of belief based on a decision maker’s willingness to make bets and to choose among lotteries given all available evidence. Therefore, by definition, this kind of probability cannot be “validated” in the classical statistical sense, at least not until one has gathered a sufficient body of experimental data, and provided that the system has remained in a steady state. This is rarely the case in innovative engineering or policy making. Instead, one has to seek justification of the model through a careful presentation of assumptions, reasoning, data and conclusions.

Incomplete Evidence Base The Bayesian RA method is thus at the root of evidence-based decisions 10 , but this does not necessarily imply that the evidence involves a complete set of classic statistical data. Again, this is true because one often has to make such decisions in the face of uncertainty (e.g., in medicine or in astronautics) before complete information can be obtained. Therefore, the method uses all the evidence that exists, imperfect as it may be when needed, as opposed to the “perfect” one that one would want to have to follow the classic statistics path. In effect, the inputs of the RA method, i.e., the best information available, may be subjective and imperfect, but it may be the best one has and the process by which the output is generated is a rigorous one. Since one often needs to use the concept of Bayesian probability based on a degree of belief, the first question is, of course: whose beliefs? At the onset of a risk analysis, the identity

Advances: Engineering Risk Analysis

Page 11 of 40

Ch 16 060502 V04

of the ultimate decision maker is seldom known, it may vary over time, along with the number of incidents, operations, systems, years of operation. Yet, the results have to be complete enough to provide information relevant to decision support under various levels of uncertainties when the event of interest can repeat itself. This implies, in particular that one needs to separate what has been traditionally referred to as “risk” and “uncertainty” but is better described as two kinds of uncertainties, “aleatory”, i.e., randomness, and “epistemic”, i.e., incomplete information about the fundamental phenomenon of interest (Apostolakis, 1990). At the end of the analysis, the probability of an event, in the face of epistemic uncertainty, is the mean future frequency of that event, a measure that is compatible with the maximization of expected utility 11 . As shown further, however, one needs to quantify and fully describe uncertainties about probabilities of various outcomes to allow decision makers to use the risk results in the case of repeated “experiments”.

Data The data that are used in risk analysis thus cover a wide range of sources. In the best of all worlds, one has access to operational data that describe a particular system or phenomenon in its actual setting, e.g., flight data for a space system, or steady-state operating room statistics for a well-known form of surgery. More often, however, in innovative circumstances, one has, at best, surrogate data regarding performance of subsystems and components in a different but similar environment. Other times, one may have to use test data and lab data (e.g., on human performance on simulators). The problem is that tests may have to be performed in an environment that cannot exactly represent the operational one, for instance micro-gravity for spacecraft. When one knows the characteristics of the loads to which the system will be

Advances: Engineering Risk Analysis

Page 12 of 40

Ch 16 060502 V04

subjected and the factors that influence its capacity, one can also use engineering models as a source of information. Finally, when facing new situation with no access to such data, in a first iteration of an analysis, or to supplement existing data, one may need to use expert opinions, provided that the questions have been phrased in such a way that the experts can actually respond based on their experience. Biases in these responses have been widely documented and require all the care of the analyst (e.g., Kahneman et al., 1982). Next, one often faces the unavoidable challenge of aggregating experts opinions, which is easier when the decision maker is known and can inject his own “weighting” (in effect, the equivalent of likelihood functions) in the exercise, and more complex when the risk analysis has to be performed for unknown decisions and decision makers 12 .

The Tool Kit The tools of RA thus include all those that allow description of the problem structure and computation of failure probabilities, in a world that is either static or dynamic. They involve event trees, fault trees, influence diagrams, Bayesian probability and descriptive statistics 13 , but also stochastic processes of various sorts depending on the system’s memory, time dependencies etc. They also include characterization of human errors and of the outcomes of the various scenarios based, for example, on economic analysis. When expanded to the analysis of risk management decisions, the tool kit includes decision trees (and the corresponding version of influence diagrams) and utility functions, single or multi-attribute (Keeney and Raiffa, 1976). Simulation is often needed to propagate uncertainties through the model in order to link uncertainties in the input and those in the output. To do so, one can then use for instance, Monte

Advances: Engineering Risk Analysis

Page 13 of 40

Ch 16 060502 V04

Carlo simulation, or often better, the Latin Hypercube method, which is based on a similar approach but allows for a more efficient search.

Extension of RA to Include Human and Management Factors: The SAM Model Most of the classic risk analyses do include human reliability in one form or another. Human errors may be included in failures or accident scenarios as basic events, or part of the data of component failures. Yet, they are not necessarily an explicit part of a scenario, and often simply weaken a component, e.g., through poor maintenance, which increases a component’s vulnerability. In addition, human errors are often based on management problems, for example, wrong incentives, lack of knowledge on the part of the technicians, or excessive resource constraints. To address these problems, a model called SAM was devised (Murphy and Paté-Cornell, 1996) based first, on an analysis of the system’s failure risk (S). The second level, involves a systematic identification and probabilistic characterization of the human decisions and actions (A) that influence the probabilities of the basic events of the model. Finally, a third level represents the management factors (M) that in turn, affect the probabilities of the human decisions and actions 14 . The main characteristic of the SAM model is thus that it starts with an analysis of the performance of the physical system. This model can be represented by a three-tier influence diagram (see Figure 16.1), in which the influences run from the top to the bottom but the analysis is performed from the bottom to the top. The equations of the SAM model can be described using the notations of Equations 1 and 2, and in addition, noting as p(Lh) the probability of the different loss levels Lh associated with various degrees of technical system failure indexed in h,

Advances: Engineering Risk Analysis

Page 14 of 40

Ch 16 060502 V04

(DAm) the probabilities of the decisions and actions of the different actors, and MNn the relevant management factors that affect peoples decisions and actions.

Management Factor #1

Management Factor #2

MANAGEMENT SYSTEM

Level 3

Decision 1

DECISIONS AND ACTIONS

Decision 2

Level 2

Initiating Event #1

Intermediate Event #1

Intermediate Event #2

Initiating Event #2

Outcomes (e.g., Failure or Loss Levels) PROBABILISTIC RISK ANALYSIS

Level 1

Figure 16.1: Generic influence Diagram representing the structure of the SAM Model.

The SAM equations are: SAM step 1: probability of system failures characterized by levels of losses: p(Lh) = Σi(p(IEi) x p(Lh |IEi)

(3)

SAM step 2: effects of human decisions and actions on p(losses) p(Lh) = Σi Σm p(DAm) x p(IEi|DAm) x p(Lh |IEi, DAm) SAM step 3: effects of management factors on p(losses)

(4)

Advances: Engineering Risk Analysis

Page 15 of 40

Ch 16 060502 V04

p(Lh|MNn) = Σi Σm p(DAm |MNn) x p(IEi|DAm) x p(Lh |IEi, DAm)

(5)

Note that the effects of management factors on the probabilities of losses are assessed through their effects on the probabilities of the decisions and actions of the people involved. Also, we assume here, for simplicity, that the different decisions and actions are mutually independent conditional on management factors (which can be easily modified if needed). In what follows, we present four examples of risk analyses, some at the formulation stage and some with results that include identification of possible risk management options, to illustrate different features of the RA model and, in three cases, of its SAM extension.

Example 1. Ship grounding risk: Influence diagram and SAM model representation

The Grounding of Oil Tankers or Other Cargo Ships The experience with the grounding of the Exxon Valdez in Alaska as well as the breaking at sea of several oil tankers and cargo ships such as the AMOCO-Cadiz off the coasts of Europe, posed some serious risk management problems. Are the best solutions technical, e.g., requiring double hulls, or essentially managerial and regulatory in nature, for instance, increased regulation of maritime traffic and Coast Guard surveillance and/or improvements of the training of the crew? In some cases, one could even imagine drastic options such as blowing up rocks that are too close to shipping lanes. Obviously, the risk depends, among other factors, on the nature of the ship and its cargo, on the skills of its crew, and on the location of maritime routes. Some areas such as the Molucca Strait are particularly dangerous because of the density of international traffic and at times, the anarchic or criminal behavior of the crews. Other sites are especially

Advances: Engineering Risk Analysis

Page 16 of 40

Ch 16 060502 V04

vulnerable because of their configuration such as Puget Sound, the San Francisco Bay, or Prince William Sound.

Problem Formulation Based on a SAM-Type Influence Diagram An analysis of the risks of oil spills due to ship grounding following loss of propulsion can be represented by an influence diagram, expanded to include human and management factors in the SAM format. To support a spectrum of risk management decisions, that diagram can be structured as shown in Figure 2 to include the elements of Figure 1. It represents the sequence of events starting with loss of propulsion, that can lead to a breach in the hull and in the case of oil tankers, release of various quantities of oil in the sea, and possibly, sinking of the ship

Advances: Engineering Risk Analysis

Resource Constraints (time and budget)

Maintenance Quality

Page 17 of 40

Ch 16 060502 V04

Level 3 Personnel Management

MANAGEMENT LEVEL

Level 2

Skill level of the Captain and the crew

HUMAN DECISIONS AND ACTIONS

Level 1 Weather

Loss of Propulsion LP

Uncontrolled/ Controlled Drift

Speed

Grounding

PROBABILISTIC RISK ANALYSIS Final System State e.g., breach in tank?

Source Term: Oil Flow

Location

Figure 16.2: Influence diagram for the risk of grounding of an oil tanker.

The lower part of Figure 16.2 represents the system’s failure risk analysis model. The accident sequence starts with the loss of propulsion at sea (initiating event). Given that this event has happened, the second event is drift control: can the crew control the drift? If not the next event is grounding of the ship: does it happen or not given the speed, the location and the weather? If grounding occurs, the next question is: what is the size of the breach in the hull? It depends on the nature of the seabed or the coast (sand, rocks etc.), on the characteristics of the hull, and on the energy of the shock. Finally, given the size of the breach, the next question is: what is the amount of oil spilled in the water? This outcome depends on the amount of oil carried

Advances: Engineering Risk Analysis

Page 18 of 40

Ch 16 060502 V04

in the first place and on the size of the breach as well as the external response to the incident. The final outcome can then be characterized by the financial loss and the environmental damage measured, for instance, in terms of number animals or length of coastline affected, or in terms of time to full recovery. The middle and upper parts of Figure 16.2 represent the decisions, actions, and organizational roots of the elements of the accident sequence represented in the lower part of the figure. The failure of a ship’s propulsion system starts with its design, but more importantly in operations, with inspection and maintenance procedures. The performance of the crew in an emergency and its ability to prevent grounding depend not only on the skills of its captain but also on the experience of the sailors, and on the ability of the group to work together and to communicate, especially in an emergency. The decisions and actions of the crew may thus depend in turn, on decisions made by the managers of the shipping company who may have restricted maintenance resources, hired a crew without proper training, and forced a demanding schedule that did not allow for inspection and repair when needed. The decisions and actions of crews are treated here as random events and variables conditional on a particular management system. In this example, the evidence base includes mostly statistics of the frequency of loss of propulsion for the kind of ship and propulsion system considered and on expert opinions.

The Overall Risk Analysis Model Based on the influence diagram shown in Figure 2, one can construct a simple risk analysis model represented by a few equations. Note LP (or not: NLP) the event loss of propulsion, and p(LP) its probability per operation; CD (or not: UD) the control of the drift, G (or not: NG) the grounding of the ship; B the random variable for the “final system state” i.e., the

Advances: Engineering Risk Analysis

Page 19 of 40

Ch 16 060502 V04

size of the breach in the hull characterized by its probability density function given grounding fB(b|G); and O (random variable) the “source term”, here, the quantity of oil released characterized by its probability density function fO(o), and by its conditional probability density function fO|B(o|b) given the size of the breach in the hull. Grounding can occur with or without B

drift control. Using a simple Bayesian expansion, the PRA model can then be written as one overall equation to represent this particular failure mode 15 : fO(o) = ∫b p(LP)x {p(UD|LP)x p(G|UD)+p(CD|LP)x p(G|CD)}x fB(b|G)x fO|B(o|b)db B

(6)

Given a total budget constraint (management decision), the maintenance quality can be represented by the frequency and the duration of maintenance operations (e.g., at three levels). Given the management policy regarding personnel, the experience of the crew can be represented by the number of years of experience of the skipper (on the considered type of ship) and/or by the number of voyages of the crew together 16 . These factors, in turn, can be linked to different probabilities of loss of propulsion, and to the probability of drift control given loss of propulsion, using expert opinions or statistical analysis. Numerical data need to be gathered for a specific site and ship, and the model can then be used to compute the probability distribution of the benefits of different types of risk reduction measures. For instance, improving maintenance procedures would decrease the probability of propulsion failure in the first place. Requiring a double hull would reduce the size of the breach given the energy of the shock. Effective control of the speed of the ship would reduce also the energy of the shock in case of grounding. Quick and effective response procedures would limit the amount of oil spilled given the size of the breach in the hull. The model can then be used as part of a decision analysis. This next step, however, also requires a decision criterion, e.g., what level of probability of grounding or an oil spill can be tolerated in the area, or, for a specified

Advances: Engineering Risk Analysis

Page 20 of 40

Ch 16 060502 V04

decision maker, his or her disutility for the outcomes, including both financial factors and environmental effects in a single- or multi-attribute utility function.

Example 2. A Two-Dimensional Risk Analysis Model: The Heat Shield of the Space Shuttle Orbiters In a study of the tiles of the space shuttle’s heat shield, funded by NASA between 1988 and 1990, the problem was to determine first, what were the most risk-critical tiles, second, what were their contributions to the overall risk of a mission failure, and third, what were the risk management options, both technical and organizational that could be considered (Paté-Cornell and Fischbeck, 1993a, 1993b). The challenge was to formulate the problem of the risk of tile debonding and of a “burnthrough” for each tile given its location on the aluminum surface of the orbiter, considering that they are all different, subjected to various loads, and that they cover areas of varying criticality. The key to the formulation was first to determine the nature of the accident sequences, and the way they could unfold. A first tile could debond either because of poor bonding in installation or during maintenance, or because it is hit by a piece of debris. In turn, adjacent tiles could come off under aerodynamic forces and the heat generated by turbulent flows in the empty cavity. Given the size and the location of the resulting gap in the heat shield, the aluminum could then melt, exposing to hot gases the subsystems under the orbiter’s skin. These subsystems, in turn, could fail and depending on their criticality, cause a loss of the orbiter and the crew. Therefore, faced with about 25,000 different tiles on each orbiter, the challenge was to structure the model to include the most important risk factors, whose values vary across the surface: aerodynamic forces, heat loads, density of debris hits and criticality of the subsystems under the

Advances: Engineering Risk Analysis

Page 21 of 40

Ch 16 060502 V04

orbiter’s skin in different locations. The solution was to divide the orbiter’s surface into areas in which the values of these factors were roughly in the same range and to represent this partition on a two-dimensional map of the orbiter 17 . Figure 16.3 is an influence diagram representing the structure of the model.

Figure 16.3: Influence diagram for an analysis of the risk of an accident caused by the failure of tiles of the space shuttle. Source: Paté-Cornell and Fischbeck, 1993a.

Data were gathered from both NASA and its main contractors. Figure 16.4 shows the result of the analysis, i.e., the risk criticality of each tile in different zones (represented by various shades of grey) as measured by its contribution to the probability of mission failure. The main results were that tile failures contributed about 10% of the overall probability of a shuttle accident, and that 15% of the tiles contributed about 85% of the risk.

Advances: Engineering Risk Analysis

Page 22 of 40

Ch 16 060502 V04

.

Figure 16.4 Map of the risk criticality of the tiles on the space shuttle orbiter as a function of their location. Source: Paté-Cornell and Fischbeck, 1993a.

The recommendations to NASA, at the end of the study, were to decrease the time pressure on the maintenance crews, prioritize inspection, and improve the bonding of the insulation of the external tank. Some of them were adopted (e.g., reduction of the time

Advances: Engineering Risk Analysis

Page 23 of 40

Ch 16 060502 V04

pressures), others not (e.g., improvements of the external tank). The key to a robust computation of the risk resided in the Bayesian model structure that was adopted as opposed to relying on the small statistical data sets that existed at the time (e.g., the number of tiles lost in flight). Such a statistical analysis led to unstable results that varied drastically later with the loss of a few tiles when evidence already existed at a deeper level to permit a more stable risk assessment.

Example 3. A dynamic analysis of accident sequences: anesthesia patient risk.

In 1993, a Stanford team was asked to analyze the different components of patient risk in anesthesia, and to identify and estimate changes in procedures that would improve the current situation (Paté-Cornell et al., 1996a, 1996b; Paté-Cornell, 1999a). This project was motivated by the occurrence of several publicized accidents that suggested that substance abuse among practitioners (drugs or alcohol) was a major source of the risk. As we showed, it turned out the reality was much closer to mundane problems of lack of training or supervision. One of the changes at the time of the study was the development of simulators that allowed training first, individuals, then operating room teams together. The focus of the study was on “healthy patients” (e.g., undergoing knee surgery) and trained anesthetists in large Western hospitals. The base rate of death or severe brain damage was in the order of 1/10,000 per operation. Severe accidents, resulting in death or brain damage, occur when the brain is deprived of oxygen for a prolonged duration (e.g., two minutes). The challenge was to structure the model so that the dynamics of accident sequences could be linked to the performance of the anesthesiologists, then to the factors that affect this performance. The data included two types of

Advances: Engineering Risk Analysis

Page 24 of 40

Ch 16 060502 V04

statistics: base rates of anesthesia accidents, and occurrences of different types of initiating events. The latter were the results of the Australian Incident Monitoring study (Webb et al., 1993). Following an initiating event (e.g., disconnection of the tube that brings oxygen to the lungs), the dynamics of accidents was linked to the occurrence of intermediate events (e.g., observation of a signal) as random variables, and to the time that it takes for these intermediate steps, i.e., to observe abnormal signals, diagnose the problem, and take corrective actions, and hopefully, for the patient to recover. Figure 16.5 shows on a time axis the evolution of both the patient and the anesthesia system in the operating room (incident occurrence, signal detection, problem diagnosis and correction). The total time elapsed determines the eventual patient state.

EVOLUTION OF THE ANESTHESIA SYSTEM

i= disconnect Detection and diagnosis phases DISCONNNECT

DETECTION

CORRECTION

Start

PHASE 1 GOOD

TIME End

PHASE 2 DETERIORATION

PHASE 3 RECOVERY

EVOLUTION OF THE PATIENT

Figure 16.5: Evolution of the patient state and of the anesthesia system following the occurrence of an accident initiator such as a tube disconnect. (Sources: Source: Paté-Cornell et al., 1996a)

One challenge was to quantify the durations of intermediate phases (and of different patient states), which were uncertain and were not documented by statistical data at the time of the

Advances: Engineering Risk Analysis

Page 25 of 40

Ch 16 060502 V04

study. They were estimated from expert opinions in order to assess their effects on the results. The analysis was then based on a Markov chain representation of the concurrent unfolding of the incident phases (occurrence and detection of the problem by the anesthesia team) and of the evolution of the patient. The combination was represented by “super states”, for example, “disconnection of the oxygen tube and patient hypoxemia”. The contribution of each possible initiating event to the overall patient risk per operation was then computed based on the probability distribution of the duration of the corresponding type of incident (see Table 16.1).

Table 16.1. Incidence Rates of Initiating Events During Anesthesia from the AIMS Database (Webb et al., 1993) and Effects on Patient Risk (Paté-Cornell, 1999a) Initiating Event

Number Report of AIMS Rate Reportsa

Breathing circuit disconnect 80 Esophageal intubation 29 Nonventilation 90 Malignant hyperthermia n/a Anesthetic overdose 20 Anaphylactic reaction 27 Severe hemorrhage n/a a out of 1,000 total reports in initial AIMS data

10% 10% 10% -10% 20% --

Probability of an Initiating Event 7.2 x 10-4 2.6 x 10-4 8.1 x 10-4 1.3 x 10-5 1.8 x 10-4 1.2 x 10-4 2.5 x 10-5

Relative Contribution To Patient Risk 34% 12% 38% 1% 8% 6% 1%

The factors influencing the patient risks (occurrence of initiating events and duration of intermediate phases) were then linked to the performance of anesthesiologists, based on their level of competence and alertness and on various problems that they can experience. For example, we considered the possibility of “lack of training among experienced anesthesiologists”, which may occur when a senior practitioner who does not operate frequently,

Advances: Engineering Risk Analysis

Page 26 of 40

Ch 16 060502 V04

forgets what can happen and what should be done in rare incidents that he/she has never had to face. Different possible policies were then identified to address these problems and improve performance, for example, regular training on a simulator, reduction of the length of time on duty, improvement of resident supervision or testing of the practitioners for substance abuse. The potential benefits of each policy were then computed, based on the changes that the policy could bring in the probability that the anesthesiologist experiences specified problems in a given operation, and on the probabilities of an accident given each problem (see Figure 16.6).

p(IEi ) SAj Anesthesiologist State

p(AA | SAj)

Mean p(AA) per Operation

p(AA | IEi ) Ok Organizational Factors, Policies and Procedures

Legend IEi: AA: SAj: Ok:

initiating events (e.g., breathing circuit disconnect); index i anesthesia accident (death or severe brain damage) state of the anesthesiologist (e.g., fatigued); index j organizational policy (e.g., time on duty if limited to 12 consecutive hours); index k.

Figure 16.6 Influence diagram showing the analysis of patient risk in anesthesia linked to human and organizational factors. Source: Paté-Cornell et al., 1996a and b.

Advances: Engineering Risk Analysis

Page 27 of 40

Ch 16 060502 V04

Note that the costs were not computed. Using the notation defined in the legend of Figure 16.6, the equations of the model are: Step 1 The probability of an anesthesia accident per operation (p(AA)) is obtained through the dynamic analysis of the different accident types and summed using the total probability theorem. p(AA) = ∑i p(IEi) x p(AA|IEi)

(7)

Step 2 The probability of an accident is computed first, for each of the potential problem that can be experienced by the anesthesiologist and the corresponding probabilities of the accident occurrence, and parameter values. p(AA | SAj) = ∑i p(IEi | SAj) x p(AA | IEi, SAj)

(8)

p(AA) = ∑j p(AA | SAj) x p(SAj)

(9)

Step 3 Finally, the probability of an anesthesia accident conditional on a particular organizational policy is computed as a function of the effect of that policy on the state (thus the performance) of the anesthesiologist. P(AA | Ok) = ∑j p(SAj | Ok) x p(AA | SAj)

(10)

Different policies were then considered (e.g., reduction of the time on duty) and their benefits were computed as shown in Table 16.2. The results showed that contrary to previous perceptions, the most beneficial policies concerned improvement of the training of both novice and experienced practitioners, as well as supervision of residents. The benefits of substance abuse control were found to be limited, first because the problem may actually be less frequent

Advances: Engineering Risk Analysis

Page 28 of 40

Ch 16 060502 V04

than perceived, and second, because the tests could be ineffective given anesthesiologists’ training.

Advances: Engineering Risk Analysis

Page 29 of 40

Ch 16 060502 V04

Table 16.2: Effects of Proposed Policy Changes on the Anesthesia Patient Risk. (Source: PatéCornell et a,l. 1996a and b)

Policy Base Case (current policies) Work schedule restriction Simulator testing for residents Simulator Training For practitioners Re-certification every 3 years

Effects of Policy

Fatigue cut 50% Cognitive problems cut 90% Personality problems cut 50% Lack of training cut 75%

Drug testing

Decreases of lack of training, aging, cognitive, personality problems For 10 re-certs: 84% reduction Decreases lack of training, aging, cognitive, personality problems For 6 re-certs: 67% reduction Affects 10% of operations: Aging, lack of training, alcohol abuse more heavily weighted Drug abuse cut 95%

Alcohol testing

Alcohol abuse cut 90%

Annual medical examination

Aging/neurol. problems cut 75% Drug, alcohol abuse cut 25% Fatigue cut 10% Lack of supervision cut 50%

Re-certification every 5 years Mandatory retirement

Risk Replace- with ment Policy (x 10-5) -7.12

Risk Reduction (%)

Problemfree New dist'n Problemfree Problemfree

6.72

6%

7.02

2%

5.98

16%

5.06

29%

Problem- 5.48 free

23%

New dist'n

6.89

3%

New dist'n New dist'n New dist'n*

7.03

1%

6.97

2%

6.92

3%

--

Supervision of Problem- 6.16 14% residents free *except “Fatigued” replaced by “Problem-free”. New distribution: new distribution of the probability of problems among practitioners with increase of the probability of “problem-free”.

Advances: Engineering Risk Analysis

Page 30 of 40

Ch 16 060502 V04

Example 4. Probabilistic Analysis of Threats of Terrorist Attacks

Shortly after the 9/11 attacks on the United States, this country faced the problem of setting priorities among the possible measures that can be taken to reduce the probabilities of different types of attack, capture signals of a possible attack, and reinforce the potential targets. The question was thus to design a general risk analysis model that allowed combination of threats and vulnerabilities and the uncertainties about them based on all existing information including that gathered by the intelligence community (Paté-Cornell and Guikema, 2002) 18 . See Figure 16.7.

Advances: Engineering Risk Analysis

Page 31 of 40

Ch 16 060502 V04

Figure 16.7 Structure of an influence diagram designed to assess the risks from different scenarios of terrorist attack on the United States (Source: Paté-Cornell and Guikema, 2002)

The model included first, the identification of the different terrorist groups known at the time and of their objectives as revealed by their own statements. The key issue was then to assess

Advances: Engineering Risk Analysis

Page 32 of 40

Ch 16 060502 V04

for each of them their “supply chain”, including people and skills, weapons, cash, communications and transportation based on all information available to the US. The possibility of insider help and its effect on that supply chain was considered. The next step was to describe the different attack scenarios by the choice a weapon, target and mean of delivery. The planning of such an attack could possibly generate signals intercepted by US intelligence and allow for counter measures. Finally, the probabilities of an attack’s effects on the US were assessed as the result of the probabilities of different scenarios and of the effectiveness of US countermeasures. The probabilities of the scenarios were assessed as a function of their attractiveness to the different terrorist groups based (1) on their ease of execution and probability of success and (2) the desirability of the consequences to the perpetrators. In turn, the desirability of different countermeasures was assessed as a function of the probability of success of the corresponding attack scenarios, their negative values (“disutility”) to the US, and the potential of the counter measures to reduce the probability and/or consequences of different attacks. The challenge in this type of exercise is to design a global model that captures the firstorder factors and allows for further development of the model given the decisions that have to be supported. It provides a “back-of-an-envelope” model structure that permits systematic thinking about the problem and its potential solutions. It also demonstrates the use of the engineering systems-analysis application to a different type of problem, of which little is known in terms of global statistics—in part because of non-stationarity—but partial information can be gathered for each of the major factors. This framework can be further developed, as far and deeply as needed, to analyze the benefits of various policies. The results, as usual are only as good as the information that can be gathered, but the method, if applied well, permits systematic thinking and best use of that information.

Advances: Engineering Risk Analysis

Page 33 of 40

Ch 16 060502 V04

At the end of this study, the most likely attack scenarios (in an illustrative application) were found to be repeated conventional attacks on urban targets, followed by “dirty bombs” and biological attacks. But the most destructive to the US remains a possible attack with a nuclear warhead, either fabricated or purchased by a terrorist group.

Conclusions

The engineering risk analysis method can be used in many different settings. The principles of risk analysis based on systems analysis and probability to address problems for which little or no statistics exist, can be applied to many questions involving new technologies and can be extended to include human and organizational factors. In this chapter, we illustrated the formulation of risk analysis problems for maritime, space, and medical systems as well as the complex issues of counter-terrorism. The key to the success of such an analysis is the formulation of the problem so that the most relevant factors are captured and can be described further if needed in subsequent analytical developments. Bayesian probability is a fundamental concept in such a model. Influence diagrams are useful tools in the formulation phase because they provide both an analytical framework and means of communication. The dynamics of accident sequences is often critical, and can be described through stochastic processes. This method allows anticipation of accidents, failures or attacks that may not have happened before, even though it requires “imagination” and can be challenged based on the credibility of new failure scenarios and the computation of their probabilities. It can be used to support pro-active risk management as an alternative to merely responding to the last event, and to set priorities among risk management under common constraints of time and money. The keys to its success

Advances: Engineering Risk Analysis

Page 34 of 40

Ch 16 060502 V04

remain imagination, i.e., the willingness and the ability to face events that have not occurred yet) and rationality, i.e., the discipline and the systematic thinking that allows structuring the models.

Bibliography Apostolakis, G. (1990). The Concept of Probability in Safety Assessments of Technological Systems. Science, 250:1359-1364. Bedford T. and R.M. Cooke (2001). Probabilistic Risk Analysis: Foundations and Methods. Cambridge University Press. Bier V.M. and Louis A. Cox (2006). Probabilistic Risk Analysis for Engineered Systems, Chapter 16 in Advances in Decision Analysis, Edward, Miles and von Winterfeldt Eds., Cambridge University Press. Budnitz R. J., Apostolakis G., Boore D.M., Cluff L.S., Coppersmith K.G., Cornell C.A., Morris P.A. (1998). Use of Technical Expert Panels: Applications to Probabilistic Seismic Hazard Analysis. Risk Analysis. 18 (4): 463-469. Clemen R.T and R.L. Winkler (2006). Aggregation of expert probability judgments. Chapter 8 in Advances in Decision Analysis, Edward, Miles and von Winterfeldt Eds., Cambridge University Press. Cooke R.M. (1991). Experts in uncertainty: opinion and subjective probability in science. Oxford University Press. Davoudian K., J.-S. Wu, and G. Apostolakis. Incorporating organizational factors into risk assessment through the analysis of work processes. Reliability Engineering and System Safety, 45: 85-105.

Advances: Engineering Risk Analysis

Page 35 of 40

Ch 16 060502 V04

Helton, J.C. (1994). Treatment of uncertainty in performance assessments for complex systems. Risk Analysis, 14: 483-511. Henley E., and H. Kumamoto, (1992). Probabilistic Risk Assessment: Reliability Engineering, Design, and Analysis, IEEE Press: New York. Howard R. (2004). Speaking of Decisions: Precise Decision Language. Decision Analysis, 1: 71-78 Kahneman, D., P. Slovic, and A. Tversky, Eds (1982). Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press. Keeney, R.L. and H.Raiffa. (1976). Decision Analysis with Multiple Objectives: Preferences and Value Trade-offs. John Wiley and Sons, New York. Murphy, D.M. and M.E. Paté-Cornell. (1996). The SAM Framework: A Systems Analysis Approach to Modeling the Effects of Management on Human Behavior in Risk Analysis, Risk Analysis, 16 (4): 501-515. National Commission on Terrorist Attacks Upon the United States (NCTA) (2004). The 9/11 Commission Report. Washington D.C. Paté-Cornell, M.E. (1985). Reduction of Fire Risks in Oil Refineries: Economic Analysis of Camera Monitoring. Risk Analysis, 5( 4): 277-288. Paté-Cornell, M.E., and P.S. Fischbeck. (1993a). Probabilistic risk analysis and risk-based priority scale for the tiles of the space shuttle. Reliability Engineering and System Safety 40(3): 221–238. Paté-Cornell, M.E., and P.S. Fischbeck. (1993b). PRA as a management tool: organizational factors and risk-based priorities for the maintenance of the tiles of the space shuttle orbiter. Reliability Engineering and System Safety 40(3): 239–257.

Advances: Engineering Risk Analysis

Page 36 of 40

Ch 16 060502 V04

Paté-Cornell M.E., L.M. Lakats, D.M. Murphy, and D.M. Gaba. (1996a). Anesthesia patient risk: a quantitative approach to organizational factors and risk management options. Risk Analysis 17(4): 511–523. Paté-Cornell, M.E., D.M. Murphy, L.M. Lakats and D. M. Gaba. (1996b). Patient risk in anesthesia: probabilistic risk analysis, management effects and improvements. Annals of Operations Research 67(2): 211–233. Paté-Cornell, M. E. (1996). Uncertainties in risk analysis: six levels of treatment, Reliability Engineering and System Safety, 54: 95-111. Paté-Cornell, M.E. (1999a). Medical application of engineering risk analysis and anesthesia patient risk illustration. American Journal of Therapeutics 6(5): 245–255. Paté-Cornell, M.E. (1999b). Conditional Uncertainty Analysis and Implications For Decision Making: The Case of the Waste Isolation Pilot Plant, Risk Analysis, 19 (5): 995-100. Paté-Cornell, M.E. (2000). “Greed and Ignorance: Motivations and Illustrations of the Quantification of Major Risks”, Proceedings of the study week on “Science for Survival and Sustainable Development”: 231-270, Pontificiae Academiae Scientiarum Scripta Varia (Report of the Pontifical Academy of Sciences), The Vatican. Paté-Cornell, M.E. (2002a). Finding and fixing systems weaknesses: probabilistic methods and applications of engineering risk analysis. Risk Analysis 22(2): 319–334. Paté-Cornell, M.E. (2002b). Fusion of intelligence information: a Bayesian approach. Risk Analysis 22(3): 445-454. Erratum published in 23(2): 423. Paté-Cornell, M.E., and S.D. Guikema. (2002). Probabilistic modeling of terrorist threats: a systems analysis approach to setting priorities among countermeasures. Military Operations Research 7(4): 5-23.

Advances: Engineering Risk Analysis

Page 37 of 40

Ch 16 060502 V04

Paté-Cornell M.E. (2006). Probabilistic Risk Analysis versus Decision Analysis: Similarities, Differences and Illustrations. Theory and Decision (in press). Paté-Cornell M.E. and L.A. Deleris (2005): Risks of Bankruptcy in the Insurance Industry, Research Report to the Risk Foundation, Department of Management Science and Engineering, Stanford University. Pietzsch, J.B., M.E. Paté-Cornell and T.M. Krummel (2004). A Framework for Probabilistic Assessment of New Medical Technologies, Proceedings of PSAM7 / ESREL04, Berlin, Germany, Springer-Verlag Pub. London UK: 2224-2229. Phimister, J.R., V.M. Bier and H.C. Kunreuther Eds. (2004). Accident Precursor Analysis and Management: Reducing Technological Risk Through Diligence, National Academies Press, Washington D.C. Press, S.J. (1989). Bayesian Statistics: Principles, Models, and Applications, John Wiley and Sons, New York, NY. Raiffa, H. (1968). Decision Analysis. Addison Wesley, Cambridge, MA. Rowe, W.D. (2003). Vulnerability to Terrorism: Addressing the Human Variables in Risk-Based Decisionmaking in Water Resources X, Haimes, Moser, Stakhiv, Ivry Zisk, Dirickson, and Zisk, Eds, ASCE/EWRI/UE, Reston, VA: 155-159. Savage, L.J. (1954). The Foundations of Statistics, Wiley: New York. Shachter, R. (2006). Influence Diagrams, Chapter 6 in Advances in Decision Analysis, Edward, Miles and von Winterfeldt Eds., Cambridge University Press. U.S. Nuclear Regulatory Commission (USNRC) (1975). WASH 1400(NUREG-75/014), Reactor Safety Study: Assessment of Accident Risk in U.S. Commercial Nuclear Plants, Washington, D.C.

Advances: Engineering Risk Analysis

Page 38 of 40

Ch 16 060502 V04

Von Neumann J. and O. Morgenstern. (1947). Theory of Games and Economic Behavior, 2nd ed. Princeton University Press, Princeton, NJ. Webb, R.K., M. Currie, C.A. Morgan, J.A. Williamson, P. Mackay, W.J. Russel, and W.B. Runciman. (1993). The Australian Incident Monitoring Study: an analysis of 2000 incident reports. Anaesthesia and Intensive Care 21: 520–528. 1

Bier and Cox, 2006.

2

There exist other forms of risk analysis, for instance those used in environmental/health risk

analysis, which often rely on “plausible upper bounds” or other “conservative” measurements to characterize, for example, dose-response relationships (Paté-Cornell, 1996). These methods generally do not involve Bayesian probability. 3

An influence diagram is a directed graph, whose nodes represent random events or variables

and the arrows probabilistic dependences among them (Shachter, 2006). 4

In industry, however, the risk of failure of an engineering project is often understood as the risk

of missing the project deadline or exceeding its budget. The technical failure risk is often “managed” through insurance. Furthermore, it is often assumed that once the specifications have been met everything will be fine, when in fact the problem may be first, in their definition and second, in ensuring that they have actually been met. In this paper, the focus is on technical failure risk and its management through actual system reinforcement. 5

The emphasis is on classes of scenarios, i.e., a partition of the scenario set that is sufficient to

capture the different levels of system failures without going into details that would make the formulation almost unmanageable and the collection of data almost impossible. 6

Alternatively, one can describe for simplicity, the marginal distribution of each of the attributes

and a measure of their correlation.

Advances: Engineering Risk Analysis

Page 39 of 40

Ch 16 060502 V04

7

One can rewrite these equations to characterize partial failures using the same concepts.

8

A risk analysis can support the decision of a “heterogeneous” group of decision makers, but

they will have to agree on either a decision process or on analytical methods of aggregation of probabilities and preferences. 9

There are many other ways of performing a normative or prescriptive “decision analysis”. The

Analytical Hierarchy process, for example, is designed to guide a decision and therefore, it can be considered prescriptive even though it does not rely on the norms of the von Neumann axioms. 10

The notion of evidence-based decisions is currently at the heart of a debate in the medical

community. I want to stress here that evidence cannot be restricted to large statistical samples and that one should not wait to save lives until complete statistics are available to make decisions that can be reasonably supported by Bayesian reasoning. 11

This is an approximation that applies first, to the case where that frequency is small given the

time unit, and second, to a repetitive event or something that can be construed as one, not to unique events or hypotheses. Therefore, this is a short cut, but one that is useful to bridge gaps in reasoning between two communities of risk analysts: one focused on frequencies and one focused on degrees of belief. 12

The aggregation of expert opinions can be based on analytical procedures such as Bayesian

weights representing likelihoods, or on interactive methods such as the Delphi method, or hybrid methods such as the SHAC method used in seismic hazard analysis, which has the advantage of focusing on the basic mechanisms and the supporting data, irrelevant of the field of application (Budnitz et al., 1998). Another approach is empirical calibration of the experts themselves (Cooke, 1991) or the use of copula models (Clemen and Winkler, 2006).

Advances: Engineering Risk Analysis

13

Page 40 of 40

Ch 16 060502 V04

Many PRA input come from classical statistics. Note however, that this requires a steady state,

and a sample of sufficient size, in which case classical and Bayesian statistics (if that they are based on weak priors) yield similar results. 14

Another approach is to start from an analysis of the work process (Davoudian et al., 1994).

15

Note the assumption here that the size of the breach depends only on grounding and not on drift

control, and that the amount of oil spilled depends only on the size of the breach when in fact, many other factors could be included e.g., the weather, tides, and especially, damage control measures. 16

One of the challenges is to ensure that, after the fact, these variables are defined precisely

enough to pass the “clarity test” of unambiguity of what the realization was (Howard, 2004). This means that once the uncertainties are resolved, the analyst can point to the realization of that variable that actually occurred. 17

Clearly, the tile system is not literally a plane; but the projection of the orbiter’s undersurface

on a flat plane allowed its two-dimensional partition for the purposes of the analysis in the same way as one can partition a geographic map into seismic zones to compute earthquake risks. 18

Another risk analysis model, focused on vulnerability to terrorist attacks, was presented by

Rowe (2003).