A Comprehensive Study of Risk Management

Risk Analysis for the Oil Industry

A supplement to:

Risk Analysis: Table of Contents

Biography

im Murtha, a registered petroleum engineer, presents seminars and training courses and advises clients in building probabilistic models in risk analysis and decision making. He was elected to Distinguished Membership in SPE in 1999, received the 1998 SPE Award in Economics and Evaluation, and was 1996-97 SPE Distinguished Lecturer in Risk and Decision Analysis. Since 1992, more than 2,500 professionals have taken his classes. He has published Decisions

v v v

Involving Uncertainty - An @RISK Tutorial for the Petroleum Industry. In 25 years of academic experience, he chaired a math department, taught petroleum engineering, served as academic dean, and co-authored two texts in mathematics and statistics. Jim has a Ph.D. in mathematics from the University of Wisconsin, an MS in petroleum and natural gas engineering from Penn State and a BS in mathematics from Marietta College. x

Acknowledgements

hen I was a struggling assistant professor of mathematics, I yearned for more ideas, for we were expected to write technical papers and suggest wonderful projects to graduate students. Now I have no students and no one is counting my publications. But, the ideas have been coming. Indeed, I find myself, like anyone who teaches classes to professionals, constantly stumbling on notions worth exploring. The articles herein were generated during a few years and written mostly in about 6 months. A couple of related papers found their way into SPE meetings this year. I thank the hundreds of people who listened and challenged and suggested during classes.

I owe a lot to Susan Peterson, John Trahan and Red White, friends with whom I argue and bounce ideas around from time to time. Most of all, these articles benefited by the careful reading of one person,Wilton Adams, who has often assisted Susan and me in risk analysis classes. During the past year, he has been especially helpful in reviewing every word of the papers I wrote for SPE and for this publication.Among his talents are a well tuned ear and high standards for clarity. I wish to thank him for his generosity. He also plays a mean keyboard, sings a good song and is a collaborator in a certain periodic culinary activity. You should be so lucky. x

Table of Contents

A Guide To Risk Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 3 Central Limit Theorem Polls and Holes . . . . . . . . . . . 5 Estimating Pay Thickness From Seismic Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Bayes Theorem Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . 12 Decision Trees vs. Monte Carlo Simulation . . . . . . . . 14 When Does Correlation Matter? . . . . . . . . . . . . . . . . . 19 Beware of Risked Reserves . . . . . . . . . . . . . . . . . . . . 23

Risk Analysis

Risk Analysis: An Overview

A Guide To
R

Risk Analysis
ance to isolated departments within organizations. Managers were notoriously unwilling to embrace results that presented probability distributions for reserves and net present value (NPV). Consultants offering services and software vendors know too well these levels of resistance. Now, finally, there seems to be broader acceptance of probabilistic methods, although as I write, my SPE Technical Interest Group digest contains strong negativism from traditionalists about probabilistic prices. Nonetheless, consider these items: the three most recent and five out of the last seven recipients of the SPE Economics and Evaluation award have been strong proponents of risk analysis; whereas the index to the last edition of the Petroleum Engineers Handbook only had two references to risk, the forthcoming edition will feature an entire chapter on the topic; my first paper on Monte Carlo simulation was presented at the Eastern Regional Meeting in 1987 and summarily rejected by the editorial committees for not being of adequate general interest. (It was a case study dealing with the Clinton formation, but the methods were clearly generic and used popular material

isk and decision analysis was born in the middle of the 20th century, about 50 years after some of the necessary statistics became formalized. Pearson defined standard deviation and skewness in the late 1890s, and Galton introduced percentiles in 1885. The term Monte Carlo, as applied to uncertainty analysis, was mentioned by Metropolis and Ulam: the Journal of the American Statistical Association in 1940. D.B. Hertz published his classic Harvard Business Review article in 1964. A couple of years later, Paul Newendorp began teaching classes on petroleum exploration economics and risk analysis, out of which evolved the first edition of his text in 1975, the same year as A.W. McCray and 2 years before R.E. Megill wrote their books on the subject. Ten years later, there was commercial software available to do Monte Carlo simulation. During this 50-year period, decision analysis, featuring decision trees, also came of age. Raiffas classic book appeared in 1968. By 1985, there were several commercial applications of software on the market. These developments, in many ways, paralleled the development of petroleum engineering, with the basics appearing in the 1930s, mature texts like

The oil and gas industry remained skeptical about

risk analysis throughout most of this development period.
Craft and Hawkins appearing in the late 1950s, and reservoir simulation and other computer based methods emerging in the 1970s and 1980s. Similarly, reservoir characterization and geostatistics came along in the second half of the 20th century. Oddly, the oil and gas industry remained skeptical about risk analysis throughout most of this development period, usually limiting their acceptRisk Analysis

balance notions). Ten years later, I published Monte Carlo Simulation, Its Status and Future in the Distinguished Author series; the most popular SPE Applied Technology Workshop focuses on probabilistic methods; SPE, SPEE and WPC are working on definitions that include probabilistic language; and

Risk Analysis: An Overview

We need to encourage papers and presentations

by people who have applied the methods for several years.
the largest service companies acquired smaller companies that market probabilistic software. So, we advocates of Monte Carlo and Decision Trees should be jumping with glee that the rest of the world has caught up to us, right? Well, sort of The problem with this newly-found technique is that we are still learning things by our mistakes. Like anything else that evolves into widespread use, risk analysis requires more maturation. Only now do we have the critical mass on proponents to refine our applications, to discover what works, to eliminate faulty logic, and to test our ideas by papers, courses, books and forums. I have: discovered things during the past 5 years that make some of my papers out of date; proposed methods that are too difficult to explain and therefore impractical in an organization; stumbled on limitations of the software I use that force us to change procedures; witnessed numerous misuses, misquotations, and misinterpretations, many of them in technical papers; and had ongoing unresolved arguments with reliable colleagues. All this spells opportunity for the larger community interested in risk analysis. There should be better papers and more of them, and good discussions and improved software. More people doing day-to-day modeling should come forward. I know of several companies where the high profile spokesmen are far less knowledgeable and less experienced than some of their low profile colleagues. I am reminded of an experience while on my first sabbatical at Stanford in 1972, studying operations research and management science. The university granted me full access to all the libraries, and I read more papers in 12 months than any 5-year period before or since. I discovered the early years of what were two journals then, Management Science and Operations Research. Many of the authors had later became f a m o u s . M a ny o f t h e p a p e r s we re l u c i d , expositions, rather than arcane dissertations. Often they raised unanswered questions and stimulated others. It was clear that these people had been mulling over these ideas and discovering them in fields that were still immature. Once the publications came into being, they exploded with their knowledge. We have made some progress toward this exposition as one can see by the increased number of papers at several of the SPE and AAPG meetings. But we need to encourage papers and presentations by people who have applied the methods for several years, testing and improving, discarding ideas that do not travel well from the cloister to the hearth, comparing alternative solutions, providing case studies, documenting lookbacks, citing success stories within organizations and touting benefits of company-wide probabilistic methods. SPE and AAPG have provided ample opportunities with forums, workshops and conferences for the general ideas to be promulgated.A natural next step would be to hold workshops for advanced users.

About the articles in this publication

This publication is in part a challenge to review some of our assumptions and check our language. It is intentionally provocative and open ended. Themes articulated: everyone should use probabilistic not deterministic methods; fallacies abound in both deterministic and probabilistic methods; teach people how to ask questions whose answers require probabilistic methods; we often abuse the language of probability and statistics; increased use of risk analysis presents more opportunity to do things wrong; and in order to talk about real options and optimizations (buzzwords de jour), we need to do probabilistic cash flow properly. x
Risk Analysis

Risk Analysis: Central Limit Theorem

Central Limit Theorem

W

Polls and Holes

be about 1/30 or 3%. Thus,if 54% of the voters surveyed said they voted for candidate A, then there is about a 95% chance that A will get between 51% and 57% of the full vote. This method of analysis is a direct consequence of the Central Limit Theorem, one of the most significant results in mathematical statistics. Suppose you have a distribution X, with mean and standard deviation . X can be virtually any shape. From X, we sample n values and calculate their mean.Then we take another sample of size n from X and calculate its mean. We continue this process, obtaining a large number of samples and build a histogram of the sample means. This histogram will gradually take a shape of a smooth curve.The amazing fact is that that curve, the limit of the sample means, satisfies these conditions: the sample means are approximately normally distributed; has mean equal , but; has standard deviation of approximately n (the mean standard error). The key here is that the mean standard error gets small as n (the sample size) gets large. We need another fact about normal distributions, namely that 68% of the values in a normal distribution lie within one standard deviation of its mean, and 95% and 99.7% of the values lie within two and three standard deviations, respectively. In our case, X is a binomial variable with exactly two possible values, 0 and 1, where the probability of 1 is the percentage, p, of people among all voters who voted for A and the probability of 0 is 1-p.The mean value of this distribution is p. The standard deviation is p(1-p).

hat do exit surveys of voters in presidential elections have in common with porosities calculated from logs of several wells penetrating a geological structure? The answer is that in both cases, the data can be used to estimate an average value for a larger population. At the risk of reviving a bitter debate, suppose that a carefully selected group of 900 voters is surveyed as they leave their polling booths. If the voters surveyed a) are representative of the population of a whole and b) tell the truth, then the ratio r = (Number of voters for Candidate A in survey)/(number of voters surveyed) should be a good estimate of the ratio. R = (Number of voters for Candidate A in population)/(number of voters in population) Moreover, by doing some algebra, the statistician analyzing the survey data can provide a margin of error for how close r is to R. In other words, you are pretty confident that (r - margin of error) < R < (r + margin of error). The margin of error formula depends on three things: the level of confidence of the error ( I am 90% or 95 % or 99% confident of this); the number of voters who chose candidate A; and the number of voters surveyed. To end this little diversion, here is the approximate formula for the margin of error in a close race (where R is roughly 45% to 55%), and we are satisfied with a 95% confidence level (the most common confidence level used by professional pollsters). Margin of error = 1/sqrt (N) approx. Where N = sample size, the number of voters polled. Thus when N=900, the margin of error would

Heres the point: the Central Limit Theorem

guarantees these distributions of average properties net pay, porosity and saturation will tend to be normal.
Risk Analysis

Risk Analysis: Central Limit Theorem

This would be a good time to ask that

important question, Why bother?
What does this have to do with porosity?
Structure 1S 1S 1S 1S 1S 1S 1S 1S 2S 2S 2S 2S 2S 2S 3S 3S 3S 3S 3S 3S 3S 3S 3S 3S Area 5010 5010 5010 5010 5010 5010 5010 5010 2600 2600 2600 2600 2600 2600 4300 4300 4300 4300 4300 4300 4300 4300 4300 4300 Well 1S1 1S2 1S3 1S4 1S5 1S6 1S7 1S8 2S1 2S2 2S3 2S4 2S5 2S6 3S1 3S2 3S3 3S4 3S5 3S6 3S7 3S8 3S9 3S10 Net pay 34 56 74 47 53 29 84 40 35 42 37 61 27 48 68 94 35 47 68 75 67 48 69 88 Porosity 0.12 0.07 0.14 0.09 0.18 0.08 0.15 0.14 0.12 0.07 0.09 0.13 0.10 0.08 0.12 0.16 0.07 0.09 0.14 0.10 0.12 0.13 0.09 0.15 Sw 0.31 0.20 0.38 0.30 0.26 0.29 0.37 0.25 0.33 0.38 0.41 0.26 0.33 0.35 0.26 0.20 0.37 0.30 0.32 0.28 0.25 0.23 0.30 0.23

Application: average net pay, average porosity, average saturations

Consider the following typical volumetric formula: G = 43,560Ah(1-Sw) /Bg*E Where A = Area h = Net pay = Porosity Sw = Water saturation Bg = Gas formation volume factor E = Recovery efficiency In this formula, there is one component that identifies the prospect, A, while the other factors essentially modify this component. The variable h, for example, should represent the average net pay over the area A. Similarly, represents the average porosity for the specified area, and Sw should represent average water saturation. Why is this? Because, even though there may be a large range of net pay values throughout a given area, A, we are interested only in that average net pay which, when multiplied by A, yields the (net sand) volume of the prospect. Consider a database of wells in some play, as shown in Table 1, grouped into structures. Structure 1 has eight wells, structure 2 has six wells, and so on. We need to take this a step further. Suppose we construct a net sand isopach map for structure 1, and then calculate the average net sand for the structure, by integrating and dividing by the base area. Then we do the same thing for porosity as well as for water saturation.This yields a new database,Table 2, which becomes the basis for a Monte Carlo simulation. In the simulation, when we sample a value for A, we then sample values for average net pay, average porosity and average saturation for that area, which gives one realization of gas in place. Although the suggested process of obtaining the averages involved integration, one could use the numerical average of the data to approximate the integrated average. Even so, since we often try to drill wells in the better locations, it would still be useful to sketch a map with the well locations and get some idea whether the well average would be
Risk Analysis

Table 1. Well database Structure 1S 2S 3S 4S Area 5010 2600 4300 h_ave 52 42 66 Por_ave 0.12 0.10 0.12 Sw_ave 0.30 0.34 0.27

Table 2. Structure database

Nevertheless, in many applications, as we shall see later, X is a continuous variable, with a shape of a normal, lognormal or a triangular distribution, for example. Thus, our samples mean (the percentage of the people in our sample who voted for A) may not be exactly the mean of the normal distribution, but we can be 95% confident that it is within two values of the mean standard error of the true mean.

Risk Analysis: Central Limit Theorem

biased compared with the averages over the entire structure. If we put a grid over the area and place a well location in each grid block, then the two notions of average essentially coincide.

How much difference will this make?

This would be a good time to ask that important question,Why bother?What if this is correct: that we should be using narrower and more symmetric shaped distributions for several of the factors in the volumetric formula? Does it matter in the final estimate for reserves or hydrocarbon in place? How much difference could we expect? Let us consider porosity.When we examine the database of porosities from all wells (for example, the average log porosity for the completion interval or from the layer interval in a multilayer system) as in Table 1, there are two possible extreme situations.At one extreme, it is possible that each structure exhibits a wide range of quality from fairway to flanks, but the average porosities for the various structures always fall between 10% and 14%. In this case, the range of all porosities could easily be several times as broad as the average porosities.That is, there are similarities among structures, but not much homogeneity within a structure. On the other hand, it is possible that each structure is relatively homogeneous, but the different structures are quite dissimilar in quality, with average porosities

These distributions of averages are normal

Heres the point: the Central Limit Theorem guarantees these distributions of average properties net pay, porosity and saturation will tend to be normal. Another consequence of the theorem is that these distributions of averages are relatively narrow, for example, they are less dispersed than the full distributions of net pays or porosities or saturations from the wells, which might have been lognormal or some other shape. The correct distributions for Monte Carlo analysis, however, are the narrower, normal-shaped ones. One objection to the above argument (that we should be using narrower, normal distributions) is that often we do not have ample information to estimate the average porosity or average saturation.This is true. Nonetheless, one could argue that it still might be possible to imagine what kind of range of porosities might exist from the best to the worst portions of the structure.

Even the main factor (area or volume)

can be skewed left.
To help your imagination, we do have ample information from many mature fields where material balance could provide estimates. We also have extensive databases with plenty of information, from which some range of average values could be calculated and compared with the broader ranges of well data. Always remember that, like all else in Monte Carlo simulation, you must be prepared to justify every one of your realizations, for example, combinations of parameters. Just as we must guard against unlikely combinations of input parameters by incorporating correlations in some models, we should ask ourselves if a given area or volume could conceivably have such an extreme value for average porosity or average saturation. If so, then there must be even more extreme values at certain points within the structure to produce those averages (unless, of course, the structure is uniformly endowed with that property, in which case, I would be skeptical of the wide disparity from one structure to the next.)
Risk Analysis

ranging from 5 % to 20%.In this case,the two distributions would be closer together. Note, however, that the average porosities will always have a narrower distribution than the complete set of porosities. Perhaps the contrast is even easier to see with net pays. Imagine a play where each drainage area tends to be relatively uniform thick, which might be the case for a faulted system. Thus, the average h for a structure is essentially the same as any well thickness within the structure. Then the two distributions would be similar. By contrast, imagine a play where each structure has sharp relief, with wells in the interior having several times the net sand as wells near the pinch outs.Although the various structures could have a fairly wide distribution of average thicknesses, the full distribution of h for all wells could easily be several times as broad. The distribution for A could easily be lognormal if the drainage areas were natural. In a faulted system, however, where the drainage areas were defined by faults, the distribution need not be lognormal. The

Risk Analysis: Central Limit Theorem

Additional complications arise because of

uncertainty about the range of driving mechanisms will there be a water drive? Will gas injection or water injection be effective?
right way to judge whether the type of distribution matters for an input variable is to compare what happens to the output of the simulation when one type is substituted for another. another common formula is: OOIP = 7,758Vb(NTG) So/Bo Vb = Bulk rock volume NTG = Net to gross ratio Here,Vb would be the dominant factor, which could be skewed right and modeled by a lognormal distribution, while the factors NTG, , So and Bo would tend to be normally distributed, since they represent average properties over the bulk volume.

What about the output of the simulation, OOIP?

Regardless of the shapes of the inputs to a volumetric model be they skewed right, or left, or symmetric the output will still be skewed right, thus approximately lognormal. In fact, as is well known, the Central Limit Theorem guarantees this. The argument is straightforward: the log of a product (of distributions) is a sum of the logs (of distributions), which tends to be normal. Thus the product, whose log is normal, satisfies the definition of a lognormal distribution.

Recovery factors
Recovery factors, which convert hydrocarbon in place to reserves or recoverable hydrocarbon, are also average values over the hydrocarbon pore volume. The recovery efficiency may vary over the structure, but when we multiply the OOIP by a number to get recoverable oil, the assumption is that this value is an average over the OOIP volume. As such, they too would often be normally distributed. Additional complications arise, however, because of uncertainty about the range of driving mechanisms: will there be a water drive? Will gas injection or water injection be effective? Some people model these aspects of uncertainty with discrete variables.

Will the Area distribution always be lognormal?

The traditional manner of describing area and treating it as a lognormal distribution is based on prospects in a play. If we were to select at random some structure in a play, then the appropriate distribution likely would be a lognormal. Sometimes, however, not even the Area parameter should be modeled by a lognormal distribution.Why? Suppose a particular prospect is identified from 3-D seismic. We have seen situations where the base case value of area or volume is regarded as a mode (most likely). When asked to reprocess and or reinterpret the data and provide relatively extreme upside (say P95) and downside (say P5) areas or volumes, the results are skewed left: there is more departure from the mode toward the downside than the upside. Because the conventional lognormal distribution is only skewed right, we must select another distribution type, such as the triangular, beta, or gamma distribution.

Summary
The Central Limit Theorem suggests that most of the factors in a volumetric formula for hydrocarbons in place will tend to have symmetric distributions and can be modeled as normal random variables.The main factor (area or volume) can be skewed left or right. Regardless of the shapes of these input distributions, the outputs of volumetric formulas, oil and gas in place and reserves, tend to be skewed right or approximately lognormal. Because the conventional wisdom is to use lognormal distributions for all of the inputs,the above argument may be controversial for the time being.The jury is still out. We could take a poll and see what users believe. Oh yes, then we could use the Central Limit Theorem to analyze the sample and predict the overall opinion. What goes around comes around. Stay tuned for other applications of the Central Limit Theorem. x
Risk Analysis

Variations of the volumetric formula

Among the numerous variations of the volumetric formulas, there is usually only one factor that serves the role of Area in the above argument. For instance,

Risk Analysis: Central Limit Theorem

Estimating Pay Thickness From

H
H

Seismic Data

ow do we estimate net pay thickness, and how much error do we introduce in the process? Usually, we specify two depths, to the top and bottom of the target interval and take their difference.The precision and accuracy of the thickness measurement, therefore, depends on the precision and accuracy of two individual measurements. The fact that we are subtracting two measurements allows us to invoke the Central Limit Theorem to address the questions of error.This theorem was stated in a previous article. A suitable version of it is given in the Appendix. First, let us remind ourselves of some of the issues about measurements in general. We say a measurement is accurate if it is close to the true value, reliable or precise if repeated measurements yield similar results and unbiased if the estimate is as likely to exceed the true value as it is to fall short. Sometimes we consider a two-dimensional analogy, bullet holes in a target. If the holes are in a tight cluster, then they are reliable and precise. If they are close to the bullseye, then they are accurate. If there are as many to the right of the bullseye as the left, and as many above the bullseye as below, then they are unbiased. With pay thickness, we are interested in the precision of measurement in a linear scale, which we take to mean the range of error. Our estimate for thickness will be precise if the interval of error is small. Consider the following situation.

distances from the top marker to the target facies and from the bottom of the facies to the platform namely 100m and 200m respectively, and the facies thickness is 600m the overall distance between the markers is 900m. In one particular offsetting anomaly, the depth measurement to the lower anhydrite is 5,000m, plus or minus 54m. The depth estimate to the platform is 5,900m, plus or minus 54m.What is the range of measurement for the thickness of the target facies? First, we should ask what causes the possible error in measurement. In the words of the geoscientists, If the records are good, the range for picking the reflection peak should not be much more than 30 milliseconds (in two-way time) and at 3,600 m/second that would be about 54m. This, of course, would be some fairly broad bracketing range, two or three standard deviations, so the likely error at any given point is much less. We would also hold that there is no correlation between an error in picking the lower anhydrite and the error in picking the platform. A further question to the geoscientists revealed that the true depth would just as likely be greater than as it would be less than the estimate.That is, the estimates should be unbiased.

Solution
People who use worst case scenario arguments would claim that the top of the reefal facies could be as low as 5,154m and the bottom as high as 5,646m, giving a minimum difference of 492m. Similarly, they say the maximum thickness would be (5,754m-5,046m) = 708m. In other words, they add and subtract 2*54 = 108 from the base

Application: adding errors

A team of geoscientists estimated the gross thickness of a reef-lagoon complex that resides between two identifiable seismic events: the lower anhydrite at the top and a platform at the bottom. Logs from one well provide estimates of the

But, is that what we really care about: the

theoretical minimum and maximum?
Risk Analysis

Risk Analysis: Central Limit Theorem

customary to regard the practical range of a normal distribution to be the mean plus or minus three standard deviations, which is 99.7% of the true (infinite) range. Thus the actual depths would be as much as 54m (=3 * 18) off from the estimated depths. Again, we must be careful to distinguish between the theoretical model limits and something of practical importance. Every normal distribution extends from to +.The standard practical range extends three standard deviations each side of the mean. Especially if we are talking about measurements, we know the values must be positive numbers. Next, we run a very simple Monte Carlo simulation, in which we select 2,000 pairs of values from these two distributions and calculate Thickness. The results can be plotted up as a distribution, which the Central Limit Theorem (see below) predicts is itself another normal distribution, namely: Thickness = Normal(600m; 25.5m) In other words, the 3-sigma range of values for Thickness is [523m; 677m], not [492m; 708m] as the worst- and best-case scenarios would have us believe. In fact, 90% of the time Thickness will be between 558m and 642m, and 99% of the time the value of Thickness would fall between 535m and 665m. Had we taken the distributions to be triangular (minimum, mode, maximum), the simulation would show that Thickness has a 99% probability range of [1,706.12ft; 2,231.08ft] or [520m; 680m]. In this case, we would use: Top = Triangular (5,046m; 5,100m; 5,154m) and Bottom = Triangular (5,646m; 5,700m; 5,754m) and Thickness = Bottom Top The result is approximated by Normal(600m; 31.5m).The P5, P95 interval is [549m; 65m].These and other approximations are summarized in Table 1. Finally, we run this model using the uniform (minimum, maximum) distribution, specifically uniform (4946m,5054m) for the top and uniform (5,846m, 5,954m) for the bottom. Thanks to the Central Limit Theorem, the difference can still be reasonably approximated by a normal (not uniform) distribution although as one can show, the triangular distribution type actually fits better namely Normal (600m; 38.5m).The interval ranges shown in the last column of the table were obtained from a simulation with 6,000 iterations.

Uncorrelated 99% confidence 95% confidence 90% confidence Correlated (r=0.7) 99% confidence 95% confidence 90% confidence

Normal [535m,665m] [551m, 649m] [558m, 642m] Normal [565m; 635m] [573m, 627m] [578m, 623m]

Triangular [520m ,680m] [539m, 661m] [549m, 651m] Triangular [555m, 645m] [566m, 634m] [572m, 629m]

Uniform [503m, 697m] [515m, 685m] [526m, 674m] Uniform [533m, 665m] [548m, 652m] [557m,643m]

Table 1. Effect of Distribution Shape on Ranges (maximum, minimum) for Net Pay Thickness

case of 600m to get estimates of minimum and maximum thicknesses. But, is that what we really care about: the theoretical minimum and maximum? We may be more interested in a practical range that will cover a large percentage, say 90%, 95% or 99% of the cases. A probabilistic approach to the problem says that the two shale depths are distributions, with means of 5,000m and 5,900m respectively. The assertion that there is no bias in the measurement suggests symmetric distributions for both depths. Among the candidates for the shapes of the distributions are uniform, triangular and normal. One way to think about the problem is to ask whether the chance of a particular depth becomes smaller as one moves toward the extreme values in the range. If the answer is yes, then the normal or triangular distribution would be appropriate, since the remaining shape, the uniform distribution, represents a variable that is equally likely to fall into any portion of the full range.Traditionally, errors in measurement have been modeled with normal distributions. In fact, K.F. Gauss, who is often credited with describing the normal distribution but who was preceded by A. de Moivre touted the normal distribution as the correct way to describe errors in astronomical measurements. If the uncertainty by as much as 54m is truly a measurement error, then the normal distribution would be a good candidate. Accordingly, we represent the top and bottom depths as: Bottom = Normal(5,900m; 18m), where its mean is 5,900m and its standard deviation is 18m Top = Normal(5,000m; 18m) Thick = Bottom Top 300m (the facies thickness is 300m less than the difference between the markers) We use 18 for the standard deviation because it is

Summary and refinements

When we subtract one random variable from another the resulting distribution tends to be
Risk Analysis

10

Risk Analysis: Central Limit Theorem

We may be more interested in a practical range

that will cover a large percentage, say 90%, 95% or 99% of the cases.
normal; its coefficient of variation (ratio of standard deviation to mean) reduces to about 30% from that of the original variables. When pay thickness is obtained by subtracting two depth measurements, the precision of thickness is better than the precision of the inputs.Thus rather than compounding errors, the effect is to reduce errors. One aspect of Monte Carlo simulation that always needs to be considered is possible correlation between input variables. In the present context, the question is whether a large value for Top would typically be paired with a large or small value for Bottom.That is, if we overestimate the depth of Top, would we be more likely to overestimate or underestimate the value of Bottom or would the values be uncorrelated (independent)? When they were asked, the geologists said that if anything, the two values would be positively correlated: a large value for Top would tend to be paired with a large value for Bottom. Essentially, they argued that they would consistently select the top or the midpoint or the bottom of a depth curve. How would that impact the results? We reran the simulation using a rank correlation coefficient of 0.7 between the two inputs in each of the three cases (normal, triangular, uniform shapes). The results, shown in the lower portion of Table 1, show a dramatic effect: the error intervals decrease by nearly 50% from the uncorrelated cases.

The Central Limit Theorem

Let Y = X1 + X2 ++ Xn and Z = Y/n where X1, X2, , Xn are independent, identical random variables each with mean and standard deviation .Then: both Y and Z are approximately normally distributed; the respective means of Y and Z are n* and ; and the respective standard deviations are n approximately n and This approximation improves as n increases. Note that this says the coefficient of variation, the ratio of standard deviation to mean, shrinks by a factor of n for Y. Even if the Xs are not identical or independent, the result is still approximately true: adding distributions results in a distribution that is approximately normal even if the summands are not symmetr ic. Moreover, coefficient of variation diminishes. When is the approximation not good? Two conditions will retard this process: a few dominant distributions, strong correlation among the inputs. Some illustrations may help. For instance, we take 10 identical lognormal distributions each having mean 100 and standard deviation 40 (thus with coefficient of variation, CV, of 0.40). The sum of these distributions has mean 1,000 and standard deviation 131.4 so CV=.131, which is very close to 0.40/sqrt(10) or 0.127. On the other hand, if we replace three of the summands with more dominant distributions, say each having a mean of 1,000 and varying standard deviations of 250, 300 and 400, say, then the sum has a mean of 3,700 and standard deviation, 575, yielding a CV of .16. The sum of standard deviations divided by the square root of 10 would be 389, not very close to the actual standard deviation. It would make more sense to divide the sum by the square of root of 3 acknowledging the dominance of three of the summands. As one can find by simulation, however, even in this case, the sum is still reasonably symmetric. x

Conclusions
The Central Limit Theorem (see below) can be applied to measurement errors from seismic interpretations. When specifying a range of error for an estimate, we should be interested in a practical range, one that would guarantee the true value would lie in the given range 90%, 95% or 99% of the time. When we estimate thickness by subtracting one depth from another, the error range of the result is about 30% smaller than the error range of the depths. If the depth measurements are positively correlated, as is sometimes thought to be the case, this range of the thickness decreases by another 50%.
Risk Analysis

11

Risk Analysis: Bayes Theorem

Bayes Theorem

Pitfalls
D

o you know how to revise the probability of success for a follow up well? Consider two prospects, A and B, each having a chance of success, P(A) and P(B). Sometimes the prospects are independent in the sense that the success of one has no bearing on the success of the other. This would surely be the case if the prospects were in different basins. Other times, however, say when they share a common source rock, the success of A would cause us to revise the chance of success of B. Classic probability theory provides us with the notation for the (conditional) probability of B given A, P(B|A), as well as the (joint) probability of both being successful, P(A&B). Our interest lies in the manner in which we revise our estimates. In particular, we will ask: how much better can we make the chance of B when A is successful? That is, how large can P(B|A) be relative to P(B); and if we revise the chance of B upward when A is a success, how much can or should we revise the chance of B downward when A is a failure? As we shall see, there are limits to these revisions, stemming from BayesTheorem. Bayes Theorem regulates the way two or more events depend on one another, using conditional probability, P(A|B), and joint probability, P(A&B). It addresses independence and partial dependence between pairs of events. The formal statement, shown here, has numerous applications in the oil and gas industry.

Bayes Theorem
1. P(B|A)= P(A|B)*P(B)/P(A) 2. P(A) = P(A&B1) + P(A&B2) + + P(A&Bn), where B1, B2, Bn are mutually exclusive and exhaustive We can rewrite part 2 if we use the fact that: P(A&B) = P(A|B)*P(B) 2. P(A) = P(A|B1)*P(B1) + P(A|B2)*P(B2) + +P(A|Bn)*P(Bn) Part 1 says that we can calculate the conditional probability in one direction, provided we know the conditional probability in the other direction along with the two unconditional probabilities. It can be derived from the definition of joint probability, which can be written backward and forward. P(B|A)P(A) = P(A&B) = P(B&A) = P(A|B)P(B) Part 2 says that if A can happen in conjunction with one and only one of the Bis, then we can calculate the probability of A by summing the various joint probabilities. There are numerous applications of Bayes Theorem. Aside from the two drilling prospects mentioned above, one well-known situation is the role BayesTheorem plays in estimating the value of information, usually done with decision trees. In that context, the revised probabilities acknowledge the additional information, which might fall into the mutually exclusive and exhaustive categories of good news and bad news (and sometimes no news). Further, we can define P(~A) to be the probability that prospect A fails (or in a more general context that event A does not occur). A rather obvious fact is that

Bayes Theorem regulates the way two or more

events depend on one another, using conditional probability, P(A|B), and joint probability, P(A&B).
12
Risk Analysis

Risk Analysis: Bayes Theorem

P(A) + P(~A) = 1, which says that either A happens or it doesnt. The events A and ~A are e x h a u s t i ve a n d m u t u a l l y exclusive. Almost as obvious is the similar situation P(A|B) + P(~A|B) = 1, which says that once B happens, either A happens or it doesnt. Armed with these simple tools, we can point out some common pitfalls in estimating probabilities for geologically related prospects.

Success B Drill B Fail B Success A 30% divest Drill A Success B Drill B Fail B Fail A 70% divest Divest 3%

60% 40%

0 0 0 0

0 0 1 0

97%

Pitfall 1: Upgrading the prospect too much

Suppose we believe the prospects are highly dependent on each other, because they have a common source and a common potential seal. Suppose P(A)=.2, P(B) = .1, and P(B|A)= .6 This is the type of revised estimate people tend to make when they believe A and B are highly correlated.The success of A proves the common uncertainties and makes B much more likely. But, consider the direct application of Bayes Theorem: P(A|B) = P(B|A)*P(A)/P(B) = (.6)*(.2)/.1 = 1.2 Since no event, conditional or otherwise, can have a probability exceeding 1.0, we have reached a contradiction, which we can blame on the assumptions.

Figure 1. Decision tree showing conditional probabilities

Pitfall 2: Estimating revised P(S) after bad news

Suppose P(A)=.3, P(B) = .2, and P(B|A)= .6 This is precisely the situation depicted in Figure 1. In other words, we believe the two prospects are dependent and upgrade the chances for B once A is a success. But what if A fails? Then, what is P(B|~A)? Clearly, one should instead downgrade Bs chance of success when A fails. Some people assign a value to P(B|~A), but they should exercise great caution when they do so. Here is why. BayesTheorem says P(B|-A)=P(-A|B)*P(B)/ P(-A) But P(~A) = 1-P(A) = 0.7 and P(~A|B)=1-P(A|B) = 1-P(B|A)*P(A)/ P(B) = 1-.6*.3/.2 = .1 Thus P(B|~A) = (.1)*(.2)/.7 = .03 The point is that this value already is completely determined from our other assumptions; we cannot just assign a value to it we are too late. Not only that, but .03 is probably much less than most people would guess. In summary, these two common examples point out both the power of Bayes Theorem to provide us with informed decisions, and the ease with which casual estimates of our chances can lead us. x

What is the problem?

When two prospects are highly correlated, they must have similar probabilities; one cannot be twice as probable as the other. Another way of looking at this is to resolve the equations: P(A|B)/P(A) = P(B|A)/P(B), which says that the relative increase in probability is identical for both A and B.

Decision tree interpretation

Conditional probabilities arise naturally in decision trees. For any branch along which there is a sequence of chance node, choice node, chance node, the branches emanating from the second chance node represent events with conditional probabilities. Figure 1 shows a simple example of a two-prospect tree.
Risk Analysis

13

Risk Analysis: Invoking Tools

Decision Trees vs.

Monte Carlo Simulation

D
D

ecision trees and Monte Carlo simulation are the two principal tools of risk analysis. Sometimes users apply one tool when the other would be more helpful. Sometimes it makes sense to invoke both tools. After a brief review of their objectives, methods and outputs, we illustrate both proper and improper applications of these well-tested procedures.

Decision trees: Objectives, methodology, results

Decision trees select between competing alternative choices, finding the choice and subsequent choices along a path which either maximizes value or minimizes cost. The output of a tree is a combination of the optimal path and the expected value of that path. Thus, decision trees yield one number. The rules of solving the tree force acceptance of the path with the highest expected value, regardless of its uncertainty. In theory, preference functions can be used in place of monetary values, but in practice, users seldom go to this level. Decision trees allow limited sensitivity analysis, in which the user perturbs either some of the assigned values or some of the assigned probabilities, while monitoring the overall tree value. Typically, the user varies one or two parameters simultaneously and illustrates the results with graphs in the plane (how the tree value changes when one assigned value is varied) or in space (how the tree value varies when two assigned

values are varied together).The traditional tornado chart also is used to show how each perturbed variable affects the tree value when all other values are held fixed. This chart takes its name from the shape it assumes when the influences of the perturbed variables are stacked as lines or bars, with the largest on top. Trees, along with their cousins influence diagrams, are particularly popular for framing problems and reaching consensus. For small to moderate size problems, the picture of the tree is an effective means of communication. One of the most important problem types solvable by trees is assessing the value of information. In this case, one possible choice is to buy additional information (seismic interpretation, well test, logs, pilot floods). Solving the tree with and without this added-information branch and taking the difference between the two expected values yields the value added by the information. If the information can be bought for less than its imputed value, it is a good deal.

Monte Carlo simulation: Objectives, methodology, results

Monte Carlo models focus on one or more objective functions or outputs. Favorite outputs include reserves, total cost, total time and net present value (NPV). Their respective inputs include areal extent, net pay and porosity; line item costs; activity times; and production

Trees, along with their cousins influence diagrams,

are particularly popular for framing problems and reaching consensus.
14
Risk Analysis

Risk Analysis: Invoking Tools

P0 P5 Area 976.3 1,300 Pay 10.75 20 Recovery 76.9 100

Mode 2,000 40 150

P95 2,700 60 200

P100 3,023.7 69.25 223.1

big Recov

25.0% 200

big Pay

25.0% 60
med Recov

50.0% 150 25.0% 100

Table 1. Defining parameters for inputs (Area, Pay, Recovery) to Monte Carlo model
big Area

small Recov

25.0% 2700
med Pay

forecasts, price forecasts, capital forecasts and operating expense forecasts. A Monte Carlo (MC) simulation is the process of creating a few thousand realizations of the model by simultaneously sampling values from the input distributions.The results of such an MC simulation typically include three items: a distribution for each designated output, a sensitivity chart listing the key variables ranked by their correlation with a targeted output, and various graphs and statistical summaries featuring the outputs. Unlike decision trees, MC simulations do not explicitly recommend a course of action or make a decision. Sometimes, however, when there are competing alternatives, an overlay chart is used to display the corresponding cumulative distributions, in order to compare their levels of uncertainty and their various percentiles.

50.0% 40 25.0% 20

small Pay

Success

99.0% 0
med Area

50.0% 2000 25.0% 1300

small Area

tree#1 Failure 1.0% % 0

Figure1. A decision tree begging to become a Monte Carlo model.

Example: A tree that can be converted to simulation model

Sometimes a decision tree can be reconstructed as a Monte Carlo simulation. This is especially true of trees with one root-decision node. The simulation would present the result as a distribution, whereas the tree solution would only give the mean value of the distribution.Take, for example, the tree in Figure 1, where the object is to estimate reserves. Note the pattern of a sequence of chance nodes without interspersed choices. This is a giveaway for conversion. Incidentally, a + sign signifies where a node and its subsequent branches and nodes have been collapsed. In this case, imagine copying the node at big area and pasting it at the med area and small area nodes.We shall convert this tree to a simulation and compare the nature and extent of information that these two models provide us. In fact, the tree of Figure 1 is not a standard decision tree. Instead of the convention of adding the values as one moves out along a path, this tree multiplies them together to get barrels.The output is simply the expected value obtained by considering the possible combinations of area, pay and recovery
Risk Analysis

obtained by following some path, multiplying each of them by the corresponding probability, which is obtained by taking the product of the branch probabilities, and summing these weighted values. In the tree, each parameter is discretized to only three representative values, signifying small, medium, and large. Thus, area can take on the values 1,300, 2,000 or 2,700 acres. Of course in reality, area would be a continuous variable, taking on any value in between these three numbers. In fact, most people would argue 1,300 is not an absolute minimum, and 2,700 is not an absolute maximum.They might be more like P10 and P90 or P5 and P95 estimates.We can think of a small area being is some range, say from 1,000 to 1,500 with 1,300 being a suitable representative of that class. Similarly, 2,700 might represent the class from 2,500 to 3,000 acres. Each of these subranges of the entire range carries its own probability of occurrence. For simplicity, we have made all the triples of values symmetric (for example, 1,300, 2,000 and 2,700 are equally spaced), but they could be anything. For instance, area could have the values 1,300, 2,000 and 3,500 for small, medium and large. Likewise, we have assigned equal weights to the small and large representatives, again for simplicity and ease of comparison. We have made another simplification: we assume all possible combinations are realizable. Sometimes, the large value of area would be paired with three

15

Risk Analysis: Invoking Tools

relatively large values of pay, in the belief these two parameters are dependent.This is simple to accommodate in a tree. between 0 and +1, for example, then during the simulation, realizations sampling larger values of area would tend to be matched with larger values of pay. This technique is routine, simple to implement and can use historical data when it is available to estimate the correlation coefficients.The resulting simulation in this case would feature more extreme values (both large and small) resulting in a larger standard deviation and wider 90% confidence interval than the uncorrelated case. Somewhat surprisingly, the mean value will also increase, but not as much as the range. Introducing correlation and examining its effect is a standard exercise in Monte Carlo classes. If anything, the simulation handles these paired relationships more easily than the tree, where the dependency is more hard-wired.

Distribution for Reserves/Sample/F6

0.100 0.090 0.080 0.070 0.060 0.050 0.040 0.030 0.020 0.010 0.000

Meanv=11.96266

Building the corresponding Monte Carlo simulation model

Converting to an appropriate Monte Carlo model requires finding a suitable distribution for 0 7 14 21 28 35 each of the three inputs area, pay 5% 5% and recovery. In light of the 20.88 4.85 discussion above, we took the Figure 2. Histogram from simulation small values to be P5 and the big values to be P95. We also selected triangular distributions is each case (which is how Percentage MMSTB we obtained our P0 and P100 values). The 5% 4.9 resulting distributions are shown in Table 1. From the tree analysis,we can calculate the expected 10% 6.0 value as well as finding the two extreme values (smallest 15% 6.8 and largest) and their respective probabilities: 20% 7.5 Expected value 12MMSTB 25% 8.2 Minimum value 2.6MMSTB, P(min) = 1/64 30% 8.9 Maximum value 32.4 MMSTB, P(max) = 1/64 35% 9.5 What we cannot determine from the tree analysis is 40% 10.1 how likely the reserves would exceed 5 MMSTB, how 45% 10.8 likely t h ey wo u l d b e between 5 MMSTB and 15 50% 11.4 MMSTB, how likely they would be less than 12 55% 11.9 MMSTB, and so on. 60% 12.6 The histogram from the Monte Carlo analysis is 65% 13.4 shown in Figure 2, and its corresponding percentiles 70% 14.2 in Table 2.Thus, while the mean value coincides with 75% 15.0 the mean from the tree analysis (in part because of the 80% 16.0 symmetry of the inputs and lack of correlation), we 85% 17.3 learn much more about the range of possibilities: 90% 19.0 90% of the values lie between 4.9 and 20.9 95% 21.2 MMSTB; Table 2. Percentiles for there is about a 56% chance of finding less than output (reserves, MMSTB) 12 MMSTB (it is close to the median); from simulation there is only about a 20% chance of exceeding 16 MMSTB; and

Example: Comparing alternative mud systems

Sometimes a problem can be described with a tree and enhanced with a simulation.A case in point would be a drilling cost estimate where there is a choice of mud systems. Consider the following problem. For 4 years, you have been drilling in a field where there is a danger of differential sticking. Of the 20 wells drilled to date, six of them encountered stuck pipe. In five of those wells, the drillers fixed the problem, spending from 4 to 18 days of rig time with an average of 9 days. In one well, they eventually had to sidetrack at a marginal cost U.S. $700,000 for materials and 12 days of rig time beyond the attempt to free the stuck pipe, which had been 18 days. Average time to drill the wells is approaching 45 days.The rig rate is $60,000/day. One option for the next well is to use oil based mud, which would greatly reduce the chance of stuck pipe (to 1/10) and the time to fix it (to 6 days), but cost $70,000/day, Ordinarily, one would look to the decision tree to compare these two alternatives. First, we calculate the expected value of drilling with the conventional water based mud system for the three cases: no problem, stuck but fixed and sidetrack. No problem cost: (45 days) * ($60,000) = $2,700,000 Stuck and fixed cost: (45 + 9 days)*($60,000) = $3,240,000 Sidetrack cost: (45 + 30)*($60,000) + $700,000 = $5,200,000 The respective probabilities we assign are 14/20, 5/20 and 1/20. Next, we estimate the costs and probabilities for the oil based mud system.
Risk Analysis

Modifying the tree to account for dependency among inputs

Consider the modified tree branches shown in Figure 3, where the user believes larger areas correspond to larger net pays. Without going into detail, the corresponding Monte Carlo simulation would handle this relationship by specifying a number between -1 and +1 to indicate the degree of correlation between the inputs. If the correlation were

16

Risk Analysis: Invoking Tools

No problem cost: (45 days)*($70,000) = $ 3,150,000 Cost when stuck pipe: (45+6)*($70,000) = $3,570,000 The respective probabilities are 9/10 and 1/10. The resulting tree is shown in Figure 3, indicating the correct decision would be to use the water based mud with an expected value of $2,960,000 rather than the oil-based alternative with an expected value of $ 3,171,000.
P(StuckWater) P(SideTrack) P(StuckOil) DayRateWater DayRateOil DaysWater_NoStuck DaysOil_NoStuck DaysSideTrack StuckWater? SideTrack? StuckOil? DayWater_Stuck DaysOil_Stuck ExtraCost_ST CostWater CostOil TimeWater TimeOil 30.0% 5% 5% Sample $ 60 $ 70 45.0 45.0 12.0 0 0 0 10.4 7.2 700.0 $ 2,700 $ 3,150 45.0 45.0

The Monte Carlo approach

The Monte Carlo model shown in Table 3 captures the same information as the tree model, but uses it differently. Specifically, the simulation model allows the user to estimate ranges for the various activities and costs. For comparison purposes, the historical minimum and maximum values are treated as P5 and P95 to build input distributions, with the option for the user to set these percentages. Most people estimating ranges of this sort, especially cost estimators, tend toward right-skewed distributions. Certainly, in this model, we should care about more than just the expected value. For instance, we may want to know the probability of exceeding a certain value, such as $4,000,000.We may also want to question other inputs (see discussion below). Figure 5 shows the comparison with the assumptions taken from the problem statement. Figure 6 shows a variation where the oil-based mud takes only 40 days on average to drill and the chance of sidetrack with water mud is 10%, not 5%. The difference between the two cases is clear: as the probability of sidetracking increases, the oil-based mud presents a less risky alternative (the density function of outcomes is much narrower). Similar analyses can be made easily in the Monte Carlo model.

LowP HighP Plow 60 70 40.5 40.5 10.8 1 = stuck 1=sidetrack 1 = stuck 4 3

5 95 Mode 60 70 45 45 12

Phigh 60 70 49.5 49.5 13.2

8 6

18 12

Table 3. Monte Carlo model to compare oil based and water based mud systems

Which model is better in this instance?

Both the tree and the simulation offer information to the user and to the larger audience of the presentations that can be prepared from them. The tree emphasizes the two contrasting choices and identifies the extreme outcomes and their probabilities of occurrence. It is both simple and effective.Yet, the tree does not explicitly acknowledge the underlying uncertainty in each of the contributory estimates (such as days for problem free drilling, cost of mud system, days for trying to free the stuck pipe, days for sidetracking). The user can handle these uncertainties one or two at a time by using the tree framework to do a trial-and-error sensitivity
Risk Analysis

analysis, and any user Decision Trees Monte Carlo simulation would be remiss in Objectives make decisions quantify uncertainty avoiding this. But the Inputs discrete scenario distributions overall uncertainty is Solution driven by EV run many cases made more explicit Outputs choice and EV distributions in the Monte Carlo Dependence limited treatment rank correlation simulation, where the very nature of the model begs the Table 4. Comparison between Decision trees and Monte Carlo user to specify the simulation range of uncertainty. On balance, if I had to pick one model, I would pick the simulation, in part because I have had far better results analyzing uncertainty and presenting the details to management when I use simulation.But whatever your preference in tools, for this problem the combination of both a tree and the simulation seems to be the most useful.

Which model is better in general?

When is Monte Carlo more appropriate? In general, Monte Carlo seems to have more varied applications. Any (deterministic) model you can build in a spreadsheet can be converted to a Monte Carlo model by replacing some of the input values with probability distributions. Thus, in addition to the popular resource and reserve volumetric product models, people build Monte Carlo models for costs of drilling and facilities,

17

Risk Analysis: Invoking Tools

big Pay big Area

25.0% 90

25.0% 2700
med Pay

No Problem
50.0% 60 25.0% 40 25.0% 80 +

70.0% 2700

Oil Mud
+

small Pay

TRUE 0

Chance 2960 Stuck/Fix

big Pay

25.0% 3240 5.0% 5200

Success

99.0% 0
med Area

med Pay

50.0% 40 25.0% 30 25.0% 60

tree#1
+

50.0% 2000
small Pay big Pay

Decision 2960

Stuck/Fix

+ +

tree#1

No Problem
med Pay small Area

50.0% 30 25.0% 15

95.0% 3150

25.0% 1300
small Pay

Water Mud
+

FALSE 0

Chance 3171 Stuck/Fix

Failure

1.0% % 0

5.0% 3570

Figure 3. Tree incorporating dependence between area and pay

Figure 4. Tree to compare oil based vs. water based mud systems

time for projects (up to a point, when project schedule software is more applicable), production forecasts, and all sorts of cashflows, including those with fiscal terms. Decision trees must come down to something that compares alternative choices under uncertainty.
Distribution for CostWater/Sample/E22
1.600 1.400 1.200 1.000 0.800 0.600 0.400 0.200 0.000 2 E22: Meanv=2937.594 E23: Meanv=3148.974

technical papers appear, many of these issues will be resolved.

Summary
Table 2 summarizes some of the features of the two methods, illustrating their principle differences. Dont be surprised to find someone using a tree solution to a problem you elect to solve with simulation or vice versa. Do try to use common sense and do as much sensitivity analysis as you can regardless of your choice. x

Combining simulation and trees

It is now possible to build a decision tree where some or all of the branches from chance nodes contain probability distributions rather than values.Then one can run a Monte Carlo simulation, where each iteration is a new solution of the tree. This technology is relatively new and relatively untested. Although it holds promise, this combination requires the user to be expert at both decision trees and simulation. Some people raise questions of logic when we solve a tree populated with values that are not mean estimates, but rather extreme possibilities of the inputs. In time, when

Values in 10 -3

Figure 5. Comparing oil and water based mud systems

Values in 10 -3

Figure 6. New cost comparison when oil based mud results in faster drilling.

18

v v v

4 Values in Thousands

Distribution for CostWater/Sample/E22

1.600 1.400 1.200 1.000 0.800 0.600 0.400 0.200 0.000 2 E22: Meanv=2833.63 E23: Meanv=2826.767

4 Values in Thousands

Risk Analysis

Risk Analysis: Correlation

When Does Correlation

Matter?
where:

What is correlation?
Often, the input variables to our Monte Carlo models are not independent of one another. For example, consider a model that estimates reserves by taking the product of area (A), average net pay (h) and recovery (R).In some environments,the structures with larger area would tend to have thicker pay.This property should be acknowledged when samples are selected from the distributions for A and h. Moreover, a database of analogues would reveal a pattern among the pairs of values A and h. Think of a cross-plot with a general trend of increasing h when A increases, as shown in Figure 1. Such a relationship between two variables can best be described by a correlation coefficient.

Cov (1/n)
x
and

(xi
varx

x)*(yi

y)

varx (1/n)

(xi

x)2

Examples of correlated variables

Typical pairs of correlated parameters are: daily high temperature and air conditioning cost for a personal residence in New Orleans; measured depth to total depth and drilling cost for wells in the North Sea; porosity and water saturation assigned to completion intervals for wells in a given reservoir; porosity and permeability assigned to completion intervals for wells in a given reservoir; operating cost and production rate; the permeability-thickness product and the recovery efficiency for gas wells in a given field; height and weight of 10-year old males in the United States; propane price and ethane price on Friday afternoon NYMEX; propane price and Brent crude price; and square footage and sales price of used houses sold in Houston in 1995.

Excel has a function, sumproduct ({x},{y}), that takes the sum of the products of the corresponding terms of two sequences {x} and {y}. Thus, covariance is a sumproduct. The value of r lies between 1 (perfect negative correlation) and +1 (perfect positive correlation). Although there are tests for significance of the correlation coefficient, one of which we mention below, statistical significance is not the point of this discussion. Instead, we focus on the practical side, asking what difference it makes to the bottom line of a Monte Carlo model (e.g., estimating reserves or estimating cost of drilling a well) whether we include correlation. As we shall see, a correlation coefficient of 0.5 can make enough of a difference in some models to worry about it. Before we illustrate the concept, we need to point out that there is an alternate definition.

Two types of correlation - ordinary and rank

When correlation was formally defined in the late 19th century, statisticians recognized one or a few points with extreme values could unduly influence the formula for calculating r. Specifically, the contribution of a pair (xi,yi) to the covariance, namely

(xi

x) * (yi

y)

Definition of correlation coefficient

Two variables are said to have correlation coefficient r if:

r
19

cov(x, y) x*

could be an order of magnitude larger than the other terms. Charles Spearman introduced an alternative formulation, which he labeled distribution-free and called it the rank-order correlation coefficient, in contrast to the Pearson coefficient defined above.To
Risk Analysis

Risk Analysis: Correlation

obtain the Spearman coefficient, one replaces the original data with their ranks then calculates the correlation coefficient using the ranks.Tables 1 and 2 provide a simple example illustrating both methods.

Table 1. Area, porosity, and gas saturation (and their ranks) for 13 reefal structures
Name S1 M1 A1 P2 K3 D U1 P1 K2 K4 K1 U2 Z1 Area, km^2 10 24 37 6 28.8 6 34 13 60 11 58 48 108 Porosity 0.12 0.12 0.13 0.14 0.14 0.15 0.15 0.16 0.16 0.17 0.18 0.22 0.22 Sg 0.77 0.85 0.87 0.81 0.91 0.61 0.82 0.81 0.78 0.83 0.80 0.75 0.89 Rank_Area 3 6 9 1 7 1 8 5 12 4 11 10 13 Rank_por Rank_Sg 1 1 3 4 5 6 7 8 9 10 11 12 12 3 10 11 6 13 1 8 6 4 9 5 2 12

Impact of correlation on the output of a model

What does correlation do to the bottom line? Does it alter the distribution of reserves or cost or net present value (NPV), which is, after all, the objective of the model? If so, how? We can make some generalizations, but remember Oliver Wendell Holmess admonition, No generalization is worth a damnincluding this one. First, a positive correlation between two inputs will result in more pairs of two large values and more pairs of two small values. If those variables are multiplied together in the model, for example, a reserves model, then this results in more extreme values of the output. Even in a summation or aggregation model (aggregating production from different wells or fields, aggregating reserves, estimating total cost by summing line items and estimating total time), positive correlation between two summands will cause the output to be more disperse. In short, in either a product or aggregation model, a positive correlation between two pairs of variables will increase the standard variation of the output. The surprising thing is what happens to the mean value of the output when correlation is included in the model. For product models, positive correlation between factors will increase the mean value of the output. For aggregation models, the mean value of the output is not affected by correlation among the summands. Let us hasten to add that many models are neither pure products nor pure sums, but rather complex algebraic combinations of the various inputs.

Table 2. Correlation and Rank Correlation coefficients for the reefal structure data.
Ordinary r Area Porosity Sg Area 1 0.67 0.36 Porosity 1 -0.02 Sg Rank r Area Porosity Sg Area 1 0.57 0.28 Porosity 1 -0.12 Sg

Each student is assigned a correlation coefficient, from 0 to 0.7, and uses it to correlate A with h and h with r (the same coefficient for both pairs).The students run the simulations using both these correlations and call out their results, which are tallied in Table Y:
r 0 .1 .2 .3 .4 .5 .6 .7 mean 17.5 17.9 18.1 18.3 18.6 18.8 19.0 19.3 StDev 7.7 8.6 9.1 9.7 10.5 10.9 11.3 11.9

Example 1. Correlating parameters in a volumetric reserves model

A simple example will illustrate the properties of a product model. A common classroom exercise to illustrate the effect of correlation on reserves uses the model: N = AhR With: A = Triangular (1000,2000,4000) h = Triangular (20,50,100) R = Triangular (80,120,200)
Risk Analysis

Now of course, even if both correlations were warranted, it would be unlikely they would be identical. Nevertheless, we must note the standard deviation can increase by as much as 50% and the mean by about 10% under substantial correlation. The message is clear: pay attention to correlations in volumetric product models if you care about the danger of easily underestimating the mean value by 5% or more, or if you care about understating the inherent uncertainty in the prospect by 30% or 40%. Almost as important: often correlation makes little difference in the results.We just need to check it out. While it may not be obvious, nearly all the correlations in reserves models cause the dispersion

20

Risk Analysis: Correlation

(and the mean) to increase. It is far less common for the correlations to have the opposite effect. In more complex models, however, it is possible for cor relations to actually reduce unPositive correlation certainty. One example 120 we discovered several 100 years ago is a capital 80 project involving con60 struction and operation 40 of a gas-fired electric 20 plant. The high positive 0 correlation between 0 500 1,000 1,500 2,00 2,500 3,000 price of natural gas (the Area principal operating cost) Figure 1. Correlation between Area and Pay and price of electricity (the principal factor in Positive correlation revenue) will cause the 0.24 output, NPV, to be far 0.22 more modest in range 0.20 (on the order of 50%) 0.18 when the correlation is 0.16 included, making the 0.14 investment less risky. 0.12 With a cost model, 0.10 0 20 40 60 80 100 120 adding cor relations Figure 2. Original data, r = 0.67 between line items is one of several refinements that can increase R the range of total costs. Correlation coefficients 13 12 in this case are often 11 10 thought to be higher 9 8 than those in reserve 6 models. For instance, 4 steel prices can influ2 ence numerous line 1 2 3 4 5 6 7 8 9 10 11 12 13 items, thereby resulting Figure 3. Ranks of data r = 0.57 in a positive correlation among the pairs. One of our clients used a Correlations for NPV cost model with a matrix 0.694 Production1/D4 of risk factors vs. line items. When the same Investment/D3 -0.528 risk factor influenced Price1/D6 two or more line items, 0.432 these items became Decline Rate/D5 -0.074 positively correlated. -1 -0.5 0 0.5 1 Positive correlation Correlation Coefficients (the usual type) among Figure 4. Sensitivity chart line items will cause the total cost to have a bigger range, but the mean cost is unaffected by correlation of any kind. It should be noted that correlation can have a serious impact on cost models in terms of contingency. Some people define contingency for the total project as the difference between some large percentile, say P85 or P90,and a base value like P50 or the mean total cost. This contingency could easily grow substantially, though, because when we correlate line items, the mean (and to a large degree the P50) remains unchanged but the dispersion (typically) increases.

Pay

Estimating the correlation coefficient

Once you decide to include correlation in a model, you must find a way to estimate the coefficients.This issue is similar to the more basic issue of what distribution to use for a given input variable. In both cases, you can resort to one or more of the following: empirical data; experience; and fundamental principles. Sometimes we have adequate and appropriate field data,in which case,we can calculate the (rank order) correlation coefficient. Likewise, we can use curve-fitting procedures (both popular Monte Carlo software packages have them) to generate the probability distribution. If we have no data or have appropriate data but only a small sample, we should temper our automatic processes with experience. What does this kind of parameter usually look like: is it symmetric or skewed, small or large? And for correlation, how do these two parameters usually relate to each other: positively or negatively? Once in a great while, we can appeal to that higher power: fundamental principles. For example: this parameter is a sum or average of other parameters; thus it should be approximately normal; this parameter is a product of other parameters it should be skewed right; these two prices represent substitutable commodities, therefore they should be positively correlated; and these two parameters represent competing quantities and should be negatively correlated. Remember, the correlation coefficient is limited to the range [-1,+1]. Anything in the range [-0.15, +0.15] could be noise and is unlikely to impact the results. It is rare to have correlation coefficients above 0.8 or below 0.8. So, make an educated guess, and run the model with several alternate
Risk Analysis

21

Risk Analysis: Correlation

values and document the results. You may not resolve the issue, but you can defuse it.

Summary
Things to remember: correlation is easy to calculate in Excel (the function Correl); there are two types of correlation coefficients: ordinary (Pearson) and rank-order Figure 5. Negative correlation: investment and NPV (Spearman). They tend to differ when one or both the variables is highly skewed; correlation might matter; it depends on the type of model and the strength of the correlation; correlation always will affect the standard deviation, often but not always increase it; correlation will affect the mean value of a product; and correlation is useful to describe sensitivity of output to inputs. x

Another use of correlation: Sensitivity

One argument for studying correlation is to better understand sensitivity analysis. Figure 4 shows one common type of sensitivity chart, sometimes called a tornado chart. In this case, the numbers represent rank correlation coefficients between the target output variable (in case a model has two or more outputs) and each of the inputs. A negative correlation coefficient indicates a model input that tends to increase as the output decreases and vice versa. Thus, as capital gets large, NPV tends to get small. The relationship is properly described by correlation, because the other three inputs also affect NPV. Simply knowing that capital is large does not force NPV to have a certain small value. But given a particular, large value of capital might limit the range of NPV to a smallish part of the entire NPV range, a fact made clear by Figure 5.

Risk Analysis

22

Risk Analysis: Risked Reserves

Beware of

R
R

Risked Reserves
isked reserves is a phrase we hear a lot these days. It can have at least three meanings: 1. risked reserves might be the product of the probability of success, P(S), and the mean value of reserves in case of a discovery. In this case, risked reserves is a single value; 2. risked reserves might be the probability distribution obtained by scaling down all the values by a factor of P(S); or 3. risked reserves might be a distribution with a spike at 0 having probability P(S) and a reduced probability distribution of the success case. Take as an example Exploration Prospect A. It has a 30% chance of success. If successful, then its reserves can be characterized as in Figure 1, a lognormal distribution with a mean of 200,000 STB (stock tank barrels) and a standard deviation of 40,000 STB. Then: definition 1 yields the single number 0.3*200,000 = 60,000 STB; definition 2 yields a lognormal definition with a mean of 60,000 and a standard deviation of 12,000 (See Figure 2); and definition 3 is the hybrid distribution shown in Figure 3. By contrast, suppose another prospect, B, has a 15% chance of success and a reserves distribution with a mean of 400,000 STB and a standard deviation of 200,000 STB.Then under definition 1,B would yield the same risk reserves as A, 0.15*400,000 = 60,000 STB. However, consider Figure 2, which shows how B would be scaled compared with A,with the same mean but larger standard deviation.And Figure 4 shows how the original distributions compare. Assigning these two prospects the same number for the purpose of any sort of ranking could be misleading. Prospect B is much riskier, both in the sense that it has only half the probability of success than does A, and also because even if it is a success, the range of possible outcomes is much broader. In fact, the P10, where P=Percentile, of Prospect B equals the P50 of Prospect A.Thus, if you drilled several Prospect A types,for fully half of your successes (on average),the reserves would be less than the 10th percentile of one prospect B. The only thing equal about Prospects A and B is that, in the long run, several prospects similar to Prospect A would yield the same average reserves as several other prospects like B. Even this is deceptive, because the range of possible outcomes for several prospects like A is much different from the range of
Risk Analysis

Figure 1.Lognormal distribution for Prospect A reserves

Figure 2.Comparing the original distributions for A and B

23

Risk Analysis: Risked Reserves

possible outcomes of B-types. For instance, if we consider a program of five wells similar to A, there is a 51% chance of at least one success, and a 9% chance of success on two or more. However, with five prospects like B, the corresponding chances are 26% and 2% assuming they are geologically independent.

Running economics
What kind of economics these two prospects would generate is another story. Prospects like A would provide smaller discoveries more consistent in size. They would require different development plans and have different economies of scale than would prospects like B. So, does that mean we should run economics? Well, yes, of course, but the question is with what values of reserves do we run economics? Certainly not with risked reserves according to definition 1,which is not reality at all.We would never have a discovery with 60,000 STB. Our discoveries for A would range from about 120,000 STB to 310,000 STB and for B from about 180,000 STB to 780,000 STB (we are using the P5 and P95 values of the distributions). So, surely, we must run economics for very different cases.We could take a few typical discovery sizes for A (or B), figure a production schedule, assign some capital for wells and facilities, sprinkle in some operating expenses and calculate net present value (NPV) at 10% and IRR (internal rate of return). My preference is not to run a few typical economics cases and then average them. Even if you have the percentiles correct for reserves, why should you think those carry over to the same percentiles for NPV or IRR? Rather, I prefer to run probabilistic economics. That is, build a cashflow model containing the reserves component as well as appropriate development plans. On each iteration, the field size and perhaps the sampled area might determine a suitable development plan, which would generate capital (facilities and drilling schedule), operating expense and production schedule the ingredients, along with prices, for cashflow. The outputs would include distributions for NPV and IRR. Comparing the outputs for A and B would allow us to answer questions like: what is the chance of making money with A or B? What is the probability that NPV>0? and what is the chance of exceeding our hurdle rate for IRR? The answers to these questions together with the comparison of the reserves distributions would give us much more information for decision-making or
Risk Analysis

Figure 3.Hybrid distribution for A showing spike at 0 for failure case

Figure 4. Original distributions for A and B

ranking prospects. Moreover, the process would indicate the drivers of NPV and of reserves, leading to questions of management of risks.

Summary
The phrase risked reserves is ambiguous. Clarifying its meaning will help avoid miscommunication. Especially when comparing two prospects, one must recognize the range of possibilities inherent in any multiple-prospect program. Development plans must be designed for real cases not for field sizes scaled down by chance of success. Full-scale probabilistic economics requires the various components of the model be connected properly to avoid creating inappropriate realizations.The benefits of probabilistic cashflow models, however, are significant, allowing us to make informed decisions about the likelihood of attaining specific goals. x

24