Sei sulla pagina 1di 9

BACI design

Eric P. Smith

Volume 1, pp 141–148

in

Encyclopedia of Environmetrics (ISBN 0471 899976)

Edited by

Abdel H. El-Shaarawi and Walter W. Piegorsch

John Wiley & Sons, Ltd, Chichester, 2002

BACI design

The purposes of impact assessment are to evaluate whether or not a stress has changed the environ- ment, to determine which components are adversely affected, and to estimate the magnitude of the effects. Evaluating change in environmental conditions is often difficult, due to several factors. It is often not clear which environmental component will be affected by the stressor, what type of change will occur and what the exposure will be. Choices must be made about where and when the potential effect will occur (i.e. define the spatial and temporal extent), what organisms will be affected (fish, plants, etc.), what the exposure will be (magnitude, duration), what any mitigating factors could be (what affects distri- bution of exposure) and how may these factors alter exposure and effect. Change in the environment is natural and variation due to natural effects may be great. For example, suppose an industrial plant wishes to discharge treated effluent into a river. The spa- tial extent of the effluent is not constant and will depend on the flow. The measurement of the effect may be difficult. Decisions need to be made about which organisms to measure and how the organisms will be affected. If fish are selected, their abundance may be limited by reductions in survival or avoidance of the affected area. Fish abundance may be increased due to compositional changes in predators and com- peting fish or increased prey (tolerant species might increase while intolerants decrease). Thus the defini- tion of impact may be difficult. The view in impact assessment has become ‘any change in means that is correlatable to the start of some new human activity must be considered an environmental impact’ [24]. The evaluation of impact involves comparative methods. Early approaches to impact assessment involved the use of computer simulation models to predict the impact [22]. Decisions were then made based on the soundness of those predictions [12]. Much of the work involved the effect of nuclear power plants on fish and other organisms. As part of plant operation, a large amount of water is removed from rivers or other water bodies to cool the plant. Fish may be harmed when passing through the cool- ing system and the warm water that is released from the plant may affect the area near where it is released. In the late 1970s, emphasis also focused on more empirical approaches for evaluating

impact [23]. Since 1970, studies of potential impacts have increased beyond nuclear power plants and include spills, and effluent studies as well as recov- ery of systems. Although the data used to interpret effects are quite varied, the methods for analysis are often quite similar and involve comparison of impact areas with control areas. When information is avail- able prior to the potential impact, the design is often referred to as a Before–After Control-Impact (BACI) design. Several variations on the basic design have been proposed and are discussed below.

Before–After Design

Green [8] described several types of designs that might be useful for detecting changes in means asso- ciated with human activity. The simplest approach involves collection of data prior to the activity and compares it with data after the activity. This corre- sponds to a before-after design (BA) (Figure 1a). In some instances, there may be no or little data prior to the activity. For example, in the case of a chem- ical spill, there may only be data on areas near the spill and most information is collected subsequent to the spill. In some instances, the data that are col- lected are over a brief period (such as 12 monthly observations before and after). The typical approach to analysis is to treat the data as independent samples and to compare the samples using a two-sample test. In other instances, there may be considerable data. For example, with air pollution studies there may be daily measurements for several years. The analyst may choose a more sophisticated methodology such as intervention analysis. Any difference found in the analysis is attributed to the activity. However, causal inference is diffi- cult for this type of study since the data collected are observational and rely on a number of assump- tions. For the analysis to be valid, the change in the measurements must be due to the activity. How- ever, trends in environmental data are common and may or may not be due to human activity. Trends may be the result of changes due to dry conditions, floods or other natural events (see Natural disas- ters). The BA design is without controls with which to compare the site, and the measured changes may actually be widespread and due to causes other than the activity. Indeed, lack of a statistical effect may actually indicate an impact since natural trends may

2 BACI design

35 35 30 30 25 25 20 20 15 15 10 10 0 0 10
35
35
30
30
25
25
20
20
15
15
10
10 0
0
10
20
30
40
50
60
10
20
30
40
50
60
(a)
Time
(b)
Time
Environmental variable
Environmental variable
40 40 30 30 20 20 10 10 0 0 10 20 30 40 50
40
40
30
30
20
20
10
10 0
0
10
20
30
40
50
60
10
20
30
40
50
60
(c)
Time
(d)
Time
Environmental variable
Environmental variable

Figure 1 Plots of patterns at site(s) describing different situations for impact assessment. (a) Example data profile for before–after analysis. Dashed line indicates time of activity. (b) Example data profile for BACI analysis, assuming no impact. (c) Example data profile for BACI analysis given an impact. (d) Example data profile for BACIP assuming impact. Lines indicate pairs of samples

be interrupted by the activity or the activity may cre- ate changes in the variance structure rather than the mean structure. A statistical model for the analysis of data, X ik , is

X ik D C ˛ i C k i

1

where is the overall mean, ˛ i is the effect of period (i D before or after), and k i represents

times within period (k D 1, 2, ,t A , for i D after

and k D 1, 2,

The analysis may be based on either a two-sample comparison test (e.g. two-sample t-test) or in terms of an analysis of variance (ANOVA) table. A skeletal

,t B for i D before).

Table 1

denoted by MS and defined as the SS/df

ANOVA table for BA model. Mean squares are

Source

SS

df

F

Period:

SS

BA

1

MS BA /MS times

Before–After

Sampling times

SS

times

t

B

C t A 2 C t A 1

 

Total

SS

Total

t

B

ANOVA table is presented in Table 1 and provides for useful extensions of the model and approach. Ideally, the times should be selected at random; however, the intervals of sampling are typically chosen to be fixed for convenience. Natural cycles such as seasonality may then confound the analysis.

BACI design

3

Before–After Design with Multiple Sites

ANOVA, given in Table 3. This model has gener- ally been referred to as the BACI model. Figure 1(b)

A

variation on this design is to sample a number

illustrates the situation where there is no effect while

of

sites rather than a single site before and after the

Figure 1(c) illustrates the impact case.

activity. A common approach is to select M sites in

the impact area and sample these in both periods. Thus, there are M t B C t A observations. A practical problem is whether to view the sites as replicates or subsamples. How they are viewed may depend partly

on the purpose of the experiment and the nature of

the potential impact. If focus is on the extent of the impact, then the sites are typically selected according

to a preset scheme and viewed as replicates. If they

are selected purely at random, then they are typically viewed as subsamples. Table 2 displays the ANOVA for the assessment, treating the sites as subsamples.

In this case, they are used to improve estimation of

the effect at the different times and only affect the test through the estimate of the mean squares for times. Green [8] suggested extending this design to in- clude information from a control site. There are two

sites or locations, one designated as the control site and the other as the impact site or site where the activ-

ity is thought to have an influence. In addition, data

are collected at both locations before and after the activity. Thus, there are a total N observations with multiple observations over time or space. Green sug- gested analyzing the resulting data using a two-factor

Table 2

ANOVA table for the BA design with subsam-

pling

Source

SS

df

F

Period:

SS

BA

1

MS BA /

Before–After

Sampling times

Replicate

Total

sites

SS

SS

SS

times

E

Total

t B C t A 2 M 1 t B

M t B C t A 1

C t A

MS times

Table 3

ANOVA table for two-factor BACI design

Source

SS

df

F

Period:

SS

BA

1

Before–After Location: Con- trol–Impact Interaction BA ð CI Error Total

SS

CI

1

SS

BACI

1

MS BACI /MS E

SS

E

N 4 N 1

SS

Total

Before–After Control-Impact Design

The above design treats the study as a fully-designed experiment. However, impact and control stations are not randomly assigned to the locations. There is only one treatment area. Hurlbert [10] criticized the anal- ysis of data from impact studies as being statistically incorrect because of this problem. An alternative is to recognize that the impact site represents a treat- ment that the researcher cannot control. One tries to assess changes in the site due to the potential impact. Thus the approach is based on comparing before and after. To account for problems of nat- ural change, the impact area is paired to another area, which is referred to as the control area. This approach was suggested by Eberhart [3] and referred to as the control–treatment pairing (CTP) design (see also [1]). The approach was popularized by Stewart- Oaten et al. [21] and became known as the BACI model, although it is better to refer to it as a BACI paired (BACIP) model to avoid confusion with the unpaired design. The layout described in [21] follows. It is assumed known when the potential impact or activity will occur. Sampling is carried out on t B occasions before the occurrence of the impact treatment and t A occa- sions after. Suppose the variable of interest is denoted by X. The data are X ijk , with i referring to the period, j the occasion and k the location of the site. The

Table 4

Data for the paired BACI design

 
 

Sampling

 

Period

occasion

Control

Impact

 

1

X

111

X

121

2

X

121

X

122

Before

Ð

Ð

Ð

t A t A C

1

X

11t A X 21 t A C1

X

12t A X 22 t A C1

t A C 2

Ð

After

Ð

Ð

N

D t B C t A

X 21 t A Ct B

X 22 t A Ct B

4 BACI design

data table for the study is given in Table 4 with example data in Table 6. Figure 1(d) illustrates the paired design where the vertical lines indicate the pairings. The structure of the problem is somewhat sim- ilar to one from the analysis of repeated measure- ments. In a typical repeated measures design, units are selected and would be assigned to a treatment (e.g. control or impact). Measurements are then made on each unit before and after the activity. In the case of the BACIP design, there is a single site with multiple times. Thus each site pair-by- time combination is treated as a unit. An important assumption is that there is independence among these combinations [19]. Given the validity of the assumptions, the model that is implied is

X ijk D C ˛ i C k i C ˇ j C ˛ˇ ij C ε ijk

2

where is the overall mean, ˛ i is the effect of period (i D before or after), k i represents times

within period k D 1, 2, ,t A , for i D after and

B for i D before , ˇ j is the effect of

k D 1, 2, ,t

location (j D control or impact), ˛ˇ ij is the inter-

action between period and location, and ε ijk repre- sents the remaining error. The ANOVA table for this experiment is given in Table 5. Stewart-Oaten et al. [21] suggested a simple way to view the analysis of this data. Note that the test on the interaction may be obtained in a simple manner in some circumstances. Suppose that the data are collected based on pairs of observations. For example, in a river a site downstream of a potential impact is paired with a control site (upstream of the potential impact). Then there are N pairs of samples with some before the effect and some after the effect.

Table 5

ANOVA table for the BACIP design

Source

SS

df

F

Period:

Before–After Times within period Location: Con- trol–Impact Interaction BA ð CI Error Total

SS BA

 

1

SS t BA

t

B

C t A 2

SS

CI

1

SS

BACI

1

MS BACI /MS E

SS

E

t

B

C t A 2

SS

Total

2 t B C t A 1

Now form differences between the pairs:

D ik D X iC j X iI k D C i C ε ik

3

where is the mean difference between control and impact, i is the change in difference from before to after, and ε ik is the error associated with the differences. Here X represents the ecologically relevant vari- able, e.g. the abundance of an organism. In many cases the data are first log transformed and the analysis may be thought of as an analysis of ratios of abundance in the control area vs. the impact area. The differences, D ik , may then be compared for the before period with the after period using a two- sample test. If there is no impact, the differences in the before period should be similar to those in the after period. If a two-sample t-test is used then the result is equivalent to the test of interaction [19, 24]. This approach has been referred to either as the paired BACI design (BACIP, [4]) or the BACI paired

Table 6 Data on fish collected to assess a possible impact. The first 12 observations correspond to the before period while the second 13 are after the plant is online

Time

Year

Control

Impact

Difference

1

75

1

1

0

2

75

0

1

1

3

75

0

0

0

4

75

2

6

4

5

75

47

103

56

6

75

63

36

27

7

75

78

6

72

8

75

16

143

127

9

75

143

145

2

10

75

28

9

19

11

75

4

15

11

12

75

0

2

2

13

76

3

1

2

14

76

28

108

80

15

76

51

117

66

16

76

17

41

24

17

76

19

23

4

18

76

40

44

4

19

76

1

4

3

20

76

2

0

2

21

77

0

0

0

22

77

0

0

0

23

77

2

26

24

24

77

2

4

2

25

77

4

10

6

BACI design

5

series (BACIPS) design [7]. An extension of this design is to take multiple measurements at each time. These would be viewed as subsamples and would add another error term to the ANOVA table [26, Table 1a]. The difference approach is valuable since sev- eral extensions are possible. Rather than viewing the study as a design and ANOVA problem, the differ- encing may be viewed as a covariate model. Mathur et al. [13] used a covariate model to study the effects of a nuclear power plant on fish. In addition to using a control site as a covariate, they also consider tem- perature and river flow. Thus a model for the analysis might be

X iI j D ˇ 0 C ˇ 1 X iC j C Z j

4

The analysis of the data would use methods from linear regression analysis (see Linear models).

Example Samples of fish were taken for a period of 12 months before and 13 months after a nuclear power plant began operations; see Table 6. The power plant is cooled by water that is drawn from a river. When the water exits the plant, its temperature is elevated. The concern is that the warmed water will adversely affect the abundance and composition of fish below the plant.

The ANOVA table for this experiment is given in Table 7. The test on interaction indicates that the effect is not significant. If the assumptions of the model are met, this would suggest lack of an impact or that the impact is small given the variation in the data. The results of an analysis based on using a t-test are given in Table 8. The test of equality of variances indicates the variances are not significantly different at the 5% level. In Table 8 note that there is little difference in the test of differences between after and before regardless of equality of variance. If a rank sum test [9] is used to analyze the data, then the test is

Table 7

ANOVA for BACIP example

 

Source

df

MS

F

P value

Before–After

1

2565

0.93

0.3438

ErrorA

23

2574

Control–Impact

1

1673

2.28

0.1444

Interaction

1

252

0.34

0.5631

ErrorB

23

733

Total

49

Table 8 Summary statistics and tests using paired data approach for example data

Period

N

Mean

SD

SE

After

13

16.07

26.82

7.44

Before

12

7.08

47.75

13.78

Variances

t

df

P value

Unequal

0.57

17.0

0.5734

Equal

0.59

23.0

0.5631

also not significant (S D 175, z D 1.01, p D 0.3124). Hence the results suggest that although there appear to be differences in the means (of about 9 fish on the average), there is large variation in the counts and hence the difference is not great enough to be significant.

Models with Measurements Over Space and Time

Models and designs applied in the area of nuclear power plant assessment often use a paired design but are more complex as additional factors are often included in the model. For example, Skalski and McKenzie [17] describe the design for the assessment of the Zion nuclear power plant. Sites in the impact zone were selected and paired with sites outside the zone. Factors affecting the pairs included: A i D status (preoperational or operational), i D 1, 2; B j D time

of sampling, j D 1, ,

pair depth,

k D 1, 2, 3; D l D relative position of the pairs (north

or south of the plant), l D 1, 2. In this case the spatial location of the sites is relevant and is treated as a factor rather than as subsampling. The analysis was based on a factorial ANOVA that included main effects and interactions of the four factors. The dependent variable was the log of the ratio of abundances. In another example, Thomas et al. [23] described a study on the potential effects of a power plant on benthic fauna in the Connecticut River near the Had- dam Neck nuclear power plant. Data were collected using grab samples at a total of nine locations, before and after the plant started operating. The data were collected at irregular intervals (generally on a weekly basis) with less data collected during the cold months. The analysis was run separately for each month and considered the following factors: A i D status (pre- operational or operational), i D 1, 2; B j D time of

12;

C k D site

6 BACI design

sampling j D 1,

control, downstream control, exposed), k D 1, 2, 3; D l k D station within site l D 1, 2 for k D 1, l D

, substrate type (sand or silt) m D 1, 2. Another variation on the BACI model that is commonly applied is the asymmetrical BACI design suggested by Underwood [24]. The BACIP model uses temporal variation as the measure of variability for evaluating change. Measurements in time are used as the units. For the above methods, model-dependent analysis is required to separate confounding factors (time and space) from potential impact effects. An extension is to use multiple sites for each impact site. Thus, instead of having a single control site, L 1 control sites and one impact site are used (for a total of L sites); see Table 9. Note that the sites are chosen to reflect locations similar to the impact site. Underwood [26] argues that the locations are selected at random and should be considered a ran- dom factor (see Random effects). He further sug- gests that the test of interaction is a general test and that alternative tests based on contrasts may be of greater interest for specific types of disturbance. For example, one may partition the BA ð CI sum of squares into components associated with before (B ð CI) and after (A ð CI). Similarly, the error may be partitioned as the sum of squares for error and inter- action associated with times within period by control- impact [T(BA) ð CI]. Underwood [25] discusses par- titioning this to test for an impact given interaction amongst the control sites over time (e.g. abundances at some control sites go up while others go down). Underwood focuses on different types of tests available through this design approach that may be valuable for different types of impacts. If the impact

12, i D 2; C k D site (upstream

,

1,

4 for k D 2, and l D 1, 2, 3 for k D 3; G m D

Table 9

ANOVA table for asymmetric BACI design

Source

SS

df

F

Period:

SS BA

 

1

Before–After Times within period Locations: Con- trols–Impact Interaction BA ð CI Error

SS t BA

t B C t A 2

 

SS

CI

L 1

SS

BACI

L

1

MS BACI /

 

MS E

SS

E

L 1 ð t B C t A 2 N 1

Total

SS

Total

effect is a pulse, then using a large number of times

is not effective and will wash out the effect. Thus,

testing the early period after the activity starts may be

warranted. Underwood provides examples of various types of hypotheses about impacts that may be tested. Stewart-Oaten and Bence [20] have criticized the additional control sites as being inefficient relative to the BACIP approach. Because measurements must be made at additional control sites, this draws resources away from the impact site. As a result, the test for an impact also has lower power.

Other Variations

There are numerous variations on the basic model that deal with different study designs and issues. Roberts [16] examined the effect of sewage treatment plants over time using a BACI model. Data were collected every six days on occurrences of fecal col- iform bacterial contamination. A factorial approach was used with an extension of the basic model, which incorporated a periodic regression. Roberts also discussed estimates of change using a covari- ate approach. Various extensions of the basic model were presented in [5] which discussed variance com- ponents and ANOVA designs for a variety of possible models. Power analysis for designs of the type con- sidered here were presented in [6] and [15].

Another extension of the basic model is to include additional impact sites. This can be quite useful when the potential impact is expected to vary over space. When the focus is primarily on the spatial extent, the design would involve selecting sites at varying distance from the source of the impact. This is often called a gradient design. Discussion and an example of this design is presented in [4]. Skalski and Robson [18] discuss detection of impact using data from mark–recapture studies (see Capture–recapture methods). For example, in the assessment of effects of power plant intakes on fish, fish are marked and sent through the intake. Estimates

of survival may then be calculated based on recovered

marks. Skalski and Robson describe how the analysis of mark–recapture information may be evaluated for

a variety of impact assessment designs. They also

discuss power analysis and study design. Multivariate extensions of the BACI approach are presented in [11], which describes an approach based on reducing the multivariate observations via methods such as canonical correspondence analysis,

BACI design

7

and using Monte Carlo tests to evaluate signifi- cance (see Simulation and Monte Carlo methods). Faith et al. [6] describe a multivariate analysis based on a gradient model and discuss design and power analysis. Although the BACI approach is typically pre- sented as a means for testing if an impact occurs, it is really a model that tests if a change has occurred (see Change, detecting). As pointed out by [14], the model may also be used to evaluate restoration projects and test if sites are improving (see Re- storation, environmental). Discussion of impact assessment for accidents is given in [27], which describes a variety of approaches for evaluating effects from accidents such as oil spills. Beyers [2] summarizes much of the concern in using these studies from a causal perspective. He cautions against using just a test statistic to infer causation and suggests a set of rules for combining the test with other evidence to make a stronger argument for causation.

References

[1]

Bernstein, B.B. & Zalenski, J. (1983). An optimum sampling design and power tests for environmental biologists, Journal of Environmental Management 30,

129–133.

[2]

Beyers, D.W. (1998). Causal inference in environmental

27–70.

[3]

impact studies, Journal of the North American Bentho- logical Society 17, 367–373. Eberhardt, L.L. (1976). Quantitative ecology and impact

assessment, Journal of Environmental Management 4,

[4]

Ellis, J.I. & Schneider, D.C. (1997). Evaluation of

[5]

a gradient sampling design for environmental impact assessment, Environmental Monitoring and Assessment 48, 157–172. Evans, J.C. & Coote, B.G. (1993). Matching sampling

[6]

designs and significance tests in environmental studies, Environmetrics 4, 413–437. Faith, D.P., Humphrey, C.L. & Dostine, P.L. (1991).

[7]

Statistical power and BACI designs in biological moni- toring: comparative evaluation of measures of commu- nity dissimilarity based on benthic macroinvertebrate communities in Rockhole Mine Creek, Northern Terri- tory, Australia, Australian Journal of Marine and Fresh- water Research 42, 589–602. Glasby, T.M. (1997). Analysing data from post-impact

[8]

studies using asymmetrical analyses of variance: a case study of epibiota on marinas, Australian Journal of Ecology 22, 448–459. Green, R.H. (1979). Sampling Design and Statistical Methods for Environmental Biologists, Wiley, Chich- ester.

[9] Hollander, M., & Wolfe, D.A. (1973). Nonparametric Statistical Methods, Wiley, New York.

[10]

Hurlbert, S.J. (1984). Pseudo-replication and the design

[11]

of ecological field experiments, Ecological Monographs 54, 187–211. Kedwards, T.J., Maund, S.J. & Chapman, P.F. (1999). Community-level analysis of ecotoxicological field studies: II. Replicated design studies, Environmental Toxicology and Chemistry 18, 158–166.

[12] Mapstone, B.D. (1995). Scalable decision rules for environmental impact studies–effect size, Type I and Type II errors, Ecological Applications 5, 401–410.

[13]

Assessment of thermal discharges on zooplankton in Conowingo Pond, Pennsylvania, Canadian Journal of Fisheries and Aquatic Science 37, 937–944. [14] Michener, W.K. (1997). Quantitatively evaluating

Mathur, D., Robbins, T.W. & Purdy, E.J. (1980).

[15]

restoration experiments: research design, statistical analysis, and data management considerations, Restora- tion Ecology 5, 324–337. Millard, S.P. & Lettenmaier, D.P. (1984). Optimal de- sign of biological sampling programs using the analysis of variance, Estuarine, Coastal and Shelf Science 22,

637–656.

[16]

Roberts, E.A. (1993). Seasonal cycles, environmental

[17]

change and BACI designs, Environmetrics 4, 209–231. Skalski, J.R. & McKenzie, D.H. (1982). A design of

[18]

aquatic monitoring systems, Journal of Environmental Management 14, 237–251. Skalski, J.R. & Robson, D.S. (1992). Techniques for

[19]

Wildlife Investigations: Design and Analysis of Capture Data, Academic Press, San Diego. Smith, E.P., Orvos, D.R. & Cairns, J.J. (1993). Impact

[20]

assessment using the before-after-control-impact (BACI): Comments and concerns, Canadian Journal of Fisheries and Aquatic Sciences 50, 627–637. Stewart-Oaten, A. & Bence, J.R. (2001). Temporal and spatial variation in environmental assessment, Ecologi- cal Monographs 71, 305–339.

[21] Stewart-Oaten, A., Murdoch, W.W. & Parker, K.R.

[22]

(1986). Environmental impact assessment: pseudorepli- cation in time? Ecology 67, 929–940. Swartzman, G., Deriso, R. & Cowan, C. (1977). Comparison of simulation models used in assess- ing the effects of power-plant-induced mortality on fish populations, in Proceedings of the Conference Assessing Effects of Power-plant-induced Mortality on

[23]

Fish Populations, Gatlinburg, Tennessee, May 3–6, W. van Winkel, ed., Pergamon Press, London. Thomas, J.M., Mahaffey, J.A., Gore, K.L. & Watson, D.G. (1978). Statistical methods used to assess biological impact at nuclear power plants, Journal of Environmental Management 7, 260–290.

[24] Underwood, A.J. (1991). Beyond BACI: experimen- tal designs for detecting human environmental impacts on temporal variations in natural populations, Aus- tralian Journal of Marine and Freshwater Research 42,

569–587.

8 BACI design

[25]

Underwood, A.J. (1992). Beyond BACI: the detection

[26]

of environmental impacts on populations in the real, but variable, world, Journal of Experimental Marine Biology and Ecology 161, 145–178. Underwood, A.J. (1994). On beyond BACI: sampling

[27]

designs that might reliably detect environmental distur- bances, Ecological Applications 4, 3–15. Wiens, J.A. & Parker, K.R. (1995). Analyzing the effects of accidental environmental impacts:

Approaches and assumptions, Ecological Applications 5, 1069–1083.

(See also Bioabundance; Ecological statistics; Ecological study design; Impact, environmental; Rivers, canals and estuaries)

ERIC P. SMITH