American Society for Quality

A Class of Three-Level Experimental Designs for Response Surface Modeling Author(s): Max D. Morris Source: Technometrics, Vol. 42, No. 2 (May, 2000), pp. 111-121 Published by: American Statistical Association and American Society for Quality Stable URL: http://www.jstor.org/stable/1271443 Accessed: 06/07/2010 17:51
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://dv1litvip.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=astata. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

American Statistical Association and American Society for Quality are collaborating with JSTOR to digitize, preserve and extend access to Technometrics.

http://dv1litvip.jstor.org

Editor's Note: This article will be presented, with oral discussion, at the 7th annual Spring Research Conference, to be held in Seattle, Washington, June 28-30, 2000.

A for

Class

Three-Level Experimental Designs Surface Response Modeling

Max D. MORRIS of Departments Statisticsand Industrial Manufacturing and Systems Engineering IowaState University Ames, IA 50011-1210
Most empirically constructed response surface models are based on polynomials containing terms of order 2 or less. Experimental designs involving three equally spaced levels of each factor are popular choices for collecting data to fit such models. Because complete three-level factorial plans require more experimental runs than can usually be accommodated in practice, smaller designs are typically used. The families of three-level designs most often used in this context are the BoxBehnken plans and various forms of the central composite designs. This article introduces a different method for constructing composite designs, motivated by notions of sequential experimentation and the minimax and maximin distance criteria used in spatial modeling. Operational and performance characteristics of some designs constructed by the method are compared to those of competing Box-Behnken and central composite plans. KEY WORDS: Augmented pairs design; Box-Behnken design; Central composite design; Maximin distance criterion; Minimax distance criterion; Response surface analysis: Sequential experiment; Small composite design.

of

In many experimental settings, the study of the relationship between a set of factors (or independent or controlled variables) and a response (or dependent variable) is based on a polynomial response surface model (e.g., Box and Draper 1987). Although higher-order terms are sometimes necessary, models composed of an intercept and linear, pure quadratic, and bilinear terms are often flexible enough to adequately represent the true underlying response surface. In such situations, experimental designs that employ three evenly spaced levels of each factor, or three-level designs, are popular because they are conceptually and operationally simple, while providing sufficient information to infer important characteristics of the response surface. Popular series of three-level designs used in this context are the BoxBehnken (1960) plans, and the "face-centered" version of the central composite designs introduced by Box and Wilson (1951). In practice, however, the estimation of a response surface model is often accomplished via an experimental process that is sequential in character. An experimental program may begin with a screening design incorporating only two levels of each factor, and subsequent experimental effort, if needed, can be focused on those factors that appear to be of practical importance. Depending on what is learned from data collected in this first stage, the design may be augmented with one or more additional stages to provide the support necessary to estimate an adequate model. Hence, three-level designs that can be formed by augmenting twolevel plans are particularly attractive because all data collected throughout the experimental process can be useful in estimating the response surface. A central composite de111

sign can be executed as a two-level factorial plan followed by additional axial points and center-point replicates in this manner. The subject of this article is a new method of constructing second-order three-level experimental plans, called augmented pairs designs, that contain standardfirst-ordertwolevel plans as subsets. Attention is focused primarily on the value of these designs as single-stage second-order response surface designs, even though their structure is motivated by the possibility that informal sequential experimentation may be desired. The construction procedure is defined and motivated in Section 1. Numerical comparisons to some competing three-level plans are presented in Section 2, and the effects of blocking are briefly addressed in Section 3. Section 4 contains an example, based on a classical textbook problem, of how an augmented pairs design might be used. 1. AUGMENTEDPAIRS DESIGNS The designs to be introduced here use three equally spaced values, coded as -1 0, and +1, for each of k factors; let A represent the design region {-1, 0, +1}k. Assume that a first-order design, preferably one that is orthogonal, has been specified in which only factor settings of -1 and +1 are used, and denote the sth run of this design by the k vector x5 and the associated n1 x k design matrix composed of these row vectors by X1. (For economy of notation, I shall
2000 American Statistical Association and the American Society for Quality TECHNOMETRICS,MAY2000, VOL. 42, NO. 2

112

MAX D. MORRIS

runs. Designs constructed in this manner shall be called even if no = 0. I also add one new design run, xst, for each augmentedpairs (AP) designs; an example in five factors is listed in Table 1. The two-level portion of this pair of runs (x,, xt), s < t, in X1 by setting plan is constructedusing five columns from an eight- 1 (Xs + Xt) run Plackett-Burman Xst (1) (1946) design; AP designs in four, six, or seven factors can also be based on this two-level Hence, for each factor, the value selected in the new run fraction. is the negativeof the averageof the two values from the sth and tth runs-- if they differ or the negativeof their 1.1 Motivation commonvalue if they are the same. Figure 1 depictsthree The goal of this article is to develop a techniquefor first-order augmenting designs to provideefficientsecondorderdesignsusing not manymorethanthe required minimumnumber runs.For relativelysmallfirst-order of plans, in nl m 1 + k, the numberof parameters the first-degree model then polynomialmodel. Because the full quadratic
Table 1. Augmented Pairs Design, k = 5 Center-point runs 0 0 0 0 0 1 -1 -1 1 -1 1 1 -1 0 0 -1 0 -1 0 1 0 1 0 0 1 0 1 0 0 1 0 1 -1 0 0 0 1 Figure 1. Three Examples in k = 3 Dimensions of How Two Points From the First-Order Design (xs and xt, denoted by open circles) Generate a ThirdAugmented Point (xst) Through Equation (1). TECHNOMETRICS,MAY2000, VOL. 42, NO. 2 0 0 0 0 0 0 0 1 1 -1 -1 1 -1 1 -1 -1 0 0 -1 0 -1 0 0 0 -1 0 -1 0 1 0 1 0 1
0

also referto the first-order designitself by X1.) First-order thatare of greatestpracticaluse here are relatively designs small but not supersaturated, with ni equal to or slightly thank + 1. greater I augment first-order the designby firstaddingno centerpoint replicates,collectivelydenotedby the no x k design matrixXo. In most cases, practitioners requirepositive will values of no, althoughsome of the designsto be discussed containreplication otherpoints,andmost arenonsingular at

examplesshowinghow an augmented designpoint(marked the darkened circle) is generatedfor a pair of existing by points (markedby the open circles) in k = 3 dimension. The n2 (1 ) new runs so determined all pairingsof by runs in X1 are collectively denotedby an n2 x k design matrix X2, and X = (XO IX)' specifies the complete IX
three-level design after augmentation, in n = no + ni + n2

Two-level design

Augmented points

0 0 0 0 0 1 1 1 -1 -1 1 -1 -1 -1 -1 0 0 -1 0 0 -1 0 0 -1 0 0 0
0

0 0 0 0 0 -1 1 1 1 -1 -1 1 -1 0 0 0 1 1 0 1 -1 -1 0 0
-1

-1
0 0

1
0

1
0

1
0

1 1
0

-1
0 0

1 1
0 0

0 -1 0 0 -1 0 0 0 -1 0 1 0
1

0
1

0 0 0 0 0 1 -1 1 1 1 -1 -1 -1 0 -1 -1 -1 0 0 0 0 0 0 1 1 1 -1 -1 0 0 0 -1 0 0 0 0 0 0 1
1 1

1
0

THREE-LEVELEXPERIMENTAL DESIGNS FOR RESPONSE SURFACE MODELING

113

contains
p

= (1 +k) +

n+

The process describedin Section 1 for constructing designs leaves open the questionof how X1 is selected.This reflects my primarygoal of suggesting a techniquefor deaugmentingfirst-orderdesigns to form second-order signs ratherthanof presentinga detailedcatalogof plans. In practice, experimentsare designed to answer several application-specific questions,many of which bear on the selectionof X1. The more generalstatisticalproperties of triples (x, xt, Xt) that are Mm, conditional on xs and xt the resultingaugmented designs, however,also dependon being fixed in advance. I have proven numericallythat the first-order plan selected.To providenumerical perforis also conditionally mM in A for k as large mancecomparisons otherdesigns,I shall consideras exto (x, Xt, Xst) as 10. (As pointedout by the associateeditorduringthe re- amplesthe AP designsbasedon X1 constructed the popas view process,similarstatements cannotbe generallymade ular orthogonalfirst-order plans characterized Plackett by when Euclideandistanceis used.) and Burman(1946). Except when k is 1 less thana multiOf course,selectinga designso thatspecified pointtriples ple of 4, these designsare not alwaysuniquebecauseonly havethese characteristics does not necessarilylead to afull a subset of the availabledesign column vectors is needed design with good overalldistanceproperties. Furthermore, to form X1. Only in the case of k = 4, however,does this the designtheorylaidout by Johnson al. (1990)was given ambiguityhave an effect on the resultspresented et here.For for spatialstochasticprocess models ratherthan for poly- k = 4, the possible designs form two internallyequivanomialmodels. Still, the notionof spreading the better designpoints lent groups; designused hereprovidessomewhat the than those in the other groupbut is not broadlyand/or uniformlythroughout design space is overallproperties in appealingand, as will be seen in the next section,leads to dominant all respects. a design construction thatis useful for quadratic rule In this study, AP designs constructedfrom minimal polynomialmodels also. Plackett-Burman to planswill be compared threeotherseries of three-level quadratic responsesurfacedesigns.These 1.2 Sequential Operation are the plans introduced Box and Behnken(1960), the by As noted in the introduction, advantage one often cited "face-centered" versionof the centralcompositedesignsof for the centralcompositedesign is the potentialto imple- Box and Wilson (1951) and the similarsmall structurally mentit in two or morestages,beginning with an orthogonal compositedesigns of Lin and Draper(1990). Examplesof two-levelplanfor screeningof factorsandpreliminary esti- the Box-Behnken(BB), centralcomposite(CC),and small mationof lineareffects. The originalversionof the central composite(SC) designs in five factorsare listed in Tables compositedesign discussedby Box andWilson (1951) uti- 2, 3, and4, respectively. designslistedin Tables1-4 are All lizes a resolution-5two-levelfractional factorialdesign (or shownwith five center-point replicates,althoughthis numfull factorialplan when k < 4), which may be executed ber can be varied on experimental depending requirements. as a first stage, followed by additional runs that allow esBB plans are constructed using two design conceptssitimationof purequadratic effects. A fractionof resolution multaneously. Whatmightbe calledthe "gross" structure of 5, however,is largerthanmost "screening" plans.In some the design is determined selectingan incompleteblock by cases it may be possibleto begin with a smallerfractionof design (IBD) in k treatments b blocks. and (This is not the resolution3 or 4, augmentingat an intermediate stage to response surfacedesign itself but an intermediate step in the resolution-5plan before addingthe final points. Other the construction is process.)A one-to-onerelationship esversions of the centralcompositeplan, called small com- tablished betweenthe treatments the IBD andthe factors of posite designs and discussed more fully later, follow the of the responsesurfaceproblem. Then,for eachblockof the general structure of the original central composite design IBD, the "fine"structure the responsesurface of design is but incorporate a smaller two-level fraction. determined selectinga (sometimesfractional) two-level by As suggested from its construction, the AP design can factorial design in the factors associatedwith these treatalso be executed in stages, beginning with a two-level frac- ments and assigningthe value of 0 to all other factors in tion. AP designs can be based on smaller (sometimes satu- theseruns.For example,the designlistedin Table2 is based
TECHNOMETRICS,MAY2000, VOL. 42, NO. 2

terms, it seems reasonable to consider augmentation schemesthat add one run for each pair of existingruns. Giventhis rationale, augmentation the specifiedin Equation (1) is at least intuitively suggested by the maximin (Mm) and minimax(mM) distancecriteriaintroduced by Johnson,Moore, and Ylvisaker(1990). The Mm criterion calls for design pointsto be selectedso as to maximizethe smallest interpairdistancebetween selected points. For a specifieddesign region, the mM criterioncalls for design points to be selected so that the maximumdistancefrom the design to a point not in the design is minimized.One definitionof distanceused in spatialanalysisis the "rectangular"or L1 distance;that is, the distancebetween xs and xt is definedas the sum of the elementsof Ixs - xtl. It is fairly easy to show that for either = {-1, 0, +l}k or A* = [-1, +1l]k,selecting xst as in Equation(1) yields

rated)two-levelfractionsthanwill usuallywork with central compositeplans. Hence, they may be more attractive when it is desirableto invest relativelylittle in the first experimental stage, and more heavily in the second if the results suggest value in continuedexperimentafirst-stage tion. Both centralcompositedesignsandAP designsallow formalblockingto accountfor differences thatmay be due to sequential the operation; blockingof AP plansis treated brieflyin Section 3.
2. 2.1 Designs NUMERICAL COMPARISONS

114

MAX D. MORRIS Table 2. Box-Behnken Design, k = 5 Table 3. Central Composite Design, k = 5 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 1 -1 1 0 0 0 0 0 0 0 0 -1 1 -1 1 0 0 0 0 -1 1 -1 1

0 0 0 0

Center-point runs

Points associated with block #1 of IBD

Points associated with block #2 of IBD

Points associated with block #3 of IBD

Points associated with block #4 of IBD

Points associated with block #5 of IBD

Points associated with block #6 of IBD

Points associated with block #7 of IBD

Points associated with block #8 of IBD

Points associated with block #9 of IBD

0 0 0 0 0 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0

0 0 0 0 0 -1 1 -1 1 0 0 0 0 0 0 0 0 0 0 0 0 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 0 0 0 0
0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 -1 1 -1 1 0 0 0 0 0 0 0 0 -1 1 -1 1 0 0 0 0 0 0 0 0 -1 -1 1 1 -1 -1 1 1
0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 1 -1 1 0 0 0 0 0 0 0 0 -1 1 -1 1 0
0 0 0

Center-point runs

Factorial portion

Axial portion

0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 -1 0 0 0 0 0 0 0 0

0 0 0 0 0 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 0 0 1 -1
0 0 0 0 0 0

0 0 0 0 0 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 0 0 0 0 1 -1 0 0 0 0

0 0 0 0 0 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 0 0 0 0 0 0 1 -1 0 0

0 0 0 0 0 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1 0 0
0 0 0 0 0 0

1 -1

Points associated with block #10 of IBD

-1 -1 1 1

-1 1 -1 1 -1 1 -1 1

aberrationresolution-5 fractional factorial designs listed in table 12.15 of Box, Hunter, and Hunter (1978, p. 410). Hartley (1959) observed that nonsingular composite designs can also be constructed using smaller fractional factorials, so long as pairs of bilinear terms (or two-factor interactions) are not confounded. Following this idea, Westlake
Table 4. Small Composite Design, k = 5 Center-point runs 0 0 0 0 0 1 -1 1 -1 -1 -1 1 1 1 -1 1 -1 1 -1 0 0 0
0 0 0 0 0

on an IBD with 10 blocks of size 2 from 5 treatments. The first block includes treatments associated with factors 1 and 2, so the first set of four runs in the response surface design is set up as a factorial design in these two factors. Each of the 10 blocks in the IBD generates 4 such runs, for a total of 40 runs aside from the added center-point runs. The BB designs used in comparisons here are, aside from the indicated blocking and number of center-point replicates, those listed in table 4 of Box and Behnken (1960); note that a design for eight factors is not given for this series. CC designs like those introduced in this article include three groups of design points (including replicates at the center). The factorial or cube portion of the design is a minimal regular fractional factorial design of resolution 5 in all k factors; for k <K4, the full factorial plan is used. The axial or star portion of the design includes two runs associated with each factor, for.which the assigned values are +1 and -1 in these two runs, whereas other factors are assigned the value of 0. The factorial portions of the CC designs used in these comparisons are the minimumTECHNOMETRICS,MAY2000, VOL. 42, NO. 2

Factorial portion (irregular)

Axial portion

0 0 0 0 0 1 1 -1 1 -1 -1 -1 1 1 1 -1 -1 0 0 1 -1 0 0 0 0 0 0

0 0 0 0 0 -1 1 1 -1 1 -1 -1 -1 1 1 1 -1 0 0 0 0 1 -1 0 0 0 0

0 0 0 0 0 1 -1 1 1 -1 1 -1 -1 -1 1 1 -1 0 0
0 0 0 0

1 -1
0 0

0 0 0 0 0 1 1 -1 1 1 -1 1 -1 -1 -1 1 -1 0 0 0 0 0 0 0 0 1 -1

THREE-LEVELEXPERIMENTAL DESIGNS FOR RESPONSE SURFACE MODELING

115

(1965), Draper(1985), and Lin and Draper(1990) introducedotherspecificcatalogsof SC designs.The SC designs employedhere are basedon two-levelfractionstakenfrom the Plackett-Burman series and specifiedin table 4 of Lin and Draper(1990). (The Lin and DraperSC listed for nine factorsis singularandso is not includedin the comparisons to follow.) Table5 lists the numberof runs and numberof degrees of freedom for replicationfor each design consideredin this comparison, assumingthat five center-point replicates are used. The value of no = 5 was selected arbitrarily as one that will provide some limited degree of background replicationfor systems in which "noise"is expectedto be moderaterelativeto "signal." shall use it in severalcomI to follow, noting situationsin which results are parisons differentif no is changed.Most designslisted substantially 4 df for "pureerror," to provide corresponding the five data valuestakenat the centerpoint.The exceptionsare the AP designs in four and five factorsand the BB design in nine factors,whichcontain6, 2, and24 pairsof replicated design points other than those at the center,respectively.When smaller design sizes are required,some of the replicated points can be eliminatedfrom these designsat the expense of some balanceand symmetry; may be especiallyapthis in the case of the BB design in nine factorswhen pealing blocking is not required.In four factors, AP designs are largerthanBB and CC plans but providemore degreesof freedom for errorestimation.The AP design in five factors is also largerthanthe CC design. For k = 3 or k > 5, however,the AP designsare smallerthantheircounterparts fromthe BB andCC series.The AP andSC designsin three factors are equivalent,and for k = 7 the AP design is acSC tually two runs smallerthanthe corresponding design. (AP designs based on saturatedfirst-order plans, such as Plackett-Burman in threeor seven factors,contain designs exactly p runs apartfrom those at the center and so are nearly saturated themselves.)For the remainingcases discussedhere,the SC designis smallerthanthe corresponding plan from any of the otherseries.
2.2 Comparison of Some Statistical Properties

n x k designmatrixX for any of the given designs,andlet xi denotethe ith elementof thatvector.(Note the departure from the notationused in Sec. 1.) I anticipatean analysis in which the responseobservedfor the corresponding exrun perimental is modeledas
k k k

Yx = 30 +
i

jiXi+ E iiEi
i

+
i<j

pijxiXj + 8,

where E is random with expectation 0 and variance a2 with independence between instances in any pair of runs, and where interest centers on least squares estimation of the p regression coefficients (e.g., bi for /i, etc.) and/or predictionof y or estimationof its expectation. To this end, let Z be the n x p model matrix in which the row corresponding x (from X) is z = to
(1,1x,...

,xk,x2,...

,x2,xlx2,.

2 displays some statisticalfigures of merit relatedto the model coefficientsfor precisionof estimatesof quadratic AP,BB, CC,andSC designscontaining = 5 center-point no
3
5

xixj, i j) elements as ZL, ZQ, and ZB, respectively; that is, Z = (1, ZL, ZQ, ZB). 2.2.1 Precision of Model CoefficientEstimates. Figure

erally, let zx be the p element row vector corresponding to any x E A, whether included in the design or not. Moreover, denotesubmatrices Z containing linof the ear (e.g., xi), pure quadratic(e.g., x2) and bilinear (e.g.,

.. Xk-lXk), and more gen-

(b 4

i SI

SI

S
n 2

I
I

(3 0
2

SI
I I I I Sl

S''-

;A

Next I shall comparethe statisticalqualityof estimates C0 associatedwith the four classes of designs describedpreviously when analysisis based on a quadratic polynomial 7 response surfacemodel. Let x denote a given row of the
Table5. Numbersof Runs and Degrees of Freedom forReplication Designs Compared in AP
k 3 4 5 6 7 8 9 10 n 15 41 41 41 41 83 83 83 dfrep 4 10 6 4 4 4 4 4 n 17 29 45 53 61 125 165 0o5

3 4 5 6 7 8 9 10 k

3 4 5 6 7 89 k
(d S,A
I

10

.' B
I

, B
I
I

3 0
CO

BB
dfrep 4 4 4 4 4 28 4 n 19 29 31 49 83 85 151 153

CC
dfrep 4 4 4 4 4 4 4 4 n 15 21 27 33 43 57 73

SC
dfrep 4 4 4 4 4 4 4

(CD

B B S S,i BS.Ai /ASI

'A

IA,I
1 3 4 5 6 7 8 9 10 k

S' '

I 'S

3 4 5 6 7 8 9 10 k

Figure2. PrecisionIndicesforAP,BB, CC,and SC Designs in 3-10 Factors,Each WithFive Center-Point Runs: (a), GSD; (b), GSDL;(c), GSDQ;(d), GSDB.
TECHNOMETRICS,MAY2000, VOL. 42, NO. 2

116

MAX D. MORRIS

replicates. Indices compared include the square root of a scaled generalized variance of the regression coefficients: GSD = /n det-/P(Z'Z). I shall call this summary measure of precision for all estimated model coefficients the generalized standard deviation. I shall also examine the square root of the scaled generalized variances of the subsets of linear, quadratic, and bilinear coefficient estimates:
GSDL -= An det(Z ZL- Z-

ZL(ZZL)-

Z ZL),

GSDQ and
GSDB

/n

det-1/

(ZQZ

- Z

Q(ZZQ)-1Z

Q)

n det -1/(x(k-'))(ZBZB

ZBZ(gBZB)

ZBZB),

where ZL is the matrix containing all columns of Z except for those in ZL, and ZQ and ZB are similarly defined. These three indices allow a more detailed examination of the precision of each group of coefficient estimates, given that the entire second-degree polynomial model must be estimated. Note that each index is defined in standard deviation units and is expressed on a per-observation basis. For a fixed value of n, designs that minimize GSD are D-optimal for the second-degree polynomial model, whereas those that minimize GSDL, GSDQ, and GSDB are Ds-optimal for estimating the corresponding subsets of coefficients for this model (e.g., Silvey 1980). Here I shall not be using these indices to identify designs that are necessarily optimal but rather as measures of comparison among alternative plans. Note that, although GSDL, GSDQ, and GSDB are intended to provide more detailed comparisons, they are still summary measures because each describes the precision with which a group of coefficients is estimated. Symmetry properties of several designs examined here are such that the standard errors of all coefficients from any one of the three groups are equal. AP designs for k = 4 5. 6, and 8, and BB designs for k = 6.9, and 10 are less symmetric, with some variation in the standarderrors of linear and bilinear terms. The largest individual coefficient standard error, however, is not more than 10% greater than the corresponding group average for the designs from these classes compared in Figure 2. Standarderrors of linear and bilinear coefficient estimates are much more variable for SC designs due to the confounding of first- and second-order effects in the fractional factorial segments of these designs, and this imbalance grows more severe for larger numbers of factors. GSD, and other monotonic functions of det(Z'Z), are probably the most often-used overall measures of fitted model precision in the response-surface-design literature. For a given value of k, panel (a) of Figure 2 shows that the generalized standarddeviation value is best (lowest) for the CC design in each case, and, except when k = 8, the BB and SC designs are worst; the AP design has a slightly
TECHNOMETRICS,MAY2000, VOL. 42, NO. 2

larger GSD than the SC plan in the case of k = 8 only. AP designs have smaller (better) values of GSD than do BB designs, except when k = 3 where the index is roughly 22% greater for the AP plan. The four design patternshave different strengths in coefficient estimation, as measured by GSDL, GSDQ, and GSDB; these indices are displayed in panels (b), (c), and (d) of Figure 2, respectively. For estimation of the coefficients of linear terms, panel (b) shows that the pattern of GSDL for the four-design series leads to a rank-ordering of preferences similar to that suggested by GSD: CC designs are best (have the lowest index values) and SC designs are worst for each value of k. Except for k = 3, where AP and SC designs are the same, and k = 6, where AP and BB designs are comparable, AP designs are superior to either BB or SC plans. The order of preference is considerably different, however, for pure quadraticcoefficients, as depicted in panel (c). Values of GSDQ produced by CC and SC designs are consistently larger than those for BB and AP designs, sometimes by a factor of 4. Precision of estimation for pure quadratic effects is especially sensitive to the number of observations taken at the center point in BB designs, as will be demonstrated more fully. When no = 5 (as depicted in Fig. 2) or less, AP designs provide the best estimates of quadratic coefficients by this measure. For larger values of no, BB designs are sometimes nearly as good as AP designs, and are the best of the four series when k = 3 and no > 2, k = 4andno > 6, or k = 7and no > 6. In all other cases, however, the AP designs offer better precision of pure quadratic coefficients. In each case CC designs provide the smallest values of GSDB, as shown in panel (d). With the marked exception of k = 3, BB designs would be generally considered worst, with AP designs generally comparable or superior to SC plans. AP designs also provide the greatest uniformity of standard errors among the different groups of coefficient estimates. For all designs listed with no = 5, the values of GSDL, GSDQ, and GSDB all fall between 1.4 and 3.4 standard deviation units for AP designs. CC designs with the same number of observations at the center point are superior for estimating first-order and bilinear coefficients, producing values of GSDL and GSDB between 1.0 and 1.6, but values of GSDQ are all more than 2.6 and increase with k to approximately 7.0 for designs with 9 and 10 factors. The reason for this deteriorationin performance with increasing k is clear from the CC design structure;information about pure quadratic effects is provided strictly through the axial runs (of which there are only two associated with each factor regardless of the number of factors considered) and runs at the center point. On a per-run basis, then, CC designs cannot be expected to perform well when k is relatively large (or even as small as 4 in some cases) and precise estimation of the response surface curvature is required. Because SC designs have the same general structure as CC designs, they also produce relatively poor estimates of pure quadraticcoefficients. Furthermore,the quality of estimates of linear and bilinear coefficients from SC designs also suffers from the lack of balance in their fractional factorial

THREE-LEVELEXPERIMENTAL DESIGNS FOR RESPONSE SURFACE MODELING

117

portions. BB designs provide more consistent precision of estimation across the groups of model coefficients than do CC designs, with values of GSDL between approximately 1.5 and 1.8 when no = 5, values of GSDQ between approximately 2.0 and 2.3, and GSDB between approximately 2.0 and 3.6 as k varies from 3 to 10. Values of GSDQ, however, are typically worse for BB designs when fewer center-point replicates are used. For example, Figure 3 shows how GSD, GSDL, GSDQ, and GSDB vary with the number of centerpoint replicates used for the four designs when k = 5. (The qualitative pattern of this collection of graphs is similar for other values of k.) Note in particular the sensitivity of GSDQ to small values of no in the BB designs. Hence, except when BB designs can be augmented with many centerpoint runs, AP designs arguably provide more uniform precision across the three types of parameterestimates than do the other three classes of designs. 2.2.2 Precision of Expected Response Estimates. For comparing the precision with which E(y) can be estimated across the experimental region, I partition A into mutually exclusive subsets, or orbits, A,: r = 0, 1. 2,..., k, where A, = {x x E A, x = r}. Hence, A, contains those points of Euclidean distance v/ from the center of A. For each such subset of points, compute the square root of the average scaled variance of the estimated expected responses, RSE,. =
3 4b)

/n

avexA,,zx(Z'Z)-l1,

3
03
C2)

and call this the response standard error for the rth orbit. As with GSD, this index is defined apart from the common factor- and is expressed on a per-observationbasis. Figure 4 contains plots of RSE, that are similar to the variance dispersion graphs of Giovannitti-Jensen and Myers (1989) for AP, BB, CC, and SC designs with no = 5 and k = 3 to 10. Within a given orbit, standard errors of the estimated response surface are relatively consistent, and often constant, for BB and CC designs. AP designs are not rotatable, and the standard errors of the estimated response surface vary more within orbits for these designs; replacing ave by max in the definition of RSE, results in increases of up to 87% for the AP plans displayed in Figure 4. Lack of rotatability, however, is even more pronounced for SC designs, for which replacing ave by max in the definition of RSE, increases these values by up to 273% in some orbits. For each value of k, the average precision of response surface estimates from CC designs is more uniform across orbits than for AP, BB, or SC designs. For larger values of k, a curved pattern is clear in which precision from CC designs is particularlygood near the center and corners of the region and worse at intermediate distances. AP, BB, and SC designs provide their most precise estimates at or near the center of the region. Especially for larger k, AP and BB designs provide more precision than CC designs near the center of the design space and less precision near the corners. SC designs are generally worst on a per-run basis except very near the center of the design region. AP designs consistently provide somewhat more precise estimates than CC plans at the center of the design space, and (again, except when k = 3) are as good as, or better than, BB designs overall by this characterization.

(3

2.3 Comparisonof Some ProjectiveProperties

Projective properties refer to characterizationsof experimental designs in k factors when only k* < k are used-that is, projections of the design onto k*-dimensional subspaces. In particular, Lin and Draper (1992) investigated the projective properties of Plackett-Burman designs because they are often used in screening contexts in which some or even most of the experimental factors are eventually discovered to have little or no impact on the response. In this context, it is of concern that the selected k-factor design be effective, no matter what k* factors turn out to be important. The projective properties of designs containing relatively larger numbers of runs may be somewhat less important because they are typically used in situations in which more (sometimes all) of the factors are important. Still, when a few factors are dominant in determining the behavior of the response, designs with spatially well-distributed projections may provide more complete descriptions of these dominant mechanisms than other experimental plans. Table 6 lists the number of design points in each twodimensional projection of AP, BB, CC, and SC designs for k = 3 to 10, excluding center-point runs. Entries include the number of design points located in each corner, side
TECHNOMETRICS,MAY2000, VOL. 42, NO. 2

1
no no

3 a 0 cn r5 CD
0r

012345678910
no

012345678910
no

Figure 3. Precision Indices for AP, BB, CC, and SC Designs in Five Factors, With 0-10 Center-Point Runs: (a), GSD; (b), GSDL; (c), GSDQ; (d), GSDB. (The BB design without center-point runs is singular.)

118 k=3 (a) k=4

MAX D. MORRIS

Side, and Center Table6. Numbersof Pointsin Each Corner, of in Two-Dimensional Projections the Designs Compared, all Excluding CenterPoints Design k
3 4 5 6 7 8 9 10 3

Corner
1 3 3 3 3 6 6 6 1

Mid-side
1 4 4 4 4 9 9 9 2

Center
2 8 8 8 8 18 18 18 0

5
LL
LA-

3
.C()
J C:

AP

rr
-

I/ I
n

I
BB 2 orbit k=5 3 2 orbit k=6 3 4

4
5 6 7 8 9 10

1
1 2

4
6 8

4
12 8

4
2 2 4 4 8 2

4
8 26 12 24 16 1

16
16 48 56 48 64 2

15

10
CI)

cn
5 c c

LU

C) 10-

CC

4
5 6 7 8 9 10 3 4 5

4
4 8 16 16 32 32 1 2 3

1
1 1 1 1 1 1 1 1 1

4
6 8 10 12 14 16 2 4 6

0 orbit k=7 40

0 orbit k=8

SC

30 25

6 7
8 9 10

4 6
9 10 12

1 1
1 1 1

8 10
12 14 16

30

midpoint, and center in the projections. (BB designs for k = 6,9, and 10 have two unique two-dimensional pro- c 15ca 20 n: Er jections, each listed in the table.) If a three-level design 10were uniformly distributed over the support points in each 10 two-dimensional projection, 1/9 of the points would project 5 onto each of the nine locations. The table highlights in italics and boldface the numbers of projected points that are 0 4 0 2 8 6 4 2 0 less than half (1/18n) or more than twice (2/9n) this numorbit orbit ber, respectively. In all cases except for k = 3, BB designs k=10 k=9 have at least some two-dimensional projections for which , , 80 30 the corners contain fewer than 1/18 of the points, and BB plans for k = 9 and 10 have large numbers of points in the 6 60 center of their projections. CC designs have better corner/ 20 to-center balance but have only one midpoint on each side LU in two-dimensional projections. SC designs, like CC plans, 140 A C) have only one run at each mid-side, and like BB plans, have 10 large numbers of runs projecting to the center point. AP \ 2 20 designs are more balanced in these projections than BB, CC, or SC designs, and although more than 2/9 of their points project to the center when k > 7, this is consider0 0 0 1 4 4 8 10 2 2 6 0 6 ably less central mass than is seen with large BB designs. orbit orbit Note that none of these patterns is improved with the addition of (k-dimensional) center-point runs to the designs, Figure4. RSEr, r = 0 -k forAP BB, CC, and SC Designs, Each WithFive Center-Point Runs:(a), k = 3; (b), k = 4; (c), k = 5; (d), k = so these projections are best-cases from the standpoint of balance. 6; (e), k = 7; (f), k = 8; (g), k = 9; (h), k = 10.
Ui
LU

20 -

TECHNOMETRICS,MAY2000, VOL. 42, NO. 2

THREE-LEVELEXPERIMENTAL DESIGNS FOR RESPONSE SURFACE MODELING

119

3.

BLOCKING

If the designs describedhere are to be used in a sequential fashion,it will be useful or necessaryin some cases to performthe experimentin blocks. Of the three classes of here,BB plansaremost easily blockedin designsexamined small groupsof runs,corresponding the structo relatively ture inheritedfrom the generating IBD as noted in Section 2. CC designscan also be blockedso thatlinearandbilinear terms are orthogonalto blocks. Here I limit consideration to executionof AP designs in two blocks, the first correspondingto the initial first-order design along with no0, to runs,andthe secondcorresponding the augcenter-point mented points along with the remainingn0,2 center-point
runs, no =n0,1 + n0,2.

Table 8. GSDo Inflation(%/)for AP Designs With Tabulated k and no,1 = no,2 = 0 - 5 no,1 = no,2 k 3 4 5 6 7 8 9 10 0 16.4 18.5 22.7 11.8 13.2 16.0 1 10.91 7.67 7.27 6.93 6.62 5.66 5.56 5.48 2 5.73 4.56 4.14 3.79 3.50 3.51 3.35 3.21 3 3.76 2.99 2.65 2.39 2.17 2.37 2.24 2.12 4 2.75 2.07 1.82 1.62 1.46 1.68 1.57 1.48 5 2.14 1.49 1.29 1.14 1.02 1.23 1.14 1.07

designs describedin Section 2 for k = 3-10, with equal of A blockingvariable, can be addedto the model,coded numbers center-point runs,rangingfrom 0 to 5, in each Xb +1 and -1 in the first and second blocks, respectively. If block. (As noted previously,this is not an optimalallocarunsfromthe standpoint minimizing X1 is an orthogonal array of strength 2-that is, if the four tion of center-point of of for possiblecombinations valueseach appear n1/4 times for varianceinflationbut may be preferred otherreasons.) each pair of factors-it is easy to show that,for complete Blocking these designs with two or more center-point observations each set resultsin inflationof less than6% in AP designs, in GSDQfor the AP designsdescribedhere. -

XbXi

-=

XbXiXj

0,

1 < i < j < k,

where sums are over all n design runs. As a result, the linear and bilinearterms are orthogonalto this blocking scheme.Purequadratic effectsarenot orthogonal blocks. to Varianceinflationdue to blocking is limited to the pure effects, however,because (1) every model effect quadratic that is not orthogonalto all pure quadraticeffects is orthogonalto Xband (2) every model effect that is not orthogonalto Xbis orthogonalto all (other)pure quadratic effects. Table7 containsthe percentincreasein GSDQresulting from this blocking scheme appliedto the design for k = 5 factors with up to 10 center-point in replicatesdistributed even increments betweenthe two blocks.Variance inflation is most severe when no center-point observationsare includedin the designin eitherblock;however,all modelcoefficientsare estimablewith this blockeddesigneven without the center-point observations. Variance inflation much is smallerwhen two center-point replicatesare addedto the is quite insensitiveto the numberof center-point runs in the second block. The additionof center-point runs to the second block is less helpful, in part,becausethis block is alreadymuch largerthanthe first. Table 8 displays the percentinflationin GSDQ resulting from this blockingscheme appliedto the the eight AP
Table 7. GSDQ Inflation (%) for k = 5 AP Design With Even- Valued no,1 and no,2
no, 1

4.

EXAMPLE

first block and is virtually negligible if n0o, > 2. GSDQ

response at xl = .563,x2 = .677,x3 = .834. The model

10 .00564

Bennettand Franklin(1954, p. 509) brieflydescribedan to experiment, originallyattributed Fuell andWagg (1949) that "was designed to investigatethe effects of the concentration detergent, of sodiumcarbonate, sodiumcarand cellulose upon the washing and suspending boxymethyl powerof a solution.The data quotedrefer to the washing is tests, whereimprovedperformance denotedby a higher Each factor was investigatedat three levels with figure. the equal spacingso thatit is convenientto separate linear and quadratic of the main effects and interaccomponents tions."The data from the 33 experiment, includingcoded factors,are repeatedin Table9. Bennett and Franklin's analysis included polynomial terms of order higher than 2; however, a second-degree polynomialfits these data reasonablywell, achieving an R2 valueof 96%.Fittedresidualvalues are plottedagainst predictedvalues based on the entire dataset in Figure 5, panel (a). Panel (b) of Figure5 displaysa plot of residuals against predictedvalues based on a second-degreepolynomial fit to data from the AP design composed of the runs denotedby asterisksin Table9. (Data from runs included in the design are plotted as filled circles, and data from runs not includedare plotted as open circles.) The fitted model based on the AP design can be seen to also predictthe excludeddata reasonablywell. Both fitted response surfacesare concave-that is, have global maxima. The model fittedto all datalocates the maximum expected

no,2 0 2 4 6 8 10

0 18.5 14.4 12.6 11.6 10.9 10.5

2 4.10 4.14 4.21 4.28 4.36

4 1.43 1.63 1.82 1.98

6 .480 .644 .800

8 .116 .215

fittedusing only datafrom the AP design predictsa maximumresponseat x1 = -.145, x2 = 12.081,x3 = 5.885, and so does not offer particularly good quantitative agreement. This is perhapsnot surprising,given that the AP design containsonly 11 runs, one more than the numberof parametersin the second-degree polynomialmodel. Of the 27 points in A, however,four of the five points at which model predictionsexceed 400 have observedvalues over
TECHNOMETRICS,MAY2000, VOL. 42, NO. 2

120 Table 9. Design and Data From Detergent Experiment of Bennett and Franklin(1954) xl -1 0 1 -1 0 1 -1 0 1 -1 0 1 -1 0 -1 -1 0 1 -1 0 1 -1 0 1 -1 0 1 X2 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 X3 -1 -1 -1 0 0 0 1 1 1 -1 -1 -1 0 0 0 1 1 1 -1 -1 -1 0 0 0 1 1 1 Y 106 197 223 198 329 320 270 361 321 149 255 294 243 364 410 315 390 415 182 259 297 232 389 416 340 406 387 AP

MAX D. MORRIS (a) 40 20

cn

0
I

._ (0

0 -20 -40

* 0* * 0

200 300 400 100 modeled responses based on all data (b)

50)0

120 60 *
C)

co o3
a)

0 -60
-I -190
C-.V

200 300 400 100 modeled responses based on AP design only

500

Figure 5. Residuals Versus Modeled Responses for the Detergent Data of Bennett and Franklin(1954): (a), Model Fitted to the Full Dataset of 27 Responses; (b), Model Fitted to Only the AP Design (solid circles) With Predictions at Other Points (open circles).

400 as well, and only one of these is included in the AP design. To examine what might happen in a larger problem, a second dataset was generated using Bennett and Franklin's data as a starting point. A "true"quadraticresponse surface model in five factors was constructed in two steps. First, the estimated model coefficients from the least squares fit of all 27 data values in Table 9 were used as their parameter counterparts in terms involving only xl,x2,
and
X3:

= 58.222,32 = 32.388, 3 = 69.056 -43.444, 022= -24.278,

33

311 =

-35.278

312 = 4.667,/313= -14.917,/323

-2.583.

rections is somewhat less than in directions corresponding to the first three variables, resulting in less precise estimation of the last two components in the response maximizer. Again, the subset of these data that compose the AP design based on the first five columns of the eight-run Plackett-Burman plan was selected and the model refitted. This 36-run (apart from those at the center) AP design contains two pairs of replicated runs, at (0 0 0 -1 0) and (0 0 0 +1 0). One replicate from each pair was deleted, and one center-point run was included, resulting in a subset of 35 runs from the full set of 243. The fitted polynomial was again negative definite, with the response maximizer estimated to be x1 = .577, 2 = .729, 3 = .850, 4 = -.311, and x5 = 1.305, quite similar to the solution found using all the data. 5.

CONCLUSION In applications involving response surface modeling, the selection of an experimental design is often a matter of As a result, the effects of (x1,x2,23) and (X4, x5) on y compromise, owing to the availability of resources and the are separable, and the location of the maximum value for variety of questions that must be answered. In many apthis (negative definite) model is found at xz = .563, 2 = plications, designs based on three equally spaced values of .677, X3 = .834, X4 = 0, and x5 = 1. At each of the 243 each experimental factor are desirable. They are conceptupoints on the full three-level lattice, one data value was ally simple, often based on one "typical"and two "extreme" generated using this polynomial, adding a pseudonormal values of each controllable variable, and are naturally sugvariate with mean 0 and standard deviation 4.5 (the root gested when the region of interest is cuboidal rather than mean squared error of the fit to the 27 data values of Ben- spherical. nett and Franklin). The polynomial fitted by least squares In this article, augmented pairs designs are proposed, to these data was also negative definite, with a maximum some of which are compared to Box-Behnken designs and located at xi = .547, x2 = .692, X3 = .858, X4 = -.427, and face-centered central composite and small composite deX5 = 1.435. Curvatureof the true model in the X4 and x5 disigns. Like the composite designs, AP plans combine a twoTECHNOMETRICS,MAY2000, VOL. 42, NO. 2

Next model coefficients for two additional terms were given values of /4 = 0,35 = 25,344 = -10,/55 = -10, and /45 = -5, which are on the order of the fitted coefficients. All other terms involving x4 and x5 were omitted.

THREE-LEVELEXPERIMENTAL DESIGNS FOR RESPONSE SURFACE MODELING

121

level factorialplan with additional runsto allow estimation terms. They can be loosely motivatedby of second-order a conceptof "distance" from spatialdesign theorybut are competitivewith standard responsesurfaceplans for estimatingquadratic polynomialmodels. AP designssharemanyof the advantages the BB, CC, of and SC designs to which they have been comparedin this article.In particular, they

TheAP designsdiscussedin this articleareespeciallyapin pealingin quadratic responsesurfaceapplications which is in experimentation conducted a sequenceof steps,the use of factorsat only threelevels is desired,the numberof exrunsshouldnot be manymorethanthe number perimental of modelcoefficients, of and/orpreciseestimation response surfacecurvature needed.Especiallyin these situations, is to they providea useful alternative BB, CC, and SC plans.

* requirethe use of only threeevenly spacedvalues in ACKNOWLEDGMENTS each factor, I appreciate valuablecontributions the made by the ed* are easily constructed simplerules, and via itor, associate editor, and a referee duringthe review of * can be blockedfor sequentialexperimentation. earlierversionsof this article.A portionof this work was Relativeto CC and SC designs,AP plans performedwhile I was a memberof the researchstaff of Oak Ridge NationalLaboratory was sponsored part and in * offer more preciseestimatesof purequadratic coeffiSciences ResearchProgram, by the AppliedMathematical cients, Officeof EnergyResearch, U.S. Department Energy,unof * can be constructedusing smallertwo-level factorial der contractDE-AC05-840R21400with LockheedMartin segmentsor blocks (i.e., resolution3) thanCC plans, EnergyResearch,Inc. and * have statisticalpropertiesthat are generallysuperior [Received June 1999. Revised November 1999.] to those of SC designs. Relativeto BB designs,AP plans
REFERENCES

* can be constructed using fewer runs, Bennett, C. A., and Franklin,N. L. (1954), Statistical Analysis in Chemistry * providegenerallysuperior comparable or staand Chemical Industry, New York: Wiley. per-run tisticalproperties, and D. Box, G. E. P.,andBehnken, W.(1960),"SomeNew ThreeLevelDesigns for the Study of Quantitative Variables," Technometrics, 455-475. 2, * requirefewer center-point runsfor preciseestimation (Corrections, 131;3, 576.) 3, of purequadratic coefficients. N. R. Box, G. E. P., and Relativeto all otherdesignsconsidered here, AP plans
(1987), Empirical Model-Building and Draper, Response Surfaces, New York: Wiley. perimenters, New York: Wiley. * providemore uniformone- and two-dimensional pro- Box, G. E. P.,andWilson,K. B. (1951),"Onthe Experimental Attainment jections in the design space. of Optimum Conditions," Journal of the Royal Statistical Society, Ser.

W. Box, G. E. P., Hunter, G., and Hunter, S. (1978), Statistics ExJ. for

Additionalresearchwill be needed to understand how AP designs should best be implementedand generalized. For example,considerthe following questions: * How shouldthe initialtwo-leveldesignbe selectedso as to producethe best overallAP plan? * When restrictionto three levels of each factoris not needed, how might the augmentedruns be rescaled so as to producea more effective five-level design? That is, how should the value of a be chosen in a to replacement Equation(1) of form
Xst=

B, 13, 145. N. Draper, R. (1985),"Small Composite Technometrics, 17327, Designs," 180. Fuell, H. J., andWagg,R. E. (1949), "Statistical Methodsin Detergency
Investigations,"Research, 2, 334.

Giovannitti-Jensen, andMyers,R. H. (1989),"Graphical A., Assessment of the Prediction of Capability ResponseSurfaceDesigns," Technometrics,
31, 159-171.

Hartley,H. O. (1959), "SmallestCompositeDesigns for Quadratic ResponseSurfaces," Biometrics, 611-624. 15, Johnson,M. E., Moore,L. M., and Ylvisaker, (1990), "Minimax D. and
Maximin Distance Designs," Journal of Statistical Planning and Inference, 26, 131-148. Technometrics,32, 187-194.

N. Lin,D. K. J., andDraper, R. (1990),"Small Response-Surface Designs,"

Technometrics,34, 423-428.

2 (x5 + t)

(2)

of (1992), "Projection Properties Plackettand Burman Designs,"

R. so as to optimizedesign performance? J. Plackett, L., andBurman, P. (1946),"TheDesign of Optimum Multifactorial Experiments,"Biometrika, 33, 305. * Can simpleblockingrulesbe constructed allow the to Silvey,S. D. and Design,London: Chapman Hall. the augmented runs to be groupedin more blocks of Westlake, (1980),Optimal W. J. (1965),"Composite Fractions DesignsBasedon Irregular smallersize, withoutsubstantial of efficiency? loss of Factorials," Biometrics, 21, 324-336.

TECHNOMETRICS,MAY2000, VOL. 42, NO. 2