Introduction to Design of Experiments

University of California at Berkeley

Mechanical Engineering Department
Summer, 2001
by Michael Montero
Part 1
Full Factorial Design and Analysis (2 levels)
Part 2
Fractional Factorial Design and Analysis (2 levels)
Part 3
Software Introduction and 3-Level or Higher Designs
Introduction to DOE - Part 3
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Available DOE Software
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Example: Minitab v11.21
Windows Based (Windows 9x, NT, and 2000)
Spreadsheet-like interface and command line interface
User-friendly menus
2
k
full and fractional factorial designs (regular and non-regular)
Response surface building
Analysis of Variance (ANOVA)
Multiple linear regression
Statistical Process Control (SPC), time-series analysis (autoregression)
Reproducibility and Repeatability (R&R)
And more...
SAS JMP Mixsoft
S-Plus Nutek Qualitek-4
Genstat StatSoft
Minitab Adept Scientific DOE_PC IV
State-Ease, Design-Expert Process Builder STRATEGY
Echip S-Matrix CARD
Statgraphics Qualitron Systems DoES
Systat RSD Associates Matrex
Umetrics MODDE 6
Commercial Software for Experimental Design
DOE Specific
General Statistical Package
Minitab Example: Injection Molding Experiment
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Injection Molding Experiment
(Box, G. E. P., Hunter, W. G., and Hunter J.S., Statistics for Experimenters: An
Introduction to Design, Data Analysis, and Model Building, Wiley Interscience, p. 413,
1978.)
Problem: Identify important factors effecting part shrinkage.
Less shrinkage is better.
DOE:
4 8
IV
2

Design Generators
Minitab: Create Factorial Design Step by Step
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
1
Minitab: Factorial Designs Dialog Box
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Predefined Designs
Custom Designs
Screening Design
Select # of Factors
Summary of Possible 2-Level Designs
Design Selection
2
5
3
4
Minitab: Summary of 2-Level Designs
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
4
Minitab: Design Selection
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
5
Minitab: Factorial Designs Dialog Box Contd
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
6
7
8
Define Factors
Output Selection
Additional Options
Minitab: Define Factors and Actual Level Values
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
6
Minitab: Additional Design Options
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
7
Choose if you want
to fold on certain
factors
Randomize order
of tests
Store design in
current worksheet
Choose which
fraction to use
A
B
C
A
B
C
I = + ABC I = - ABC
Minitab: Output Selection
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
8
Display confounding pattern
up to selected order
Generators, defining relation,
and design matrix displayed
Minitab: Command Session Window
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Minitab: Worksheet or Data Window
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Design Matrix
Type in or paste in response values
Minitab: Analyze Factorial Design
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
9
Minitab: Select Response
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
9
Select terms in
model by order
10
11
12
Response, effects
and residual plots
Store effects, residuals,
etc. in worksheet
Minitab: Select Terms for Effects Calculation and Store
Results in Worksheet
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
10
11
Store effects in
worksheet
Store residuals
and fits in
worksheet
Minitab: Graphical Options
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
12
Normal Probability
Plot of Effects
Pareto Chart
Select Confidence
Residual plots
Minitab: Effect Calculations
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Minitab: Plots
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Normal Plot of Effects
Pareto Chart of Effects
Minitab: Factorial Plots
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Minitab: Setup of Factorial Plots
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Minitab: Main and Interaction Plots
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
All Main Effects Plot
AE Interaction Plot
M
i
n
i
t
a
b
:

C
a
l
c
u
l
a
t
o
r
M
.

G
.

M
o
n
t
e
r
o
UC-Berkeley, Mechanical Engineering
Minitab: Calculator Used to Construct AE Column
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Minitab: Multiple Regression
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
M
i
n
i
t
a
b
:

L
i
n
e
a
r

F
i
t
M
.

G
.

M
o
n
t
e
r
o
UC-Berkeley, Mechanical Engineering
Three level or Higher Factor Levels
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Laser-assisted Composite Mfg. Experiment
(Mazumdar and Hoa, 1995.)
Problem: Verify if laser power truly effects composite
strength (measured by short-beam-shear test)
DOE: 3
1
ANOVA: Analysis of variance indicates that laser power does significantly effect (F
calc
> F
crit
)
composite strength. Next, look into whether the relationship between the factor and response is
linear or quadratic over the three levels.
Replicates will allow for
estimate of error
= 0.10
Linear and Quadratic Contrasts
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
linear contrast =y
3
- y
1
=-1y
1
+0y
2
+ 1y
3
quadratic contrast =(y
3
- y
2
) - (y
2
- y
1
) =1y
1
- 2y
2
+1y
3
=
To define the quadratic contrast, one can use the following argument. If relationship
is linear, then (y
3
- y
2
) and (y
2
- y
1
) should approximately be the same:
=1y
1
- 2y
2
+1y
3
0 if relationship is linear
So, in vector form:
linear contrast =(-1 0 1)(y
1
y
2
y
3
)
T
quad. contrast =(1 -2 1)(y
1
y
2
y
3
)
T
Linear Contrast Vector (u)
Quad. Contrast Vector (v)
Where dot product of contrast vectors equals 0. Contrast vectors are orthogonal to one
another ensuring that the contrasts are independent of one another:
u v =(-1 0 1) (1 -2 1) =(-1)(1) +(0)(-2) +(1)(1) = 0
Response Vector
0.0034
Linear and Quadratic Effects
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Length (u) = [(-1)
2
+ (0)
2
+ (1)
2
]
1/2
=
Scale vectors so that they both have unit lengths. Hence, divide contrast vector
by length of vector:
2
Length (v) = [(1)
2
+ (-2)
2
+ (1)
2
]
1/2
=
6
Linear and Quadratic Effect Estimates:
( ) ( ) 636 . 8 = =
T
3 2 1 l
y y y 1 0 1
2
1
A
( ) ( ) 109 . 0 = =
T
3 2 1 q
y y y 1 2 1
6
1
A
= 0.10
ANOVA:
Linear term is significant (F
calc
>F
crit
) while
quadratic term is not (F
calc
<F
crit
)
AVE = 30.921
21.249
0.003
Predictive Model and Orthogonal Polynomials
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
To be able to predict composite strength through the range of the design space (40 to 60
Watts), we must extend the notion of orthogonal contrasts to orthogonal polynomials:
+ + + =
6
) (
2
) (
2 1
x P
A
x P
A AVE y
q l
Where:
( ) 1 0 1
10
50
1
=

=
x
!
m x
(x) P
( ) 1 2 1
3
2
10
50
3
3
2
3
2 2
2
=

'
|
=

'
|
=
x
!
m x
(x) P
, when x ={40,50,60} respectively
, when x ={40,50,60} respectively
x laser power
distance between consecutive levels
m middle level
Example: What is composite strength when laser is powered at 55 Watts?
( ) [ ]
6
3
2
10 50) (55 3
0.109
2
10 50) (55
8.636 30.921 y
2

+ =
34.03 .0556 3.053 30.921 y = + + =
Extending Orthogonal Polynomials
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Model can be extended for any level equally spaced (4, 5, 6, etc.)
Analysis is the same using factorial plots, normal plot of effects, and
confidence intervals or ANOVA for statistical testing
Analysis of equal level DOEs with more than one factor is the same but we
must also consider interaction estimates (For example 3
2
)
A
l
x B
l
A
l
x B
q
A
q
x B
l
A
q
x B
q
Also in mixed-level designs (For example 2
1
3
1)
A x B
l
A x B
q
Polynomials with fourth and higher degrees, however, should be avoided
unless responses behavior can be justified by a physical model
Data can be well fitted by using higher-degree polynomial model but
the resulting fitted model will lack predictive power
In regression analysis, this is referred to as overfitting
Average variance of the regression parameter estimates is proportional
to the number of regression parameters in the model.
Overfitting inflates variance and lowers accuracy of predictive
model (Draper and Smith, 1998)
Higher degree polynomials become harder to interpret
Further Topics: 3-Level Fractional Factorial Designs
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
3
k-p
designs rely on generators and defining relation not based on
multiplicative column but modulus calculus:
Example: 3
4-1
(Generator: D = ABC) where x
D
= x
A
+ x
B
+ x
C
(mod 3)
So:
3/3 = 1 remainder 0
1/3 = 0.3 remainder 1
2/3 = 0.6 remainder 2
Where:
x = coded value (0, 1, or 2)
Column Ds coded pattern is generated by the
x
A
+ x
B
+ x
C
(mod 3) relation
Further Topics: 2
m
4
n
Mixed Designs
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
2
m
4
n
designs can be generated from fractional factorial 2
k-p
designs by
method of column replacement:
Example: 2
7-4
(Generators Not Shown)
2
4
4
1
Statistical Literature
M. G. Montero
U
C
-
B
e
r
k
e
l
e
y
,

M
e
c
h
a
n
i
c
a
l

E
n
g
i
n
e
e
r
i
n
g
Box, G. E. P., Hunter, W. G. and Hunter, J.S., Statistics for Experimenters: An Introduction to
Design, Data Analysis, and Model Building, Wiley Series in Probability and Statistics, 1978.
Devor, R. E., Chang, T. and Sutherland, J. W., Statistical Quality Design and Control:
Contemporary Concepts and Methods, Macmillan, 1992.
Ross, P. J., Taguchi Techniques for Quality Engineering, McGraw Hill, 2nd Edition, 1996.
Wu, C. F. J. and Hamada, M., Experiments: Planning, Analysis, and Parameter Design
Optimization, Wiley Series in Probability and Statistics, 2000.
Myers, R. H. and Montgomery, D. C., Response Surface Methodology: Process and Product
Optimization Using Designed Experiments, Wiley Series in Probability and Statistics, 1995
Experimental Design and Optimization
Statistics and Multiple Linear Regression
Walpole, R. E., Myers and R. H., Myers, S. L., Probability and Statistics for Engineers and
Scientists, Prentice Hall, 6th edition, 1998.
Sen, A. and Srivastava, M., Regression Analysis: Theory, Methods, and Applications, Springer-
Verlag, 1990.