19-Aug-14

1
Design of Industrial Experiments
Introduction
Why should Industrial Engineers be interested in designing
experiments?
2
Push the quality issue further and further upstream
The need to focus on and continually study process
performance with an eye toward identifying
opportunities for improvement
19-Aug-14
2
Experimentation
3
A test or a series of tests in which purposeful changes are
made to input variables of a process or system to observe and
identify the reasons for changes that may be observed in the
output response.
Plays an important role in
Product design
Process development
Process improvement
One of the objectives is to develop a robust process
Minimally affected by external sources of variability
What is the difference between testing and
experimentation?
4
Life testing
Subjecting many identical units to field conditions for the
purpose of determining the expectancy of life or performance
Experimentation
Purposefully varying design parameters for the purpose of
identifying particular combinations that optimize certain
design criteria or performance measures
Both are important cannot substitute one for the other
19-Aug-14
3
SPC-DOE Relationship
5
Root cause analysis often requires experimentation with the
process, purposely changing certain factors with the hope of
observing corresponding changes in the response of the
process
A quieter process will allow the effects of small changes in
process parameters to be more readily observed
When the process is in control, the future is more predictable
Improvement actions are more likely to be realized in future
operations of a process in control
The Process Model
6
19-Aug-14
4
Objectives of experimentation
7
Determine which variables are most influential on the
response y
Determine where to set the influential xs so that y is
almost always near the desired nominal value
Determine where to set the influential xs so that
variability in y is small
Determining where to set the influential xs so that the
effects of the uncontrollable variables (zi) are minimized
Experimentation Approaches
8
Best-guess approach
May work if the experimenter has a great deal of technical and
theoretical knowledge about the process as well as considerable
practical experience
No guarantee of success could continue for a long time
Temptation to stop testing if the initial guess produces an acceptable
result
One factor at a time
Vary each factor with all other factors held constant
Fails to consider interactions
The correct approach factorial or other designs
19-Aug-14
5
Experimentation The Problem
9
Complexity
Necessary to consider simultaneously the influence of a number of 'input
variables
With k factors and p responses there are (k x p) entities to consider
Interactions
Experimental error
Variability not explained by known influences is called experimental error
Only a small part of experimental error is attributable to errors in
Measurement
Variations in raw materials, sampling, and in the settings of experimental factors
often provide larger components
Differentiation between correlation and causation
Statistical Design of Experiments
10
Statistical methodology is the only objective way when
the data involved are subject to error
Process of planning the experiment so that appropriate
data the can be analyzed by statistical methods is
collected, resulting an valid and objective conclusions.
19-Aug-14
6
Application Characterizing a Process
11
Soldering of circuit boards
Several controllable (7) and uncontrollable factors (5)
Which factors control the occurrence of defects on the circuit
boards?
Can design an experiment to estimate the magnitude and
direction of factor effects
Does changing factors together produce different results?
Screening designs typically fractional
Application Optimizing a Process
12
Improving the yield of a chemical
process
From a screening design we know
that temperature and reaction
time are important factors
Develop an empirical model of the
process
Experiment to locate the optimal
values of the factors
Response contours / response
surface
19-Aug-14
7
Basic Principles
13
Replication
Repetition of the basic experiment
Allows the experimenter to
estimate experimental error
Helps in determining statistical
difference
May help in the reduction of error
(e.g. for mean

)
Repeated measurement is NOT
replication (represents inherent
variability in the measurement
system GRR)
Randomization
Allocation of experiment material
and order of performance
Statistical methods require
independent trials
Averaging out of extraneous
factors removal of bias
Blocking
Block is a set of relatively
homogenous experimental
conditions
Eliminate or reduce variability due
to nuisance factors
Guidelines for Designing Experiments
14
Recognition of and statement of the problem
Solicit input from all concerned parties: engineering, quality
assurance, manufacturing, marketing, management, the customer,
and operating personnel (who usually have much insight and who
are too often ignored) For this reason a team approach to designing
experiments is recommended
Prepare a list of specific problems or questions to be addressed by
the experiment
Keep the overall objective in mind
Is this a new process or system - the initial objective is likely to be
characterization or factor screening - or is it a mature or reasonably well-
understood system that has been previously characterized-in which case
the objective may be optimization?
One large comprehensive experiment is unlikely to answer the key
questions and a sequential approach using a series of smaller
experiments is a better strategy
19-Aug-14
8
Guidelines for Designing Experiments
15
Choice of factors, levels, and range
Classify as potential design factors or nuisance factors.
Potential design factors are those factors that the experimenter may wish to vary
in the experiment
Nuisance factors, may have large effects that must be accounted for, yet we may
not be interested in them
Often classified as controllable, uncontrollable, or noise factors.
A controllable nuisance factor is one whose levels may be set by the experimenter.
Alternative classification: design factors, held constant factors, and
allowed-to-vary factors
Choose the ranges over which these factors will be varied, and the
specific levels at which runs will be made
Process knowledge is required to do this
For screening designs, keep the number of levels low generally two
Range must be broad
Guidelines for Designing Experiments
16
Selection of the response variable
Be certain that this variable really provides useful information
about the process under study
Most often, the average or standard deviation (or both) of the
measured characteristic will be the response variable
Multiple responses are not unusual.
Gauge capability (or measurement error) is an important factor
If gauge capability is inadequate, only relatively large factor effects
will be detected by the experiment or perhaps additional replication
will be required
Where gauge capability is poor, the experimenter may decide to
measure each experimental unit several times and use the average of
the repeated measurements as the observed response
19-Aug-14
9
Guidelines for Designing Experiments
17
Choice of experimental design
Consideration of sample size (number of replicates)
Selection of a suitable run order for the experimental trials
Whether or not blocking or other randomization restrictions are
involved
Performing the experiment
Ensure that everything is being done according to plan
Prior to conducting the experiment a few trial runs or pilot runs are
often helpful
These runs provide information about consistency of experimental material, a
check on the measurement system, a rough idea of experimental error, and a
chance to practice the overall experimental technique
Also provides an opportunity to revisit the decisions made in steps 1-4
Guidelines for Designing Experiments
18
Statistical analysis of the data
Statistical methods cannot prove that a factor (or factors) has a particular effect.
They only provide guidelines as to the reliability and validity of results.
Statistical methods do not allow anything to be proved experimentally, but they
do allow us to measure the likely error in a conclusion or to attach a level of
confidence to a statement.
Statistical methods is that they add objectivity to the decision-making process
Good engineering or process knowledge and common sense in addition to
statistical techniques are required for sound conclusions.
Present the results of many experiments in terms of an empirical model
An equation derived from the data that expresses the relationship between the
response and the important design factors
Residual analysis and model adequacy checking are also important analysis techniques
Graphical methods, confidence intervals, hypothesis testing may also be useful
19-Aug-14
10
Guidelines for Designing Experiments
19
Conclusions and recommendations
Draw practical conclusions about the results and recommend a course of action
Follow-up runs and confirmation testing should also be performed to validate
the conclusions
Keep in mind
Experimentation is iterative
We tentatively formulate hypotheses, perform experiments to investigate these
hypotheses, and on the basis of the results formulate new hypotheses
Major mistake to design a single, large, comprehensive experiment at the start
of a study
We usually experiment sequentially, and as a general rule, no more than about
25 percent of the available resources should be invested in the first experiment
This will ensure that sufficient resources are available to perform confirmation runs and
ultimately accomplish the final objective of the experiment.
Summary: Using Statistical Techniques in Experimentation
20
Use your non-statistical knowledge of the problem
Choosing factors, determining factor levels, deciding number
of replicates, interpreting results, etc.
Keep the design and analysis as simple as possible
Recognize the difference between practical and statistical
significance
Iterate
19-Aug-14
11
Statistical Design A Brief History
21
Four eras in the development of statistical experiment design
The First Era Agricultural
Led by the pioneering work of Sir Ronald A. Fisher in the 1920s and early 1930s
Responsible for statistics and data analysis at the RothamstedAgricultural
Experimental Station near London, England
Recognized that flaws in the way the experiment that generated the data had
been performed often hampered the analysis of data from systems (in this case
agricultural systems)
Developed insights that led to the three basic principles of experimental design:
randomization, replication, and blocking
Systematically introduced statistical thinking and principles into designing
experimental investigations, including the factorial design concept and the
analysis of variance
Statistical Design A Brief History
22
The Second Era Industrial
Catalyzed by the development of response surface methodology (RSM)
by Box and Wilson (1951)
Recognized and exploited the fact that many industrial experiments are
fundamentally different from their agricultural counterparts in two
ways:
The response variable can usually be observed (nearly) immediately
The experimenter can quickly learn crucial information from a small group of runs
that can be used to plan the next experiment.
Box (1999) calls these two features of industrial experiments immediacyand
sequentiality
Over the next 30 years, RSM and other design techniques spread throughout the
chemical and the process industries, mostly in research and development work.
The application of statistical design at the plant or manufacturing process level
was still not widespread
Inadequate training in basic statistical concepts and methods for engineers
Lack of computing resources and user-friendly statistical software
19-Aug-14
12
Statistical Design A Brief History
23
The Third Era Quality
Taguchi advocated using designed experiments for robust parameter
design
Making processes insensitive to environmental factors or other factors
that are difficult to control
Making products insensitive to variation transmitted from components
Finding levels of the process variables that force the mean to a desired
value while simultaneously reducing variability around this value
Suggested highly fractionated factorial designs and novel statistical
methods
Highly controversial (not peer reviewed adequately when
introduced)
Subsequent peer review indicated Taguchi' s engineering concepts and
objectives were well-founded but there were substantial problems with
his experimental strategy and methods of data analysis
Statistical Design A Brief History
24
The Fourth Era
The Taguchi controversy made DOE popular
Widely used in the discrete parts industries, including
automotive and aerospace manufacturing, electronics and
semiconductors, etc.
Many new approaches, including alternatives to Taguchis
methods
Formal education in statistical experimental design is
becoming part of many engineering programs