MEASUREMENT SYSTEM

ANALYSIS
ONE DAY TRAINING
PROGRAM

23 DEC 16 IAS - MSA TRAINING 1

 Objectives
Discuss the need for Measurement System Analysis
Identify and define the various components of measurement
error
List and discuss the sources of error in measurement
systems used in processes
Perform an Attribute Measurement System Analysis
Perform a Variables Measurement System Analysis

23 DEC 16 IAS - MSA TRAINING 2

 What Is An MSA?
Scientific and objective method of analyzing the
validity of a measurement system
A tool which quantifies:
1. Equipment Variation
2. Appraiser (Operator) Variation
3. The Total Variation of a Measurement System
MSA is NOT just Calibration
MSA is NOT just Gage Repeatability & Reproducibility
(R&R)
Measurement System Analysis is often a project within a project

23 DEC 16 IAS - MSA TRAINING 3

 Fundamentals
Measurement Systems Analysis is used to:
validate that a measurement system is producing correct values
Determine the source of measurement system inaccuracy for
measurement system improvements.
Focus on the measurement systems associated with process output
to start.
Involve people who gather, input, and analyze the data.

Evaluate the entire measurement system from initial data input to final use.

23 DEC 16 IAS - MSA TRAINING 4

 THE MEASUREMENT PROCESS

the term measurement is defined as the assignment of numbers to

material things to represent the relations among them with respect to
particular properties.The process of assigning the numbers is defined as
the measurement process, and the value assigned is defined as the
measurement value.

From these definitions it follows that a measurement process should be

viewed as a manufacturing process that produces numbers (data) for its
output.
Measurement Systems Analysis
Detroit: AIAG Press, 1995.

23 DEC 16 MSA TRAINING 5

 TERMINOLOGY
Gage: any device used to obtain measurements

Measurement System: the collection of operations, procedures, gages and

other equipment, software, and personnel used to assign a number to the
characteristic being measured; the complete process used to obtain
measurements.

23 DEC 16 IAS - MSA TRAINING 6

 Why Measure?
To understand a decision:
Meet standards & specifications
Detection/reaction oriented
Short-term results
Stimulate continuous improvement:
Where to improve?
How much to improve?
Is improvement cost effective?
Prevention oriented
If you strategy
Long-term cannot measure, you cannot improve!
Taguchi

23 DEC 16 IAS - MSA TRAINING 7

Components Of Measurement Error
Resolution/Discrimination
Accuracy (bias effects)
Linearity
Stability (consistency)
Repeatability-test-retest (Precision)
Reproducibility (Precision)

Each component of measurement error can contribute to variation, causing

wrong decisions to be made

23 DEC 16 IAS - MSA TRAINING 8

 Main Sources Of Variation
Materials
Methods
Machines
People
Environment
Measures

Measurement systems are the most neglected

23 DEC 16 IAS - MSA TRAINING 9

Measurement System As A Process
Material Method Machine
Cleanliness Sequence Cleanliness
Temperature Temperature
Dimension Design
Weight Positioning Precision
Corrosion Location Calibration
Hardness Resolution
Set-up
Conductivity Stability
Density Preparation Wear
Measurement
Cleanliness Compliance-procedure Error
Fatigue
Vibration Attention
Calculation error
Atmospheric pressure Interpretation
Lighting Speed
Coordination
Temperature Knowledge-instrument
Dexterity
Humidity Vision

Environment People

23 DEC 16 IAS - MSA TRAINING 10

 Possible Sources of Process Variation
Observed Process Variation

Actual Process Variation Measurement Variation

Long-term Short-term Variation Variation due Variation due

Process Variation Process Variation w/in sample to instrument to operators

Repeatability Calibration Stability Linearity

To address actual process variability, the variation

due to the measurement system must first be
identified and separated from that of the process

23 DEC 16 IAS - MSA TRAINING 11

 Possible Sources of Process Variation
Accuracy:degree of conformity measured
2
actual
2
ms
2 Precision:degree of refinement
of a measure to a standard or a with which an operation is
true value Measured Process Variation performed or a measurement
stated.

Actual Process Variation Measurement Variation

Long-term Short-term Variation Variation due Variation due

Process Variation Process Variation w/in sample to instrument to operators

Repeatability Bias Stability Linearity

Repeatability is the variation due to

the instrument and/or the positional Reproducibility shows how consistent
variation of the part inBias
the is the difference
Linearity
instrument. between
isvariability
a measure the
isofbetween operators.
the difference
Stability is variationinin the
This is variation withinobserved average
accuracy
an operator. of the measurements
or precision over thebias
range of time
systems over
and the
thereference value.
instruments capability.
23 DEC 16 IAS - MSA TRAINING 12
 MEASUREMENT SYSTEM ERROR
Measurement System Variance Measurement System Bias
Measured Variance = Actual Variance + System Variance Measured Value = Actual Value + Bias Error

2 measured = 2 actual + 2 error m measured = m actual + m error

2 error = 2 repeatibility + 2 reproducibility m error = m

stability + m bias + m linearity

Variance Error Tool = R&R Study Measurement Error Tool = Calibration Study

Bias =
Mean (m)
Shifted
Variance =
Spread ()
Increased

-3 -2 -1 0 1 2 3
-6 -4 -2 0 2 4 6

23 DEC 16 IAS - MSA TRAINING 13

 Terminology
Accuracy related terms
True value
Bias
Precision related terms
Repeatability
Reproducibility
Linearity
Discrimination
Stability (over time)
Correlation

23 DEC 16 IAS - MSA TRAINING 14

 Accuracy
Accuracy - Extent to which the average of the
measurements deviate from the true value
True value
Theoretically correct value
NIST standards
Bias
Distance between average value of all
measurements and true value
Amount tool is consistently off target
Systematic error or offset

23 DEC 16 IAS - MSA TRAINING 15

 Bias
Difference between the reference value and
the observed average of the measurements.

Reference Actual
90
80
This measurement 70 Bias
system has a bias of 60
1.0mm. Could be due 50

to poor calibration, 40

operator training, 30
20
clamping sequence, etc.
10
0

-1 0 1 2 3
Data

23 DEC 16 MSA TRAINING 16

 Stability
Difference in the measurement systems accuracy or
precision over time. Done by measuring a master
part over time.
There are no indices for stability. Control charts can
be used to assess stability. I and MR Chart to show stability
Chart shows a measurement 10
1
1

Individual Value
1 111 1 11
1 1
system with poor stability. There 0
UCL=4.883

Mean=0.8220

is an upward trend over time 111 1

11 1
LCL=-3.239

1 11
possibly due to temperature, Subgroup
-10
0 100 200 300 400 500

humidity, etc. The system then 15

1
goes out of control which may
Moving Range

1
1
10 11 1
1 1
1 11 1111 111 111
mean the measurement system 5
1 1 1 1 1 1 1111
UCL=4.988

requires maintenance. 0
R=1.527
LCL=0

23 DEC 16 IAS - MSA TRAINING 17

 Linearity
Difference in the accuracy or precision over the
operating range of the measurement system.
Gage Linearity Study for a bad gage Gage Linearity Study for a good gage

4 Gage Linearity 0.5 Gage Linearity

Linearity: 3.21387 Linearity: 0
3 %Linearity: 107.129 %Linearity: 0.000
R-Squared: 0.652 R-Squared:
2
Gage Bias Gage Bias
Bias: 0.264014 Bias: 0
1 %Bias: 8.800 %Bias: 0.000
Bias

Bias
0 0.0
Percent of Process Variation Percent of Process Variation
-1
0.5
100

-2
Percent

Percent
-3 0.0
50

-0.5
-4

-1 0 1 -5 0 5
0 -0.5
Master Part Measurement Master Part Measurement
Linearity Bias Linearity Bias

This measurement system has a linearity This measurement system does not have
problem. As parts reach the operating a linearity problem. No bias is
limits, the bias the measurement system introduced through the operating range
introduces increases. of the part.

23 DEC 16 IAS - MSA TRAINING 18

 TYPES OF R&R STUDIES
R&R STUDY

ATTRIBUTE R&R VARIABLE R&R

STUDY STUDY

Defect count or Continuous Data

Go/No Go (Variable Data)
Measurement Measurement
Systems Systems

Examples Examples
Visual Defects Dimensional Measures
Inventory Counts Plating Thickness
Attribute Gages Turned Diameter
Misplaced Components

23 DEC 16 IAS - MSA TRAINING 19

 Precision
total product repeatability reproducibility
2 2 2 2

Repeatability: the variation between successive

measurements of the same part, same characteristic, by the
same person using the same instrument. Also known as
test - retest error; used as an estimate of short-term
measurement variation.
Master Value

Poor Repeatability
Good Repeatability

23 DEC 16 IAS - MSA TRAINING 20

 Precision
2
total 2
product 2
repeatability 2
reproducibility

Reproducibility: The variation that results when

different conditions are used to make the same
measurements
Different operators
Different set-ups
Different test units
Different environmental conditions
Long-term measurement variation
Estimated by the standard deviation of the averages of
measurements from different measurement conditions
23 DEC 16 IAS - MSA TRAINING 21
 Precision
Reproducibility: The difference in the average of the
measurements made by different persons using the
same or different instrument when measuring the
identical characteristic on the same part
Master Value

Inspector A
Inspector B
Inspector B
Inspector A

Inspector C
Inspector C

23 DEC 16 IAS - MSA TRAINING 22

 Discrimination
The number of decimal places that can be measured by
the system. Increments of measure should be at least
one-tenth of the width of the product specification or
process variation.
Poor Discrimination

1 2 3 4 5

Good Discrimination

1 2 3 4 5

23 DEC 16 IAS - MSA TRAINING 23

 # of Distinct Categories

Establishes if the measurement system has good or poor

discrimination (resolution).
The rule of thumb is for the increments of measure to be 1/10th of
the specification or process variation.
This metric should be used in a production environment when
multiple parts can be gathered that represent the process and
product variation can be calculated.

23 DEC 16 IAS - MSA TRAINING 24

23 DEC 16 IAS - MSA TRAINING 25
23 DEC 16 IAS - MSA TRAINING 26
23 DEC 16 IAS - MSA TRAINING 27
23 DEC 16 IAS - MSA TRAINING 28
23 DEC 16 IAS - MSA TRAINING 29
23 DEC 16 IAS - MSA TRAINING 30
23 DEC 16 IAS - MSA TRAINING 31
23 DEC 16 IAS - MSA TRAINING 32
23 DEC 16 IAS - MSA TRAINING 33
23 DEC 16 IAS - MSA TRAINING 34
23 DEC 16 IAS - MSA TRAINING 35
23 DEC 16 IAS - MSA TRAINING 36
23 DEC 16 IAS - MSA TRAINING 37
23 DEC 16 IAS - MSA TRAINING 38
23 DEC 16 IAS - MSA TRAINING 39
 Gage name: LH Rear door to hinge fixture
Date of study: May 7, 2002
Gage R&R (ANOVA) for Lwr Hinge Reported by: John Wheeler
Tolerance: +/- 0.5mm
Misc: After pins were removed

Components of Variation By Part

1
%Contribution
400
%Study Var
0
Percent

300 %Tolerance

200 -1
100
-2
0
Gage R&R Repeat Reprod Part-to-Part Part 1 2 3 4 5 6 7 8 9 10

R Chart by Operator By Operator

0.10 Gary Jack 1
UCL=0.09475
Sample Range

0
0.05
R=0.029 -1

0.00 LCL=0 -2
0 Operator Gary Jack

Xbar Chart by Operator Operator*Part Interaction

1 Gary Jack 1 Operator
Gary
Sample Mean

Jack
0 0
Average

-1 UCL=-0.8445
Mean=-0.899
LCL=-0.9535 -1

-2 -2
0 Part 1 2 3 4 5 6 7 8 9 10

23 DEC 16 IAS - MSA TRAINING 40

 Gage name: LH Rear door to hinge fixture
Date of study: May 7, 2002
Gage R&R (ANOVA) for Lwr Hinge Reported by: John Wheeler
Tolerance: +/- 0.5mm
Gage R&R info Misc: After pins were removed

Components of Variation By Part

1
400 Break down of variation. %Contribution
%Study Var Averages all the data by part
We want the part-to-part 0
Percent

300 %Tolerance

200 variation to be high. -1

100
-2
0
Gage R&R Repeat Reprod Part-to-Part Part 1 2 3 4 5 6 7 8 9 10

R Chart by Operator By Operator

0.10 Gary Jack 1
UCL=0.09475
Shows error between
Sample Range

Shows error within an 0

0.05 operator (Repeatability) operators (Reproducibility)
-1
Must be in control. R=0.029

0.00 LCL=0 -2
0 Operator Gary Jack

Xbar Chart by Operator Operator*Part Interaction

1 1 Operator
We want the X-bar chart to
Gary Jack
Gary
Sample Mean

Jack
0 be out of control. If in Average
0 Shows if an operator has a
-1
control there is either UCL=-0.8445
Mean=-0.899
LCL=-0.9535 problem with measuring a
-1
inadequate resolution or too specific part.
-2
much measurement error -2
0 Part 1 2 3 4 5 6 7 8 9 10

23 DEC 16 IAS - MSA TRAINING 41

 Attribute Terminology
Attribute Data: Qualitative (go / no go) data that can
be tallied for recording and analysis.
Attribute Measurement System: A measurement
system that compares each part to a standard and
accepts the part if this standard is met.
Screen: 100% evaluation of product using inspection
techniques (an attribute measurement system).
Screen Effectiveness: The ability of the attribute
measurement system to properly discriminate good
from bad.
Customer Bias: Operator has a tendency to hold back
good product.
Producer Bias: Operator has a tendency to pass
defective product.
23 DEC 16 IAS - MSA TRAINING 42
 Purpose of Attribute R&R
To assess your inspection or workmanship standards against your
customers requirements
To determine if inspectors across all shifts, all machines, etc use
the same criteria to determine good from bad
To quantify the ability of inspectors to accurately repeat their
inspection decisions
To identify how well these inspectors are conforming to a known
master which includes:
how often operators decide to ship truly defective product
how often operators do not ship truly acceptable product
Discover areas where:
training is needed
procedures are lacking
standards are not defined

23 DEC 16 IAS - MSA TRAINING 43

 Attribute R&R -- The Method
Step 1 - Select a minimum of 30 parts from the process.
50% of the parts in your study should have defects
50% of the parts should be defect free
If possible select border line (or marginal) good and bad
samples
Step 2 - Identify the inspectors which should be qualified.
Step 3 - Have each inspector, independently and in random order, assess
these parts and determine whether or not they pass or fail.
Step 4 - Enter the data into the Inspection.xls spreadsheet to report the
effectiveness of the attribute measurement system.
Step 5 - Document the results. Implement appropriate actions to fix the
inspection process if necessary.
Step 6 - Re-run the study to verify the fix.
Note: A 30 piece sample will yield an estimate of inspector efficiency and
capability which has a fair amount of uncertainty.

23 DEC 16 IAS - MSA TRAINING 44

 Critical and significant
characteristics ?
Critical characteristics are those product
requirements or process parameters that can
affect compliance with government
regulations and/or the safety of the vehicle,
which require special controls or monitoring
actions.

23 DEC 16 IAS - MSA TRAINING 45

 Critical and significant
characteristics ?
Significant characteristics are those product,
process and test requirements that increase
customer satisfaction if the variation is
reduced.

23 DEC 16 MSA TRAINING 46

 Types of Variation ?
1. Special Cause of Variation
2. Common Cause of Variation

23 DEC 16 IAS - MSA TRAINING 47

 Special Cause of
Variation
A source of variation that is
intermittent, often unpredictable,
unstable;sometimes called an
assignable cause. It is signaled by a
point beyond control limits or a run
or other non-random pattern of
points with in the control limits.

23 DEC 16 IAS - MSA TRAINING 48

 Common Cause of
Variation
A source of variation that affects all
the individual values of the process
output being studied; in control
chart analysis it appears as part of
random process variation.

23 DEC 16 IAS - MSA TRAINING 49

Special Causes Examples from job...

car breaks
down Common Causes
accident on traffic lights
freeway weather
amount of traffic
23 DEC 16 IAS -MSA TRAINING 50
 VARIABLES and ATTRIBUTES

Data can be classified in a very general way by how it is collected.

Measured data is called Variable Data.

Counted or Classified data is called Attribute Data

Variable Data. Attribute Data.

Is measured in physical units.
e.g. Millimeters (size)
C (Temperature)
Nm (Torque)
Is continuous.
Can be compared numerically
Is countable. It either exists or
e.g. defects on a painted surfac
Uses pass/fail, OK/Not OK
e.g. Engine starts or does not.

23 DEC 16 IAS - MSA TRAINING 51

 Why VARIABLE measurements are better

than ATTRIBUTE measurements.

Go/ No Go measuring device arent

able to detect the variation in the process -
the main indicator of IMPROVEMENT

GO

NO GO

Controlling a process depends on the ability to detect the differences

(Variation in the process.)

Quality Assurance rely on being able to detect differences in the

OK band to improve the product.

Much more information about a process can be obtained by taking

variable measurements wherever possible.

23 DEC 16 IAS - MSA TRAINING 52