easurement systems are routinely analyzed using traditional gage

repeatability and reproducibility (R&R) studies. These studies are

ANOVA (analysis of variance) methods used to quantify the varia-
tion attributable to gage repeatability and operator reproducibility.
The gage R&R study as it applies to continuous data is widely used and
written about. But another form of this toolthe attribute gage R&Rcan
improve process yields and reduce costs dramatically.
Most processes require at least some form of subjective inspection or val-
idation. It could be a check for blemishes on a painted or plated finish of
a part or a judgment concerning the color, taste or smell of a product. In
some cases, measuring equipment is available to access the acceptability of
such characteristics.
Many times, however, test equipment is not used due to cost or is simply
not available. For example, although profilometers may be available dur-
ing the inspection of a machined surface finish, the surface finish may be
judged using a fingernail test.
This fingernail type of inspection method has the potential for variability
among inspectors and even variability by the same inspector over a period
of time. Any variability in the measurement system will affect the measured
process variability, thereby affecting the measure of process capability.
Although the math is different, the effect of misreporting process capa-
bility is the same for both continuous and attribute gage studies.
One advantage of the attribute gage study is that, unlike the variable
gage study, it can easily be applied to transactional processes. For example,
a study could be performed on how customer service representatives inter-
pret a customer complaint or the way a customer requirement is convert-
ed into an internal order.
In the following case study during the measurement phase of a Black
Belt project, the simplest form (short method) of the attribute gage R&R
is credited with saving a company more than $400,000 annually.
Background
An electroplating company supplying silver plated machined parts for a
telecommunications company was experiencing a rejection rate of just
over 16,000 ppm at the customer facility.
The parts were 100% visually inspected at the silver plating facility for
defects consisting of pits, blisters, voids and rough surfaces. When accept-
ed, the parts were wrapped and shipped to the customer, where they were
sampled and inspected by the customer. If parts were rejected, they were
returned to the supplier for a process referred to as strip and replate.
In this process the existing silver was removed, the part cleaned and new
MEASUREMENT SYST EMS
Attribute Gage R&R
A SIMPLE BUT
ROBUST TOOL SAVED
ONE COMPANY
$400,000 A YEAR.
M
By Samuel E.
Windsor,
Delta Sigma
Solutions LLC
S I X S I G M A F O R U M M A G A Z I N E
I
A U G U S T 2 0 0 3
I
23
silver applied. The strip and replate process cost the
plater two times the cost of the initial plating due to
the cost of removing the existing silver and applying
new silver.
For example, a part with a surface area of 100
square inches would cost 10 cents per square inch to
plate, at a total cost of $10 per part. The cost of the
strip and replate process would be 20 cents per square
inch. This means the rework cost $20 per unit.
Investigation
An investigation was conducted to determine what
specifications were used by both the customer and sup-
plier. In both cases the specification required the parts
to be free of blisters, voids, scratches and roughness.
Even with identical specifications, nearly 2% of parts
plated were rejected at the customers facility. Further
investigation revealed no part was or could be expect-
ed to be completely free of blisters, voids, scratches
and roughness. In addition, there was no real reason
for the parts to be perfect.
There was a need, however, for the blemishes and
other defects to be minimal. The difficulty became
defining minimal.
The attribute gage R&R was employed to investigate
and determine the actual compliance of both the cus-
tomers and suppliers attribute measuring system. As
the project was initiated by the customer, the initial
gage R&R was performed at the customers location
using a slight variation of the short method attribute
gage R&R study, referenced in the Automotive
Industry Action Groups (AIAG) Measurement System
Analysis text.
1
Results
The results were analyzed using a simple spread-
sheet (many statistical software packages also have this
capability.) But a spreadsheet was not required
because the data could also be analyzed manually.
The study was conducted with a sample of 30 parts
selected by their degree of compliance to the actual
engineering requirement. Eight of the 30 parts were
considered unacceptable to varying degrees, and 22
were considered acceptable, some marginally.
Acceptability was determined by the agreement of
two of the customers product engineers. Each part
was numbered and the engineering decision recorded
for each part as the standard.
Two experienced inspectors from the customers
receiving department were chosen to participate.
Each inspector would evaluate each part in the morn-
ing and afternoon of the same day, yielding a total of
120 inspection data points. As the study was conduct-
ed, the results were recorded for each piece next to
the appropriate number on the data collection sheet.
The results are shown in Table 1.
The data analysis indicates inspector one agreed
with himself in 83% of the cases and with the standard
in 53% of the cases. Inspector two agreed with his own
results 90% of the time and with the standard 23% of
the time.
In total, the percentage of time both inspectors
agreed with their own results and with the standard
was 13%. In 33% of the cases, the inspectors agreed
with each other on both trials but not necessarily with
the standard.
An identical experiment with the same 30 parts was
At t r i but e Gage R&R
24
I
A U G U S T 2 0 0 3
I
W W W . A S Q . O R G
Table 1. Summary Results for Initial
Customer Study
Inspector Inspector Both
one two inspectors
Agreed with
own results
83% 90%
Agreed with
standard
53% 23%
Agreed with
each other on 33%
both trials
Agreed with
each other and 13%
with standard
Table 2. Summary Results for Initial
Supplier Study
Inspector Inspector Both
one two inspectors
Agreed with
own results
83% 90%
Agreed with
standard
40% 43%
Agreed with
each other on 33%
both trials
Agreed with
each other and 70%
with standard
At t r i but e Gage R&R
conducted at the supplier using experienced final
inspectors, with the results shown in Table 2.
Comparisons of the study results indicate the sup-
pliers inspection is actually more consistent with the
customers engineering requirements. (Instances in
which both inspectors agreed with each other and
with the standard were 13% for the customer vs. 70%
for the supplier).
S I X S I G M A F O R U M M A G A Z I N E
I
A U G U S T 2 0 0 3
I
25
Table 3. Initial Customer Data
Sample Inspector one Inspector two
number Standard Try one Try two Try one Try two
1 R R R R R
2 A A A R R
3 A R R R R
4 R R R R R
5 A A A A R
6 A A A R R
7 R A A R R
8 A R R R R
9 R A A A R
10 A R R R R
11 R A R R R
12 A A A R R
13 A A A R R
14 R A A A A
15 R A A R R
16 A A A R R
17 A A A R R
18 R R R R R
19 A A R R R
20 A A A R R
21 A A A R R
22 A R R R R
23 A A R R R
24 A R R R R
25 A A R R R
26 A A A A A
27 A A R R R
28 A A A R R
29 A A A R R
30 A A A A R
A = accept; R = reject.
Table 4. Initial Supplier Data
Attribute gage
R&R known
population Inspector one Inspector two
sample number Standard Try one Try two Try one Try two
1 R R R A A
2 A A R R R
3 A R R R R
4 R A R R R
5 A A A A A
6 A A A R A
7 R A A A A
8 A A R R R
9 R R R R R
10 A R R R R
11 R R R R R
12 A R R R R
13 A A A A A
14 R A A A A
15 R A A A A
16 A R R R R
17 A A A A A
18 R R R R R
19 A R R R R
20 A A A A A
21 A R R R R
22 A R R R R
23 A A R A R
24 A A A A A
25 A A A A A
26 A R R R A
27 A A A A A
28 A R R R R
29 A R R A A
30 A A R A A
Inspector one Inspector two
Agreed with own results 83.33% 90.00%
Inspector vs. standard 40.00% 43.33%
A = accept; R = reject.
Further study of the data indicates inspector two in
the initial customer study agreed with himself 90% of
the time but with the standard only 23% of the time.
A detailed review of the results, shown in Tables 3 and
4 (p. 25), also indicates customer inspector two is much
more likely to reject a part than is customer inspector
one. Also, customer inspector one rejected acceptable
parts (type one error) in both trials on five occasions,
while accepting discrepant parts (type two error) on
both trials on four occasions. Inspector two consistent-
ly rejected acceptable parts on 19 of the 30 occasions.
Explanation of Results
The agreed with own results was calculated as the
percentage of agreement over each of the two trials.
In the initial customer study, inspector one had agree-
ment between the first and second attempt on 25 of
the 30 parts (83.3%).
The agreement with standard percentage shows
how often each inspector agreed with himself and
with the standard. In this case, inspector one had 16
cases of 30 when the results were consistent between
the trials and with the standard (53.3%).
At t r i but e Gage R&R
26
I
A U G U S T 2 0 0 3
I
W W W . A S Q . O R G
Table 5. Customer Results After Training
Inspector Inspector Both
one two inspectors
Agreed with
own results
93% 90%
Agreed with
standard
93% 90%
Agreed with
each other on 83%
both trials
Agreed with
each other and 83%
with standard
Table 6. Supplier Results After Training
Inspector Inspector Both
one two inspectors
Agreed with
own results
97% 97%
Agreed with
standard
93% 90%
Agreed with
each other on 83%
both trials
Agreed with
each other and 83%
with standard
Table 7. Customer Data After Training
Attribute gage
R&R known
population Inspector one Inspector two
sample number Standard Try one Try two Try one Try two
1 R A R R R
2 A A A A A
3 A A A A A
4 R R R R R
5 A A A A A
6 A A A A A
7 R R R A R
8 A A A A A
9 R R R R R
10 A A A A A
11 R R R R R
12 A A A A A
13 A A A A A
14 R R A R R
15 R R R A R
16 A A A A A
17 A A A A A
18 R R R A R
19 A A A A A
20 A A A A A
21 A A A A A
22 A A A A A
23 A A A A A
24 A A A A A
25 A A A A A
26 A A A A A
27 A A A A A
28 A A A A A
29 A A A A A
30 A A A A A
Inspector one Inspector two
Agreed with own results 93.33% 90.00%
Inspector vs. standard 93.33% 90.00%
A = accept; R = reject.
At t r i but e Gage R&R
The overall effectivenessor inspector one vs.
inspector two vs. the standard is the percentage of
time each inspector agreed with himself and with the
standardin this case four of 30 times (13.3%).
What the Data Say
The most important part of any such exercise is to
turn the raw data into either a validation of the system
or an action plan to fix the system. In this case the
measurement system is in clear need of repair or
replacement.
Replacement is not an option as there are no known
methods other than human inspection for the
process. The challenge is to correct the present system
to provide consistency between the customer and sup-
plier results.
The data show customer inspector two rejected
many more parts than necessary, indicating he did not
understand the requirements, took a more critical
view of the requirement or was just afraid to accept
any part that had a small inconsequential defect.
Operator one accepted more parts but still made the
correct decision only about 53% of the time.
To address the problem, the customers engineering
and quality representatives worked with the suppliers
quality group to create a standard for the most com-
mon defect types along with minimum/maximum
type photos.
This information was included with the specifica-
tion for silver plating at both the customer and sup-
plier facilities. All inspectors were trained in this spec-
ification, and the actual requirements were discussed
in great detail.
In the weeks after the training session, the gage
R&R study was performed again using the original
parts, with the results shown in Tables 5 and 6.
Original data for the last three studies are shown in
Tables 7 and 8.
Savings
In the measurement phase of any Six Sigma Black Belt
project you must evaluate all forms of measurement, not
only the measurement systems with variable outputs.
You have to ask whether a fingernail type test is repeat-
able within a given inspector and reproducible between
inspectors at both customers and suppliers facilities.
While work continues to further improve the meas-
urement system used in the case study (according to
AIAGs Measurement System Analysis,
2
agreement
should be 100%), the attribute gage R&R yielded sig-
nificant results by dramatically improving the agree-
ment between the customer and supplier and among
inspectors within the two organizations.
S I X S I G M A F O R U M M A G A Z I N E
I
A U G U S T 2 0 0 3
I
27
Table 8. Supplier Data After Training
Attribute gage
R&R known
population Inspector one Inspector two
sample number Standard Try one Try two Try one Try two
1 R A A R R
2 A A A A A
3 A A A A A
4 R R R R R
5 A A A A A
6 A A A A A
7 R R R A A
8 A A A A A
9 R R R R R
10 A A A A A
11 R R R R R
12 A A A A A
13 A A A A A
14 R R A R R
15 R R R A R
16 A A A A A
17 A A A A A
18 R R R A A
19 A A A A A
20 A A A A A
21 A A A A A
22 A A A A A
23 A A A A A
24 A A A A A
25 A A A A A
26 A A A A A
27 A A A A A
28 A A A A A
29 A A A A A
30 A A A A A
Inspector one Inspector two
Agreed with own results 96.67% 96.67%
Inspector vs. standard 93.33% 90.00%
A = accept; R = reject.
At t r i but e Gage R&R
In this case, the gage R&R resulted in an annualized
savings of nearly $400,000 on just the cost of the strip
and replate operation. If the time lost and transporta-
tion costs of returning the material to the supplier had
been tracked, the reported savings would have been
even greater.
The application of attribute gage R&R demon-
strates the variation in inspection methods between
experts when inspection standards are not utilized.
The control phase of a project involving visual
repeatability and reproducibility is an important con-
sideration. Publication and ongoing document con-
trol for visual standards, along with periodic training,
are critical to ensure visual inspection methods
remain consistent.
The tool can be used in this very simple form or
expanded to include confidence intervals and proba-
bilities of defects within particular ranges, as explained
in AIAGs attribute gage R&R long method.
3
The case study in this article shows how the tool can
be used without using software. But the data can also
be analyzed using popular statistical software packages
or with a simple spreadsheet that will perform the cal-
culation for you.
An attribute gage R&R can normally be performed
at very low cost with little impact on the process.
Significant benefits can be gained from looking at
even our most basic processes.
REFERENCES
1. Automotive Industry Action Group, Measurement System Analysis,
www.aiag.org.
2. Ibid.
3. Ibid.
28
I
A U G U S T 2 0 0 3
I
W W W . A S Q . O R G
WHAT DO YOU THINK OF THIS ARTICLE? Please share
your comments and thoughts with the editor by e-mailing
godfrey@asq.org