measure Vol. 7 No.

1 March 2012

IN THIS ISSUE:
Risk Mitigation Strategies for
Compliance Testing

Experimental Study and Computer

Modeling of the Triple Point of
Argon System

Economic Impact of Equivalence

of Measurement Standards

Traceability and Quality Control in a

Radiation Thermometry Laboratory
 measure Vol. 7 No. 1 March 2012

Contents
Welcome to NCSLI Measure, a metrology journal published by
NCSL International for the benefit of its membership.

Features
24 SPECIAL FEATURE
A Conversation with the Director of NIST

TECHNICAL NOTE
 0 Calculation of Dual Circle Closure Calibrations using
3
Computer Spreadsheets
Hy D. Tran

TECHNICAL PAPERS
38 Risk Mitigation Strategies for Compliance Testing
Jonathan Harben and Paul Reese

50 Wattmeter Calibration for Use in Standby Power Testing to IEC 62301

Ilya Budovsky and Dimitrios Georgakopoulos

56 Experimental Study and Computer Modeling of the Triple Point of

NCSL International Argon System
Workshop & Symposium R. Ding, M. J. Zhao, T. Nielson, E. Nerdrum, and D. Farley
Page 18
62 Economic Impact of Equivalence of Measurement Standards
Takashi Usuda and Andy Henson

72 Traceability and Quality Control in a Radiation Thermometry Laboratory

Frank Liebmann and Tom Kolat

Departments
3 Letter From the Editor
4 NMI News
8 Metrology News
78 New Products and Services
80 Advertisers Index

NCSL International Craig Gulka, Executive Director

2995 Wilderness Place, Suite 107 Boulder, CO 80301 (303) 440-3339

Vol. 7 No. 1 March 2012 NCSLI Measure | 1

 measure
NCSLI Measure (ISSN #19315775) is a metrology journal published by NCSL International (NCSLI).
The journals primary audience consists of practitioners and researchers in the field of metrology,
including laboratory managers, scientists, engineers, statisticians, and technicians. NCSLI
Measure provides NCSLI members with practical and up-to-date information on calibration
techniques, uncertainty analysis, measurement standards, laboratory accreditation, and quality
processes, as well as providing timely metrology review articles. Each issue contains peer reviewed
metrology articles, new product and service announcements, technical notes, national metrology
institute news, and other metrology information. Author instructions are available at www.ncsli.org.
If you are interested in submitting new product/service announcements or purchasing advertising,
please visit www.ncsli.org for more information.

Managing Editor:
Michael Lombardi, National Institute of Standards and Technology (NIST), USA,
lombardi@ncsli.org

Associate Editors:
Dr. Klaus Jaeger, Jaeger Enterprises, jaegerenterprises@comcast.net
Harry Moody, Harry J. Moody Enterprises, harryjmoody@cs.com
Dr. Leslie R. Pendrill, SP Technical Research Institute of Sweden, leslie.pendrill@sp.se
Dr. Alan Steele, National Research Council of Canada, alan.steele@nrc-cnrc.gc.ca

NMI/Metrology News Editor:

Dr. Richard B. Pettit, NCSL International, 7808 Hendrix Rd., NE Albuquerque, NM 87110
USA, randepettit@comcast.net

New Product/Service Announcements & Advertising Sales:

Linda Stone, NCSL International, 2995 Wilderness Place, Suite 107, Boulder, CO
80301-5404 USA, lstone@ncsli.org

Technical Support Team:

Norman Belecki, NIST (Retired), 7413 Mill Run Dr., Derwood, MD 20855 USA
Carol Hockert, National Institute of Standards and Technology (NIST), USA
Dr. James K. Olthoff, National Institute of Standards and Technology (NIST), USA
Dr. Salvador Echeverria-Villagomez, Centro Nacional de Metrologia (CENAM), MX
Dr. Seton Bennett, National Physical Laboratory (NPL), UK
Dianne Lalla-Rodrigues, Antigua/Barbuda Bureau of Standards, Antigua W.I.
Dr. Angela Samuel, National Measurement Institute (NMI), Australia
Pete Unger, American Association for Laboratory Accreditation (A2LA), USA
Dr. Michael Khne, Bureau International des Poids et Mesures (BIPM), FR

Copyright 2012, NCSL International. Permission to quote excerpts or to reprint any

figures, tables, and/or text from articles (Special Reports/Features, Technical Papers, Review
Papers, or Technical Notes) should be obtained directly from the author. NCSL International, for
its part, hereby grants permission to quote excerpts and reprint figures and/or tables from articles
in this journal with acknowledgment of the source. Individual teachers, students, researchers,
and libraries in nonprofit institutions and acting for them are permitted to make hard copies of
articles for use in teaching or research, provided such copies are not sold. Copying of articles
for sale by document delivery services or suppliers, or beyond the free copying allowed above, is
not permitted. Reproduction in a reprint collection, or for advertising or promotional purposes, or
republication in any form requires permission from one of the authors and written permission from
NCSL International.

2 | NCSLI Measure www.ncsli.org

Letter From the Editor
Greetings! My name is Michael
Lombardi, and this is my first
issue serving as the managing
editor of NCSLI Measure.
Ive been involved with the
journal since its inception,
having previously worked as
an associate editor and news
editor. And I have a long involvement in metrology, having worked
in the Time and Frequency division of NIST for about 30 years. I
hope to maintain the same high standards that my friend and mentor
Dick Pettit established during our first six years of publication. Dick
now occupies my old position as news editor, and continues to make
invaluable contributions to the journal.
There are some other staff changes Id like to announce. We are
fortunate to have two new associate editors who are both distinguished
metrology professionals. Alan Steele is the director of Metrology at
the National Research Council Institute for National Measurement
Standards (NRC-INMS) in Ottawa, Canada. Leslie Pendrill is a
research director at the SP Technical Research Institute of Sweden
and the chair of the European Association of National Metrology
Institutes (EURAMET). Alan and Leslie join our long-time associate
editors Klaus Jaeger and Harry Moody. Klaus and Harry are each
past presidents of NCSLI, and have made countless contributions to
our organization for many years. Another NCSLI past president,
Carol Hockert of the NIST Weights and Measures division, continues
to help produce the special features for NCSLI Measure as she has
since 2008. We all benefit from Carols knowledge and her extensive
list of contacts.
Ive been reflecting lately on why metrology is such a rewarding
field. Few, if any of us, started out in life thinking that wed be
metrologists. Somewhere along the way, however, we discovered that
metrology has an unmistakable allure. To me, the allure of metrology
comes from its purity and from its connection for the search for truth.
When I first arrived at NIST, we had an equipment rack full of
LF phase tracking receivers. I had long been interested in radio, so
I immediately wanted to know what these radios did. What, for
example, could you listen to? When I asked my boss that question,
he explained that you couldnt listen to anything; there wasnt enough
bandwidth on the signal to carry audio. These receivers produced
only one thing, a sine wave at a known frequency. At first, that
HOW TO REACH US
NCSLI Measure, 2995 Wilderness Place, Suite 107, Boulder, CO 80301-5404 USA
www.ncsli.org measure@ncsli.org
Vol. 7 No. 1 March 2012 NCSLI Measure |
seemed incredibly dull, but I soon learned how accurate and stable
those sine waves were. The receivers did only one thing, but they
were well engineered machines that did that one thing exceptionally
well. I discovered the purity involved in working with fundamental
quantities, in working with standards.
Later, I learned that the search for truth is the foundation of
metrology. We live in a world where we are bombarded with messages
from advertisers, political pundits, bloggers, talk show hosts, and so
on. We are used to hearing things that might not be absolute lies
but that are not completely true, and nobody warns us, for example,
when their words are only 60 % true. Metrologists, in contrast, are
trained to always acknowledge that there is some uncertainty in their
measurement results. They then provide a quantifiable analysis of
that uncertainty, and continually try to reduce the uncertainty to get as
close to the truth as possible.
We will attempt to capture the allure of metrology, including its
purity and its search for truth, in each issue of NCSLI Measure.
This issue features six excellent technical papers, including a
noteworthy contribution from Takashi Usuda of NMIJ/AIST in
Japan and Andy Henson of the BIPM, entitled Economic Impact of
Equivalence of Measurement Standards. This significant addition to
the literature describes how the work of national metrology institutes
can benefit their nations economy. This issue also features an
interesting interview with Pat Gallagher, the director of NIST.
Id like to close by reminding you that authors and peer reviewers
are the life blood of the journal, and we encourage and welcome your
contributions. If you currently publish metrology related papers or
would like to begin publishing, please consider NCSLI Measure for
your future publications. If you have an idea for a paper that you
would like to discuss, please send me an email. Or, if you want to
be added to our list of peer reviewers, please contact me and provide
information about your area of expertise.
Sincerely,
Michael Lombardi
Managing Editor
lombardi@ncsli.org
3
NMI NEWS
PTB Develops Quantum-Based
Impedance Bridges
Schematic set-up of the quantum bridge. The quantum
voltages are adjusted by varying the microwave frequen-
cies 1 and 2 , and the bridge is balanced. By synchroniz-
ing current source 1 and current source 2 with the delay
electronics, the phase between the AC voltages can be
adjusted. Credit: PTB.
Traditional impedance bridges make use of inductive dividers and
achieve, in their frequency range from 500 Hz to 10 kHz, excellent
relative measurement uncertainties of only a few parts in 109. The
bridges must, however, be adjusted for each frequency of operation
and this involves a complex manual procedure.
The Physikalisch-Technische Bundesanstalt (PTB) newly
developed Josephson impedance bridge can be easily adjusted. The
AC voltage amplitudes from two Josephson arrays are adjusted over
their microwave frequency range, and the phase angle between the
synthesized voltages is adjusted via delay electronics with a resolution
of 10 ps. Both processes are fully automatic. The utilization of
quantum standards to generate a voltage on both sides of the bridge
renders new adjustments at all other frequencies unnecessary. It is
therefore possible to perform precise measurements at 20 different
frequencies within just 30 minutes.
The efficiency of the new procedure has been demonstrated by
measuring the 1:1 ratio between two 10 k resistors and two 100 pF
capacitors. The resistance ratio over the frequency range from 25 Hz
to 10 kHz was determined with a measurement uncertainty of about
2 108. For capacitance ratios, the uncertainty lies in the kHz range
below 1 108. With decreasing frequency, the uncertainty increases
as a function of the impedance 1/C. At 25 Hz, the uncertainty is
2 107, which is still 20 times smaller than when measuring with
traditional bridges.
In further developments, the new Josephson bridge will be used
for ratio measurements where a resistor is compared to a capacitor.
By integrating the frequency-independent quantum Hall resistor, the
frequency response of capacitors could be calibrated up to the range of
technical frequencies (in Germany: 50 Hz) with high precision.
For more information, contact Luis Palafox: luis.palafox@ptb.de
4 | NCSLI Measure
NIST Delivers Zero Gravity Metrology
Sensor to NASA
The National Institute of Standards and Technology (NIST) Engineering
Laboratory (EL) researcher Nick Dagalakis recently delivered the
prototype of a zero gravity volumetric sensor to engineers from the
NASA Goddard Space Flight Center (NASA/GSFC). NASA funded
the development of this experimental sensor, called Cryogenic Fuel
Gauge for zero gravity environments using capacitance tomography,
for the purpose of creating better fuel tank imaging sensors under
low gravity operating conditions. NIST designed, fabricated, and
performed dynamic testing of the sensor for NASA. Part of the work
involved the fabrication of innovative flexible arrays of specially-
arranged micro devices by EL guest researcher Dr. Seung Ho Yang.
The electronic circuitry was developed with the collaboration of
researchers from the PML Quantum Electrical Metrology Division,
Dr. Rae Duk Lee and Dr. Yicheng Wang. The sensor was successfully
tested and then moved to a new NASA/GSFC laboratory where work
will continue in order to make the sensor practical and space-worthy
for use in NASA space craft fuel tanks.
For more information, contact Nick Dagalakis at
nick.dagalakis@nist.gov
PTB Celebrates 125-year anniversary in
March 2012
After a 125-year success story, Physikalisch-
Technische Bundesanstalt (PTB) will cele-
brate its anniversary on March 28, 2012,
which is the very same day it was granted its
first imperial budget. There will be a political
ceremonial event in the Braunschweig
Civic Centre and a scientific symposium on March 27, 2012 titled
Metrology, the Universe and Everything. The 2012 Helmholtz
Prize will be awarded on that same date.
The two fathers of the PTB are the industrialist Werner von
Siemens, who initiated its founding, and the scientist Hermann von
Helmholtz, its first president. This connection will be celebrated
on October 17, 2012 in Berlin, when PTB will host lectures at the
Audimax of the TU Berlin and the official dedication of its restored
Observatory.
Finally, PTB has had its book series reprinted on the history of the
institute and is making the classical experimental physics reference
book Praktische Physik by Friedrich Kohlrausch available on its
website.
For more information, visit:
www.ptb.de/cms/en/presseaktuelles/125-years-precisely.html
www.ncsli.org
 NMI NEWS

PTB Guide for NMIs on How To Obtain New CENAM Vacuum Mass Comparator
International Recognition
Guide No. 6/2011
Installed
1

Physikalisch-Technische Bundesanstalt
(PTB) has recently published a guide
titled Towards Mutual Recognition of
Metrological Competence by Christin
Bauer (Guide #6). The guide is intended
to give an insight into the steps necessary
when a national metrology institute (NMI)
TOWARDS MUTUAL RECOGNITION OF
METROLOGICAL COMPETENCE
Christin Bauer wishes to gain international recognition
of both its national measurement
standards and its calibration and measur-
ement capabilities (CMCs) under the
International Committee for Weights and
Measures Mutual Recognition Arrangement (CIPM MRA). The new vacuum mass comparator at CENAM, with Luis
The formal, up to date and authoritative guidance is that issued Manuel Pea Prez (L) and Luis Omar Becerra Santiago
by the CIPM and/or the Joint Committee of the Regional Metrology (R). Credit: Axel Becerril, CENAM
Organizations (JCRB) is available from the Bureau International des
A vacuum mass comparator, called M-1, was installed at Centro
Poids et Mesures (BIPM) website, specifically those pages addressing
Nacional de Metrologa (CENAM), making Mexico the first country
the CIPM MRA at www.bipm.org/en/cipm-mra/
in Latin America to have this equipment. The equipment will provide
greater accuracy for measurements related to the value of mass taking
The 12 page guide is divided into three sections:
place in Mexico.
1. Why have internationally recognized measurement capabilities?
Currently, CENAM maintains 67 national measurement standards
2. Building mutual recognition of measurement capabilities.
for Mexico. One of these is the national standard mass, referred to as
3. The steps to mutual recognition.
the No. 21, which is a kilogram that is traceable to the international
prototype of the kilogram maintained by the Bureau International des
The guide is available from the PTB website:
Poids et Mesures (BIPM). At one kilogram, the new mass comparator
www.ptb.de/en/org/q/q5/index.htm
has a resolution of 1 part in 1010, which is equal to 0.1 g. This
technological innovation places CENAM at the level of the leading
scientific laboratories in the world.
NIST Solicits Proposals For Research
Grant Programs The balance will improve the accuracy in mass measurements and
strengthen the link of the traceability of these measurements within
The National Institute of Standards and Technology (NIST) is Mexico. This activity was supported by the division of Mass and
soliciting proposals under the Measurement Science and Engineering Density in the Mechanical Metrology area of CENAM.
(MSE) Research Grant Programs for financial assistance for FY 2012
under the following nine program areas: For more information, contact Luis Omar Becerra Santiago
(1) Material Measurement Laboratory Grant Program; or Luis Manuel Pea Prez: lbecerra@cenam.mx
(2) Physical Measurement Laboratory Grant Program; or lpena@cenam.mx
(3) Engineering Laboratory Grant Program;
(4) Fire Research Grant Program;
(5) Information Technology Laboratory (ITL) Grant Program; NIST Device Measures Absolute Optical
(6) NIST Center for Neutron Research Grant Program; Power in Fiber at Nanowatt Levels
(7) Center for Nanoscale Science and Technology Grant Program;
(8) Standards Services Group Grant Program; and
(9) Office of Special Programs Grant Program.

For all programs, except the Fire Research Grant Program,

proposals must be received by June 1, 2012. Details of each program
can be found in the full announcement for 2012-NIST-MSE-01, which
is available at:

www.nist.gov/director/ocfo/grants/upload/2012-mse-ffo.pdf

Colorized micrograph of multi-walled carbon nanotubes,

each 40 m long, which absorb more than 99.9 % of the
light inside NISTs prototype fiber-coupled radiometer.

Vol. 7 No. 1 March 2012 NCSLI Measure | 5

You have a big job.
Now theres a complete calibration
software system to match.

New Fluke Calibration MET/CAL Plus 8.0 software: the only all-in-one calibration
software solution that measures up to the task.
Your job is tough. You need to increase productivity and throughput with a more complex, higher-
volume workload. And you need to reduce costs while maintaining high accuracy, traceability, and
extensive documentation. New Fluke Calibration MET/CAL Plus Calibration Management Software
version 8.0 includes everything you need to address those needs.
Its comprehensive calibration automation, tracking, documentation, and reporting capabilities can
increase productivity by up to 80 %.

See for yourself. Download free MET/CAL Plus 8.0

demo software and get more product information at
www.flukecal.com/metcal8

Fluke Calibration. Precision, performance, confidence.

Electrical RF Temperature Pressure Flow Software 2011 Fluke Calibration. Ad 4092141A


Researchers at the National Institute of Standards and Technology
(NIST) have demonstrated a prototype device capable of absolute
measurements of optical power delivered through an optical fiber.
The device is the worlds first fiber-coupled cryogenic radiometer
that links optical fiber power measurements directly to fundamental
electrical units and national standards. It uses a microscopic forest of
carbon nanotubes to measure values that are about one-thousandth of
the levels typically attained with a cryogenic radiometer lacking direct
fiber input capability. With improvements in temperature control and
speed, the device could meet the needs for ultraprecise calibrations
at ultralow power in telecommunications, medical devices and other
industries.
Optical power and energy are traceable to fundamental electrical
units. Radiometers absorb optical energy and convert it to heat. Then
the electrical power needed to induce the same temperature increase
is measured. Because optical and electrical heating are not exactly
equivalent, measurement uncertainties can be relatively large from a
metrology point of view.
The demonstration is also a step toward converting radiometry
from a classical practice based on electrical units to a quantum practice
based on single particles of light (photons).
The new radiometer is about 70 mm long and incorporates a
1.45 mm thick optical fiber capped by a light-trapping cavity at
one end with the nanotube absorber and a heater. The ultra-dark
nanotubes are grown on a tiny X-shaped piece of micromachined
silicon. Light absorption was so high it was difficult to determine
measurement uncertainties; Lehman travelled to a special facility at
the National Physical Laboratory in the United Kingdom to make
some measurements.
Experiments and calculations indicate the new radiometer
can measure a power level of 10 nanowatts with an uncertainty of
0.1 %. By comparison, typical measurements of optical power
delivered through fiber have an uncertainty of 3 % or more at similar
power levels. More importantly, these commercial devices rely on a
series of calibrations to establish traceability to national standards.
NIST aims to develop an absolute quantum standard for optical
power and energy based on single photons. The effort includes
development of sources and detectors spanning a wide range of
optical power measurements, from single photon counts to trillions of
photons. Single photons are already used in quantum communications
systems, which offer novel capabilities such as detecting extremely
weak optical signals and providing quantum guarantees on security.
NPL Launches Carbon Measurement
Center
The National Physical Laboratory (NPL)
will open the worlds first center for carbon
measurement to support the science of
emissions reduction, climate change and
low carbon technology. The new center
will provide the necessary measurement
infrastructure for the UKs transition to a low
carbon economy. Environment expert Jane
Jane Burston, NPL. Burston has been appointed to establish the
center and head up the work.
Vol. 7 No. 1 March 2012 NCSLI Measure |
NMI NEWS
Plans for the center for Carbon Measurement were drawn up
in response to the threat of global climate change, which requires
policies to reduce emissions through low carbon energy generation,
improved efficiency, development of low carbon technologies and
behavioral change.
The center for Carbon Measurement will focus on three areas:
Improving climate data: Improve the accuracy and consistency
of climate data through data auditing, setting standards, devel-
oping measurement technology, and instrumentation testing.
Carbon trading and pricing: Develop the underpinning mea-
surement science and technology to support carbon reporting
and trading. This will enable regulation, ensure a fair and stable
market, support businesses in reporting and managing emis-
sions, and provide confidence to establish and operate interna-
tional agreements.
Low carbon technologies: Provide access to the best measure-
ment techniques for developers of low carbon technology, in or-
der to allow scientific validation of their performance.
For more information, visit: www.npl.co.uk
Primary Reference Gas Mixtures Support
New Developments in Natural Gas
Industry
In Europe and elsewhere, there is a growing
demand to open gas grids for the transportation
of natural gas for alternative fuels, such as biogas
and other energy carrying gases. In response to
this demand, VSL, the Dutch National Metrology
institute participates in a project to support the
development of metrological services for gas
transport companies and related industries.
As part of the project Characterization of
Energy Gases, funded in the frame of the
European Metrology Research Program, VSL
developed primary reference gas mixtures and
measurement methods for measuring hydrogen and oxygen in natural
gas. These reference gas mixtures can be used by laboratories for
calibrating and checking their measurement equipment.
Traditionally, oxygen is practically absent in natural gas, but it
occurs in potential alternatives, such as biogenic gas (0.1 % to 2 %;
molar basis), and coal mine methane. The first set of mixtures was
measured with different gas chromatographic configurations, and
the results over time indicate that there are no stability issues. The
uncertainty of measurement of oxygen in natural gas is approximately
0.4 % relative.
Hydrogen is naturally occurring in natural gas, but usually in very
small amounts. In various regions in Europe, hydrogen is blended in
natural gas. A first suite of mixtures was prepared gravimetrically with
4 % to 14 % (molar basis) hydrogen and subsequently checked for
consistency using two different gas chromatographs. The uncertainty of
measurement of oxygen in natural gas is approximately 0.2 % relative.
These two extensions supplement VSLs existing range of primary
and certified reference materials for the natural gas industry.
7
METROLOGY NEWS
Joint BIPM/OIML/ILAC/ISO Declaration lished in the relevant areas of the KCDB;
on Metrological Traceability Within the OIML Mutual Acceptance Arrangement (MAA), ac-
creditation should be provided by bodies which are signatories
The International Bureau of Weights
to the ILAC Arrangement and the above policies on traceability
and Measures (BIPM), the International
to the SI should be followed.
Organization of Legal Metrology (OIML),
the International Laboratory Accreditation
These principles should be used whenever there is a need to dem-
Cooperation (ILAC) and the International
onstrate metrological traceability for international acceptability.
Organization for Standardization (ISO) are
each involved in metrology, accreditation
For more information and a copy of the Joint Declaration,
and standardization efforts worldwide.
visit: bipm.org/en/bipm/mou/bipm-oiml-ilac-iso_joint_declara-
Because each organization realizes that
tion.html
metrologically traceable measurement
results are essential to their core mission, a joint policy document was
approved and published on November 9, 2011. Members of the four
Impact of New SI on Legal Metrology
organizations, as well as others for whom traceability is important,
A recently published paper, by Dr Richard Davis, BIPM, discusses the
are encouraged to adopt the joint policy document. Other standards
impact of the proposed changes to the International System of Units
bodies are encouraged to declare their support for the principles and
(SI) on legal metrology. The paper, titled SI Units - Proposed change
practices embodied in this declaration wherever possible.
to the definition of the kilogram: Consequences for legal metrology,
The document builds on the tripartite statement* issued by the
was published in International Organization of Legal Metrology
BIPM, OIML and ILAC on January 23, 2006 on the relevance of
OIML Bulletin, vol. LII, no. 4, pp. 5-12, October 2011.
various international agreements on metrology to trade, legislation
The paper, which is a progress report, begins by discussing the
and standardization.
present definition of the kilogram and the practical problems it poses
The International Vocabulary of Metrology - Basic and General
for mass metrology and for science generally. Next, the concept
Concepts and Associated Terms (VIM) defines metrological
of a redefinition of the kilogram based on fundamental constants
traceability as:
is presented, which shows that any reasonable redefinition can be
realized by at least two different approaches. Then the prospects for a
property of a measurement result whereby the result can be related
redefinition of the kilogram in approximately five years is discussed;
to a reference through a documented unbroken chain of calibrations,
with an emphasis on the impact (if any) on legal metrology.
each contributing to the measurement uncertainty.
Up to date information on the new SI, including Draft Chapter 2 of
the 9th SI brochure (September 2010), is available from: www.bipm.
The definition indicates that metrological traceability embodies
org/en/si/new_si/
the concepts of measurement uncertainty and calibrations against a
hierarchy of reference standards. Therefore, the BIPM, OIML, ILAC,
The full paper is available from:
and ISO endorse the following recommendations:
In order to be able to rely on their international acceptability,
kcdb.bipm.org/NL/16/OIML_Davis_2011.pdf

calibrations should be performed either in National Metrology

Institutes (NMIs) that should normally be signatories to the In-
New KCDB Reports Web Page
ternational Committee for Weights and Measures Mutual Rec-
A new Key Comparison Data Base (KCDB) web page gives access to
ognition Arrangement (CIPM MRA) and have Calibration and
a presentation outlining the structure and importance of the KCDB.
Measurement Capabilities (CMCs) published in the relevant
The 21 page presentation is by Dr. Claudine Thomas, BIPM KCDB
areas of the Key Comparison Data Base (KCDB), or in labora-
Coordinator. In addition to the presentation, users can access the
tories accredited by accreditation bodies which are signatories to
biannual reports that the KCDB Office prepares, details concerning
the ILAC Arrangement;
Measurement uncertainty should follow the principles estab-
publication of key and supplementary comparisons, and approved sets
of CMCs.
lished in the Guide to the Expression of Uncertainty in Mea-
In the latest KCDB Newsletter (No. 16, December 2011), it is
surement (GUM);
The results of the measurements made in accredited laboratories
reported that the number of CMCs as of September 2011 was 24,196!
The evolution of the CMCs over the past 10 years, which were around
should be traceable to the International System of Units (SI);
11,000 in March 2002, is shown in the figure. Note that over a period
NMIs providing traceability for accredited laboratories should of 9.5 years, the CMCs have increased by 120 %, or an average of
normally be signatories to the CIPM MRA and have CMCs pub-
more than 1,400 new CMCs each year. In addition, there are 50
member states of the BIPM with CMCs listed in the KCDB.
*
See www.bipm.org/cc/CIPM/Allowed/95/BIPM_ILAC_OIML_300106.pdf

8 | NCSLI Measure www.ncsli.org

 METROLOGY NEWS

EURAMET Launches its 2020 Strategy

The European Association of National
European Association of National Metrology Institutes

Metrology Institutes (EURAMET) has

EURAMET
published a new document that outlines its
2020 Strategy
Issued:
Version:
strategy from now until the year 2020. This
15-09-2011
1

strategy document discusses the vision, mission

and strategic objectives, and concentrates on
helping member nations and Europe to meet
future needs related to innovation, growth and
societal well-being, through the provision and
development of quality assured and traceable measurement.
EURAMETs vision is: To be the leader in the development and
application of measurement enabling Europe to be competitive, healthy
and sustainable through innovation. In addition, its mission is:
Historical plot of the total number of CMCs from March
2002 to September 2011. To develop and disseminate an appropriate, integrated and cost ef-
fective measurement infrastructure for Europe taking into account
In addition, the KCDB website on CMCs in Chemistry was the needs of end users in industry, business and government.
modified. The acronym QM and the expression Amount of To ensure that the European measurement infrastructure is in-
substance are generally not familiar to the industrial community, ternationally competitive and recognized, and is based on robust
and have been replaced by the word Chemistry. The CMCs and high quality science and R&D.
covering Physics and Chemistry are also clearly separated, and some To support members in meeting their own national requirements
new functionality is offered to users in order to facilitate access to through collaboration and a balanced European measurement
information in these areas. infrastructure.

For a copy of the KCDB presentation and biannual reports, visit A copy of the 2020 strategy can be obtained from the
the BIPM/KCDB website: bipm.org/jsp/en ViewKCDBReport.jsp EURAMET website: euramet.org

Under Pressure To Make Connections Fast,

Calibrate Faster!
Calibrate Faster? For transmitters, gauges and pressure
sensing instruments
No wrenches or thread sealants
Eliminates thread damage
Reduces contamination risks
Twist-to-connect, sleeve and handle actuated
connectors save time
Leak-tight sealing from vacuum to 10,000 psi
1/8" to 1/2" NPT, AN37 flare, sanitary style
flange and metric with additional sizes
available

Improve your lab efficiency

calibration connection manifold
for calibration of up to four
instruments simultaneously!

www.fastestinc.com
800.444.2373

Vol. 7 No. 1 March 2012 NCSLI Measure | 9

METROLOGY NEWS

EURAMET Publishes Legal Time The document is organized into the following main chapters: (1)
Regulations and Practices Introduction to Proficiency Testing; (2) Selection of PT Schemes; (3) Use
of PT by Laboratories; (4) How a PT Provider Evaluates the Laboratorys
European Association of National Metro-
Performance; and (5) Laboratory Interpretation of PT Results.
logy Institutes (EURAMET) has recently
published a summary of the regions legal
To obtain a copy of the guide, visit the EURACHEM website:
time metrology regulations and pract-
www.eurachem.org/index.php/publications/guides/usingpt
ices. The document, titled EURAMET
Countries Legal Time Regulations and
Practices, covers the regions three time
EURAMET Countries' Legal Time EURACHEM Guide to VIM3 for
Analytical Measurements
Regulations and Practices
zones.
A survey conducted among EURAMET
1

Focus for Analytical Chemistry in Europe (EURACHEM) has

members in 2009 and 2010 revealed that
published a new guide titled Terminology in Analytical Measurement:
more than half of these countries have their legal time covered by
Introduction to VIM 3. This publication is a guide for analytical
legislation, while other countries use UTC (Coordinated Universal
scientists for the third edition of the International Vocabulary of
Time) as their de facto legal time. Typically, the national metrology
Metrology - Basic and General Concepts and Associated Terms (VIM3).
institute (NMI) or a designated institute (DI) in a country is responsible
This EURACHEM Guide was written because in VIM3 there are
for maintaining and disseminating legal time or de facto legal time.
substantial changes to terminology in order to accommodate chemical
There are a significant number of countries, however, where this is
and biological measurements. In order to make the new VIM3
not the case.
accessible to analysts, the publication provides context and additional
The traceability of time in 22 European countries is covered
examples which relate to the new concepts to chemical and biological
through an ongoing international key comparison, maintained by
measurements.
the Bureau International des Poids et Mesures (BIPM); in addition,
Examples are provided over and above those in the Notes
two countries obtain their traceability through another NMI, while a
accompanying the definitions in VIM3. The document is organized
significant one fifth of surveyed countries have no traceability of time
into the following chapters: (1) General Metrology; (2) Metrological
in place. Finally, comments and remarks of participating NMIs related
Traceability; (3) Measurement Uncertainty; and (4) Validation and
to legal time issues are summarized at the end of the document.
Method Performance.
The guide focuses on the most likely concepts from VIM3 that
The publication is available from:
might be encountered in an analytical laboratory. It covers chemical,
www.euramet.org/index.php?id=documents
biological, and clinical measurements, and is intended for laboratory
staff, accreditation bodies, for users of measurements.

Updated EURACHEM/EUROLAB/EA To obtain the guide, visit:

Proficiency Testing Guide www.eurachem.org/index.php/publications/guides/48-gdtam11
The EURACHEM/EUROLAB/EA Proficiency
Testing Working Group has released a revised
CMSC 2011 Report How Behavior Im-
Selection, Use and
Interpretation of Proficiency edition of its well-known guide Selection, Use
est g ((PT
Testing T)) Schemes
Sc e es
and Interpretation of Proficiency Testing (PT) pacts Your Measurement
Schemes. CMSC 2011 Measurement Study ReportThe Coordinate Metrology Society
This document represents the current state-
(CMS) published a report in July 2011
of-the-art with respect to the selection and
Second Ed
dition 2011 2011 CMSC Measurement Study Report that summarizes the results of their
use of proficiency testing schemes, and the How Behavior Impacts Your Measurement

large-scale, interactive measurement

interpretation of results and evaluations given
study conducted at the 27th Coordinate
in proficiency testing schemes. Although this is
Metrology Systems Conference (CMSC).
aimed primarily at staff in analytical laboratories, it is also useful for
The 58-page report, titled How Behavior
customers of laboratories, assessors working for accreditation bodies 27th Annual
Coordinate Metrology Impacts Your Measurement, focuses on
and other external users of PT scheme results. Systems Conference

Phoenix, Arizona
measurement strategies and behaviors
July 2011
of coordinate metrologists. More than
The main topics covered by the Guide are:
100 conference attendees participated
1. Aims and benefits of participation in PT schemes.
in the data collection activity coordinated by the National Physical
2. Selecting the most appropriate PT scheme.
Laboratory (NPL) assisted by members of the CMS Certification
3. Understanding the basic statistics and performance scoring used
Committee.
by the PT providers.
In 2009, the Certification Committee of the Coordinate Metrology
4. Using and interpreting the PT results in order to improve the
Society (CMS) was formed to consider the need for professional
overall performance of the laboratory.
certification in the field of metrology. After market research and
an investigation of existing certifications, the group determined a

10 | NCSLI Measure www.ncsli.org

 Discoverthe
BlueBox
Difference

Model 6242T Thermometry Bridge

0.01 to 100k range Accuracy < 10 x 10-8

6 channel scanner Linearity < 5 x 10-9
Keep Warm Currents Choice of reference resistor

www.mintl.com

321A-EXhlfPgAd.qxp:Layout 1
12 | NCSLI Measure
GUILDLINE INSTRUMENTS LIMITED
6625T T EMPERATURE M EASUREMENT S YSTEM
5/31/11 2:41 PM Page 1
BetaGauge 321A-EX
Dual Sensor
Intrinsically Safe for Use in
Hazardous Areas
. Accuracy: 0.025%
Also
Available
.. ClearBrite LCD Display
in Single
Sensor
.. Single or Dual Sensor
25 Pressure Ranges
BetaGauge .. Pt100 RTD Connector
Read mA
311A-EX
. 3NIST
Key Martel Menu System
Traceable Calibration
Martel Electronics Corporation
Ex ia IIB T3 Gb (Ta=10... +45C)
KEMA 10 ATEX 0168X 0344 Tel: 800-821-0023 / 603-434-1433
Ex ia IIB T3 Gb (Ta=10... +45C) sales@martelcorp.com
II 2 G IECEx CSA 10.0013X
www.martelcalibrators.com
www.ncsli.org

properly structured certification program would be of value to the
CMS. The committee began the development of a preliminary Body of
Knowledge. In 2010, the CMS Certification Committee performed a
statistical study at CMSC aimed at identifying skill gaps in the general
metrology community. The open measurement workshop enticed
hundreds of attendees to test their knowledge of core metrology
principles based on a variety of hand tools used in dimensional
measurement.
These studies lead to the CMS to commission a follow-up study
based on coordinate metrology equipment. The research project
collected measurement results and focused on core knowledge,
understanding, and behavioral requirements. On two days the attendees
were observed performing 3D coordinate measurements, while on the
third day, the participants were invited to the measurement workshop
where the study group presented their findings.
Authors Keith Bevan, NPL, and Trevor Toman, Coventry
University, performed statistical analysis of the study and compiled
their conclusions into the report. The authors examined various
training and assessment techniques, such as the evaluation of the
participants prior learning and experience level, questioning methods,
practical task monitoring, and demonstration. Nearly a quarter of the
participants had less than three years of measurement experience, a
dozen fell in the midrange of three to seven years of practice, while 69
participants were veteran metrologists with more than seven years of
hands-on experience with various measurement systems.
The outcome of this study drives home the importance of defined
best practices, and understanding measurement fundamentals
Vol. 7 No. 1 March 2012 NCSLI Measure |
METROLOGY NEWS
that enable an individual to make informed judgments about a
measurement, irrespective of the technology used whether it is hand
tools or 3D portable metrology systems. Questioning and planning
the requirements of the measurement help reduce the possibility of
making poor measurements.
During the two-day study, participants were asked to perform three
tasks using portable 3D coordinate metrology equipment for data
acquisition: (1) measure a door using a combination of a Laser tracker,
retro reflectors, and software; (2) measure an engine compartment
using a combination of an articulating arm, probe and software; and
(3) measure a vehicle using a combination of a Laser Tracker, probing,
and laser scanning system.
The final report concluded that whether the metrologist uses
hand tools or the most sophisticated coordinate measuring system,
questioning and planning all the requirements of the measurement
help to reduce the possibility of generating poor measurements. The
keys to informed measurement results are: (1) not taking things for
granted, and (2) implementing behavioral questions and techniques
for the measurement process.
To obtain the report, visit:
www.cmsc.org/stuff/contentmgr/files/0f7dbf9282c3245d7573d
89eb82030080/files/cmsmeasurementreport2011.pdf
13
METROLOGY NEWS

Norman Ramsey Dies at 96; Research R221, Specific Requirements: Forensic Examination Accreditation
Led to the Atomic Clock Program-Testing.
Forensic Testing Laboratories, Inc.s Scope of Accreditation
Norman F. Ramsey, 1989 Nobel prize
includes presumptive testing for blood, semen and saliva as well as
winning physicist at Harvard University,
DNA testing of forensic samples for nuclear DNA. The forensic
and a giant in the field of time and frequency
examination program will help to bridge the gap between what is
metrology, passed away on November 4,
normally considered forensic examination (e.g., DNA, fingerprints,
2011. He was 96 years old. He invented
and crime scene examination) and the often overlooked work of
the separated oscillatory field method for
the commercial industry for civil litigation (e.g., environmental
studying atoms and molecules in 1949,
investigation, forensic engineering, failure analysis, and accident
leading to the development and construction
analysis).
Norman F. Ramsey. of atomic clocks. In this separation method,
The Forensic Examination Accreditation Program consists of two
(IEEE) atoms are exposed to a magnetic field only as
separate accreditation options. The first is for the accreditation of
they enter and exit a device now known as the Ramsey Cavity. This
forensic testing laboratories. Laboratories seeking accreditation under
method is known today simply as the Ramsey Method.
this option will be assessed for compliance to ISO/IEC 17025 and
During World War II, Ramsey worked at the MIT Radiation
A2LA policies and requirements, including the A2LA supplemental
Laboratory and headed a group that developed 3 centimeter radar.
document, R221, Specific Requirements: Forensic Examination
Soon after, he went to Los Alamos, New Mexico to work on the
Accreditation ProgramTesting.
Manhattan Project.
The second accreditation option is for the accreditation of
Later, while working at Harvard, he established a molecular beam
forensic inspection bodies (e.g., crime scene units, environmental
laboratory. Working with atomic hydrogen, he invented the atomic
inspection units, forensic engineering structural failure inspection
hydrogen maser, which was used for accurate measurements of the
units). Organizations seeking accreditation under this option will
hyperfine structure of hydrogen, deuterium, tritium, and so on. Along
be assessed for compliance to ISO/IEC 17020 and A2LA policies
with colleagues including Robert Vessot and others, he converted the
and requirements including the A2LA supplemental document,
hydrogen maser to a clock with unprecedented stability.
R309, Specific Requirements: Forensic Examination Accreditation
He led a National Research Council committee that investigated
ProgramInspection.
the assassination of President John F. Kennedy. The committee
A2LA is currently accepting new applications for accreditation
concluded in 1982 that the acoustical evidence did not support the
under the Forensic Examination Accreditation Program. Further,
existence of a second gunman. After he retiring from Harvard in 1986,
A2LA is working with existing accredited organizations to add
he remained active in physics, working as a research fellow at JILA,
compliance with these new requirements to their existing Scopes
part of the University of Colorado.
of Accreditation. Organizations already accredited by A2LA should
In addition to the Nobel Prize, Ramseys work has been recognized
direct questions on expanding their Scope (through the use of A2LA
with many prestigious awards, including: Presidential Certificate
form, F108, Request for Expansion of Scope of Accreditation
of Merit (1947); E. O. Lawrence Award (1960); Davisson-Germer
Testing) to their current Accreditation Officer.
Prize (1974); Columbia Award for Excellence in Science (1980);
and the IEEE Medal of Honor (1984). The IEEE award reads: For
For additional information regarding this program, contact
fundamental contributions to very high accuracy time and frequency
Karin Athanas at: kathanas@a2la.org
standards exemplified by the cesium atomic clock and hydrogen
maser oscillator.
Free ISO Online Tutorial for
For more information about Dr. Ramsey, visit the Nobel Prize
ISO/IEC 17021
website (www.nobelprize.org) and the Institute of Electrical and
A free tutorial on the second edition of ISO/IEC 17021:2011,
Electronics Engineers (IEEE) website (www.ieee.org).
Conformity Assessment Requirements for Bodies Providing
Audit and Certification of Management Systems, is available on
the International Organization for Standardization (ISO) website.
A2LA Accredits First Forensic This standard sets new requirements for the auditing of management
Examination Testing Laboratory systems and for auditor competence is available below.
The American Association for Laboratory The purpose of ISO/IEC 17021:2011 is to increase the value
Accreditation (A2LA) has accredited its first of management system certification to public and private sector
laboratory in the field of Forensic Testing. The organizations worldwide. The online tutorial on the standard is
first accreditation was granted on November made available by the ISO Committee on Conformity Assessment
11, 2011 to Forensic Testing Laboratories, (ISO/CASCO). It was developed by Randy Dougherty and Alister
Inc. located in Las Cruces, New Mexico. Dalrymple, the two leaders of the working group responsible for the
The location is a branch facility of Genetics Testing Laboratory, Inc., standard.
an A2LA-accredited testing laboratory. Forensic Testing Labs. was The tutorial, which is available in English, French and Spanish,
assessed to ISO/IEC 17025 and the A2LA supplemental document, consists of three PowerPoint presentations, including notes: (1)

14 | NCSLI Measure www.ncsli.org

Vol. 7 No. 1 March 2012 NCSLI Measure | 15
METROLOGY NEWS
An Overview (27 slides); Audit Process (62 slides), and Auditor
Competence (19 slides). The tutorial explains the differences to the
first edition, published in 2006, and the new requirements.
Published on 1 February 2011, ISO/IEC 17021:2011 is intended to
increase trust in certificates issued attesting conformity to management
system standards (MSS), such as ISO 9001 (quality management),
ISO 14001 (environmental management) and ISO 22000 (food safety
management).
For a copy of the tutorial, visit: www.iso.org/tutorial17021
Co-Nanomet Publishes European
Nanometrology 2020
Co-Nanomet Co-Nanomet (Coordination of Nanometrology
Co-ordination of Nanometrology in Europe
in Europe) has recently published a new report
titled European Nanometrology 2020.
European
Nanometrology 2020 This report provides a guide to the many
bodies across Europe in their activities or
responsibilities in the field of nanotechnology
and related measurement requirements. It
supports the commercial exploitation of
nanotechnology, as it transitions through this
next exciting decade.
The report defines a common strategy for European nanometrology
such that future nanometrology development in Europe will build upon
its many current strengths. In this way, European nanotechnology will
be supported in order to reach its full and exciting potential.
Since the field of nanotechnology covers a breadth of disciplines
each having specific and varying metrological needs, the report focuses
on a set of four core technology fields: 1. Engineered Nanoparticles;
2. Nanobiotechnology; 3. Thin Films and Structured Surfaces; and 4.
Modeling & Simulation. Specific objectives underpinning these four
core fields are defined, plus the underlying assumptions.
The report is organized with the following sections:
A. Policy-Relevance of Nanometrology
B. Current and Emerging Regulatory Issues
C. Nanometrology and Standardization
D. Metrology for Laboratory Quality Assurance
E. Technical Challenges and Opportunities
F. Infrastructure
G. Education and Skills
Of particular interest is Section E which provides a detailed
scientific review across eight special nanometrology areas:
Biological; Chemical; Dimensional; Electrical; Mechanical; Modeling
and Simulations; Nanostructured Materials; and Thin Films. The
emerging technical challenges within these areas are cross-cutting and
underpinning to the four priority themes detailed above.
To obtain the report, visit: www.euspen.eu/page1406/Resources/
Online-Publications/European-Nanometrology-2020
16 | NCSLI Measure
New Book by Robert P. Crease:
World in the Balance
A new book, published by W. W. Norton &
Co, Inc., deals with the history of metrology
and measurements. The book is entitled
World in the Balance - The Historic Quest for
an Absolute System of Measurement and was
written by Robert P. Crease of Stony Brook
University.
According to the books overview: Mill-
ions of transactions each day depend on a
reliable network of weights and measures.
This network has been called a greater
invention than the steam engine, comparable only to the development
of the printing press.
Robert P. Crease traces the evolution of this international system
from the use of flutes to measure distance in the dynasties of ancient
China and figurines to weigh gold in West Africa to the creation
of the French metric and British imperial systems. The former
prevailed, with the United States one of three holdout nations. Into
this captivating history Crease weaves stories of colorful individuals,
including Thomas Jefferson, an advocate of the metric system, and
American philosopher Charles S. Peirce, the first to tie the meter to
the wavelength of light. Tracing the dynamic struggle for ultimate
precision, World in the Balance demonstrates that measurement is
both stranger and more integral to our lives than we ever suspected.
The introduction to the book starts with the amusing story titled
The Noonday Canon. In this legend, an inquisitive boy asks
the local artilleryman how he knows when to fire the noonday
cannon. Answer: On the order of his commanding officer. And
the commanding officer keeps his precision watch accurately set by
passing the towns watchmaker store each day. Finally, the boy learns
that the watchmaker sets his store clock by the noonday cannon!
For more information, visit:
books.wwnorton.com/books/detail.aspx?ID=20538
NLA 2012 Test & Measurement
Conference
The 2012 Test and Measurement
Conference, sponsored by National
Laboratory Association (NLA) South
Africa, will be held from September
2-5, 2012 at the Elangeni Hotel &
Conference Centre in Durban, South
Africa. The theme for T&M 2012 is:
The impact of reliable measurements
on our environment.
Topics of discussion will include how developing nations need to
comply with the Kyoto Protocol to reduce greenhouse gas, and new
regulations for the automobile and medical industries.
There are special discounts available for those whose final
manuscripts are submitted by July 1, 2012.
For more information, visit: www.nla.org.za/conferences/2012
www.ncsli.org
 Joe D. Simmons
NIST (NBS) (1963-1994)
1992 NCSLI, William A. Wildhack
Founded in the memory of Joe Simmons to support the study of
award winner.
1995 MSC, Andrew J. Woodington
measurement science and metrology related quality topics.
award winner.
Co-founder and Chair of
ASQ Measurement Quality Division.
QUALIFIED STUDENTS ARE ENCOURAGED TO APPLY
$3000 S C H O L A R S H I P
Completed applications are due March 1

For application forms or more information contact your

www.ncsli.org advisor, student aid office, or the Scholarship itself at:
SimmonsScholarship@ncsli.org
www.ncsli.org (search Simmons)
Submit to:
Simmons Scholarship
PROMOTING 2995 Wilderness Place, Suite 107
ACADEMIC
EXCELLENCE IN Boulder, CO 80301
METROLOGY

Vol. 7 No. 1 March 2012 NCSLI Measure | 17

C o n f erence Sponsors
 KEYNOTE Making the Economic Case for Quality
ADDRESS James J. Rooney is Director of the Department of Energy Programs for the Public Sector
Division of ABS Consulting. Jim currently serves as Chair of the American Society for Quality
(ASQ). Mr. Rooney has more than 30 years of experience in quality, risk assessment, and
process safety management for the chemical process, pharmaceutical, petroleum, medical
device, nuclear power, and nuclear weapons industries.

The economics of quality cant be denied, especially during these challenging times. Increasing
evidence concludes that when company executives choose to invest in quality, they are investing
in their companys bottom line. Implementing quality tools, processes and principles reduces
error, waste and defects. While quality ultimately leads to a better product or service, customer
and employee satisfaction is increased along the way. Consequently, customer satisfaction
promotes a good reputation and that is reflected in revenue and customer retention rates.
This presentation details the economic case for quality and why no company can afford not
to focus on quality and the company-wide benefits it offers. Reducing costs and producing
James J. Rooney, P.E.
revenue are measurable effects of quality that organizations cant deny, while service value
and satisfaction are effects of quality that the customer cant deny. Maintaining quality is just
as important as implementing it. This presentation will also identify risks of using quality as a
quick fix, as opposed to allowing it to transform practices and become a permanent institution
for long term financial prosperity.
 NCSLI Workshop & Symposium
2012 REGISTRATION RATES
Early Bird Regular Late Rate
FULL REGISTRATION Ends Rate Ends Begins
RATES March 1 May 30 June 1
Member Rate $1,055 $1,105 $1,205

Non-Member Rate $1,255 $1,305 $1,405

REGISTER NOW! *Book your room at a HQ hotel and receive $65 off one full conference registration rate.

One Day Registration $425 $425 $425

NCSL INTERNATIONAL HQ HotelS
Hyatt Regency Sacramento la carte Extra Luncheon Tickets $35 Guest Banquet Tickets $90
1209 L Street
Sacramento, California 95814
Phone: 916-443-1234
Single/Double $160 2012 TUTORIAL PROGRAM RATES
Phone In Reservations USA: 888-421-1442
International: 402-592-6464
TUTORIAL Regular Rate Ends Late Rate Begins
PROGRAM RATES June 30 July 1

Sheraton Grand Sacramento Hotel two DAY tutorial RATE

1230 J Street
Sacramento, California 95814 Member $675 $700
Phone: 916-447-1700 Non-Member $725 $750
Single/Double $160 FULL DAY tutorial RATE
Member $400 $425
Full Conference Registration INCLUDES: Non-Member $450 $475
Sunday: Exhibitor Welcome Reception HALF DAY tutorial RATE
(badges required - guest badges are available)
Member $225 $250
Monday - Thursday: Continental Breakfast Non-Member $275 $300
Monday - Thursday: Technical Program including Lunch
Monday - Wednesday: Entrance into the Exhibit Hall *Present your HQ hotel reservation code to receive your $65 discount when purchasing your
full conference registration.
1 Banquet Ticket
(Guest tickets are available to purchase - limited availability) All attendees and guests attending any NCSL International function must be registered.

ncsli.org 303-440-3339 info@ncsli.org

T UTO R I A L P RO G R A M
Friday, July 27, 2012 8:00 AM - 5:00 PM full day
T1 Train the Trainer: I Wish I Would Have Thought of That!
Georgia Harris, NIST
This train the trainer tutorial will start with review of good instructional design con-
cepts using the ADDIE model and Blooms taxonomy; we will then review three key
training essentials: 1) writing learning objectives, 2) selecting learning activities, and
3) assessing student understanding. During this session, participants will share their
own examples of learning objectives, ideas for activities, and assessment methods. Par-
ticipants need to bring relevant metrology content from a course they will teach or
have taught to work on during the session so that they can share a best practice ac-
tivity during the workshop portion. The activity workshop is an opportunity to share
good metrology related ideas and examples in groups among metrology instructors
and to give each other encouragement and constructive feedback through peer review.
Saturday and Sunday, July 28-29, 2012 8:00 AM - 5:00 PM
two day
T2 An Introduction to Instrument Control and Calibration Automation in LabVIEW
Laura Ramirez, National Instruments, Paul Packebush, National Instruments
Automation of instrument calibration improves repeatability of service, reduces manpower
requirements, and improves service margin. Many calibration laboratories understand the
benefits of automation, but feel they lack the appropriate software development skills to
perform coding tasks. Modern programming environments and tools are extremely power-
ful, but can seem daunting to laboratory engineers. Alleviating these concerns, a number
of software companies supply simplified tools and libraries to ease the interactions between
calibration standards and automation software. Additionally, programming tools exist that
are designed with the automation engineer in mind. During this two day hands-on tutorial,
session participants will explore the LabVIEW environment, learn to develop, instrument
control, data-logging, and measurement analysis applications. At the end of the course,
they will be able to create applications using basic design templates and architectures to
automate processes, acquire, process, display, and store real world data.
T3 Balance Calibration and Use in an Analytical Environment
Mark Ruefenacht, Heusser Neweigh, Val Miller, NIST
This NIST Seminar will discuss the sources of weighing errors in analytical environments,
methodologies for quantifying the errors, and computation of balance calibration uncer-
tainty. Attention will be given to error sources, selection of standards, and various cali-
bration and testing procedures used in the balance industry. The focus will then move
to use of balances in an analytical environment where compliance with pharmaceutical
(FDA/ USP) and international requirements will be discussed and practiced. Method-
ologies for process measurement assurance techniques in analytical weighing will also
be covered. Participants should leave with a working knowledge of balance calibration
methods, uncertainty estimation, measurement assurance concepts and minimum bal-
ance load computations that can be applied directly to their applications. This NIST
seminar is designed for beginner to advanced users of balances or calibration managers
wanting a better understanding of balances and associated uncertainties in organizations
where analytical weighing is an integral part of operations. Limited to 30 participants
(Laptop with spreadsheet capability and/ or scientific calculator strongly encouraged.)
Saturday, July 28, 2012 8:00 AM - 5:00 PM full day
T4 Running an Effective Laboratory: Measuring Performance tutorial will address three key areas: measurement uncertainty, conformance decision rules,
Jesse Morse, Morse Metrology, Malcolm Smith, WesCan
This How To tutorial will be of interest to owners, managers, and supervisors of calibra-
tion laboratories, both in-house and commercial. The tutorial will cover five areas where
performance measurement is important in the running of a laboratory: 1) productivity, 2)
quality, 3) finance, 4) service levels, and 5) customer satisfaction. The range of measures
and associated tools that can be used to establish goals and monitor performance in each
of these areas will be explained and discussed. Suggestions on how these tools might be
used in practice will be reviewed. Examples of measurements needed for effective process
improvement projects will also be given.
T5 Pressure Metrology
Mike Bair, Fluke Calibration
This full day Pressure Metrology tutorial will cover pressure measurement concepts and
terminology, standards used in pressure metrology, measuring and calibration methods,
sources and calculation of uncertainty and other practical considerations.
T6 Understanding ISO/IEC 17025 Requirements and Most Common Deficiencies
Pam Wright, A2LA
This full day tutorial covers ISO/IEC 17025 requirements as well as the ten most commonly
cited deficiencies for laboratories that are currently accredited, are in the process of obtain-
ing their ISO/IEC 17025 accreditation or for those who are interested in applying the ISO/
IEC 17025 requirements in their laboratory.
Saturday, July 28, 2012 8:00 AM -12:00 PM half day morning
T7 Fundamentals of Gas Flow Measurement
Bob DeRemer, CSA International
The main thrust of the fundamentals of gas flow measurement tutorial will be the com-
parison of volumetric flow meters and mass flow meters used in gas flow measurement
applications. Specific topics that will be covered will include principles of operation of
various types of flow meters, factors that influence when to choose a mass flow meter or a
volumetric flow meter, how to interpret performance specifications, how the various types
of meters are calibrated, and an example of measurement uncertainty.
T8 Fundamentals of Calibration in Dimensional Metrology
Jim Salsbury, Mitutoyo Corporation, Amosh Kumar, Mitutoyo Corporation
This tutorial provides an overview of calibration techniques and key issues in dimensional
metrology. This tutorial will include a variety of hands-on practical calibration exercises.
All the major types of dimensional calibrations will be discussed. We will start by stud-
ying the calibration, use, and traceability issues of the standards used in the field, such
as gage blocks, ring gages, optical flats, thread wires, and length standards. We will then
discuss the calibration of common small measuring tools such as micrometers, calipers,
dial indicators, and height gages. Hands-on exercises in the calibration of small tools will
include procedures, worksheets, and certificates. In addition to presenting specific cali-
bration methods, the bigger goal of the tutorial is on developing understanding of the
principles behind the dimensional calibration methods thereby giving the attendee the
necessary tools to extend the tutorial concepts to other types of calibrations. Recom-
mended procedures including optional methods, sources of errors and methods to reduce
them, and techniques for estimating the uncertainty of measurement will be presented.
Saturday, July 28, 2012 1:00 PM - 5:00 PM half day afternoon
T9 Root Cause Analysis
Rob Knake, A2LA
This half day tutorial will provide tips and tools that can be used to identify the true root
cause for identified problems/non conformities. Many standards such as ISO/IEC 17025
and ISO 9001 require your quality management systems have a procedure for investigating
and eliminating the root cause of a nonconformance during the corrective action process.
T10 Applying RP-16 to Metrology Lab Design and Construction
Douglas Cooper, Precision Environments, Inc.
RP-16 was conceived to provide guidance to metrologists for testing metrology lab environ-
ments. This tutorial will be interactive and hands on. An overview of applying RP-16 to cre-
ating design specifications will be given pertaining to the specific topics presented during
the course of the tutorial. Vibration testing will be demonstrated applying RP-16 practices
with the use of accelerometers. Attendees will participate in constructing an operating lab
using these concepts. Construction will include architectural, mechanical, electrical and
temperature control system installations. Different components of the mechanical system
will be shown and discussed so the attendees understand the basis for component selec-
tions. Different temperature controls will be present with sensor installation in the lab. The
lab will be tested using the principals of RP-16. Attendees will then be divided into groups
to create their own specification for environmental control of a lab. Ideas will be combined
to create a comprehensive final specification with all groups participating.
T11 Modern Calibration Concepts: Radical Rethinking of Measurement Uncer-
tainty, Decision Risk Rules, and Proficiency Testing
Jim Salsbury, Mitutoyo Corporation, Amosh Kumar, Mitutoyo Corporation
Recent research and international standardization in measurement uncertainty has brought
forward new and fundamental concepts that are challenging the status quo in the calibra-
tion field. The resulting changes are most apparent in the calibration and performance
verification of measuring instruments. While the concepts in this tutorial are advanced
in nature, the treatment is at a fundamental level and anybody with at least a basic un-
derstanding of calibration and measurement uncertainty will benefit from attending. This
and proficiency testing. The measurand in the calibration of measuring instruments will be
carefully considered, along with the myth of resolution and repeatability in measure-
ment uncertainty. The use of measurement uncertainty in decision rules will then be ad-
dressed from both a risk management and uncertainty management perspective. Novel ap-
proaches to proficiency testing will also be presented to support the new, and much lower,
measurement uncertainty estimates in the calibration of measuring instruments. This tuto-
rial is most suitable for quality and technical representatives from ISO/IEC 17025 accredited
calibration laboratories, but anyone involved with calibration or testing will benefit.
T12 Good, Bad, or Indeterminate: Who Makes the Call?
Jeff Gust, Fluke Calibration
When calibration data is provided on a certificate, ISO/IEC 17025 requires a statement of
the measurement uncertainty. In addition, if a statement of compliance with specifications
is made, the uncertainty of measurement must be taken into account. ANSI/NCSL Z540.3
requires the false accept risk to be less than 2 % if a TUR of greater than 4:1 cannot be main-
tained. This tutorial will describe some practical ways to consider uncertainties and their as-
sociated false test decision risk when making in-tolerance or out-of-tolerance declarations.
T13 Knowledge Retention Techniques for Lab Managers
Walter Nowocin, Medtronic, Inc.
Biomedical and other industries are facing similar issues in how to retain the loss of knowl-
edge when technical employees leave their position or retire. Lost knowledge is a sig-
nificant issue facing todays leaders as shifting demographics are threatening the future
capabilities of their workforce. The scope of this tutorial is to examine a knowledge reten-
tion strategy from four perspectives: human resources processes and practices; knowledge
transfer processes; IT applications to capture, store and share knowledge; and knowledge
recovery initiatives. We will use David W. DeLongs methodology as a practical application
of a knowledge retention strategy and process. We conclude that the use of a knowledge
retention framework creates a roadmap for organizations on how to mitigate the risk of the
loss of knowledge in an aging technical workforce.
T UTO R I A L P RO G R A M
Sunday, July 29, 2012 8:00 AM - 5:00 PM full day
T14 Increasing Personal and Organizational Effectiveness
Dean Williams, Duke Energy
This hands-on interactive one day workshop provides participants with a key to unlocking
and activating nine insights for increasing their personal and their organizations effective-
ness. The nine insights are broken into three easy to grasp and apply learning modules. The
three principles provide insights for establishing a framework for greater effectiveness. The
three practices provide insights on how to apply daily habits of greater effectiveness. The
three pragmatics provide insights into how to overcome the inevitable obstacles to greater
effectiveness. Each insight is explained, illustrated, and the learning reinforced with hands-
on and group learning activities.
T15 Auditing, Traceability, and Auditing Traceability
Barbara Belzer, NVLAP
This tutorial will be of interest to managers and staff of laboratories with new or mature
quality systems. It will cover what constitutes objective evidence by stepping through the
internal audit process, including skills that an auditor needs to have to be effective. The
tutorial will emphasize the importance of records for all aspects of the management sys-
tem including reference documents, method validation and their interdependency with
metrological traceability and reporting results. Using the described internal audit process
prior to an onsite assessment companies new to the accreditation process as well those with
mature quality systems will be able to prepare better for an upcoming onsite assessment.
Sunday, July 29, 2012 8:00 AM - 12:00 PM half day morning
T16 Microwave Measurement Basics
Ron Ginley, NIST edge useful for obtaining laboratory accreditation and establishing and maintaining cost
This tutorial will cover the basic aspects of microwave scattering-parameter (s-parameter)
measurements, microwave power measurements, and the care and feeding of microwave
connectors. Special emphasis will be on s-parameter measurements using vector network
analyzers (VNAs) and transmission line basics for the s-parameter portion. The power por-
tion will cover different types of power detectors and power meters that are used today as
well as understanding the measurement process including mismatch factor correction. The
cleaning and proper use of microwave connectors will also be covered.
T17 Fundamentals of Torque Calibration
Henry Zumbrun, Morehouse Instrument Co., Bill Lane, Morehouse Instrument Co.
This presentation is a review of the fundamentals of torque calibration. Topics include an
overview of torque standards including ASTM E2428 and BS 7882, uncertainty of torque
calibration standards, Type A and B uncertainty analysis, torque calibration equipment,
calibration and testing of torque transducers, proper calibration techniques, error sources
associated with torque calibration, and why proper torque measurement is more than just
a traceable length and mass calibration. This segment will cover torque transducers as well
as proper torque wrench use. There will be a hands-on demonstration on how to properly
use a torque wrench and the errors associated with improper handling. This segment is in-
tended for those who are involved with torque calibration, those wanting to minimize the
errors associated with improper use of torque equipment, and for those who have questions
that need to be answered.
T18 Fundamentals of Temperature Calibration
Tom Wiandt, Fluke Calibration and FDA regulations. This four hour tutorial provides the basic training needed to get start-
This presentation is a review of the fundamentals of temperature calibration. Topics include
calibration equipment, calibration techniques, curve fitting issues, and the mathematics
important to thermometry. Types of thermometers covered include platinum resistance
thermometers, thermistors, thermocouples, and combined thermometer/readout systems.
This segment is intended for those who are new to temperature calibration, those who need
to validate what they already know, or those who just have some nagging questions that
need to be answered.
T19 Humidity Calibration
Ken Soleyn, GE Sensing lecture covering the basics of radiation temperature measurement and infrared thermom-
The humidity measurement tutorial provides metrologists, engineers and technicians as well
as those specifying and operating metrology, process, and building automation instruments
and controls with an overview of humidity measurement and instrumentation fundamen-
tals. Humidity measurement and control is a very important for energy management, proc-
ess control, product testing, and process control and validation of various parameters such
as process yield, heat transfer, dimensional stability, emissions control and power manage-
ment. The science of humidity measurement is often very difficult to understand as humid-
ity may be expressed in a variety of units and is influenced by temperature, pressure and gas
composition. A review of various sensing technologies is presented and emphasis is placed on
providing insight into the design and calibration of humidity and trace moisture instrumen-
tation. The workshop includes discussions of various humidity measurement technologies.
A CD-ROM with humidity parameter conversion software and technical papers is provided.
Sunday, July 29, 2012 1:00 PM - 5:00 PM half day afternoon
T20 Force Calibration
Mike Tovey, Tovey Engineering
Force calibration is a special discipline with many considerations not common to other
areas of metrology. Often measurement uncertainties are underestimated due to the omis-
sion of significant error sources. Metrologists must consider both mechanical and electrical
boundary conditions to achieve calibrations with low measurement uncertainties. Factors
such as second order material responses, and interaction of undesired parasitic loading due
to fixture characteristics, misalignment of load frame components, stiffness, etc. can have
significant influence on the measurement result. This tutorial will cover the characteristics
of force transducers, force calibration methods, force calibration standards ASTM E74 and
ISO 376 and measurement uncertainty models for primary standards, secondary standards
and field transfer standards. The tutorial begins at a basic level and leads to discussion of
more complex issues.
T21 Intermediate Dimensional Metrology
Ted Doiron, NIST
There are a large number of books on dimensional metrology, and a few classes and tutori-
als, but nearly all of them are at the beginner level how to use the instruments for inspec-
tion. Thermal expansion, elastic deformation, stability, refractive index of air, closure and
reversal methods, and the large collection of tricks-of-the-trade that make up dimensional
metrology are generally ignored. This tutorial will be an overview of important techniques
and concepts not covered in books and classes. Each concept will be presented with exam-
ples of how the techniques make measurements more accurate, and in some cases, more
efficient.
T22 Uncertainty and Risk Analysis for Compliance with ANSI/NCSL Z540.3
and other Standards
Howard Castrup, Integrated Sciences Group
Because of its specific risk control requirements, the publication of ANSI/NCSL Z540.3
has stimulated considerable interest over the past two years in measurement decision risk
analysis. In giving courses on this subject, it has become clear that, to be effective, its
treatment requires a working knowledge of measurement uncertainty analysis. This tuto-
rial is intended to present the highlights of both analytical disciplines to pave the way for
further study. The outcome of such will be useful not only for satisfying the requirements of
ANSI/NCSL Z540.3, but those of other standards as well. Additional benefits include knowl-
effective measurement quality assurance.
T23 Extending Electrical Calibration Techniques to RF and Microwave
Richard Ogg, Agilent, Jian Jason Liu, Agilent
This tutorial is for the technician familiar with DC and low frequency calibration tech-
niques and takes them into RF and microwave measurements. Differences can be signifi-
cant and the attendee will gain an appreciation for those dissimilarities. The course begins
with a discussion of transmission lines from a practical understanding of the basic con-
cepts. The focus within this tutorial is on recognizing when transmission line concepts
might be violated and undermine the integrity of the measurement. The course also looks
at RF power measurements up to millimeter wavelengths and spectrum concepts and meas-
urements using a spectrum analyzer. This tutorial will not address the S-plane, S-parameter
measurements, or vector network analyzer topics.
T24 Basic Training for New Calibration Technicians in the Biopharma and
Medical Device Industries
Jay Bucher, Bucherview Metrology Services
Most biopharma and medical device companies give calibration responsibilities to the low-
est ranking member of their quality assurance department. That is how it used to be done,
but this is the 21st Century and times have changed. A quality calibration program, no mat-
ter if all their test equipment is sent out for calibration/repair, or accomplished in house,
must include a training program which is a mandatory requirement under ISO standards
ed in the right direction for any calibration/metrology department. The learning objectives
for this workshop are: The real meaning of calibration and traceability; what the regulations
and standards actually say; calibration records and what needs to be included in them; The
five steps of a quality calibration program; and surviving an audit/inspection data.
T25 Fundamentals of Radiation Thermometry Calibration
Frank Liebmann, Fluke calibration
This presentation is an overview of the basic knowledge necessary to perform radiation
thermometer calibrations. The presentation is divided into two parts. The first part is a
etry calibration. The second part is a hands-on portion which covers the steps necessary
to make a calibration measurement, plus a number of tests to determine measurement
uncertainty, and the computation of uncertainties following international standards. The
attendee will be provided with a spreadsheet to facilitate the computation of uncertainty.
The type of radiation thermometer covered in this presentation is an infrared thermometer
with a thermopile detector and an 8 to 14 m bandwidth. However, the principals taught
are applicable to other classifications of radiation thermometers to include radiation ther-
mometers with pyroelectric detectors and thermal imagers. The presentation is geared to
those who are new to radiation thermometer calibration, those who need a refresher on the
subject, and to those who would like to perform better calibrations.
T E C H N I C A L sessions
ABOU T THE CONFERENCE
The NCSLI
annual Conference will
include five tracks and the
following Sessions,
Panels and Plenary Sessions.
Track A: Learning and Development Track C: Management/Quality
Metrology Ambassadors Proficiency Testing - Managing Systems
Panel: Metrology Ambassador Workshop Panel: Proficiency Testing - Requirements
Speaking the Language of Metrology and Benefits
Best Practices Using Presenting/Training Tools Panel: Traceability
Measurement Assurance Management and Quality
What Is a Metrologist? Managing M & TE
Healthcare Metrology Proficiency Testing -Technical Reports
Panel: Meet the NCSLI Learning and Workplace Management
Development Committees Management/Quality Dimensional
Laboratory Operations
Track B: Metrology Skills - Technician/ Laboratory Management
Bench Applications
Electrical - Resistance Measurements Track D: Global Perspectives
Metrology Skills Dimensional Metrology Integration into Manufacturing
Gas/Flow Metrology Plenary - The New SI
Mass Metrology Applying Metrology to Business
Electrical - Resistance Measurements Regional Metrology Organization News
Pressure Panel: Addressing Challenges in the Global
Standard Reference Materials Testing Marketplace
Time/Frequency Weights and Measures: Impacting All of Us
Panel: Dimensional Testing and Dimensional Applying Metrology-from the NMI to Production
Calibration Panel Discussion: ILAC P14: Trials and
Electrical - Power Tribulations
Phasors/Magnetics Panel: Data Evaluation Concepts
Track E: Metrology Concepts - Theory
Metrology Potpourri
Workshop & Symposium Metrology Theory - Dimensional
The Business End of Metrology Electrical - Current Measurements
Temperature
Quality and Testing
Electrical - Power
July 29 - August 2, 2012 Uncertainty
RF/Microwave
Uncertainty and Conformity Assessment
Electrical - Resistance

e x hibitors
A.K.O. Inc., Torque Specialties Div. Guildline Instruments Limited Ohm-Labs, Inc.
A2LA - American Association for Laboratory Heusser Neweigh On Time Support, Inc.
Accreditation IET Labs, Inc. Paroscientific, Inc.
Acucal, Inc. INSCO Metrology Pond Engineering Laboratories, Inc.
Additel Corporation Interface Pratt & Whitney Measurement Systems
Agilent Technologies Sponsor International Accreditation Service Precision Environments, Inc. Sponsor
Alltest Instruments Isotech Quality Vision Services, Inc.
Ametek Kaymont Consolidated Ind. Inc. Radian Research, Inc.
Andeen-Hagerling King Nutronics Corporation Ralston Instruments
ANSI-ASQ National Accreditation Board/ ACLASS Laboratory Accreditation Bureau RH Systems
ASL U.S. Lockheed Martin Rice Lake Weighing Systems
AssetSmart Lockheed Martin Technical Operations Rohde & Schwarz USA, Inc.
Bios International Corporation Mahr Federal Inc. Rotronic Instrument Corp.
Bruel & Kjaer Martel Electronics Corporation Sartorius Mechatronics Corporation
Burns Engineering, Inc. Measurements International SIMCO Electronics
Clarke-Hess Communications Research Meatest Standards Council of Canada
Colorado Engineering Experiment Station Inc. Mensor Corporation Sponsor Tegam, Inc.
(CEESI) Mettler-Toledo Tektronix Service Solutions
Condec Michell Instruments, Inc. Test Equipment.Com
Crystal Engineering Corporation Micro Motion TestEquity LLC
Data Proof Mitutoyo America Corporation The Bionetics Corporation
Dynamic Technology, Inc. / Trescal Sponsor Morehouse Instrument Co. The Boeing Company Sponsor
Edison ESI/So. Calif. Edison Sponsor National Instruments The Modal Shop
Electronic Development Labs, Inc. National Physical Laboratory Thunder Scientific Corporation
Essco Calibration Lab National Research Council of Canada Transmille
Exelon PowerLabs NAVAIR Metrology Tovey Engineering, Inc.
FasTest, Inc. Naval Surface Warfare Center, Corona Troemner, LLC
Flexim NIST Measurement Services U.S. Army Primary Standards Laboratory
Fluke Calibration Sponsor NIST/NVLAP Vaisala, Inc.
Fred V. Fowler Co. NMIJ / AIST Western Environmental Corporation
GE Measurement & Control Solutions Northrop Grumman Corporation WorkPlace Training, Inc.
SPECIAL FEATURE

A Conversation with the

Director of NIST
Introduction
Dr. Patrick Gallagher was confirmed as the 14th Director of the U.S. Department of Commerces
National Institute of Standards and Technology (NIST) on November 5, 2009. He also serves as
Under Secretary of Commerce for Standards and Technology, a new position created in the America
COMPETES Reauthorization Act of 2010, signed by President Obama on January 4, 2011.
Gallagher provides high-level oversight and direction for NIST. The agency promotes U.S. inno-
vation and industrial competitiveness by advancing measurement science, standards, and technology.
NISTs FY 2012 resources include $ 750.8 million from the Consolidated and Further Continuing
Appropriations Act, 2012, [Minibus] H2112, (Public Law 112-55), $62.7 million estimated service
fees and $128.9 million from other agencies. The agency employs about 3,100 scientists, engineers,
Dr. Patrick Gallagher technicians, support staff, and administrative personnel at two main locations in Gaithersburg,
Director, NIST Maryland, and Boulder, Colorado.

Q. NCSLI is very pleased to interview you for its metrology journal
NCSLI Measure. As Director of NIST, what is your vision for the
organization?
A. I see NIST as the national lab for industry. We are the only federal
research laboratory specifically charged with helping companies to in-
novate and improve their products. We help ensure that our U.S. man-
ufacturers and service companies have the data, standards, measure-
ment tools, and technologies they need to be competitive in a global
economy. We see this task as an interdisciplinary, public-private col-
laborative effort. Industry helps determine our research agenda and
direct collaboration with researchers from industry, academia, and
other government agencies helps ensure that our work has the greatest
possible impact for advancing industry and science.
So many of the truly difficult technology challenges and opportu-
nities today require sophisticated, complex systemsadvanced man-
ufacturing, the smart electric power grid, electronic health records,
cybersecurity, advanced manufacturing. NISTs role is to convene the
right players, agree on the needed standards, improve interoperabil-
ity, and provide the right measurement tools to ensure confidence that
these new technologies work as intended.
Q. How do you view the relationship between NIST and NCSLI, and
do you think it will change in the future?
A. NCSLI is one of NISTs key partners in identifying where mea-
surement science can enable innovation. Our two organizations have
a long and very close history, and I see that close relationship continu-
ing, and becoming more important. We rely on NCSLI to help us work
with numerous public and private-sector organizations and to help us
translate our laboratory work into meaningful use in the field. NCSLI
can help us connect with practitioners, understand their needs, and
understand what the industry drivers are. You can also help us explain
our value to our stakeholders and to build a constituency that under-
stands the great value that we provide to society.
Q. In your two years as NIST Director youve initiated a major re-
structuring and reorganization of the laboratories. Can you tell us your
reasoning behind these changes? Have these changes had a positive
impact on NISTs performance? Have they accomplished your goals?
A. Over the 20 years since our last major reorganization, our responsi-
bilities and programs had changed substantially. The number of orga-
nizational units within NIST had also grown substantially.
We needed to remap the organization to support our mission and
we needed to improve customer service, so the reorganization was
about getting the organization better aligned. Weve moved away
from an activity-based structure for the agencys laboratories toward
a mission approach. The realignment is helping us improve delivery
of our services and strengthen the connections between our research
programs and our congressionally assigned mission goals.
The end goal is a better NIST that is more nimble and able to re-
spond more quickly to our customers needs.
Q. Many NCSLI members are NIST customers and depend on the
calibration services provided by NIST. Can customers expect any
changes to the NIST calibration program, and if so, what might those
changes be?
A. Yes, hopefully they will be improvements. All NIST measurement
service programs continually strive to evolve and improve to better
meet the demands of our customers. This evolution is often reflected
in new services, reduced uncertainties, expanded parameter spaces,
or reduced calibration times. Sometimes the changing needs of our
customers will result in the termination of unnecessary or underuti-
lized services, but this is always done with significant advanced notice

24 | NCSLI Measure www.ncsli.org


NIST receives thousands of gage blocks, highly accurate short length measuring tools, every year for calibration against
NIST masters from manufacturers and government agencies around the world. (Barry Gardner/NIST)
and by seeking out other alternatives. This continual evaluation and
technical improvement of measurement services has been a constant
at NIST for some time, and will continue.
The recent reorganization of NIST has for the first time put the
responsibility for calibration services within a single organizational
unit, our Physical Measurement Laboratory. With this consolidation,
we have the opportunity to greatly improve the customer interaction
and administrative components of NISTs calibration programs. Cus-
tomers can expect to see more informative and functional web pages,
development of a single point of contact for questions and guidance,
and easier access to NIST technical staff.
Q. NIST was the beneficiary of a substantial amount of American Re-
covery and Reinvestment Act funding. Can you talk about some of the
major projects this money funded?
A. The American Recovery and Reinvestment Act funding we re-
ceived allowed us to undertake some much-needed upgrades to our
existing facilities, and to build new facilities that will allow us to im-
prove our measurement science in several areas.
The act provided $360 million in direct appropriations for con-
struction of research facilities, half for NIST projects and half for a
competitive grant program for science research facilities outside of
NIST. At our Boulder campus, the funding is supporting completion
of the new Precision Measurement Laboratory, set to open in April of
this year. The lab will enhance research productivity and enable the
sophisticated measurements needed by U.S. industry and the scientific
community in national priority areas.
In Gaithersburg weve been able to upgrade several facilities and
Vol. 7 No. 1 March 2012 NCSLI Measure |
SPECIAL FEATURE
build new ones. Were adding the 22,000-square-foot (2,040-square-
meter) National Fire Research Laboratory, which will provide a unique
capability for testing full-scale structural elements, subassemblies,
and systems under realistic fire conditions. The Net-Zero Energy Resi-
dential Test Facility, which resembles a typical suburban Maryland
home, will enable the development and demonstration of the measure-
ment science needed to achieve a house that can produce as much
energy from renewable resources as it consumes over the course of a
year. The 7,500-square-foot (697-square-meter) Robotics Test Facility
will be a test bed for robot performance metrics. A 20,000-square-
foot (1,860-square-meter) addition to an existing building will pro-
vide production, packaging and storage space for the Standard Refer-
ence Materials Program. And a Grid-Connected Photovoltaic System
will generate up to 600 kW of electricity that will offset a portion of
NISTs electrical power needs.
The act provided $220 million in direct appropriations for NIST
laboratory research, competitive grants, research fellowships, and ad-
vanced research and measurement equipment and supplies. With this
funding we were able to increase positions for early-career scientists
through the competitive Postdoctoral Research Associates Program,
which is administered in cooperation with the National Academy of
Sciences/National Research Council. These young researchers began
two-year tenures at NIST in 2010. The funding also allowed us to
extend the appointments of a number of postdocs nearing the end of
their terms for 17 months.
We also received funding transferred from other federal agencies
that included $20 million for standards-related work on electronic
medical records to assist in modernizing the health care system and
$10 million for collaborative efforts to develop a comprehensive
25
SPECIAL FEATURE

The Multi-Axis Crystal Spectrometer (MACS) at the NIST
Center for Neutron Research uses an intense incident
beam of cold neutrons and a multichannel detection
system to provide unprecedented access to atomic scale
structural and dynamic information in a wide range of
materials. NIST and Johns Hopkins scientists use MACS
to understand the fundamental physics of novel magnet-
ic and superconducting materials with unique properties.
( Robert Rathe (licensed for any NIST purpose))
Artists conception of the first extreme ultraviolet (EUV)
frequency comb, which creates a clear series of evenly-
spaced spectral lines. Laser-generated frequency combs
are the most accurate method available for precisely mea-
suring frequencies, or colors, of light. The new tool can
aid in the development of nuclear clocks based on ticks
in the nuclei of atoms, and measurements of previously
unexplored behavior in atoms and molecules. The EUV
comb was created at JILA, a joint institute of NIST and
the University of Colorado, Boulder. (Brad Baxley/JILA)

framework for a nationwide, fully interoper-
able smart grid for the U.S. electric power
system.
Q. Your position as Director of NIST was
recently elevated to the Undersecretary of
Commerce for Standards and Technology.
Can you tell us what the implications of this
change are?
A. This change raises the profile of the po-
sition to one of the key advisors to the Sec-
retary. It signifies that the director of NIST
also has a larger role to play in helping the
Department and the Administration to shape
technology, innovation, and manufacturing
policy.
Q. As Undersecretary of Commerce for Stan-
dards and Technology, what changes do you
anticipate for the role standards play in the
measurement community?
A. Standards really are the bedrock that our
entire modern society is built on. In a global
economy you can no longer operate success-
fully as a technology island. Technologies
must work together and must be able to prove
their reliability and effectiveness by meeting
agreed upon standards. This raises the profile
and the impact of the work being done by
the measurement community. This work has
always been important. It is now also recog-
nized by policy leaders as critical to econom-
ic growth, innovation, and competitiveness.
Q. As budget debates continue in Washing-
ton, D. C., how is NIST positioned to handle
the future budget uncertainty? Have there
been any impacts to NIST programs at this
point?
A. I believe our recent realignment has bet-
ter positioned us to handle the future. And the
NIST laboratory programs have been fortu-
nate in that the president and Congress recog-
nize the value of our work and our most recent
budgets have reflected that. Like all agencies,
we are evaluating our costs and finding places
where we can reduce spending and be a more
efficient and focused organization.
Two of our programs did not receive fund-
ing in our fiscal year 2012 appropriations.
The Technology Innovation Program is in the
process of an orderly shutdown. The Baldrige
Performance Excellence program is work-
ing with the Baldrige Foundation to continue
its operations at NIST with private and fee-
based funding. The lack of federal appropria-
tions means that some Baldrige services for
helping organizations improve their opera-
tions and competitiveness will be curtailed
and that some services will now require pay-
ment of fees.
The Obama Administrations recently re-
leased fiscal year 2013 budget request would
increase our budget to $857 million, $106.2
million more than our 2012 enacted budget. It
would allow us to continue our current efforts
and to focus on a number of initiatives, in-
cluding advanced manufacturing, cybersecu-
rity, forensic science, disaster resilience, and
advanced communications networks.
Q. NIST user facilities are available for col-
laborative joint research and outside use by
industry and university researchers. Talk
about the success of these programs and the
benefits obtained.
A. We have two user facilities available to re-
searchers from industrylarge and small, ac-
ademia and other government agencies. The
NIST Center for Neutron Research (NCNR),
which is where I began my career at NIST,
supports a wide range of neutron-based re-
search tools, from activation analysis used for

26 | NCSLI Measure www.ncsli.org

 SPECIAL FEATURE

Developed by international shipper FedEx and tested with help from NIST, the SenseAware device connects to cell
phone networks to provide users with near real-time information on a packages precise location, temperature, humidity,
pressure, acceleration, elevation and exposure to light. NIST researchers plan to deploy the technology as part of a pilot
project to monitor and improve climate measurements.

detecting trace levels of chemicals in the en-
vironment to low-energy cold neutron scat-
tering for studying the structures of advanced
materials like proteins or superconductors.
The NCNR currently has 25 experiment
stations and each year more than 2000 re-
search participants from all areas of the coun-
try are served by the facility. Users who in-
tend to publish their research do not pay for
beam time, but full operating costs are recov-
ered for proprietary research. Twice a year, an
independent beam time allocation committee
peer-reviews proposals and chooses the up-
coming projects.
The NCNR is preparing to resume opera-
tions after an 11-month outage to complete a
major upgrade that will increase the centers
cold neutron measurement capability by more
than 25 percent.
The Center for Nanoscale Science and
Technology (CNST) was established in May
of 2007 to accelerate innovation in nanotech-
nology-based commerce. It is a resource for
nanoscale fabrication and measurement and
for developing innovative nanoscale mea-
surement and fabrication capabilities. The
CNSTs NanoFab facility provides access
to more than 90 different specialized instru-
ments and other resources that companies
dont typically have available. It can make
a real difference for a small company testing
out a new idea, or even an established large
company that would like to speed up a devel-
opment process.
Users can gain access to the facility in as
little as two weeks, thanks to an online appli-
cation system, and we continue to attract new
users working on advanced research projects
in polymers, biology, nanoelectromechanical
systems, ceramics, radiation physics, atomic
physics, optics, and more.
Q. NIST has an active technology transfer
program. Can you provide an example where
NIST has successfully transferred a metrology
related product or service to U. S. industry?
A. Its difficult to pick one example, so I will
tell you about a few recent ones. There are
now chip-scale atomic clocks on the mar-
ket that were inspired by NISTs pioneer-
ing inventions in that field. We developed
the first phantom for calibrating magnetic
resonance imaging (MRI) machines that are
traceable to standardized values, and in 2011
awarded a grant to develop manufacturing
methods for producing them at low cost.
We worked with a pair of industry collabo-
rators to develop a now commercially avail-
able device able to transmit highly accurate
real-time information on a packages precise
location, temperature, humidity, pressure, ac-
celeration, elevation and exposure to light.
We alerted the semiconductor industry to a
potentially serious problem that could have
upset the transition to next-generation chip-
making equipment and devised methods to
overcome the problem. This smoothed the
way for timely introduction of the advanced
optical tools needed to produce integrated cir-
cuits with nanoscale features. And of course,
theres our time keeping. NIST official time
is used more than 11 billion times per day to
time-stamp hundreds of billions of dollars
worth of financial transactions every business
day and to synchronize telecommunications
and computer networks and radio-controlled
clocks. The number of requests increases by
about five percent every month. Each day
our researchers are working with industry to
translate our measurement science into inno-
vative new products or processes.

Vol. 7 No. 1 March 2012 NCSLI Measure | 27

SPECIAL FEATURE

fundamental metrology necessary for the de-

velopment of quantum computing, and more.
In order to facilitate these interactions
so that NIST can effectively respond to the
metrology needs of our sister agencies, we
have established a Reimbursable Agreements
Coordination Office (RACO) to improve and
streamline our internal procedures for accept-
ing reimbursable funding. This office has de-
veloped the guidance to assist in meeting the
legal requirements for the transfer of funds,
and is assisting NIST staff and other agency
sponsors in efficiently meeting these require-
ments.

Q. Is there something else new or exciting

happening at NIST that you want to commu-
Colorized micrograph of multi-walled carbon nanotubes, each 40 microm- nicate with the members of NCSLI?
eters long, which absorb more than 99.9 percent of the light inside NISTs
prototype fiber-coupled radiometer, a device capable of absolute measure-
ments of optical power delivered through an optical fiber. A. This is an exciting time for NIST. With
(Huang/NanoLab, colorized by Talbott/NIST) our long history of working with industry and
other types of research partners and accel-
erating standards development, weve been
asked to take leadership roles in several ar-
eas of national importance, such as advanced
manufacturing. Weve also seen an increased
demand for measurement services. In Janu-
ary, the White House issued a memo that lays
out principles for federal engagement in stan-
dards activities that address national priori-
ties. It highlights NISTs expertise in this area
and statutory authority to coordinate confor-
mity assessment activities of federal, state
and local governments, and the private sector.
Were looking forward to taking on these
new challenges, and continuing to evolve
our existing programs to make sure we are
meeting the needs of our customers and other
stakeholders. We value our partnership with
NIST performs comprehensive investigations of disasters that result in struc- NCLSI and hope we can continue to rely on
tural failure and loss of life such as the tornado that struck Joplin, Missouri
your help to engage the measurement com-
on May 22, 2011. Using data collected on the characteristics of the cause,
the performance of structures and lifelines affected, the performance of and munity and ensure our programs are aligned
response to emergency communications, and injuries and deaths, NIST will with its needs. Together we can keep mea-
make recommendations, if warranted, for improvements to building codes, surement science strong, and in doing so help
standards and practices. (NASAs Terra satellite)
ensure that U.S. industry is a strong competi-
tor in the global marketplace.
Q. About 20 percent of NISTs available Institutes of Health, the Department of En-
funding for 2012 comes from other agency ergy, the Department of Defense, and others.
funding. Can you talk about the type of work In many cases, these relationships or agree-
that is funded by other agencies and how the ments involve other federal agencies fund-
new Reimbursable Agreements Coordination ing NIST to perform research that addresses Dr. Patrick Gallagher
Undersecretary of Commerce for
Office will affect this funding? their unique measurement challenges. These
Standards and Technology
programs cover a vast array of technical ar-
National Institute of Standards and
A. NIST has relationships with many govern- eas including calibration of satellite-based Technology
ment agencies, including the Department of sensors to monitor global climate change, 100 Bureau Drive, MS 1000
Homeland Security, National Institute of Jus- development of new calibration techniques Gaithersburg, MD 20899, USA
tice, Health and Human Services, National for unique instrumentation, advancement of patrick.gallagher@nist.gov

28 | NCSLI Measure www.ncsli.org

OHMLABSsmartresistorAD.12.11_Layout 1 12/21/11 10:48 AM Page 1

S MARTR ESISTOR

A RESISTANCE LAB IN A BOX FULLY PORTABLE
EIGHT STANDARDS, CONSTANT TEMPERATURE BATH
& PRECISION THERMOMETER
CONTAINS LOW THERMAL EMF SCANNER
REAL-TIME CORRECTIONS FOR TEMPERATURE & DRIFT
FULLY AUTOMATED VIA ETHERNET OR GPIB
LOWER COST, MORE COMPACT THAN
SEPARATE COMPONENTS
ACCREDITED CALIBRATION INCLUDED

SEE WWW.SMARTRESISTOR.COM FOR DETAILS

611 E. CARSON ST. PITTSBURGH, PA 15203

TEL 412-431-0640 FAX 412-431-0649
WWW.OHM-LABS.COM

Vol. 7 No. 1 March 2012 NCSLI Measure | 29

TECHNICAL NOTE
Calculation of Dual Circle Closure
Calibrations using Computer
Spreadsheets
Hy D. Tran
Abstract: This technical note will detail a procedure for simultaneous calibration of two rotary devices by circle closure, along
with the necessary calculations. These calculations can be implemented using numerical analysis software, or with computer
spreadsheets. We first detail an example procedure for a dual closure for two rotary tables with six sectors, and then provide a
detailed derivation of the mathematics.
1. Introduction
The calibration of angular devices, such as rotary tables, is a class of
calibration where circle closure may be used to help validate data [1].
Circle closure is a well known technique and a type of self-calibration
[2], but can be tedious to implement.
In 1998, Estler published an analysis of uncertainty for calibration
of indexing rotary tables by circle closure [3]. In general, a circle
closure angle calibration divides a circle into n angular segments
(where n is an integer). The sum of the n angles has to equal 360 (or
2 radians). This property can be used to calibrate the angle device.
We can illustrate a simple example of circle closure by showing how
to calibrate a single 60 angle, such as a protractor as shown in Fig. 1
(adapted from Moore [1]). By indexing the protractor six times around
a circle, we should get 360 because 60 is a 1/6 sector of a circle.
Note that 360 has the same orientation as 0, and that any deviation
from returning to the cardinal zero will be six times the error of the unit
under test (UUT). These basic principles are easy to understand, but
the actual mechanical implementation is more difficult.
When we calibrate an indexing rotary table (such as the Moore
1440 indexing table*) at Sandia National Laboratories, we use two
indexing tables and an autocollimator. This process is called dual
closure in Estler [3]. Through the use of indexing rotary tables, we
can position objects at discrete angles. Figure 2 shows Sandias setup
for performing a dual closure calibration for two Moore 1440 indexing
rotary tables.
The remainder of this paper will detail a procedure for simultaneous
calibration of two rotary devices by circle closure, along with the
necessary calculations, drawing extensively from the work of Estler
[3]. The calculations can be implemented using numerical analysis
tools, such as MATLAB [4], or computer spreadsheets, such as
Microsoft Excel. We begin by detailing an example procedure for a
dual closure for two rotary tables with six sectors. We then discuss
validation of the method, and provide a detailed derivation of the
mathematics.
30 | NCSLI Measure
2. Procedure
A rotary table has degree markings. Calibration of the rotary table
establishes the error of the angular markings. Specifically, the Moore
1440 rotary indexing table has 1,440 positions in degree increments:
0, 0.25, 0.50, 0.75, , 359.50, 359.75. Because of the constraint
that the circle has 360, the sum of all the errors at each individual
degree setting is zero. When performing a calibration of an indexing
rotary table, we do not calibrate each individual angle setting. For the
Moore 1440 indexing tables, we typically calibrate at 30 intervals: 0
to 30, 30 to 60, 60 to 90, 330 to 360. We have found, based
on historical calibration data, that a sampling of 12 intervals provides
sufficient data to check the performance of this type of indexing rotary
table. To reduce mathematical complexity, the procedure illustrated
here is for six intervals (sectors).
In the dual closure procedure, two rotary tables are stacked on
top of each other. One table is rotated clockwise, the other is rotated
counterclockwise. The sketch in Fig. 3 shows the setup for the
measurements. For this example, each rotary table is divided into six
sectors. We number the sectors from 1 through 6 going clockwise from
0. If the rotary table markings also proceed clockwise, then, sector 1
represents the sector from angle 0 to 60. Note that if desired, we can
also apply this procedure to optical polygons, although the spreadsheet
matrices will be different. This is because an optical polygon has
defined angles from one side to the next side, and the measurement
equations (which are derived in Section 4) will be slightly different.
The two rotary tables are stacked and aligned in an open setup,
using test indicators to align the rotary axes of the rotary tables. In
the open setup, we also align the autocollimator axis to the rotary
axes of the rotary table. Misalignment of the rotary tables to each
other would introduce non-repeatability, as the autocollimator would
*
Certain commercial equipment, instruments, or materials are identified
in this paper in order to adequately describe the procedure. Such
identification does not imply recommendation or endorsements by the
authors, Sandia National Laboratories, or NCSL International, nor does
it imply that the materials or equipment identified are the only or best
available for the purpose.
Actually, the Moore 1440 uses a vernier scale with markings at 0, , , and
degrees for reading the degree marks.
www.ncsli.org
 TECHNICAL NOTE

Figure 1. In this illustration, we wish to calibrate the angle
q , which is nominally 60. We step the angle six times
around a circle (a through f). The deviation a is necessar-
ily six times the deviation of the angle q from the nominal.
Figure 2. Sandias setup for a dual closure calibration of
two Moore 1440 indexing tables. In general, it is necessary
to have an autocollimator and two indexing tables. The fan
in the back helps stabilize readings with the autocollimator.
When performing a dual closure calibration, both indexing
tables are always calibrated, even if the customer only sub-
mits one indexing table for calibration.

Figure 3. Illustration of measurement setup. Two rotary
tables are stacked and aligned, and the autocollimator is
also aligned.
Figure 4. Taking the first measurement: Rotate the bottom
table through one sector (sector B1), and rotate the top
table through one sector (the last sector, which is T6). The
first measurement point is B1\T6. Continue making mea-
surements and recording data; second measurement point
is B2\T5, etc. Do not rezero the autocollimator between
readings.

potentially hit different parts of the return axis would introduce cross-coupling errors of
mirror; misalignment of the autocollimator x-angle (vertical axis) and y-angle (horizontal
axis). The rotary tables are also clocked so
that their zero angle readings are aligned. We
Author mount a mirror for an autocollimator on the top
rotary table, and align the autocollimator such
Hy D. Tran
that the zero reading for the autocollimator is
Sandia National Laboratories
aligned with the zero angle reading on the top
MS-0665, PO Box 5800
Albuquerque, NM 87185
rotary table.
hdtran@sandia.gov We adopt the convention that looking
from above, a clockwise rotation of the
mirror yields a positive reading on the
autocollimator. We have found that forced
convection in the laboratory (using a fan)
helps the stability of the autocollimator
readings, and also decreases the effect of the
operators body temperature on the setup and
measurements [5].
For the dual closure, a total of n2 (where
n is the number of sectors to be calibrated
on the rotary table) measurements are made.
The first n measurements are made with the

Vol. 7 No. 1 March 2012 NCSLI Measure | 31

TECHNICAL NOTE
Figure 5. Reposition and fixture the autocollimator mirror
on the top rotary table to align with the start of T2. Then,
rotate the top rotary table until you get a nominally zero
reading on the autocollimator. Zero the autocollimator.
Figure 7. Sample worksheet entry into MS-Excel for dual closure calibration of six sectors on two rotary tables.
autocollimator mirror fixtured to align with the zero position (left side
of T1 per the sketch in Fig. 3). The first measurement is taken by
rotating the bottom table counterclockwise through one sector (sector
B1), and the top table through one sector clockwise (sector T6). The
autocollimator measurement is then recorded on a worksheet. The
autocollimator is not rezeroed. The metrologist then rotates the
bottom table one sector counterclockwise, and the top table one sector
clockwise, and records the next reading (see Fig. 4). The metrologist
continues this procedure for n readings. The nth reading is made
back at the starting position, and should read zero. It will typically
not read zero, due to drift and noise, but if the reading is acceptably
close to zero (we use the nominal specification for repeatability of the
indexing table as an acceptable drift), we accept the measurements
and continue. Otherwise, the measurements are repeated until the
acceptance criteria are met.
We then refixture the autocollimator mirror clockwise by one sector
on the top rotary table so that it is now aligned with the beginning of
sector T2, as shown in Fig. 5. The top rotary table is rotated until sector
32 | NCSLI Measure
Figure 6. Obtain the next series of measurements by mak-
ing counterclockwise rotations of the bottom table and simi-
lar clockwise rotations of the top rotary table, similar to the
illustration in Fig. 4. In this illustration, the mirror is aligned
to T2. The first measurement is B1\T1; the second measure-
ment is B2\T6; and so on.
T2 points to the autocollimator, and the autocollimator is zeroed.
The metrologist then makes the next series of n measurements
and records each series in a similar fashion. For each series of
measurements, the bottom table is rotated through one sector
counterclockwise, and the top table is rotated through one sector
clockwise as illustrated in Fig. 6.
Again, the autocollimator mirror is refixtured clockwise by one
sector on the top rotary table, and more measurements are taken, until
n measurements have been taken for n mirror fixturings.
The data are recorded on a worksheet and transcribed to a
spreadsheet, or directly into a spreadsheet as shown in Fig. 7. Then,
the spreadsheet calculates the individual deviations of each sector
from nominal using a least squares fit.
If you use an autocollimator with a data interface, you could have the data
entered into the worksheet for you automatically, and avoid transcribing
data.
Derivation and details are in the derivation section of this technical note.
www.ncsli.org
 TECHNICAL NOTE

Drift correction is performed assuming linear drift during one se-

ries of n measurements. The first entry in a row is the zero entry when
Solution Autocolli- the autocollimator is zeroed. The last entry in a row is after each
A-98765 Operator:
Vector mator: rotary table has been rotated through 360 (counterclockwise for the
bottom table; clockwise for the top table). Therefore, a non-zero read-
Dates data
1 14.787 ing is attributable to drift during the calibration process. Looking at
taken:
the row marked 0/T1 in Fig. 7, the first data entry is the zero reading
2 -5.013 from the autocollimator. The last entry (B6\T1) is 5.8. This indicates
3 -4.726 R-23456 arc-sec arc-sec a drift in the readings from 0 to 5.8. The individual entries between
Solution 0/T1 and B6\T1 are corrected by removing the linear interpolation
4 9.802 Sector Cumulative
(top) straight line from 0 to 5.8.
5 -5.089 T1 0.0981 0.0981 The next tab in the spreadsheet performs the least square calcula-
tion and estimates uncertainty from a quadrature sum of the variance
6 -9.762 T2 -0.0487 0.0494
from the least squares fit (Type A), the drift (Type A), and the autocol-
1 9.806 T3 -0.0976 -0.0482
limator calibration uncertainty (Type B). The Type A terms are a less
2 -4.868 T4 0.0996 0.0514 rigorous approach than Estler [3], in that an analysis of variance is not
3 -9.760 T5 0.0485 0.0999 performed, and therefore, potentially correlated terms are treated as
4 9.962 T6 -0.0999 0.0000 independent. Uncertainties associated with misalignment of the ro-
Sectors are numbered clockwise from tary axes of the tables, and the non-parallelism with the autocollimator
5 4.850
0 degrees axis (Type B) have not been evaluated. A sample calculation results
tab is shown in Fig. 8.
6 -9.990
R-12345 arc-sec arc-sec 3. Validation
Solution
Sector Cumulative To validate the spreadsheets being used, we created two artificial data
(bottom) sets for two rotary tables. The measurement process was simulated,
B1 0.1479 0.1479 and the simulated results were entered into the spreadsheet. The
B2 -0.0501 0.0977 calculation correctly identified the artificial data sets. A second step
of validation was performed by entering data collected from previous
B3 -0.0473 0.0505
calibrations of rotary tables and previous calibration certificates. The
B4 0.0980 0.1485
resulting data were compared, and the spreadsheet gave the same
B5 -0.0509 0.0976 results.
B6 -0.0976 0.0000
Sectors are numbered clockwise from 4. Derivation
0 degrees For the derivation, we repeat the procedure, along with the necessary
mathematics. The autocollimator is zeroed with the rotary tables at
zero. Then, the bottom rotary table is rotated counterclockwise by one
other notes: 4. Derivation
4. Derivation
sector, and the top rotary table is rotated clockwise by one sector, as
4. Derivation
For the
the derivation,
derivation, we repeat the the Since
procedure, along with with the the necessary mathematics. The
For illustratedwe repeat
in Figure 4. procedure, the bottom along table necessary
is rotated by an mathematics.
angle The
For the derivation,
autocollimator
autocollimator is
is we repeat
zeroed
zeroed with
with the
the
the procedure,
rotary
rotary tables
tables alongat
at with the
zero.
zero. Then,
Then, necessary
the
the mathematics.
bottom
bottom rotary
rotary The
table
table is
is
arc-sec B1, this is a counterclockwise
autocollimator
rotated counterclockwise
rotated is zeroed
counterclockwise with
by one
by one rotary rotation,
the sector,
sector, tables
and the
and theattop
which
zero.
top rotary
rotary
is a negative
Then, table
table theis isbottom
reading. table is
rotatedrotary
rotated clockwise by
clockwise by
stdev of fit rotated The top table is by
counterclockwise rotated by an angle
one sector, and the T6.top
the The first table
rotary rotation of the top
is rotated
0.0027 one sector,
one sector, as illustrated
as illustrated in in Figure
Figure 4. Since
4. Since the bottom
bottom table is
table is rotated
rotated by clockwise
by an angle
an angle B1, by
B1,
u of Elcomat 0.007 one
this sector,
is a as
table illustrated
counterclockwise
is the last in Figure
rotation,
sector of 4.
which
that Since
table. is the
a bottom
negative
The
this is a counterclockwise rotation, which is a negative reading. The top table is rotated table
reading.
autocollimator is rotated
The
readingtopby an
table
will angle
is B1,
rotated
this
by an
by is angle
an aangle
counterclockwise
be B1 T6. +The
T6. The Forrotation,
T6.first
first the nominal
rotation
rotation which
of the
of the istop
condition,
topa negative
tableB1
table is reading.
is=theB2last
the = B3
last The= top
sector
sector T1 =table
of that
of that is rotated
T2table.
table. The
The
u_drift 0.0444 by an angle T6. The first rotation of the top table is the last sector of that table. The
autocollimator
autocollimator = T3 reading
reading will
The calibration be
will be B1willB1 + T6.
+ T6.giveFor For
us the the nominal
the deviations
nominal condition,condition,
from nominal. B1
B1 = B2 = B2
If == B3B3 ==
rss u (k=1) 0.0450558 autocollimator
T1 == T2
T1 T2 ==the T3
T3 readingThe
The will be
calibration
calibration B1 +
will
will T6.
give
give For us
us the
the
the
actual angle of sector B1 (for this example) is greater than 60 by nominal
deviations
deviations condition,
from
from B1
nominal.
nominal. = B2If
If =the
the B3 actual
actual=
T1 = T2 =sector
T3B1 The calibration will give us thethan deviations from nominal. If the actual
Figure 8. Sample results from a calibration. Note that
angle
angle
angle
of sector
of
of b ,
sector and B1
B1
(for
(for
the(foractualthisangle
this
this
example)
example)
example)of sector is greater
is
is
greater
T6 is greater
greater
than
than
60
60
thanby
60
by60bbb11,,by
by , and
and
and
the actual
the
t6, then,
the
actual
the angle
actual
angle
angle of
of
of
sector T6
sector T6 actualis greater
is greater
1
than
than 60
60 by
by tt6 , then, the actual resulting measurement
6, then, the actual resulting measurement m11 will be: 1 m will be:
the spreadsheet cell formats report more digits than are sector T6 is greater than 60
resulting by t6, then,mthe
measurement 1
will be resulting measurement m1 will be:
actual
significant.
mm1 bb1 tt6 (1) (1)
(1)
m11 b11 t66 . (1)
Specifically, the spreadsheet:
Calculates and corrects drift during each series of measurements We do We not
do do not re-zero
re-zero thethe autocollimator.We
autocollimator. Wethen then continue
continue by by taking
taking aa second second
We not re-zero the autocollimator. We then continue by taking
measurementWea second
measurement doby
by notrotating
re-zerothe
rotating
measurement thebottom
the autocollimator.
bottom
by rotatingtable counterclockwise
table We then
counterclockwise
the bottom tablecontinue
anotherbyincrement
counterclockwise
another taking a and
increment second
and the
the
for a particular orientation of the autocollimator mirror measurement
top rotary
top rotaryanother
table
table byclockwise
rotating
clockwise the
byand
by bottom
thethe
the same
same table counterclockwise
increment.
increment. Because
Because another
we
we do
do
by notnotincrement
re-zero,
re-zero,
same the andnext
the the
next
Calculates the least square fit for the deviations of each top
increment top rotary table clockwise the
rotaryincrement.
sector
measurement
measurement tablemm22clockwise
is: Because
is: by wethe do samenot increment.
re-zero, the Because next measurement we do notmre-zero, is the next
measurement m2 is: m m b t
2
(2)
for each rotary table based on the measurements m m b t (2)
Calculates an estimate for type A uncertainty [6] by calculating
2 1 2 5
m22 m11 b22 t55 . (2) (2)
the variance of the actual measurements from the measurements In In general,
general, for nn sectors,
for
In general, sectors, the measurement
the measurement at the kkthth rotation
at the rotation
kth rotation is:
is:
In general, for n for n sectors,
sectors, the measurement
the measurement at theatkththe rotation is: is
predicted by the least squares fit kk kk
mk
m k b bii k tt
(3) (3)
(3)
mkk ii11 bi
nnii .
ii11 tn i (3)
i 1 i 1

We wish
We wish to
to find
find the
the deviations
deviations for
for each sector, tt11,, tt22,, ,
each sector, , ttnn and
and bb11,, bb22,, ,
, bbnn.. We
We
Vol. 7 No. 1 March 2012 NCSLI
t1, t2, Measure
, | 33 b
know Wewe
know that
that wish
we tohave
will
will find more
have the
more deviations for each
measurements
measurements sector,
than
than unknowns,
unknowns, so aatnleast
so least bsquare
1, b2, ,
and square n. We
calculation
calculation
know
[7,8] thatprovide
[7,8] will
will we willaahave
provide more measurements
reasonable
reasonable estimate for
estimate than
for the
the unknowns,
unknowns
unknowns tti and so babileast
and square calculation
1n).
(i(i=1n).
 In general, for n sectors, the measurement at the kth rotation is:

TECHNICAL NOTE k k
mk bi tn i (3)
i 1 i 1
We wish to find the deviations for each sector, t1, t2, , tn and distinct measurements for each mirror alignment. We also have n
We wish b1, to b2,find
, bthe . deviations
We know that for each we will sector, have t1, more
t2, ,measurements
tn and b1, b2, , than bn. We possible mirror alignments. Therefore, we are making n2 - n distinct
n
now that we unknowns,
will have more measurements than unknowns, so a least square calculation
so a least square calculation [7,8] will provide a reasonable measurements (not counting either the initial zeroing or the nth
7,8] will provide a reasonable estimate for the unknowns ti and bi (i=1n).
estimate for the unknowns ti and bi (i=1n). measurement at each mirror alignment at return to zero), and have to

We form We form xan(column
an array array x (column vector) containing
vector) containing the unknowns: the unknowns solve for 2n unknowns in x .
From inspecting equations (5) and (7), each measurement m is just
From inspecting a differentequations linear combination (5) and (7), of theeachelements measurement of the unknownm is just vector a different
b1 linear combination of the elements of the unknown vector x . If we assemble the
From inspecting x . If we equations assemble the (5) measurements
and (7), each m in a columnmvector
measurement is justy ,athen, different
measurements
linear combination mwe i in aof column
the vector
elements y ,
of then,the we can
unknown write
i
vector x . If we assemble the
can write
bn measurements m i in a column vector y , then, we can
write
, From
(4) From inspecting
inspecting equations
equations (5)y(5) A
and andx(7),,,(7), each each measurement
measurement m is m justis just a(8) (8)
different
a different
x (4)
linearlinear combination
combination of of the the elements
elements yof of Ax the
the unknown
, unknown vectorvector x . x .If we If we assemble
assemble (8)the the
t where the matrixmiAin represents thevector
linear
1 measurements
measurements m iain column
a column vector y combinations
y , then, , then,we we cancan write resulting
write in the measurements.
where the matrix A represents the linear combinations resulting in
The
where matrixthe matrix A is not square, as the
Ameasurements.
represents we linear
have more combinationsmeasurements thaninwe thehave unknowns.
t The least squares theestimate [9] as forwe Thebematrix
x can found Aby not resulting
iscalculating: square, as we measurements.
have more
n The matrix A is not square, have yAxmeasurements
ymore A x than
, , The least squares estimate we have unknowns.(8) (8)
measurements than we have unknowns.
The least squares estimate [9] for x can be found by calculating:

1
[9] for x can be foundx by calculating
A T
A A T
y (9)
Where bi is the deviation of the ith sector on the bottom, and t is where
the where the
deviation matrix
the matrix
of A represents
A represents the linear
the linear combinations
combinations resulting
resulting in thein the measurements.
measurements.
where bi is the deviation of the ith sector on the bottom,i and tThe is the
A estimate

1
th
he i sector on the top. For ourth example where n = 6, b2 would be the deviation The i matrixmatrix
of
While A
the A
sector is not
solution square,
is not of square, as
the least we
asx have
we T more
Ahave
squares more measurements
A T y measurements
.is thanthan
straightforward, we we have
there haveunknowns.
are(9)unknowns.
(9)
a few
deviation of the i sector on the top. For our example where The
importantn The=least
6, least2squares
bdetails squaresto estimate
estimate
consider: [9] [9]
for for x can x be found
can be found by calculating:
by calculating:
would be the deviation of sector B2 (60 to 120) of the bottom table While the solution While the of solution
the leastofsquares the least estimate
squaresis estimate straightforward, there are a few
is straightforward,
B2 (60 to 120) of the bottom table from a nominal 60, and t1 wouldimportant be the deviation details of to consider:
x A Amto Ay be
1 T
B2 (60
sector T1 (0tofrom
120)
to 60) a of ofthe
thebottom
nominal top60,tabletable
andfrom t1from
would a nominal
a nominal be the60. 60, and of
deviation t1 would
sector T1 be
the (0 deviation
Numerical
to there 11values ofare aoffew theimportant
measurements x details T T
iA should
A
1
consider: T
y scaled to minimize round off(9) (9)
sector T1 (0 60)toof 60)
theof topthe top from
table table afrom nominal a nominal 60. 60. errors Numerical in computation
values of the measurements m should be scaled to minimize round off
While While the the solution
solution of theof the least leastsquares
squares estimate estimate is straightforward,
is straightforward, there are are
there a fewa few

i
B2 (60
Thetofirst
The
120) fewof
The thefew
first bottom
measurements table
measurements from(Equations
(Equations a nominal
(1) and (1) 60,and
(2)) and
are (2)) t1 are
then would then
simply: be
importantthe
simply: deviation
errors
Formulating
important in
details of
computation
details toA and
Numerical
consider:
to calculating
consider: values A
of A
T the1measurements
A T
must be donem carefully,
should be and
scaled one must
to
(0first few measurements
of the top table(Equations (1) and (2)) are then simply:
i
sector T1 to 60) from a nominal 60.
Tand A isin
1
that AAT A
Formulating minimize
and round offAerrors Tcomputation.
Awell-conditioned.
check iscalculating must be done carefully, and one must
m1 1 0 0 0 0 0 | 0 0 0 0 0 1 x (5) Numerical
Numerical values values
Formulating
both full rank
of the
of measurements
the
A measurements
and calculating m i should
m(A A)be
i should
T -1
scaled
AbeT scaled
must to be minimize
to done
minimize round
care- roundoff off
The first few 1 0 0 (Equations
m1measurements 0 0 0 (1) | 0 and 0 (2)) 0 are 0 then0 1simply: x check
To that
formulate (5) T
Afully,
theA Aisand both
matrix, full rank
note thatand it is
must well-conditioned.
have 2n columns (recall that n is the
m2 1 1 0 0 0 0 | 0 0 0 0 1 1 x errors errors in computation
in
(5) computation we must check that (A A) is both full rank T
and is well-
m2 1 1 0 0 0 0 | 0 0 0 0 1 1number x Toof sectors the thatA we are note calibrating; 1 we 1 are calibrating two tables; each
m1 1 0 0 0 0 0 | 0 0 0 0 0 1measurement
numberx of
formulate
Formulating
Formulating
m
sectorsi is a A and A matrix,
conditioned.
linear
that we
andcalculating
calculating
combination
are

that
of
calibrating;
it Tmust
A Tthe
A AA
T
deviation
we
have
AmustT

are
must2n
bethe
in
columns
done
be
calibrating
done
sectors
(recall
carefully,
carefully,
from
two
that
and
each
tables;
one
and n is
one themust
must
table).
each
Note thatNote m0 thatwas m the0 initial zeroing of the autocollimator andThe
was the initial zeroing of the autocollimatoratand m (mm 6 nfor this(5) the
a linear T Aiscombination
A TApattern both
rowsn at in Amfollow
m0 2was the
1theinitial
1bottom and
measurement
xtopattables,mn check
(mcheck that
i isthisthat is inboth equations
full rank
full ofrankand
the(5) isand
and
deviation is (7).
well-conditioned.
well-conditioned.
in the sectors from each table).
example),Note we(mare that
forat m 360
this for
example), we0 are 0 andat0360
zeroing the0of for
| 0 the
the
top 0 bottom
0 and
autocollimator
tables, 0 and 1 1the
the measurement for
6should To formulate the A matrix, note that it must have 2n columns
example), we are at 360 for the bottom and the top tables, and the The rows
measurement Toin A follow
should
formulate the
the pattern
A matrix, in equations
note that (5)
it and
must (7).
have 2nare columns (recallwe thatthatn isnthe
To formulate 1) the n isAthe matrix, noteofthethat it must have 2n columns in the(recall is the
6
return to zero. andAlso, note that because
the measurement should thesereturn are to angular
zero. sectors
Also, note on athat rotarybecause tablefirst
The which
(recall(n - that rows of number
y are sectors
actual that we
measurements calibrating; firstare mirror
return to
Notezero.
form a circle,these that
the sum Also,
m note
was that
the because
initial zeroingthese of
0of the deviations of the sectors has to sum to zero:
are
the angular sectors
autocollimator on and a rotary
number
atnumber
m n
table
(m of 6 of
for which
sectors sectors
this that that we we are arecalibrating;
calibrating; we we are arecalibrating
calibrating two two tables;
tables; eacheach
alignment with calibrating
respect to twotop
the tables;
table: y eachare measurement
mi-ais1)
the The first mi(nis a rows of thetheofactual measurements in the first mirror
form a circle, are angular sectors on a of rotary table which toform a circle, sum mi is inathe linear combination of
example), we the are sumat 360 of the fordeviations
the bottom and the the sectorstop has tables, sum
and to thezero: measurement
measurement
measurement should linear linear combination
combination of deviation
the deviation in the sectors sectors from from each eachtable).
table).
return to zero.of theAlso, deviations
note that of the sectors
because has to
n these aresum to zero:
angular sectors onalignment aThe The
rotaryrows with
rows in Ain
table respect
the follow
whichAdeviation
follow tothethe the intop
pattern thetable:
pattern in equations
sectors from each
in equations (5) (5) andtable).(7).(7).
and The rows in A follow the
form a circle, the sum of the deviations oftithe m1 inequations 1 0 (5) and 0 (7). 0 | 0 0 0 1 b1
n 0
sectors has to sum to zero: pattern
i 1 ti 0 The -first
inthe
The first
The(6) first(n 1)
m21 (n - 1(n rows
1)-0rows of
1 rows
1) y
0 of of are
0y y0are the
are| the actual
the0 actual 0 measurements
10 1 b1 in
measurements
measurements inthe first
the first mirror
first mirror
n i 1
(6) to the top table: (10)
alignment (6)
alignment mirrorwith with respect
n
bni 0 m2 respect 1to the
alignment 1 withtop
0 table: to |the0 top table: 1 1 t1
respect
i 1
tbi 00
i 1
i 1
i
mn(6) 1
1 1 1 0 | 0 1 1 1 t1 (10)
n m1 m 1 1 10 0 0 00 0| 0| 00 0 0 01 1 b1 b1
In our example, we first make sixmeasurements bi six
0 measurements by rotating the bottom table m n m1 m 1 th1
1 11 10 0
1 0 | 0 1 1
| 0| 0
1
1 11 1 that
In ourInexample,
counterclockwise our example,
through we sectors
six we make
first first
andmake six
i
measurements
rotating
1 the top table by rotating
by rotating
clockwise theWe thedo not
bottom
through table
sixwrite 2 the n measurement, as that is the closure constraint
2
(10) the two (10)(10)
sectors. The bottom
counterclockwisenext step table
through
is tocounterclockwise
six sectors
reposition the and through
mirror rotating sixthe
clockwise sectors
top
on theand
table rotating
top rotarythe
clockwise top
tables
table, through return
so that six
to after
zero
th 360 of rotation. | In
| symbolic form, we t tcould
rewrite
We do not write the n measurement, as that is the closure constraint 1

1 that the two
thesectors. The
mirrorInis table
our next
aligned step
withisthe
clockwise
example, tothrough
we reposition
firstofmake
end T1 the
six sectors.
(orsix mirror
beginning The clockwise
measurements next stepon
of T2), by
asisthe topthese
to reposition
rotating
shown rotary
inthe Fig. table,
measurements
the
bottom
5. return sotable
Then, thatas:
rotary tables mto nm1zero
n 1
1 11360
after 1 ofrotation. 1 10 0| 0In 1 11 1 form,
| 0symbolic 1 1we could rewrite
wethe mirror
counterclockwise
make is
another aligned series with
through of the
six end
sectorsof T1
measurements and(or
mirror clockwise on the top rotary table, so that the mirror beginning
rotating
(six the oftopT2),
counterclockwise as
table shown
clockwise
these
in
rotations Fig. 5.
through
measurements as:
is aligned of Then,
the six
we make
sectors. The another
next series
step ofreposition
six measurements
isoftocorresponding theclockwise
mirror (six
clockwise counterclockwise
onofthe toptoprotary rotations
table, ofas the L Ras1 that
bottom
bottom
rotary with
rotary
table theand
table
endsix
and six
T1 (or beginning
corresponding
of T2),
clockwise
rotations
as shown
rotations
intheFig.
of the
5.rotary
top
Then, We
rotary
table)
Wedosonot
table)
that
doWe aswrite
not dowrite
notthewrite th m
the n1thmeasurement,
nth nmeasurement,
the measurement,
x as
as that is the closure
is the
that is theclosure constraint
closure constraint thatthat
constraint (11)
the twotwo
the
the mirror
illustrated in we is aligned
Fig.make with the end of T1 (or beginning
6. another series of six measurements (six counterclockwise of T2), as shown in Fig. 5. Then,
illustrated in Fig. 6.
we make another series of six measurements (six counterclockwise
rotaryrotary tables
rotations
tables return
that
of the as:
thetotwo
return zero
to zero after
rotary after m
360
tables
1360
of L R
rotation.
of rotation.
return 1to x zero In symbolic
In
after 360form,
symbolic ofform, we we
rotation. could could (11)
Inrewriterewrite
rotations these these measurements
Where m
measurements 1 is a vector as: of n - 1 measurements, L and R1 are (n -as1) n arrays**, and
bottomNotrotary
counting table of andthe
the sixbottom
initial
rotary table and sixrotations
corresponding
zeroing of clockwise the autocollimator of
corresponding clockwise
andthethetopclosing rotary table) symbolic
repeat as form, we could rewrite these measurements
Notrotations
inatcounting m6the initial zeroing
table) of the autocollimator Fig. 6. and x theis
thethe desired
closing solution
mrepeat vector of 2n angular sectors. The
1 is a vector of n - 1 measurements, L and
subscripts 1 denote that **
, this
illustrated
measurement Fig.
360, 6. of Where , R1 are (n - 1) n arrays and
the
, thetop rotary
first few measurements as illustrated after in repositioning mirror are:
measurement Not at 360, m
counting 6 , the first few measurements
the initial zeroing of the autocollimator after repositioning x is the
the
and mirror
desired
the are:solution vector of 2n
m
angular
1
m L
L R 1 R x x
1 sectors. 1 The subscripts 1 denote that this (11)
(11) (11)
**
Notclosing counting
m7 repeat 1 measurement
the initial
0 0 0 at zeroing of
0 360, the autocollimator
0 |1m6,0the0first 0 few and ** x
0 0measurements L and
the R are
closing for Left
repeat and Right, and should not be confused with the similarly named LR
m7 m61, the 0the 0 L xand
0 0 decomposition for where
Lower Left m1 is a vector
triangular of n - Right 1 measurements, L and used R are (n - 1) ** **
measurement afteratmrepositioning
360, 0few0measurements
firstmirror 0 0 |1 after 0 0 repositioning theRmirror
Where
are Where
for are: m1 m
Left isanda**vector
1 is a vector
Right, ofandnand- n1Upper
ofshould -measurements,
1notmeasurements,
triangular matrices
be confusedLwith and
Lthe Rsimilarly
and 1R are in1linear
(n
1 are -(n1)- 1)
named
algebra.
narrays
LR n arrays , and, and
8 1 1 0 0 0 0 |1 0 0 0 0 1 decomposition
are: x (7)
nLower arrays and x is and
,triangular theUpperdesired solution vector ofused2n angular sectors.
for Left Right triangular matrices in linear algebra.
m8 1 1 0 0 0 0 |1 0 0 0 0 1 x x is the desired x is the desired
The(7)
solution vector
solution vector
subscripts
of
1 denote
2n angular
of 2nthat angular sectors.
this set sectors. The subscripts
The subscripts
of measurements
1
is for
denote
1 thedenote
that 13
this
first that this
m9m7 1 11 01 00 0 0 0 0 0|1 |10 00 00 01 01 0x x
m9 1 1 1 0 0 0 |1 0 0 0 1 1 **x ** positionand of Right,
the mirror with not respect to thewith topthe rotary table. The first 13
m8 1 1 0 0 0 0 |1 0 0 0 0 1 xL and R
(7)are
L and R forare Left
for(7)Left and Right,and and should should be notconfused
be confused with similarly
the similarly named named LR LR
For each mirror alignment relative to the top table (mirror aligned decomposition
with start
decomposition for
(nof Lower
- 1)Lower
for Left
(nLeft-triangular
1)triangular
elements and andUpper
ofUpper Right
the LRightarray
triangular arematrices
triangular a lower
matrices usedtriangular
in linear
used in linearmatrix
algebra.
algebra.
T1, start ofForT2, each andm 1alignment
9 on),
mirror
so we1make 1 relative
n0measurements
0 to 0the|1top(where 0table n0 is1the1aligned
0 (mirror x
number with
of sectors start
withof-1 in the lower triangular elements, and zero in the remaining
T1, n measurements (where n at 13 13
that westart of T2, and
calibrate). so on),the
However, we nmake th
measurement
th
is the condition is 360
the number where of sectors
the two
elements; the last (nth) column of the L array contains zeros. The R1
that we calibrate).For each mirror alignment
However, the n relative to is
measurement thethe topcondition
table (mirror at 360 alignedwhere the two
For have
rotary tables each returned
mirror alignment to their starting relativeposition.to the topTherefore, table (mirror we have aligned n -with 1 distinctstart
arrayofis the mirror image (flipped left-right) of the L array, but with +1
rotary tables
T1, start
measurements ofwithT2,have
for start
and
eachreturned
ofmirror
so T1, start
on), to
we their
of T2,
make
alignment. startingandWeposition.
n measurements soalso
on),have Therefore,
we(where
nmake we
nn ismeasurements
possible the have
mirrornumber n - of 1 distinct
alignments. sectors
measurements n counting in the nonzero elements.
that we calibrate).
Therefore, we areformaking
(where neach
However,
is the mirror
n2number n alignment.
-2the nthofmeasurement
distinct We
measurements
sectors thatalso is the
we have(not
calibrate). possible
condition at 360
However, mirror
either where alignments.
the
the the two
initial
Therefore, th thare making n - n distinct measurements (not counting either the initial
we
rotaryortables
zeroing then nhave returned istothe
measurement
measurement
th
attheir
each starting
mirroratposition.
condition alignment
360 where Therefore,
at the
return twotowe havetables
zero),
rotary n - have
and 1 distinct to
L and R are for Left and Right, and should not be confused with the
zeroing orhavethe nfor measurement atstarting
each mirror We alignment have atn return we to zero), n -and 1 have to
**

measurements
solve for 2n unknowns eachinmirror
returned xto. theiralignment. position. also Therefore, possible have mirror alignments. similarly named LR decomposition for Lower Left triangular and Upper
solve for 2n unknowns in x .
2
Therefore, we are making n - n distinct measurements (not counting either the initial Right triangular matrices used in linear algebra.
zeroing or the nth measurement at each mirror alignment at return to zero), and have to
solve for 2n unknowns in x .
34 | NCSLI Measure www.ncsli.org
12
12
 TECHNICAL NOTE

n=6; %Number of sectors

L1=ones(n-1,n); %Set up left-side "L"
L=-tril(L1); %"-" for ccw of bottom table
R1=-fliplr(L); %Set up right-side "R"
A1=[L R1]; %First mirror position
R2=circshift(R1,[0 1]); %One cw rotation of mirror by 1 sector
A2=[L R2]; %next "n-1" measurements
R3=circshift(R1,[0 2]); %Continue for "n" mirror positions
A3=[L R3]; %Note that you could do this as a
R4=circshift(R1,[0 3]); %loop based on "n" positions, i.e.
A4=[L R4]; %A(n,:,:)=[L R(n,:,:)] etc.
R5=circshift(R1,[0 4]);
A5=[L R5];
R6=circshift(R1,[0 5]);
A6=[L R6];
c1=[ones(1,n) zeros(1,n)]; %Set up first constraint sum(bottom)=0
c2=[zeros(1,n) ones(1,n)]; %Set up 2nd constraint sum(top)=0
A=[A1;A2;A3;A4;A5;A6;c1;c2];
set of measurementsSolverMat=(A'*A)\A';
is for the first position of the mirror with respect to the %Solve
top rotary for "solver" to export to Excel
of
oftable. The first (n
measurements is -for 1) the (nfirst - 1) position
elementsof ofthe L arraywith
the mirror are arespect
lower triangular
to the top matrix
rotary with
ble. of-1measurements
measurements
in the
The first lower
(n 1)
is
is
triangular
-- Figure
for
for
(n
the
the
--9. 1)
first
first position
position
elements,
elements of and
the
of
of zeroL
the
the mirror
mirror
array inthe the(A
are
with
aawith
remaining
lower
respect
respect to
to the
elements;
top
thematrix
topthe rotary
rotary
last (nfor th
) the deviations of the sectors of the rotary tables.
ble. The first (n 1) (n 1) Script
elements that of generates
the L array are
T
A)-1 Atriangular
lower
T
matrix
triangular which
matrix with
solves
with
ble. in The
column
the first
lower of (n
the - 1)
L
triangular

array (n - 1) elements
contains
elements, zeros.
and of the
zero The L
in array
R
the 1
are
array
remaining a is lower
the triangular
mirror
elements; image matrix
the with
(flipped
last (n th
th ) left-
in the
theoflower
inright) lower triangular
triangular elements,
elements, and zero in
in the remaining elements; the
the last
last (n th)
lumn
lumn of ofthethe
the L Larray
L arrayarray, contains
contains

but withzeros.+1 and
zeros. in theThe zerononzero
Themirror R
R111 array
theelements.
array
remaining
is
is the
the mirror mirror
elements;image
image (flipped
(flipped
(n
left-
left-
)
lumn
ht) of of
the the
L L array
array,
After contains
but
the autocollimator
with +1 zeros.
in the The
nonzero R array is
is repositioned
elements. the mirror onimage
the top (flipped
rotary left- as follows:
ht)
ht) of the L
of the L array,
array,
table thebut
but
and with
withmore +1 in
in the
+1measurements nonzero
themirror
nonzero elements.
elements.
is taken, as in on Fig.the5, top the rotary
next set of and more m
m1
After autocollimator repositioned table m11
measurements
After taken, as in(as
themeasurements
autocollimator Figure
mirror 5,is
in Equation the next
repositioned
(7) are: set of on measurements
the top rotary (as table
in Equation
and more (7) are:
After
After the the autocollimator
autocollimator mirror
mirror is
is repositioned
repositioned on
on the the top top(as rotary
rotary table
table andand more
more
easurements taken, as in Figure 5, the next set of measurements in Equation (7) are:
easurements
easurements taken, taken, as as in in Figure
Figure 5,
5, the
the next
next set
set of
of measurements
measurements (as
(as in
in Equation
Equation (7)
(7) are:
are: mnn11 m
1 0 0 0 | 1 0 0 0 n1 m
1 0 0 0 || 1 0 0 0 1
1
0 1 1 00 0 1 | 0 1 0 0 0 mn 1
m 2 1 0 0 0 | 1 0 0 0 x (12)
m
1 1 1 1 0 || 1 1 | 0
0 1

1 x
(12) mn 1
m
m 2

1 1 0
0 | 1 0 1 x (12)
(12) n 1
m22 1 ||| 0 | 1 0 1 x 1 (12) yyy (16)
1 1 mm2 n 1 (16)
(16)
1 1 1 0 | 1 0 1 1
m2n 1 2 n 1
1
1
1
0
0 ||first 1 0
0 of1 1 (16)
Notecompared that 1 compared 1 with 1 the 1 set 1measurements, 1
Note that with the first set of measurements, the Lthe array is unchanged.
L array
The NoteR2 that
array is the original
is compared
unchanged. with The R1Rfirst
the array set
array that ishas
of thebeen
measurements,
original circularly R1the array shifted
L that tois
array hasthe right (that is,
unchanged.
been mm2 n 1
Note that compared with the first set of measurements, the L array is unchanged.
Note that compared with the first set of measurements, the L array is unchanged. m2 n 1
2 2 n 1
ee Rthe
R right-most
2 array
array is the
iscircularly column
the original
original shifted has
R
R been
1 array
to
array thethatmoved
thatright has
has(that to the
been
been firstright-most
is,circularly
the
circularly column shifted
shifted on thethe
to
column
to theleft,right
has
rightand
been each
(that
(that is,other
is,
e R 22 array is the original R 11 array that has been circularly shifted to the right (that is,
column hascolumn
ee right-most
right-most been shifted
moved
column tohas
has thebeen one column
first
been columnto
moved
moved to the
on
to the right).
the first
left,
first column
and
columnWeeach could on
other
on rewrite
the
the left,
column
left, equation
and
and haseach
been
each (12)
other
other as: 0
elumn right-most
has been column
shifted has one been
column movedtothethe to the
right). firstWe column
could on
rewrite the left,
equation and each
(12) other 0
lumn
lumn has has been shifted one column
columntoto to the right). We could rewrite equation (12) asas:
as: 0
been shiftedone one theright).
right).We We could rewrite equation (12) as: 000
shifted column could rewrite equation (12)
m2 L R 2 x (13)

m 2 L R 2x (13) (13)
L R 22 xx
.
m
m22 L R We
We
(13)
assemble
(13)
assemble We the
the matrix
assemble A as
A the follows:
matrix A as follows:
In general, we can describe the relationship for Rk (the subscript k
In general, we can describe the relationship for Rk (the subscript Wedenotes
assemble the the matrix
matrix A as follows:
as follows:
orientation
In
In general, of the
we mirror
can describewith respect
the to the
relationship top rotary
for R
R table):
(the subscript k denotes the
In general, we can describe the relationship for Rkk (the subscript to thek topdenotes
rotary the
k
L R1
entation general,
of the we
kmirror
denotescan with describe
therespect the
orientationto relationship
the oftop therotarymirror fortable):with (the subscript
respect k denotes the L L R
R1
entation
entation of of the
the mirror
mirror as with with respect respect to to thetheRtop toprotaryrotary table): 1
table) ror(R ktable):1 ) (14) L
L R R 2
k

L R 2
2

R ror( R ) (14) A (17) (17)
R
R kk ror(R
ror( R kk 11 )) . (14) A
A (17)
k k 1 (14) (14)
(17)
Where ror( ) is a rotate-right (or circular-shift-right) operation. So, in general, we 11 11 00 00
canWhere
write:ror(
10 10 01 01
ror( ))) is is aaa rotate-right
rotate-right (or (or circular-shift-right)
circular-shift-right) operation. operation. So, So, in in general,
general, we
Where ror( ) is a rotate-right (or circular-shift-right) operation. So,
Where
Where ror( is rotate-right (or circular-shift-right) operation. So, in general, we
we 00 00 11 11
nn write:
write:
n write:
in general, we can write m k L R k x (15)
m
mkk
m L
L R
L R kk xxx .
R We
(15)
(15)now
can
(15)
Wesolve can fornowthe solve
least for the least
squares deviation squares fromdeviation
nominalfrom
sectornominal
anglesin in xx
Again, the subscript k denotes k
thek orientation of the mirror with We
(15)
Wethe can
can
top now
now
rotary solve
solve for
for the
the least
least squares
squares deviation
deviation from
from nominal
nominal sector
sector angles
angles x
incan
by
by the calculating
calculating per
sector equation
per equation
angles
equation (9).
in x
(9).by
(9). AfterAfter
calculating
After
solving for solving per for the
equation
for the angle
the angle
(9). deviations
After
angle deviations
solvingx ,
deviations xx ,, we we
for
we cancan
table
Again,and ranges
the from
subscript 1 to
k n.denotes the orientation of the mirror withby calculating
calculate top rotary
the per
variance. This is done by solving
predicting estimated measurements:
Again, the the subscript
subscript k
Again, the k denotes
denotes the
subscript k the orientation
orientation of
denotes the of thethe mirror
orientation mirror with
of with
the the top
mirror
calculate topthe
with rotary the
variance.
angle deviations
This is
y , we can calculate the variance. This is done by
ble andAgain,
rangesthe from top111rotaryto n. the
calculate therotary
variance. This is done
done by by predicting
predicting estimated estimated measurements:
measurements:
ble
ble and and ranges
ranges from
from 2to to n.
n. table and ranges from 1 to n. predicting estimated measurements
We have n - n distinct measurements (n sets of n - 1 nonzero measurements), and z x (18)
zz
xx , (18)
2
2We have n - n distinct measurements (n sets of n - 1 nonzero
weWeWeaddhave two nnconstraints:
have 22 - n distinct From
- n distinct equation (n
measurements
measurements (6),sets
(n setstheofofsum nn -- of the tis (which
11 nonzero
nonzero are not
measurements),
measurements), Where measured)
and
and z is the vector of predicted measurements in the absence of (18) (18)
disturbances.
We have tonzero, - n distinct
measurements), measurements
and we theadd sumtwo (n
of sets bof
constraints: nmust
- 1tnonzero from measurements),
equation (6),Where
the and z is iswethe
the vector of predicted measurements in the absence of disturbances.
must
add
add two
two sumconstraints:
2constraints:
and From
From likewise,
equation
equation (6),
(6), the
the the
sum
sum is the
of
of the ttiiis
salso sum
(which
(which toThe
are
are
The
zero.
not
not Therefore,
measured)
Where
variance
measured)
variance iszthen
then
2is where
vector of predicted measurements in the absence of disturbances.
simply:
simply:
ust addsum two constraints:
n zero,
to n +and 2 rowslikewise, From i in A.the
equationsum (6), the sum
thebbimeasurement of the s (which are
The not measured)
variance
have n we is then
n+2 simply:
x
ust have sum of
Therefore, of the s
s must
must also also sum vectorto
to zero. y must Therefore,
sum of the t s (which are not measured) must sum to zero, and likewise, is the vector of predicted measurements in the absence of
ust sum
sum
2 to
to zero,
zero, and
and likewise,
likewise, the
the sum of the
the biito
s must also sum
sum to zero.
zero. Therefore,
Therefore,
2 2
we
we
ve n 2 n
measurements:+ 2 rows
the sum in
The of A.
A. first
the Therefore,
b s 2must
i n - n values the
also measurement
sum are the recorded
zero. vector y
vector measurements
Therefore, must
y must have n(not
we havehave n n n + 2
2 n including
2 n + disturbances. The variance 1 is then simply
ve n
ve n2 n n+ + 222rows
rows in
innA.
in Therefore,
A.nmeasurement
the measurement
th 2Therefore, the measurement vector y must have2 n n + 2
+2 1
1 i (((zzzi i yyyi i)))222 , (19)
easurements:
initial zero, The
or the first - n values are
at the
360), recorded
and the measurements
last twox values (not
of n yincluding
are
n +zero. We (19)

rows 22Therefore, the measurement vector must have ( o 1)
easurements:
easurements: The
The first n
th first n - n values are - n values are the
the recorded
recorded measurements
measurements (not
(not including
including ( o 1) i i (19) (19)
tial zero, or
assemble nth measurement
2 measurements.
the measurement Theatfirst
vector 360),
y as nand
n -follows:
2
values the lastare the two
tworecorded values
values of yy are
measurements zero. We (o 1) i i
tial
tial zero,
zero, or the nnth measurement
or the measurement at at 360),
360), and and the the last
th last two values of
of y are
are zero.
zero. We We
semble
semble the measurement
(not including vector initialy as follows:
zero, or the n measurement at 360),where and o theis the number
where o is of
the measured
number ofpoints, measured or (n (n222 n).
points, n).
or (n2The
n).variance can also be
semble the the measurement
measurement vector vector yy as as follows:
follows: where oo is
where is the
the number
number of of measured
measured points, points,T or or (n n). The variance
The variance can also be
last two values of y are zero. We assemble the measurement calculated
calculated
vectorby taking the vector dot product zTT y and scaling by (1/(o-1)). can also be
calculated by by taking
taking the the vector
vector dotdot product
product zz yy and and scaling
scaling byby (1/(o-1)).
(1/(o-1)).
Computer spreadsheet
Computer spreadsheet programs,
programs, such asas MS-Excel, include matrix math.
|math. The
Vol. 7 No. 1 March 2012 Computerfunction
TRANSPOSE spreadsheet programs, such
in MS-Excel
MS-Excel such as MS-Excel,
MS-Excel,
transposes
include
NCSLI
include
an array;
array;
matrix
Measure
matrix
the MMULT
MMULT
35 The
math. The
function
TRANSPOSE 14
function in transposes an the function
TRANSPOSE
multiplies two function in matrix
arrays using MS-Excel transposesrules,
multiplication an array;
and thethe MMULT calculates
MINVERSE function
TECHNICAL NOTE
Computer spreadsheet programs, such as MS-Excel, include
matrix math. The TRANSPOSE function in MS-Excel transposes
an array; the MMULT function multiplies two arrays using matrix
multiplication rules, and the MINVERSE calculates the matrix
inverse. In our case, we actually created and assembled the A matrix
and the solution matrix (ATA)-1 AT using MATLAB. A script showing
the creation of the A matrix is shown in Fig. 9. We used MATLAB to
check the condition number of the resulting matrix. A low condition
number indicates a well-conditioned numerical problem. A condition
number on the order of 10p indicates that p digits of precision might
be lost during calculation. The condition number of A is on the order
of 103 for a six-sector problem, indicating that with double precision
arithmetic, no more than 12 digits of precision should be expected
from the results. We used MS-Excel to do drift correction, assembly
of the measurements from a datasheet to a column vector, and solution
of the results per Equation (9).
5. Conclusions
Circle closure calibration is a well-known technique; however, it
is tedious to implement and the resulting calculations are time-
consuming. Linear algebra tools are easily implemented in computer
spreadsheets, which take the drudgery out of the calculations. We
have shown an example procedure for six-sector calibration of rotary
tables, but the process and the derivation shown can be generalized
for any number of sectors, limited by the patience of the metrologist.
6. Acknowledgements
Sandia National Laboratories is a multi-program laboratory managed
and operated by Sandia Corporation, a wholly owned subsidiary of
Lockheed Martin Corporation, for the U.S. Department of Energys
National Nuclear Security Administration under contract DE-AC04-
94AL85000.
36 | NCSLI Measure
7. References
[1] W. R. Moore, Foundations of Mechanical Accuracy, Bridgeport,
CT: Moore Special Tool Company, 1970.
[2] C. J. Evans, R. J. Hocken, and W. T. Estler, Self-Calibration:
Reversal, Redundancy, Error Separation, and Absolute
Testing, Annals CIRP, vol. 45, pp. 617-634, 1996.
[3] W. T. Estler, Uncertainty Analysis for Angle Calibrations
Using Circle Closure, J. Res. Natl. Inst. Stan., vol. 103, pp.
141-151, 1998.
[4] MATLAB, vol. R2011a, ed. http://www.mathworks.com:
The Mathworks Inc., 2011.
[5] T. Doiron, D. McLaughlin, A. Schneider, S. Adams, and E.
Stanfield, Use of Air Showers to Reduce Soaking Time for
High Precision Dimensional Measurements, Proceedings
of the NCSLI Annual Workshop and Symposium, Saint Paul,
Minnesota, 2007.
[6] BIPM/JCGM, Evaluation of measurement data - Guide to the
expression of uncertainty in measurement, 2008.
[7] C. Moler, Numerical Computing with MATLAB, Philadelphia,
PA: SIAM, 2004.
[8] NIST, NIST/SEMATECH e-Handbook of Statistical Methods,
June 2010. (available from http://www.itl.nist.gov/div898/
handbook/)
[9] G. Strang, Linear Algebra and its Applications, New York, NY:
Academic Press, 1980.
www.ncsli.org
Metrology

We measure
for your safety

World Metrology Day

20 May 2012
www.worldmetrologyday.org
TECHNICAL PAPERS

Risk Mitigation Strategies

for Compliance Testing
Jonathan Harben and Paul Reese

Abstract: Many strategies for risk mitigation are now practiced in calibration laboratories. This paper presents a modern look
at these strategies in terms of compliance to ANSL/NCSLI and ISO standards. It distinguishes between Bench Level and
Program Level risk analysis techniques, which each answer different questions about risk mitigation. It investigates concepts
including the test uncertainty ratio (TUR) and end of period reliability (EOPR) that are directly related to risk, as well as the math-
ematical boundary conditions of false accept risk to gain a comprehensive understanding of practical, efficient risk mitigation.
The paper presents practices and principals that can allow a calibration laboratory to meet the demand of customers and manage
risk for multifunction instrumentation, while complying with national and international standards.

1. Background accuracy1 specifications. The customer is asking for an in-tolerance

Calibration is all about confidence. In some scenarios, it is important
to have confidence that the certified value of a laboratory reference
standard is within its assigned uncertainty limits. In other scenarios,
confidence that an instrument is performing within its published ac-
curacy specifications may be desired. Confidence in an instrument is
often obtained through compliance testing, which is sometimes called
conformance testing, tolerance testing, or verification testing. For
these types of tests, a variety of strategies have historically been used
to manage the risk of falsely accepting non-conforming items and er-
roneously passing them as good. This type of risk is called false
accept risk (also known as FAR, probability of false accept (PFA),
consumers risk, or Type II risk). To mitigate false accept risk, sim-
plistic techniques have often relied upon assumptions or approxima-
tions that were not well founded. However, high confidence and low
risk can be achieved without relying on antiquated paradigms or un-
necessary computations. For example, there are circumstances where
a documented uncertainty is not necessary to demonstrate that false
accept risk was held below certain boundary conditions. This is a
somewhat novel approach with far-reaching implications in the field
of calibration.
While the importance of uncertainty calculations is acknowledged
for many processes (e.g. reference standards calibrations), it might be
unnecessary during compliance tests when historical reliability data is
available for the unit under test (UUT). Many organizations require
a documented uncertainty statement in order to assert a claim of met-
rological traceability [1], but the ideas presented here offer evidence
that acceptance decisions can be made with high confidence without
direct knowledge of the uncertainty.
In the simplest terms, when measurement & test equipment
(M&TE) owners send an instrument to the calibration laboratory they
want to know, Is my instrument good or bad? During a compliance
test, M&TE is evaluated using laboratory standards to determine if it
is performing as expected. This performance is compared to specifi-
cations or tolerance limits that are requested by the end user or cus-
tomer. These specifications are often the manufacturers published
or out-of-tolerance decision to be made, which might appear to be
a straightforward request. But exactly what level of assurance does
the customer receive when statements of compliance are issued? Is
simply reporting measurement uncertainty enough? What is the risk
that a statement of compliance is wrong? While alluded to in many
international standards documents, these issues are directly addressed
in ANSI/NCSL Z540.3-2006 [2].
Since its publication, sub-clause 5.3b of the Z540.3 has, under-
standably, received a disproportionate amount of attention compared
with other sections in the standard [3, 4, 5]. This section represents
a significant change when compared to its predecessor, Z540-1 [6].
Section 5.3b has come to be known by many as The 2 % Rule
and addresses calibrations involving compliance tests. It states:
Where calibrations provide for verification that measurement quan-
tities are within specified tolerances, the probability that incorrect
acceptance decisions (false accept) will result from calibration tests
shall not exceed 2% and shall be documented. Where it is not prac-
ticable to estimate this probability, the test uncertainty ratio shall be
equal to or greater than 4:1.
Much can be inferred from these two seemingly innocuous state-
ments. The material related to compliance testing in the ISO 17025
[7] is sparse, as that standard is primarily focused on reporting uncer-
tainties with measurement results, similar to Z540.3 section 5.3a. Per-
haps the most significant reference to compliance testing in ISO 17025
is found in section 5.10.4.2 (Calibration Certificates) which states that
When statements of compliance are made, the uncertainty of mea-
surement shall be taken into account. However, practically no guid-
ance in provided regarding the methods that could be implemented to
take the measurement uncertainty into account. The American Asso-
1
The term accuracy is used throughout this paper to facilitate the classical
concept of uncertainty for a broad audience. It is acknowledged that
the VIM [1] defines accuracy as qualitative term, not quantitative, and that
numerical values should not be associated with it.

38 | NCSLI Measure www.ncsli.org


Figure 1. Five possible bench level calibration scenarios.
ciation of Laboratory Accreditation (A2LA)
further clarifies the requirements associated
with this concept in R205 [8]:
When parameters are certified to be within
specified tolerance, the associated uncer-
tainty of the measurement result is properly
taken into account with respect to the toler-
ance by a documented procedure or policy
established and implemented by the labora-
tory that defines the decision rules used by
the laboratory for declaring in or out of tol-
erance conditions.2
Moreover, the VIM [1] has recently added
a new Note 7 to the definition of metrologi-
cal traceability. This note reiterates that the
International Laboratory Accreditation Coop-
eration (ILAC) requires a documented mea-
Authors
Jonathan Harben
The Bionetics Corporation
M/S: ISC-6175
Kennedy Space Center, FL 32899
jonathan.p.harben@nasa.gov
Paul Reese
Covidien, Inc.
815 Tek Drive
Crystal Lake, IL 60014
paul.reese@covidien.com
Vol. 7 No. 1 March 2012 NCSLI Measure |
surement uncertainty for any and all claims of
metrological traceability. However, simply re-
porting the uncertainty along with a measure-
ment result may not satisfy customer require-
ments where compliance tests are desired.
Without having a quantifiable control limit
such as false accept risk, this type of reporting
imparts unknown risks to the customer.
The methods presented in this paper pro-
vide assurance that PFA risks are held be-
low a specified maximum permissible value
(2 %) without direct knowledge of the un-
certainty. However, they may not satisfy the
strict language of national and international
documents, which appear to contain an im-
plicit requirement to document measurement
uncertainties for all calibrations.
Where compliance tests are involved,
the intent of the uncertainty requirements
may (arguably) be to allow an opportunity
to evaluate the risks associated with pass/
fail compliance decisions. If this is indeed
the intent, then the ideas presented here can
provide the same opportunity for evalua-
tion without direct knowledge of the uncer-
tainty. Because considerable effort is often
required to generate uncertainty statements,
it is suggested that accreditation bodies ac-
cept the methods described in this paper as
an alternative solution for compliance testing.
TECHNICAL PAPERS
2. Taking the Uncertainty
Into Account
What does it mean to take the uncertainty
into account and why it is necessary? For
an intuitive interpretation, refer to Fig. 1. Dur-
ing a compliance test on the bench, what are
the decision rules if uncertainty is taken into
account? For example, during the calibration,
the UUT might legitimately be observed to be
in-tolerance. However, the observation could
be misleading or wrong as illustrated in Fig. 1.
It is understood that all measurements are
only estimates of the true value of the measur-
and; this true value cannot be exactly known
due to measurement uncertainty. In scenario
#1, a reading on a laboratory standard volt-
meter of 9.98 V can confidently lead to an in-
tolerance decision (pass) for this 10 V UUT
source with negligible risk. This is true due to
sufficiently small uncertainty in the measure-
ment process and the proximity of the mea-
sured value to the tolerance limit. Likewise,
a non-compliance decision (fail) resulting
from scenario #5 can also be made with high
confidence, as the measured value of 9.83 V
is clearly out-of-tolerance. However, in sce-
narios #2, #3, and #4, there is significant risk
that a pass/fail decision will be incorrect.
2
The default decision rule is found in
ILAC-G8:1996 [9], Guidelines on Assessment
and Reporting of Compliance with Specification,
section 2.5. With agreement from the customer,
other decision rules may be used as provided for
in this section of the requirements.
39
TECHNICAL PAPERS
In scenarios, #2, #3, & #4, this uncer-
tainty makes it possible for the true value of
the measurand to be either in or out of toler-
ance. Consider scenario #3, where the UUT
was observed at 9.90 V, exactly at the lower
allowable tolerance limit. Under such condi-
tions, there is a 50 % probability that either an
in-tolerance or out-of-tolerance decision will
be incorrect, barring any other information.
In fact, even for standards with the lowest
possible uncertainty, the probability of being
incorrect will remain at 50 % in scenario #33.
This concept of bench level risk is addressed
in several documents [9, 10, 11, 12].
The simple analysis of the individual
measurement results presented above is not
directly consistent with the intent of The
2 % rule in Z540.3, although it still has
application. Until now, our discussion has
dealt exclusively with bench level analysis
of measurement decision risk. That is, risk
was predicated only on knowledge of the
relationship between the UUT tolerance, the
measurement uncertainty, and the observed
measurement result made on-the-bench.
However, the computation of false accept
risk, for strict compliance with the 2 % rule
in Z540.3, does not depend on any particular
measurement, nor does it depend on its prox-
imity to a given UUT tolerance limit. Instead,
the 2 % rule in Z540.3 addresses the risk at
the program level, prior to obtaining a mea-
surement result. To understand both bench
level and program level false accept risk, the
intent underlying the 2 % rule and its relation-
ship to TUR and EOPR4 must be examined.
3. The Answer to Two Different
Questions
False accept risk describes the overall prob-
ability of false acceptance when pass/fail
decisions are made. False accept risk can be
interpreted and analyzed at either the bench
level or the program level [4]. Both risk lev-
els are described in ASME Technical Report
B89.7.4.1-2005 [13]. The ASME report refers
to bench level risk mitigation as controlling
3
Bayesian analysis can result in false accept risk Z540.3 section 5.3b does not directly address
other than 50 % in such instances, where the a model UUT voltage sources are scheduled
priori in-tolerance probability (EOPR) of the UUT
is known in addition to the measurement result
and uncertainty.
4
The subject of measurement decision risk
includes not only the probability of false-accept
(PFA), but the probability of correct accept
(PCA), probability of false reject (PFR) and the
probability of correct reject (PCR). While false
rejects can have significant economic impact to
the calibration lab, the discussion in this paper is
primarily limited to false accept risk.
40 | NCSLI Measure
the quality of individual workpieces, while
program level risk strategies are described
as controlling the average quality of work-
pieces. Bench level risk can be thought of as
an instantaneous liability at the time of mea-
surement, whereas program level risk speaks
more to the average probability that incorrect
acceptance decisions will be made based on
historical data. These two approaches are
related, but result in two answers to two dif-
ferent questions. Meeting a desired quality
objective requires an appropriate answer to
an appropriate question, and ambiguity in the
question itself can lead to different assump-
tions regarding the meaning of false accept
risk. Many international documents discuss
only the bench level interpretation of risk,
and require an actual measurement result to
be available [9, 10, 11, 12]. These documents
describe the most basic implementation of
bench level risk, where no other pre-mea-
surement state of knowledge exists. They
address the instantaneous false accept risk
associated with an acceptance decision for a
single measured value, without the additional
insight provided by historical data. This most
basic of bench level techniques is sometimes
called the confidence level method. How-
ever, if a-priori data exists, a more rigor-
ous type of bench-level analysis is possible
using Bayesian methods. By employing prior
knowledge of reliability data, Bayesian anal-
ysis updates or improves the estimate of risk.
The Z540.3 standard, however, was in-
tended to address risk at the program level
[14]. When this standard requires the
probability that incorrect acceptance deci-
sions (false accept) will result from calibra-
tion tests shall not exceed 2%.., it might
not be evident which view point is being ad-
dressed, the bench level or the program lev-
el. The implications of this were significant
enough to prompt NASA to request interpre-
tive guidance from the NCSLI 174 Standards
Writing Committee [15]. It was affirmed that
the 2 % false accept requirement applies to
a population of like calibration sessions or
like measurement processes [14]. As such,
the probability of false accept to any single,
discrete measurement result or individual
workpiece and supports the program level
view of risk prior to, and independent of, any
particular measurement result.
In statistical terms, the 2 % rule refers to
the unconditional probability of false accep-
tance. In terms of program level risk, false
accept risk describes the overall or average
probability of false acceptance decisions to
the calibration program at large. It does not
represent risk associated with any particular
instrument. The 2 % rule speaks to the fol-
lowing question: Given a historical collec-
tion of pass/fail decisions at a particular test-
point for a population of like-instruments (i.e.
where the EOPR and TUR are known), what
is the probability that an incorrect acceptance
decision will be made during an upcoming
test? Note that no measurement results are
provided, and that the question is being asked
before the scheduled measurement is ever
made and the average risk is controlled for
future measurements. Even so, the question
can be answered as long as previous EOPR
data on the UUT population is available, and
if the measurement uncertainty (and thus
TUR) is known. In certain circumstances, it
is also possible to comply with the 2 % rule
by bounding or limiting false accept risk us-
ing either:
EOPR data without knowledge of the
measurement uncertainty.
TUR without knowledge of EOPR data.
To understand how this is possible, a
closer look at the relationship between false
accept risk, EOPR, and TUR is helpful.
4. End of Period Reliability (EOPR)
EOPR is the probability of a UUT test-point
being in-tolerance at the end of its normal
calibration interval. It is sometimes known as
in-tolerance probability and is derived from
previous calibrations. In its simplest form,
EOPR can be defined as
Number of in-tolerance results
EOPR = Total number of calibrations
. (1)
If prior knowledge tells us that a significant
number of previous measurements for a pop-
ulation of UUTs were very close to their tol-
erance limits as-received, it can affect the
false accept risk for an upcoming measure-
ment. Consider Fig. 2 where two different
for calibration, model A and model B. The
five previous calibrations on model As have
shown these units to be highly reliable; see
Group A. Most often, they are well within
their tolerance limits and easily comply with
their specifications. In contrast, previous
model B calibrations have seldom met their
specifications; see Group B. Of the last five
www.ncsli.org
 TECHNICAL PAPERS

Figure 2. Previous historical measurement data can influence future false accept risk.

Figure 3. The possibility of a false accept for a measurement result.

calibrations, two model Bs were recorded
as being out-of-tolerance and one of them
was barely-in. Therefore, making an in or
out of tolerance decision will be a precari-
ous judgment call, with a high probability of
making a false accept decision.
In Fig. 3, imagine the measurement result
is not yet shown on the chart. If it was known
ahead of time that this upcoming measure-
ment result would be near the tolerance limit,
it can be seen that a false accept would indeed
be more likely given the uncertainty of the
measurement. The critically important point
is this -- if the historical reliability data in-
dicates that in-tolerance probability (EOPR)
of the UUT is poor (up to a point5), the false
accept risk increases.
The previous scenarios assume familiar-
ity with populations of similar instruments
that are periodically recalibrated. But how
can EOPR be reconciled when viewed from
a new laboratorys perspective? Can a
5
Graphs of EOPR vs. false-accept risk can reveal
a perceived decrease in false-accept risk as the
EOPR drops below certain levels. This is due to
the large number of out-of-tolerance conditions
that lie far outside the UUT tolerance limits. This
is discussed later in this paper.

Vol. 7 No. 1 March 2012 NCSLI Measure | 41

TECHNICAL PAPERS
new laboratory open its doors for business
and meet the 2 % false accept requirement
of Z540.3 without EOPR data? The answer
is yes. However, the new laboratory must
employ bench level techniques, or techniques
such as boundary condition methods or
guardbanding. Such methods are described
later in this paper. This same logic would ap-
ply to an established laboratory that receives
a new, unique instrument to calibrate for the
first time. In the absence of historical data,
other appropriate techniques and/or bench
level methods must be employed.
If EOPR data or in-tolerance probability
is important for calculating risk, several other
questions are raised. For example, how good
must the estimate of EOPR be before pro-
gram level methods can be used to address
false accept risk for a population of instru-
ments? When is the collection of measure-
ment data complete? What are the rules for
updating EOPR in light of new evidence?
Sharing or exchanging EOPR data between
different laboratories has even been proposed
with varying opinions. Acceptance of this
generally depends upon the consistency of
the calibration procedure used and the labo-
ratory standards employed. The rules used
to establish EOPR data can be subjective
(for example, how many samples are avail-
able, are first-time calibrations counted, are
broken instruments included, are late calibra-
tions included, and so on). Instruments can
be grouped together by various classifica-
tions, such as model number. For example,
reliability data for the M&TE model and
manufacturer level can be used to conserva-
tively estimate the reliability of the M&TE
test point. This is addressed in compliance
Method 1 & 2 of the Z540.3 Handbook [16].
5. Test Uncertainty Ratio
It has been shown that EOPR can affect the
false accept risk of calibration processes.
However, test uncertainty ratio (TUR) is like-
ly to be more familiar than EOPR as a metric
of the quality of calibration. The preced-
ing examples show that a lower uncertainty
generally reduces the likelihood of a false
accept decision. The TUR has historically
been viewed as the uncertainty or tolerance
of the UUT in the numerator divided by the
uncertainties of the laboratorys measurement
standard(s) in the denominator [17]. A TUR
greater than 4:1 was thought to indicate a ro-
bust calibration process.
The TUR originated in the Navys Produc-
42 | NCSLI Measure
tion Quality Division during the 1950s in
an attempt to minimize incorrect acceptance
decisions. The origins of the ubiquitous 4:1
TUR [18] assume a 95 % in-tolerance prob-
ability for both the measuring device and the
UUT. In those pre-computer days, these as-
sumptions were necessary to ease the compu-
tational requirements of risk analysis. Since
then, manufacturers specifications have of-
ten been loosely inferred to represent 2 or
95 % confidence for many implementations
of TUR, unless otherwise stated. In other
words, it is assumed that all UUTs will meet
their specifications 95 % of the time (i.e.
EOPR will be 95 %). Even if the calibration
personnel did not realize it, they were relying
on these assumptions to gain any utility out of
the 4:1 TUR. However, is the EOPR for all
M&TE really 95 %? That is, are all manufac-
turers specifications based on two standard
deviations of the product distribution? If they
are not, then the time-honored 4:1 TUR will
not provide the expected level of protection
for the consumer.
While the spirit of Z540.3 is to move away
from the reliance on TUR altogether, its use is
still permitted if adherence to the 2 % rule is
deemed impracticable. The use of the TUR
is discouraged due to the many assumptions it
relies on for controlling risk. However, given
that the false accept risk computation requires
the collection of EOPR data, the use of TUR
might be perceived as an easy way for labs
to circumvent the 2 % rule. Section 3.11 in
Z540.3 redefines TUR as:
The ratio of the span of the tolerance of a
measurement quantity subject to calibration,
to twice the 95% expanded uncertainty of the
measurement process used for calibration.
At first, this definition appears to be simi-
lar to older definitions of TUR. The defini-
tion implies that if the numerator, associated
with the specification of the UUT, is a plus-
or-minus () tolerance, the entire span of the
tolerance must be included. However, this is
countered by the requirement to multiply the
95 % expanded uncertainty of the measure-
ment process in the denominator by a factor
of two. The confidence level associated with
the UUT tolerance is undefined. This quanda-
ry is not new, as assumptions about the level
of confidence associated with the UUT (nu-
merator) have been made for decades.
There is, however, a distinct difference
between the TUR as defined in Z540.3 and
previous definitions. This difference centers
on the components of the denominator. In
Z540.3, the uncertainty in the denominator is
very specifically defined as the uncertainty
of the measurement process used in calibra-
tion. This definition has broader implica-
tions than historical definitions because it
includes elements of the UUT performance
(for example, resolution and process repeat-
ability) in the denominator. Many laborato-
ries have long assumed that the uncertainty
of the measurement process, as it relates to
the denominator of TUR, should encompass
all aspects of the laboratory standards, envi-
ronmental effects, measurement processes,
etc., but not the aspects of the UUT. His-
torically, the TUR denominator reflected the
capability of the laboratory to make highly
accurate measurements, but this capability
was sometimes viewed in the abstract sense,
and was independent of any aspects of the
UUT. The redefined TUR in the Z540.3 in-
cludes everything that affects a laboratorys
ability to accurately perform a measurement
on a particular device in the expanded uncer-
tainty, including UUT contributions. This
was reiterated to NASA in another response
from the NCSLI 174 Standards Writing Com-
mittee [19].
The new definition of TUR is meant to
serve as a single simplistic metric for evaluat-
ing the plausibility of a proposed compliance
test with regard to mitigating false accept
risk. No distinction is made as to where the
risk originates, it could originate with either
the UUT or the laboratory standard(s). A
low TUR does not necessarily imply that the
laboratory standards are not good enough.
It might indicate, however, that the measure-
ment cannot be made without significant false
accept risk due to the limitations of the UUT
itself. Such might be the case if the accuracy
specification of a device is equal to its resolu-
tion or noise floor. This can prevent a reliable
pass/fail decision from being made.
When computing TUR with confidence
levels other than 95 %, laboratories have
sometimes attempted to convert the UUT
specifications to 2 before dividing by the
expanded uncertainty (2) of the measure-
ment process. Or, equivalently, UUT specs
were converted to 1 for division by the
standard uncertainty (1) of the measure-
ment process. Either way, this was believed
by some to provide a more useful apples-to-
apples ratio for the TUR. Efforts to develop
an equivalent or normalized TUR have been
www.ncsli.org

Figure 4. The probability density of possible measurement results.
Figure 5. Topographical contour map with tolerance limits (L) and regions
of incorrect compliance decisions.
Vol. 7 No. 1 March 2012 NCSLI Measure |
TECHNICAL PAPERS
documented by several authors [18, 20, 21,
22]. However, the integrity of a TUR depends
upon the level of effort and honesty demon-
strated by the manufacturer when assigning
accuracy specifications to their equipment.
It is important to know if the specifications
are conservative and reliable, or if they were
produced by a marketing department that was
motivated by other factors.
6. Understanding Program Level False
Accept Risk
Investigating the dependency of false accept
risk on EOPR and TUR is well worth the ef-
fort involved. The reader is referred to several
papers that provide an excellent treatment of
the mathematics behind the risk requirements
at the program level [3, 4, 23, 24, 25]. These
publications and many others build upon the
seminal works on measurement decision risk
by Eagle, Grubbs, Coon, & Hayes [18, 26, 27]
and should be considered required reading.
This discussion is more conceptual in
nature, but a brief overview of some funda-
mental principles is useful. As stated earlier,
M&TE tolerance limits are often set by the
manufacturers accuracy specifications. The
device may be declared in-tolerance if the
UUT is observed to have a calibration result
eobs that is within the tolerance limits L. This
can be written as L eobs L. The observed
calibration result eobs is related to the actual or
true UUT error euut and the measurement pro-
cess error estd by the equation eobs = euut + estd.
Note that the quantity euut is the parameter be-
ing sought when a calibration is performed,
but eobs is what is obtained from the measure-
ment. The value of euut is always an estimate
due to the possibility of measurement process
errors estd described by uncertainty U95. It is
not possible to determine euut exactly.
Errors (such as euut and estd), as well as
measurement observations (such as eobs), are
quantities represented by random variables
and characterized by probability density
functions. These distributions represent the
relative likelihood of any specific error (euut
and estd) or measurement observation (eobs)
actually occurring. They are most often of
the Gaussian form or normal distribution and
are described by two parameters, a mean or
average , and a standard deviation . The
standard deviation is a measure of the vari-
ability or spread in the values from the mean.
The mean of all the possible error values
will be zero, which assumes systematic ef-
fects have been corrected. Real-world mea-
43
TECHNICAL PAPERS
surements are a function of both () charac-
terized by the UUT performance uut and the
measurement eobs with associated uncertainty
, where s obs = s uut + s std . The relative like-
lihood of all possible measurement results is
represented by the two dimensional surface
area created by the joint probability distribu-
tion given by (, eobs) = () (std). Fig- risk results. In this case, the angle is given by
ures 4 and 5 illustrate the concept of prob-
ability density of measurement and represent
the relative likelihood of possible measure-
ment outcomes given the variables TUR and
EOPR. It is assumed that measurement un-
certainty and the UUT distribution follow a
normal or Gaussian probability density func-
tion, yielding a bivariate normal distribution.
Figure 5 is a top-down perspective of Fig. 4,
when viewed from above.
The height, shape, and angle of the joint
probability distribution change as a function
of input variables TUR and EOPR. The dy-
namics of this are critical, as they define the
amount of risk for a given measurement sce-
nario. The nine regions in Fig. 5 are defined
by two-sided symmetrical tolerance limits.
Risk is the probably of a measurement oc-
curring in either the false accept regions or
the false reject regions. Computing an actual
numeric value for the probability (PFA or
PFR) involves integrating the joint probabil-
ity density function over the appropriate two
dimensional surface areas (regions) defined
by the limits stated below. Incorrect (false)
acceptance decisions are made when euut >
|L| and L eobs L. In this case, the UUT
is truly out of tolerance, but is observed to
be in tolerance. Likewise incorrect (false) re-
ject decisions are made when eobs >|L| and L
euut L, or where the UUT is observed to
be out of tolerance, but is truly in tolerance.
Integration over the entire joint probability
region will yield a value of 1, as would be
expected. This encompasses 100 % of the
volume under the surface of Fig. 4. When
the limits of integration are restricted to the
two false accept regions shown in Fig. 5, a
small portion of the total volume is computed
which represents the false accept risk as a
percentage of that total volume.
In the ideal case, if the measurement un-
certainty was zero, the probability of mea-
surement errors estd occurring would be zero.
The measurements would then perfectly re-
flect the behavior of the UUT and the distri-
bution of possible measurement results would
be limited to the distribution of actual UUT
errors. That is, (obs) would equal (uut)
44 | NCSLI Measure
and the graph in Fig. 5 would collapse to a
straight line at a 45 angle and the width in
Fig. 4 would collapse to a simple two dimen-
sional surface with zero volume. However,
since real-world measurements are always
hindered by the probability of errors, obser-
vations do not perfectly reflect reality and
While a full explanation is beyond the scope
s
tan(q ) = obs , where 45 90.
s uut
7. Efficient Risk Mitigation
In order for a calibration laboratory to comply
with Z540.3 (5.3b), the program level PFA
must not exceed 2 % and must be document-
ed. However, computing an actual value for
PFA is not necessarily required when demon-
strating compliance with the 2 % rule. To un-
derstand this, consider that the boundary con-
ditions of PFA can be investigated by varying
the TUR and EOPR over a wide range of val-
ues and observing the resultant PFA. This is
best illustrated by a three dimensional surface
plot, where the x and y axis represent TUR
and EOPR, and the height of the surface on
the z-axis represents PFA (Fig. 6 and 7).
This surface plot combines both aspects
affecting false accept risk into one visual
representation that illustrates the relationship
between the variables TUR and EOPR. One
curious observation is that the program level
PFA can never be greater than 13.6 % for any
combination of TUR and EOPR. The maxi-
mum value of 13.6 % occurs when the TUR
is approximately 0.3:1 and the EOPR is 41
%. Any change, higher or lower, for either
the TUR or EOPR will result in a PFA lower
than 13.6 %.
One particularly useful observation is that,
for all values of EOPR, the PFA never ex-
ceeds 2 % when the TUR is above 4.6:1. In
Figures 6 and 7, the darkest blue region of the
PFA surface is always below 2 %. Even if the
TUR axis in the above graph were extended
to infinity, the darkest blue PFA region would
continue to fall below the 2 % threshold. Cal-
ibration laboratory managers will find this to
be an efficient risk mitigation technique for
compliance with Z540.3. The burden of col-
lecting, analyzing, and managing EOPR data
can be eliminated when the TUR is greater
than 4.6:1.
This concept can be further illustrated by
rotating the perspective (viewing angle) of
the surface plot in Fig. 6, allowing the two
dimensional maximum outer-envelope or
boundary to be easily viewed. With this per-
spective, PFA can be plotted only as a func-
tion of TUR (Fig. 8). In this instance, the
worst-case EOPR is used whereby the maxi-
mum PFA is produced for each TUR.
The left-hand side of the graph in Fig.
8 might not appear intuitive at first. Why
would the PFA suddenly decrease as the
TUR drops below 0.3:1 and approaches zero?
of this paper, the answer lies in the number of
items rejected (falsely or otherwise) when an
extremely low TUR exists. This causes the
angle of the joint probability distribution to
rotate counter-clockwise away from the ideal
45 line, shifting areas of high density away
from the false accept regions illustrated in
Fig. 5. For a very low TUR, there are indeed
very few false accepts and very few correct
rejects. The outcome of virtually all mea-
surement decisions is then distributed over
the correct accept and false reject regions as
approaches 90. It would be impractical for a
calibration laboratory to operate under these
conditions, although false-accepts would be
exceedingly rare.
Examining the boundary conditions of
the surface plot also reveals that the PFA is
always below 2 % where the true EOPR is
greater than 95 %. This is true even with ex-
tremely low TURs (even below 1:1). Again,
if the perspective of the PFA surface plot in
Fig. 6 is properly rotated, a two dimensional
outer-envelope is produced whereby PFA can
be plotted only as a function of EOPR (Fig.
9). The worst-case TUR is used for each and
every point of the Fig. 9 curve, maximizing
the PFA, and illustrating that knowledge of
the TUR is not required.
As was the case with a low TUR, a simi-
lar phenomenon is noted on the left-hand
side of the graph in Fig. 9; the maximum PFA
decreases for true EOPR values below 41 %.
As the EOPR approaches zero on the left
side, most of the UUT values lie far outside
of the tolerance limits. When the values are
not in close proximity to the tolerance limits,
the risk of falsely accepting an item is low.
Likewise on the right-hand side of the graph,
where the EOPR is very good (near 100 %),
the false accept risk is low. Both ends of the
graph represent areas of low PFA because
most of the UUT values have historically
been found to lie far away from the tolerance
limits. The PFA is highest, in the middle of
the graph, where EOPR is only moderately
poor, and where much of the data is near the
tolerance limits.
www.ncsli.org
 TECHNICAL PAPERS

Figure 6. Surface plot of false accept risk as a function of TUR and EOPR.

Figure 7. Topographical contour map of false accept risk as a function of TUR and EOPR.

8. True Versus Observed EOPR
Until now, this discussion has been limited to
the concept of true EOPR. The idea of a
true EOPR implies that a value for reliabil-
ity exists that has not been influenced by any
non-ideal factors, but of course, this is not the
case. In the calibration laboratory, reliability
data is collected from real-world observa-
tions or measurements. The measurements
of UUTs are often made by comparing them
to reference standards with very low uncer-
tainty under controlled conditions. But even
the best available standards have finite uncer-
tainty, and the UUT itself often contributes
noise and other undesirable effects. Thus, the
observed EOPR is never a completely accu-
rate representation of the true EOPR.
The difference between the observed and
true EOPR becomes larger as the measure-
ment uncertainty increases and the TUR
drops. A low TUR can result in a significant
deviation between what is observed and what
is true regarding the reliability data [23, 28,
29, 30]. The reported or observed EOPR
from a calibration history includes all influ-
ences from the measurement process. In this
case, the standard deviation of the observed

Vol. 7 No. 1 March 2012 NCSLI Measure | 45

TECHNICAL PAPERS

Figure 8. Worst case false accept risk vs. TUR.

Figure 9. Worst case false accept risk vs. EOPR.

distribution is given by s obs = s uut 2 + s std 2
where and std are derived from statisti-
cally independent events. The corrected or
true standard deviation can be approximat-
ed by removing the effect of measurement
uncertainty and solving for s uut = s obs 2 - s std 2
where uut is the true distribution width rep-
resented by standard deviation.
The above equation shows that the stan-
dard deviation of the observed EOPR data is
always worse (higher) than the true EOPR as noise, and the EOPR is the measurand,
data. That is, the reliability history main-
tained by a laboratory will always cause the
UUT data to appear to be further dispersed
than what is actually true. This results in an
89 % observed EOPR boundary condition
where the PFA is less than 2 % for all pos-
sible values of TUR6 (Fig. 10).
If measurement uncertainty is thought of
then the observed data will have greater vari-
ability or scatter than the true value of the
6
When correcting EOPR under certain conditions,
low TUR values can result in imaginary values
for uut. This can occur where uut and std are
not statistically independent and/or the levels of
confidence associated with std and/or uut have
been misrepresented.

46 | NCSLI Measure www.ncsli.org

 TECHNICAL PAPERS

Figure 10. PFA assumes worst case TUR for true EOPR and observed EOPR.

EOPR. Measurement uncertainty always at face value (analogous to Type B or heu- One particularly useful method employ-
hinders the quest for accurate data; it never ristic evaluations). However, the influence of ing a guardbanding technique is described
helps. The true value of a single data point the measurement process is always present. in Method 6 of the Z540.3 Handbook [16,
can be higher or lower than the measured This method of removing measurement un- 25]. This method does not require EOPR
value, it is never known whether the mea- certainty from the EOPR data is a best esti- data to be available because it relies on using
surement uncertainty contributed a positive mate of the true reality or reliability which is worst-case EOPR, computed for a specified
error or negative error. Therefore, it is not sought through measurement. TUR value. Using this approach, a guard-
possible to remove the effect of measurement band multiplier is computed as a function of
uncertainty from a single measurement result. 9. Guardbanding TUR. The acceptance limits are expressed
However, EOPR data is a historical collection It is sometimes helpful to establish acceptance as follows: A = L MU95 , where A is the
of many pass/fail compliance decisions that limits A at the time-of-test that are more strin- newly established acceptance limits, L is the
can be represented by a normal probability gent than the manufacturers tolerance limits original tolerance limits, U95 is the expanded
distribution with a standard deviation obs. L. Acceptance limits are often called guard- measurement process uncertainty, and M is
Sometimes the measurement uncertainty std band limits or test-limits. It is only necessary the multiplying factor that yields a risk of a
will contribute positive errors and sometimes to implement acceptance limits A, which dif- specified maximum target. Figure 11 graphs
easurement uncertainty contributed
it will contribute a positive
negative errorIforthe
errors. negative
mean error. Therefore,
fer from it
the tolerance limits L, when the false guardband multipliers for varying levels of
to remove the effect eof
of these measurement uncertainty from a single
std errors is assumed to be zero, the
measurement
accept risk is higher than desired or as part risk. The risk level for Z540.3 is specified at
er, EOPR data is a historical collection of many pass/fail compliance decisionsto keep risk below a specified
effect of measurement uncertainty on a popu- of a program 2 % but could vary depending upon the agree-
presented by a normal probability distribution with a standard deviation .
measurementlation of EOPR data
uncertainty willcan be removed
contribute as previ-
positive errors andlevel. Acceptance
sometimes it limits may be chosen to ment with the customer. M2% was previously
negative errors.
ouslyIf the mean The
shown. of these errors isfunction
inverse normal assumedisto bemitigate
zero, therisk
effect
at either the bench level or the calculated by Dobbert [25] by fitting a line
t uncertainty on
useda population
to estimate of EOPR data can
obs from be removed
observed EOPR as previously
programshown.
level. PFA calculations may be used though the data points that mitigate risk to a
rmal function data
is used to
[31] estimate from observed EOPR data [31]
to establish acceptance limits based on the level of 2 % and is given by the following
mandated risk requirements. In most instanc- simplified formula
, (2)
es, where guard(2) bands are applied, the toler-
( ) M 2% = 1.04 - e0.38 ln(TUR)-0.54 . (3) (3)
ance limits are temporarily tightened or
where
esents the inverse -1 represents
normal distribution.the inverse normal dis- reduced to create acceptance limits needed to It can be seen that the line is a good fit for
tribution. meet a PFA goal. The subject of guardband- the condition where 1 TUR 15. The intent
s a numerical quantity
EOPRarrived at by statistical
is a numerical quantitymeans applied
arrived at to empirical data was to keep the equation simple while cover-
ing is extensive and novel approaches exist
Type A evaluation in the language of the GUM [32]. The
by statistical means applied to empirical data for establishing data comes from ing the range of TUR values that make physi-
urements made over time rather than from accepting manufacturers claims at face acceptance limits to mitigate
us to Type B or analogous to a Type AHowever,
heuristic evaluations). evaluation theininfluence
the lan- of the measurement
risk, even where EOPR data is not available cal sense. It has been shown in this paper that
ays present. This
guage method
of theofGUM removing measurement
[32]. The data comes uncertainty
from [25].from the EOPR in the simplified case of no
However, for TUR 4.6, PFA is always < 2 %. To verify
stimate of the true realitymeasurements
repeated or reliability which
madeisoversought through
time rath- measurement.
guardbanding, the acceptance limits A are set that TUR = 4.6 is a boundary condition, set
er than from accepting manufacturers claims equal to the tolerance limits L (A = L ). M2% = 0 and solve for TUR. It is worth noting
ing
helpful to establish acceptance limits A at the time-of-test that are more stringent
acturers tolerance
Vol.limits
7 No.L. 1Acceptance
March limits are often called guardband limits or
2012 NCSLI Measure | 47
only necessary to implement acceptance limits A, which differ from the tolerance
the false accept risk is higher than desired or as part of a program to keep risk
TECHNICAL PAPERS
Figure 11. Guardband multiplier for acceptable risk limits as a function of TUR.
that, for 4.6, the multiplier M2% is < 0. This
implies that a calibration lab could actually
increase the acceptance limits A beyond the
UUT tolerances L and still comply with the
2 % rule. While not a normal operating proce-
dure for most calibration laboratories, setting
guard band limits outside the UUT tolerance
limits is possible while maintaining compli-
ance with the program level risk requirement
of Z540.3. In fact, laboratory policies often
require items to be adjusted back to nominal
for observed errors greater than a specified
portion of their allowable tolerance limit L.
10. Conclusion and Summary
Organizations must determine if risk is to be
controlled for individual workpieces at the
bench level, or mitigated for the population of
items at the program level7. Computation of
PFA at the program level requires the integra-
tion of the joint probability density function.
The input variables to these formulas can be
reduced to EOPR and TUR. The 2 % PFA
maximum boundary condition, formed by ei-
ther a 4.6:1 TUR or an 89 % observed EOPR,
can greatly reduce the effort required to man-
age false accept risk for a significant portion
of the M&TE submitted for calibration. Ei-
ther or both boundary conditions can be lev-
eraged depending on the available data, pro-
viding benefit to practically all laboratories.
However, there will still be instances where
the TUR is lower than 4.6: 1 and the observed
48 | NCSLI Measure
EOPR is less than 89 %. In these instances,
it is still possible for the PFA to be less than
2 %. A full PFA computation is required to
show the 2 % requirement has not been ex-
ceeded. However, other techniques can be
employed to ensure that the PFA is held below
2 % without an actual computation.
There are six methods listed in the Z540.3
Handbook for complying with the 2 % false
accept risk requirement [16]. These methods
encompass both program level and bench
level risk techniques. This paper has specifi-
cally focused on some efficient approaches
for compliance with the 2 % rule, but it does
not negate the use of other methods nor imply
that the methods discussed here are necessar-
ily the best. The basic strategies outlined here
for handling risk without rigorous computa-
tion of PFA are:
Analyze EOPR data. This will most like-
ly be done at the instrument-level, as op-
posed to the test-point level, depending on
data collection methods. If the observed
EOPR data meets the required level of 89
%, then the 2 % PFA rule has been satisfied.
If this is not the case, then further
analysis is needed and the TUR must
be determined at each test point. If the
analysis reveals that the TUR is greater
than 4.6:1, no further action is neces-
sary and the 2 % PFA rule has been met.
If neither the EOPR nor TUR threshold
is met, a Method #6 guardband can be
applied.
Compliance with the 2 % rule can be ac-
complished by either calculating PFA and/or
limiting its probability to less than 2% by the
methods presented above. If these methods
are not sufficient, alternative methods of miti-
gating PFA are available [16]. Of course, no
amount of effort on the part of the calibration
laboratory can force a UUT to comply with
unrealistic expectations of performance. In
some cases, contacting the manufacturer with
this evidence may result in the issuance of
revised specifications that are more realistic.
Assumptions, approximations, estima-
tions, and uncertainty have always been part
of metrology, and no process can guarantee
that instruments will provide the desired ac-
curacy, or function within their assigned tol-
erances during any particular application or
use. However, a well-managed calibration
process can provide confidence that an in-
strument will perform as expected and within
limits. This confidence can be quantified via
analysis of uncertainty, EOPR, and false ac-
cept risk. Reducing the number of assump-
7
Bayesian analysis can be performed to determine
the risk to an individual workpiece using both the
measured value on the bench and program-level
EOPR data to yield the most robust estimate of
false accept risk [31].
www.ncsli.org

tions and improving the estimations involved during calibration can
not only increase confidence, but also reduce risk and improve quality.
11. Acknowledgements
The authors thank the many people who contributed to our under-
standing of the subject matter presented here. Specifically, the con-
tributions of Perry King (Bionetics), Scott Mimbs (NASA), and Jim
Wachter (Millennium Engineering and Integration) at Kennedy Space
Center were invaluable. Several graphics were generated using PTCs
MathCad 14. Where numerical methods were more appropriate,
Microsoft Excel was used incorporating VBA functions developed
by Dr. Dennis Jackson of the Naval Surface Warfare Center in Corona,
California.
12. References
[1] JCGM, International vocabulary of metrology Basic
and general concepts and associated terms (VIM), JCGM
200:2008, 2008.
[2] ANSI/NCSLI, Requirements for the Calibration of Measuring
and Test Equipment, ANSI/NCSL Z540.3:2006, 2006.
[3] D. Deaver and J. Somppi, A Study of and Recommendations
for Applying the False Acceptance Risk Specification of
Z540.3, Proceedings of the Measurement Science Conference,
Anaheim, California, 2010.
[4] H. Castrup, Risk Analysis Methods for Complying with
Z540.3, Proceedings of the NCSL International Workshop and
Symposium, St. Paul, Minnesota, 2007.
[5] M. Dobbert, A Pragmatic Method for Pass/Fail Conformance
Reporting that Complies with ANSI Z540.3, ISO 17025, and
ILAC-G8, Proceedings of the NCSL International Workshop
and Symposium, Providence, Rhode Island, 2009.
[6] ANSI/NCSLI, Calibration & Measurement & Test Equipment
- General Requirements, ANSI/NCSL Z540.1: 2002, 2002
[7] ISO/IEC, General requirements for the competence of testing
and calibration laboratories, ISO/IEC 17025:2005(E), 2005.
[8] A2LA, Specific Requirements: Calibration Laboratory
Accreditation Program, A2LA:R205, 2011.
[9] ILAC, Guidelines on Assesment and Reporting of Compliance
with Specification (based on measurements and tests in a
laboratory), ILAC-G8:1996, 1996.
[10] UKAS, The Expression of Uncertainty and Confidence in
Measurement (Appendix M), UKAS:M3003, 2007.
[11] ASME, Guidelines for Decision Rules: Considering
Measurement Uncertainty in Determining Conformance to
Specifications, ASME B89.7.3.1-2001, 2001.
[12] ISO, Geometrical Product Specifications (GPS) - Inspection by
measurement of workpieces and measuring equipment - Part 1:
Decision Rules for proving conformance or non-conformance
with specifications, ISO-14253-1:1998(E), 1998.
[13] ASME, Measurement Uncertainty Conformance Testing: Risk
Analysis, ASME B89.7.4.1-2005 (Technical Report), 2006.
Vol. 7 No. 1 March 2012 NCSLI Measure |
TECHNICAL PAPERS
[14] NCSLI, Response to NASA Interpretation Request (IR2),
NCSLI 174 Standards Writing Committee, March 2008.
[15] S. Mimbs, Request for Interpretation; Correspondence Letter,
July 2007.
[16] ANSI/NCSLI, Handbook for the Application of ANSI/NCSL
Z540.3:2006 - Requirements for the Calibration of Measuring
and Test Equipment, ANSI/NCSL Z540.3 Handbook, 2009.
[17] J. Bucher, ed., The Metrology Handbook, American Society for
Quality, Measurement Quality Division, ASQ Quality Press,
ISBN 0-87389-620-3, 2004.
[18] J. Hayes, Factors Affecting Measuring Reliability, U.S.
Naval Ordnance Laboratory Technical Memorandum No. 63-
106, October 1955.
[19] NCSLI, Response to NASA Interpretation Request (IR1),
NCSLI174 Standards Writing Committee, March 2008.
[20] M. Nicholas and L. Anderson, Guardbanding Using Automated
Calibration Software, Proceedings of the NCSL International
Workshop and Symposium, Salt Lake City, Utah, 2004.
[21] Fluke Corporation, Calibration: Philosopy in Practice, ISBN:
978-0963865007, May 1994.
[22] T. Skwircznski, Uncertainty of the calibrating instrument,
confidence in the measurement process and the relation between
them, International Organization of Legal Metrology (OIML)
Bulletin, vol. XLII, no.3, July 2001.
[23] NASA, Estimation and Evaluation of Measurement Decision
Risk, NASA Measurement Quality Assurance Handbook
ANNEX 4, NASA-HDBK-8739.19-4, July 2010.
[24] M. Dobbert, Understanding Measurement Decision Risk,
Proceedings of the NCSL International Workshop and
Symposium, St. Paul, Minnesota, 2007.
[25] M. Dobbert, A Guard Band Strategy for Managing False
Accept Risk, Proceedings of the NCSL International Workshop
and Symposium, Orlando, Florida, 2008, .
[26] A. Eagle, A Method for Handling Error in Testing and
Measuring, Industrial Quality Control, March 1954.
[27] F. Grubbs and H. Coon, On Setting Test Limits Relative to
Specification Limits, Industrial Quality Control, March 1954.
[28] S. Mimbs, Measurement Decision Risk - The Importance
of Definitions, Proceedings of the Measurement Science
Conference, Anaheim, California, 2008.
[29] J. Ferling, The Role of Accuracy Ratios in Test and
Measurement Processes, Proceedings of the Measurement
Science Conference, Long Beach, California, 1984.
[30] I. Lira, A Bayesian approach to the consumers and producerss
risk in measurement, Metrologia, vol. 36, pp. 397-402,
October 1999.
[31] H. Castrup, Analytical Metrology SPC Methods for ATE
Implementation, Proceedings of the NCSL Workshop and
Symposium, Albuquerque, New Mexico, 1991.
[32] ISO/IEC, Uncertainty of measurement -- Part 3: Guide to the
expression of uncertainty in measurement, ISO/IEC Guide 98-
3:2008, 2008.
49
TECHNICAL PAPERS

Wattmeter Calibration for Use in

Standby Power Testing to IEC 62301
Ilya Budovsky and Dimitrios Georgakopoulos

Abstract: A growing number of household appliances include a standby mode, in which the appliance consumes a small
amount of electrical power. Standard 62301 of the International Electrotechnical Commission (IEC) describes the measurement
of standby power and the requirements for wideband wattmeters used for this purpose. These requirements include the ability to
measure power with uncertainty of less than 0.01 W for current waveforms having crest factors up to 10. The National Measure-
ment Institute, Australia, has developed a system for the calibration of wideband wattmeters used in the measurement of standby
electrical power in accordance with IEC 62301. The system has uncertainties significantly lower than those required by the stan-
dard and has been tested with current waveforms having crest factors up to 20.

1. Introduction
To reduce power consumption and be
convenient for the user, many household
electrical appliances now have a standby
mode. The standard 62301 of the International
Electrotechnical Commission (IEC) [1]
defines the standby mode as the lowest
power consumption mode which cannot be
switched off (influenced) by the user and that
may persist for an indefinite time when an
appliance is connected to the main electricity
supply and used in accordance with the
manufacturers instructions.
Although the average power consumed
by an appliance in standby mode is small,
the current can often take the form of
pulses or spikes. The crest factor of the
current waveform, defined as the ratio of the
maximum to the rms value of the waveform,
can be up to three and in some circumstances
as high as 10 [1]. Figure 1 shows, as an
example, the measured standby current of
a battery charger. The crest factor of this
waveform is 4.6. IEC 62301 describes the
measurement of standby power of household
electrical appliances and specifies the
requirements for wideband wattmeters used
for this purpose. A wattmeter must be capable
of measuring electric power with crest factors
of the current waveform up to 10.
The measurement uncertainty must be less
than 2 % for measured power above 0.5W and
less than 0.01W for measured power below
0.5W. To achieve this, the performance of the
wattmeter must be traceably verified for the
specific conditions described in IEC 62301.
Therefore a calibration system capable of
accurately measuring power under these
conditions is necessary.
This paper reports on a measurement
system developed at the National Measure-
ment Institute, Australia (NMIA), for the
calibration of wattmeters used to measure
the power consumed by electrical appliances
in standby mode. Current waveforms that
comply with IEC 62301 are discussed, the
measurement setup is described and results of
the evaluation of the new system are given
and summarized in an uncertainty budget.
2. Calibration Waveforms
The test voltage waveform specified in IEC
62301 is a sine wave with maximum distor-
tion of 2 % (up to and including the 13th har-
monic, which corresponds to a frequency of
650 Hz for a 50 Hz power system and 780
Hz for a 60 Hz system) and a crest factor
from 1.34 to 1.49 (note that the crest factor
of a pure sinusoidal waveform is 1.41). Such
amount of distortion usually has insignificant
influence on the errors of the wattmeter under
test because the total power caused by har-
monics contained in both voltage and current
is small (less than 2 %). Hence, in our system,
we use a sinusoidal voltage of low total har-
monic distortion (typically less than 0.02 %).
The standard specifies the current
waveform in terms of its crest factor. This
specification does not define uniquely the
waveform in the time (or the frequency)
domain. For example, Figure 2 shows two
waveforms: a sinc waveform, sin(t)/(t),
and a pulse waveform. They both have a crest
factor of about 3.57 and the same magnitude
of the fundamental but different harmonic
content, peak values and rms values, when
the harmonics are included. The Isinc current
waveform has a band-limited spectrum, while
the spectrum of the Ipulse current waveform
is much broader. Both signals comply with
the crest factor requirement of IEC 62301
but a wideband signal such as Ipulse is a more
challenging test for the sampling mechanism
(signal conditioning electronics and sampling
rate of the sampling and digitizing electronics)
of the wattmeter current channel.
We have chosen to use a pulsed current
waveform to thoroughly test the calibration
system. The pulsed current waveform has
the additional advantage that it resembles
typical current waveforms of appliances more
closely, particularly those with switch-mode
power supplies. Furthermore, a crest factor
of 10 is more easily achieved with a pulsed
current waveform.
Figure 3 shows the current waveform
used in this work for unity power factor (zero
angle between the fundamentals of voltage
and current). The peaks of the pulse are
Authors
Ilya Budovsky
Dimitrios Georgakopoulos
National Measurement Institute
Bradfield Road, West Lindfield,
NSW 2070, Australia
ilya.budovsky@measurement.gov.au

50 | NCSLI Measure www.ncsli.org

 TECHNICAL PAPERS

(a) Time domain (b) Spectrum

Figure 1. Measured standby current of a battery charger.

1.5 0.18
I sinc 0.16 Pulse

magnitude
1
1.5 0.18
0.14 Sinc
II pulse
sinc 0.16
0.12 Pulse

magnitude
0.5
Amplitude
1
0.14
0.1 Sinc
I pulse
0 1.5
0.5 0.12
0.08 0.18
Amplitude

Harmonic
0 0.2 0.4 0.6 0.8 1I sinc 1.2 0.1 0.16
0.06 Pulse

Harmonic magnitude
-0.5
0 1
0.08
0.04 0.14

Harmonic
0 0.2 0.4 0.6 0.8 1I pulse 1.2 Sinc
-1 0.5 0.06 0.12
0.02
-0.5
Amplitude

0.04
0 0.1
-1.5
-1 0 0.02 0.08
0 20 40 60 80 100
0 0.2Normalised
0.4 0.6
time (s)0.8 1 1.2 0 0.06
-1.5 -0.5 Harmonic number
0
0.04 20 40 60 80 100
Normalised time (s)
(a) Time -1
domain. (b) Frequency
0.02
0
domain
Harmonic number

Figure
(a) Time2.domain.
-1.5
CurrentNormalised
waveforms
time (s)
with the same crest factor
(b) and
Frequency 0 rms value
domain
20 as the 60fundamental.
40 80 100

Harmonic number
Figure 2. Current waveforms with the same crest factor and rms value as the fundamental.
Figure
(a) Time3domain.shows the current waveform used in this work for unity
(b) Frequency domainpower factor (zero angle
between
Figure
Figure the fundamentals
2.3 Current of voltage
waveforms
shows the current andsame
with the
waveform current).
in The
usedcrest peaks
factor
this work ofrms
andfor the pulse
value
unity asare
powerthecentred on T/4angle
fundamental.
factor (zero and
3T/4,
betweenwhere T is the period
the fundamentals of of the test
voltage andsignal. Different
current). The peaks values of power
of the pulse areandcentred
poweron factor are
T/4 and
achieved
3T/4, Figure
where by varying
T3 is thethe
shows themagnitude
period current of I1 and
of thewaveform
test its phase
used
signal. relative
in this
Different work
valuesto for
the voltage
of unity
powerpower waveform.
and factor factor
power (zero ang
are
between
achieved bythe fundamentals
varying the magnitudeof voltage
of I1 and
and current).
its phase The peaks
relative to of
thethe are centred on T/4 a
pulsewaveform.
voltage
3T/4, where T is the period i(t) of the test signal. Different values of power and power factor a
t t
T t T
achieved by varying the magnitude of I1 andt its 4phase
p

relative
p
2 4 2to the voltage waveform.
i(t)
1 2

(a) Time domain Im

(b) Frequency domain t13 T3T p p
44 2 2
tt t t
t 2t 4T3T p p
4 4 2 2
Im i(t) t p 3tT t t t t
t3 2 1 p 4 3 t 4 3T p
4 2t 4
t
t2
t p t12 T
t1 4 t 4 2t3 t2 T p
p

Figure 2. Current waveforms with the same crest factor and rms value as the fundamental. 4 2
Im t1 t2 tt 3 3T t t
p 4
T
t 4 3T
ttp
3 4 2 4 2
t1 t2 tt 3 t
p 2 t1 t 4t4 t3 T t
centered on T/4 and-Im 3T/4, where T is the period of the test signal.
Different values of t1 power
t2 t3 t4 achieved
T t by varying
-Im power and factor are
Figure
the3.magnitude
Current excitation
of I1 and waveform
its phasefor the calibration
relative of power
to the voltage meters to IEC 62301.
waveform.
Figure 3.With
Current
reference to-IFig.
excitation mwaveform for show
3, we can the calibration of power
that the rms valuemeters
of theto IEC 62301.
cur-
With reference to Fig. 3, we can show that the rms value of the current waveform is
rent waveform is (see appendices)
(appendix):
Figure
With 3. Current
reference excitation
to Fig. 3, we waveform
can showfor that thethecalibration
rms value of power
of themeters to IEC
current 62301.is
waveform
(appendix): tp
With reference to Fig. 3, we can Ishow rms I mthat 2 the rms value of the current
tT
(1) waveform (1)
(appendix): I rms I m 2 p (1)
and the crest factor is: T
and the crest
and the crest factor is: factor is: t
I Tp
CF I rmsm Im 2 T . (2)(
IIrms
m 2Tt p
and the crest factor is: CF . (2) (2)
For the waveform shown in Fig. 3, when the I rms voltage 2t p waveform is a sinusoidal waveform in
Im T
phase
Forwith
the the current waveform,
waveform shown in Fig.the 3,
average
whenCFactive
voltage
the power
waveform is. (appendix):
is a sinusoidal waveform in(
phase with theForcurrent
the waveform
waveform,shown 3,I when
in Fig.active
the average rms power 2t pisvoltage
the waveform is a
(appendix):
Forsinusoidal
the waveform shown in
waveform in phase
Fig. 23,with
whenthe the voltage
waveform,
current waveform theis a average
sinusoidal waveform
Pthe V I sin
average (3)
phase with thepower
active currentiswaveform, 2 m m active
(see appendices) 2(CF ) 2 is (appendix):
power
P Vm I m sin 2
(3)
where Vm is the amplitude of the fundamental of the2(CF )
sinusoidal voltage and Im is the peak of the
pulsed
where Vcurrent waveform. 2
m is the amplitude of the fundamental
P Vm Iof the sinusoidal
m sin voltage and Im is(3)
2
the peak of the(
pulsed current waveform. 2(CF )
where Vm is the amplitude of the fundamental of the sinusoidal voltage and Im is the peak of t
Figure 3. Current excitation waveform for the calibra- pulsed where
currentVwaveform.
m
is the amplitude of the fundamental of the sinusoidal voltage
tion of power meters to IEC 62301. and I m
is the peak of the pulsed current waveform.

Vol. 7 No. 1 March 2012 NCSLI Measure | 51

TECHNICAL PAPERS
Figure 4. Block diagram of the system for the calibration of wattmeters used in standby power measurements.
3. Calibration System
The measurements are based on a high frequency thermal power
comparator (TPC) developed at NMIA [2]. Figure 4 shows the block
diagram of the system and Fig. 5 shows the physical layout of the
measurement section.
The dual channel voltage source contains two arbitrary waveform
generators (AWG) that generate voltage and current waveforms. For
the generation of the voltage waveform, the output of the corresponding
AWG is connected to a low noise voltage amplifier. For the generation
of the current waveform, the corresponding AWG output is connected
to a transconductance amplifier (TCA) with 100 kHz bandwidth.
The TPC (Fig. 6) uses multijunction thermal voltage converters
and is, therefore, free of the accuracy limitations due to aliasing of
the current and voltage waveforms that are common to sampling watt-
meters. The output of the TPC, measured by digital voltmeter DVM1,
is proportional to the difference between the unknown ac power ap-
plied to inputs ACV and ACI of the TPC, and the known dc power
generated by two dc sources (DS1 and DS2 in Fig. 4) and measured by
DVM2 and DVM3. The ac voltage applied to the unit under test, UUT,
is converted to the input level of the TPC using a voltage divider, VD.
The VD can be either inductive or resistive. The current is convert-
ed to voltage using wideband current shunts of NMIA design [3, 4].
4. Traceability and Uncertainty Analysis
The measurement system has been characterized for amplitude and
phase up to 200 kHz and is traceable to the Australian national stan-
dards of voltage, current, resistance, electrical power and frequency.
The ac-dc difference and the phase shift of the current shunts were
measured at frequencies up to 1 MHz and 200 kHz, respectively, using
the techniques described in [4] and [5]. The inductive voltage divider
was calibrated using the method described in [6]. The typical in-phase
52 | NCSLI Measure
and quadrature errors in the output voltage are of the order of 0.01
V/V and 0.1 rad, respectively, at power frequencies. The resistive
voltage divider can be used to measure voltages up to 240 V at fre-
quencies up to 200 kHz. Its characterization is described in [7].
Table 1 shows the uncertainty budget (reporting relative uncer-
tainty) of the system at CF = 3 and unity power factor, calculated
in accordance with ISO guidelines [8]. Table 2 shows the absolute
uncertainties for the calibration of a commercial wattmeter at different
crest factors and phase angles between the fundamentals of voltage
and current. The uncertainties required by the IEC 62301 for the cor-
responding power levels are included.
5. Conclusion
A traceable measurement system for calibration of wattmeters used
in standby electrical power testing has been developed. The measure-
ment system is based on a thermal power comparator, precision cur-
rent shunts, inductive or resistive voltage dividers and precision am-
plifiers. Each of these components has been evaluated in a traceable
way. The expanded uncertainty of the system is better than 100 W/
VA at a crest factor of three. The system has been tested for current
waveforms with crest factors up to 20. When the system is used for
calibrating power meters according to the IEC 62301, the uncertainty
varies from 0.0005 W to 0.06 W, which is significantly smaller than
the requirements for the corresponding applied power.
6. References
[1] IEC, Household electrical appliances Measurement of
standby power, IEC 62301, ed. 1.0, 2005.
[2] I. Budovsky, A. M. Gibbes and D.C. Arthur, A high-frequency
thermal power comparator, IEEE Trans. Instrum. Meas., vol.
48, pp. 427-430, 1999.
www.ncsli.org
 TECHNICAL PAPERS

Figure 5. Calibration of a wideband wattmeter.

Figure 6. Schematic diagram of the thermal power comparator circuit. TC1 and TC2 are dualheater multijunction thermal
converters.

Distribu- Ui ui ciui
Component ki ci ui
tion (W/VA) (W/VA) (W/VA)

TPC AC-DC difference Normal 67 2 1 33.6 33.6 30

Shunt Normal 10 2 1 5.0 5.0 30
Resistive voltage divider Normal 20 2 1 10.0 10.0 30
DVM1 Normal 8 1 1 8.0 8.0 30
DVM2 Normal 8 1 1 8.0 8.0 31
Type A Normal 50 2 1 25.0 25.0 19
Source adjustment Normal 10 2 1 5.0 5.0 30
Combined standard uncertainty, uc (W/VA) 45.1
Effective degrees of freedom, ueff 65
Coverage factor, k 2.00
Expanded uncertainty U = kuc
(W/VA) 90.1
Table 1. Uncertainty budget of the NMIA measurement system for a crest factor of three and unity power factor.

Vol. 7 No. 1 March 2012 NCSLI Measure | 53

TECHNICAL PAPERS

Uncertainty
Current Current Nominal
Current Phase Angle Uncertainty required
Range Applied Power
Crest Factor (degrees) (W) by IEC62301
(Apk) (Apk) (W)
(W)

3 0 112.564 0.06 2.3

3 + 60 56.282 0.03 1.1
5 3 3 - 60 56.282 0.04 1.1
5 0 40.703 0.03 0.81
10 0 10.182 0.06 0.20
3 0 22.513 0.012 0.45
3 + 60 11.256 0.006 0.23
3 - 60 11.256 0.006 0.23
1 0.6
5 0 8.141 0.005 0.16
10 0 2.036 0.012 0.04
20 0 0.509 0.006 0.01
3 0 5.628 0.003 0.11
3 + 60 2.814 0.002 0.06
3 - 60 2.814 0.002 0.06
0.2 0.15 5 + 90 0 0.001 0.01
5 - 90 0 0.001 0.01
10 0 0.509 0.003 0.01
20 0 0.127 0.001 0.01
3 0 1.126 0.0008 0.02
3 + 60 0.563 0.0006 0.01
3 - 60 0.563 0.0006 0.01
5 0 0.407 0.0006 0.01
0.05 0.03
5 + 90 0 0.0006 0.01
5 - 90 0 0.0005 0.01
10 0 0.102 0.0008 0.01
20 0 0.025 0.0005 0.01
3 0 0.188 0.0005 0.01
3 + 60 0.094 0.0005 0.01
3 - 60 0.094 0.0005 0.01
5 0 0.068 0.0005 0.01
0.01 0.005
5 + 90 0 0.0005 0.01
5 - 90 0 0.0005 0.01
10 0 0.017 0.0005 0.01
20 0 0.004 0.0005 0.01

Table 2. Typical uncertainties of the calibration of commercial wattmeter using the NMIA system compared to the IEC 62301.

[3] I. Budovsky, Standard of electrical power at frequencies up to [7] I. Budovsky, A.M. Gibbes and G.M. Hammond, Voltage
200 kHz, IEEE Trans. Instrum. Meas., vol. 58, pp. 1010-1016, divider characterization at frequencies up to 200 kHz, CPEM
2009. 2000 Digest, Sydney, pp. 662-663, May 2000.
[4] I. Budovsky, A micropotentiometer-based system for low- [8] ISO/IEC, Uncertainty of Measurement part 3: Guide to
voltage calibration of alternating voltage current standards, the Expression of Uncertainty Measurement, ISO/IEC Guide
IEEE Trans. Instrum. Meas., vol. 46, pp. 356-360, 1997. 98:2008, 2008.
Appendix
[5] I. Budovsky, Measurement of phase angle errors of precision
current shunts in the frequency range from 40 Hz to 200 kHz, 7. Appendices
IEEE Trans. Instrum. Meas., vol. 56, pp. 284-288, 2007. A.1
A.1 Timing
Timingcharacteristics
characteristicsofofaapulsed
pulsedcurrent
currentachieving
achievingmaximum
maximumaverage power
sinusoidal voltage for a given phase shift
[6] I.F. Budovsky, G.H. Small, A.M. Gibbes and J.R. Fiander, average power with a sinusoidal voltage for a given phase shift
Calibration of 1000 V / 50 Hz inductive voltage dividers and With
With reference
referencetotoFig.
Fig.2,3,
ratio transformers, CPEM 2004 Digest, London, pp. 322-323, tp tp tp t
t 4 T T , t 3 T T
, and t1 T T p
June 2004. 2 2 t2 T T 2 2
Find , to maximize the average electrical power.
T
1
54 | NCSLI Measure P v(t )i (t )dt www.ncsli.org
T0
t2 t4
 A.1 Timing characteristics of a pulsed current achieving maximum average power with a
ics of a pulsed
A.1current
A.1 Timing
Timingachieving maximum
characteristics
characteristics of aaaverage
ofphase
pulsed
pulsed power with
current
current a
achieving
achieving maximumaverage
maximum averagepower
powerwith
withaa
sinusoidal voltage for a given shift
given phase shift
sinusoidalvoltage
sinusoidal voltagefor
foraagiven
givenphase
phaseshift
shift
With reference to Fig. 2, TECHNICAL PAPERS
Withreference
With reference t to toFig.
Fig.2,2, t t t
tp t4 T T tp tp , t3 T T tttpp 2 , t2 T T tp and t1 T T ttp
T , t TTTT t pp2and
2 t2find t T
2 2 ,, tt331TTthe
T
TT pp 2 ,, tt2TTTTt pp2 and and tt1TTTT pp2
Find , Tto
t44 T,
to maximize
maximize
2 average
the average 22electrical
electrical
2 power.
power. 22 1 22
e average electrical Find
Find power.
T, ,to tomaximize
maximizethe theaverage
averageelectrical
electricalpower. power.
1
P 1TTv(t )i (t )dt
T1
P 0vv((tt))ii((tt))dt
P dt
Tt 2T 00 t4
1 1
11t22t 2Vm I m sin t dt 11t4t 4Vm I m sin t dt
4

Vm I m sin t dtTT t1VVmmIImmsin sinttdt
dtT t3VVmmIImmsin ttdt
sin dt
3 T t11t1 TT t33t3 V I
V I P Vm Im cost 4 cost3 cost 2 cost1 (A.1) (A.1)
P m m cost 4 cost3 cost 2 cosPV t mI m cost cost (A.1)

2 P 12mm cos t 44 cost33 cos tt22cos
cos cos t1t1 (A.1)
(A.1)
But 22
But
But
But 1 1 t
1 cos 1 t4 cos t3 2 sin 11t 4 t3 sin t p 1t4 t3 2 sin 2 1 sin ttp (A.2)
2 sin t 4 t3 cos sin
cos
2tt444cos
t t
cos
3tt3322sin
2 sin 2
sin 2 1 sin
tt44tt33sin
sin 1
2
tt44tt3322sin
(A.2) sin2211sin sin Tpp (A.2)
2 T (A.2) (A.2)
2 2 22 T T
Similarly,
Similarly,
Similarly,
Similarly, tp
tcos p t 2 cos t1 2 sin 1 sin tt p (A.3)
cost 2 cost1 2 sin 1 sin cos

cos tt22coscos t1t122sin (A.3) sin Tp
sin11sin (A.3)
(A.3) (A.3)
T
From (A.1), (A.2) and (A.3), TT
.3), From(A.1),
From
From (A.1),(A.2) (A.2)
(A.2)and and (A.3),
and(A.3),
(A.3), V I tp
V I tp P m m sin sin 1 sin 2 1 (A.4)
P m m sin sin 1 sin 2 1 P VVmmIImm sin tTtppsin 1 (A.4)sin 2 1
T P sin sin 1 sin 2 1 (A.4)
(A.4) (A.4)
T T
brackets

For maximum power, the term in the square of (A.4) must be maximum, which happens
term in the square brackets of
Formaximum
maximum (A.4)the
power, must
the term bein maximum,
inthe
thesquare
square which happens
brackets of(A.4)
(A.4)must
For power, term brackets of 1mustbe bemaximum,
maximum,
1 whichhappens
which happens
For maximum
when sin 1 power, 1 and the
1 sin term2 1in1the square
1 , that brackets
is whenof (A.4) 1 and must be
1maximum,
. which happens when sin p (1 - a ) = 1 and
sin 2 1 when1 , sin
when sin1111and
that is when and sin22114111,,that
and . thatisiswhen
when 1
2 and and 4 .. 1
2 sin
( ) on 322T . t1 and 4t42 must be centered on T and t3, t4 centered on 3T .
1
Therefore,
sin 2 p 1 - t1band
and t3, tt4t1centred
be centred on Therefore,
TTherefore,
= t-1
2 must
, thatT
beiscentred
on 3 becentred .
when aon=T 4 and and t3,bt4=centred
. Therefore, 4
4 and t must
1 and t22must be 4 centredon on TT2 and andtt33,,t4t4centred
centred
4 onon33TT .. 4 4
4 44 4 44
A.2 Average active power of a sinusoidal voltage and an in-phase pulsed current
A.2 Average active power of a sinusoidal voltage and an in-phase pulsed current
The
er of a sinusoidal
A.2 average
voltage active
Averageand
A.2Average active
active power
an power isofdefined
in-phase
powerof apulsed as voltage
current
asinusoidal
sinusoidal voltage
T andan
and anin-phase
in-phasepulsed
pulsedcurrent
current
1
active power is defined as P 11TTv(t )i (t )dt . For a sinusoidal voltage and the pulsed
T
1
The average
r is defined asThePThe
T0
average v(t )i (active
average tactive
)dt . For a sinusoidal
power
power is
is defined
defined voltage
as
as PPand
T 0vthe
T 0
v((tt))ipulsed
i((tt))dtdt..For
Foraasinusoidal
sinusoidalvoltagevoltageandandthe
thepulsed
pulsed
current of Figure 2, the average power can be 0written as: T
verage power current
can
For be
current Twritten
a sinusoidal
ofFigure
of Figureas:2,
voltage
2,the and the
theaverage
average pulsed
power
power cancurrent
can bewritten
be written of Figureas: 2, the average power can be written as:
as:
2 TT22 Vm I m
V I P 2 2Vm I m sin t dt V I cos t2 cos t1
m m cos t2P cos 2 t sin t dt VmmI mm cos t cos t
P T 0VVmm1IImmsin t dt cost22 cost11
TT 00
V I 1 1
Vm I m 1 1 P 2 VVmmIImm sin 11t 2 t1 sin 11t 2 t1 (A.5)
2 sin t 2 t1 sin tPP
2
22t1

m m sin
sin

2
tt t t


(A.5)
sin
sin
2
tt tt
(A.5)
(A.5) (A.5)
But 2 2
22
2 1 2 1
22
2 1 2 1

1 1 t 1
sin t2 t1 sin 1 and sin t2 t1 sin p sin
But Hence from
2
But 2 2 2 T 2(CF )
But
(A.5) 1the averagepowercan be written 1as: t 1
sin t2 t1 sin 1 and sin t2 t1 sin t p sin Hence from
12 2 12 Vm I m Tp 2(CF 1 )2 (A.6)
sin t2 t1 sin 1 and sin P 2t2 t1 sin
sin 2 sin Hence from (A.6)
2
(A.5) 2 2
the average power can be written 2
as: 2(CF T
) 2 (CF )
Hencethe
(A.5) from
average(A.5)power
the average power as:
can be written can be written as:
Vm I m
P2 sin (A.6)
V I 2
P 2 m m sin 2(CF )2 (A.6)
2(CF )
A.3 RMS value of a pulsed waveform and relation of the CF factor to the timing
characteristics
A.3 RMS valueofofa pulsed a pulsed signal
waveform and relation of the CF factor to the timing characteristics of a pulsed signal
A.3 RMS value of a pulsed waveform and relation of the CF factor to the timing
With reference to Fig. 2,
characteristics
A.3
With RMS value to
reference of Fig.
a pulsed
3, signalwaveform and relation of the CF factor to the timing
characteristics of a pulsed signal
to Fig. 2, T 4 p 2
t t
With reference
T 3T p
4 2

1 1

T 0 T T ttpp t m or
2 2
With
I rms reference i t to Fig.
dt 2, I 2
m dt I dt
T T4 2 33TT tpp
44 22
1 1 T 4t p 2 2
T i t dt 3T p I dt or
t
1 1 m mt p
2 2
I rms I dt
tI m I 2 2 dt or
4 2 4 2
I rms T 0it dt T T t p I m2 dt I rms (A.7)
3T p
2 (A.7)
2 m T
T 0 T T 4 t p 2 4
t
4

2 I m 3T 4 p 2 t p the CF can be written as
The crest factor
the crest factor is is defined
definedas asCF I I .Using
CF mI rms . I
Using2
(A.7)
(A.7) the CF can be written as (A.7)
I rms m t
Tp
I rms I m 2
rms
(A.7)
T I m CF T T (A.8) (A.8)
The= crest
CF factor
. is defined as CF .Using (A.7) the CF can be written as
2t p I I rms 2t p
The crest factor is defined as CF m .Using (A.7) the CF can be written as
I rms T
CF (A.8)
2Tt p (A.8)
CF
2t p

Vol. 7 No. 1 March 2012 NCSLI Measure | 55

TECHNICAL PAPERS

Experimental Study and Computer

Modeling of the Triple Point of
Argon System
R. Ding, M. J. Zhao, T. Nielson, E. Nerdrum, and D. Farley

Abstract: The triple point of argon is a defined fixed point in ITS-90 for calibration of standard platinum resistance thermome-
ters (SPRTs). A new triple point of argon system with multiple re-entrant wells was developed. This system was tested to evaluate
the duration and quality of the argon triple-point temperature plateau. The testing results showed that the plateau can be as long
as 100 hours with the temperature change less than 0.05 mK. The re-entrant wells uniformity testing showed that the temperature
is consistent among the multiple re-entrant wells. Uncertainty analysis shows that the combined uncertainty of the argon system
is 0.26 mK (k = 2). In order to study the heat transfer process and the influence on the thermal equilibrium of the argon system
during the realization of the triple-point of argon plateau, finite element analysis (FEA) modeling was carried out to simulate the
thermal conductivity, convection, and radiation inside the argon system. The FEA simulation results are described and discussed
in this paper.
1. Introduction argon gas, a vacuum shield, a heater shield with a 12 W flexible heater
The triple point of argon (T90 = 83.8058 K) is one of the most attached on the outside surface of the shield, and a central argon cell
commonly used fixed points for the calibration of standard platinum with four stainless steel re-entrant wells. The immersion depth of the
resistance thermometers (SPRTs) at low temperatures below 273.16 re-entrant wells is 160 mm. The inside diameter of the re-entrant wells
K. In the past twenty years, a few argon apparatus for realization of is 8.0 mm. High-purity argon (certified purity: 99.9999 %) is used in
the triple point of argon were developed by other researchers [1-5]. the argon system. A total of 13.3 moles of argon is permanently sealed
In order to calibrate SPRTs at the triple point of argon efficiently and inside. The pressure of the argon system is 1.172 103 kPa at room
accurately, a new triple point of argon system based on the design of temperature. A PRT sensor is attached at the inside surface of the heater
the argon triple point apparatus by NIST [1] with multiple re-entrant shield. This sensor, along with the flexible heater and the temperature
wells and accurate triple-point plateau temperature was recently controller, is used to control the temperature of the heater shield.
developed at Fluke Corporation. This newly developed argon system When inserting SPRTs into the re-entrant wells or when moving
was tested under different experimental conditions. The immersion SPRTs during the calibration process, helium gas flows through the re-
profile of the re-entrant wells was measured and compared with the entrant wells to prevent SPRTs from getting stuck inside the re-entrant
ITS-90 theoretical hydrostatic values. The temperature consistency wells due to condensation and freezing of moisture in the air. The
among re-entrant wells was also checked. The experimental results volume of the liquid nitrogen Dewar inside the triple point of argon
are reported and the influence of different experimental parameters on system is 40 L. After the Dewar is filled, the liquid nitrogen can last up
the performance of the argon system is discussed in the paper. to 20 hours with the argon system operating at the triple-point plateau.
In order to study the heat transfer process and the influence on the
thermal equilibrium of the argon system, finite element analysis (FEA) 3. Experimental Procedures and Results
modeling was carried out to simulate the complex heat transfer process The first step for realization of the triple point of argon is to fill liquid
that includes heat conductivity, convection, and radiation occurring nitrogen into the argon system one day in advance to freeze the argon
in the argon system. Based on the simulation results, suggestions are system. It takes about 15 minutes to fill the liquid nitrogen Dewar. The
provided in order to quickly reach the thermal equilibrium and obtain argon gas inside the two argon gas cylinders will be condensed into
a long, stable plateau. the central argon cell overnight. The temperature can be read from the
controller or the pre-inserted monitoring SPRTs. The second step is to
2. Introduction of the Triple Point of Argon System raise the argon system temperature to a certain point (e.g. -190.0 C)
Figure 1 is the schematic drawing of the triple point of argon system. that is below, but close to, the triple point of argon (-189.3442 C).
The main structure of this system includes a high-accuracy temperature The system is left at this temperature for about two hours to stabilize
controller with a resolution of 1.0 mK for the temperature setting point, the system. The temperature is then raised to about 1.5 to 2.0 C above
a stainless steel liquid nitrogen Dewar vessel, two argon gas reservoir the triple point of argon (-187.344 C) to pre-melt the outer layer of
cylinders with the volume of 13.4 L for each cylinder for storage of the argon cell. The pre-melting time is about one hour. The fourth

56 | NCSLI Measure www.ncsli.org

 TECHNICAL PAPERS

Re-entrant wells To vacuum pump and helium gas

To liquid nitrogen dewar

Ar cell Heater shield

Vacuum shield

Argon gas Argon gas

cylinder cylinder

1 2
Liquid nitrogen

Figure 1. Schematic drawing of the triple point of Figure 2. Plateau with the maintenance temperature
argon system. 20 mK above the triple point of argon.

step is to drop the argon system temperature In order to investigate the influence of
to a maintenance temperature that is about the maintenance temperature on the triple-
20 to 60 mK higher than the triple point of point plateau of the argon system, a few
argon. The argon system will remain at the experiments were carried out at different
triple point of argon plateau for a certain maintenance temperatures. Figure 2 shows
period (from a few hours up to a few days the tested plateau with the maintenance
depending on the conditions) until the three- temperature 20 mK above the triple point of
phase equilibrium at the triple point of argon argon. It can be seen that the plateau duration
is broken because of melting and vaporization is over 100 hours with the temperature change
of solid and liquid argon. less than 0.05 mK.
In this study, standard platinum resistance The immersion profile of the argon system
thermometers (Fluke SPRT models 5681 and re-entrant wells was tested in this study. One
5683) were used to measure the triple point 25 SPRT (Fluke model 5681) was placed
of argon melting plateaus. A resistance bridge at the bottom of a re-entrant well. It was
(Measurements International DC Automated measured with currents of 1.0 mA, 1.414 mA,
Thermometry Bridge, model 6010T, and 1.0 mA again to correct for self-heating.
accuracy 0.05 ppm) was used to measure After testing with the SPRT at the bottom
the resistances of the SPRTs. A Tinsley 10 of the re-entrant wells, the SPRT was raised
AC/DC standard resistor was used with the 10.0 mm above the bottom and measured
bridge. A triple point of water cell (Fluke again. This testing process was repeated at
model 5901) with a maintenance bath (Fluke heights of 20 mm, 30 mm, 40 mm, 50 mm,
model 7312) provided the temperature and 60 mm above the bottom. After testing at
comparison standard. the height of 60 mm, the SPRT was lowered
back down to the heights of 50 mm, 40 mm,
30 mm, 20 mm, 10 mm, and the bottom. The
mean value of the two measurements at each
Authors height was calculated to obtain the immersion
R. Ding, M. J. Zhao, T. Nielson profile of the re-entrant wells.
E. Nerdrum, and D. Farley Figure 3 shows the tested immersion
Fluke Calibration profile of one of the four re-entrant wells
799 East Utah Valley Drive compared with the ITS-90 hydrostatic
American Fork, Utah 84003-9775 values. The tested results agree with the ITS-
rong.ding@flukecal.com
90 hydrostatic values in the range from the
bottom of the re-entrant well to a height of 30
mm. The discrepancy increases with height
above 30 mm. The closer the SPRT is to the
top of the central argon cell, the lower the
measured temperature. The reason is that the
heat transfer of the liquid nitrogen through
the stem of the SPRT and the wall of the re-
entrant wells cools down the upper area of the
re-entrant wells.
The four re-entrant wells in the triple point
of argon system allow up to four SPRTs to
be calibrated at the same time. Temperature
homogeneity of the re-entrant wells is very
important for calibration accuracy and
consistency. In order to test the temperature
homogeneity of the four re-entrant wells of
this argon system, the testing was carried
out using three SPRTs (serial numbers 1614,
4223, 4227). The fourth re-entrant well was
used to monitor the triple point plateau of
the argon system during the overall testing
process. Each of the three re-entrant wells
was tested three times using three SPRTs by
moving the three SPRTs among the three re-
entrant wells. The testing results are presented
in Table 1 and show that the temperature
difference among the three re-entrant wells
is very small. The largest difference is 0.12
mK between re-entrant well #1 and #2. This
indicates that the temperature is consistent
among the re-entrant wells.
The estimated uncertainties of the triple
point of argon system are listed in Table 2.

Vol. 7 No. 1 March 2012 NCSLI Measure | 57

TECHNICAL PAPERS

Figure 3. Comparison of the tested immersion profile of the re-entrant wells and the ITS-90 hydrostatic values.

Well #1 Well #2 T (C) Well #3 T (C)

SPRT
(, ohm) (, ohm) (#2 #1) (, ohm) (#3 #1)

SPRT 1614 5.5143700 5.5143748 0.043 5.5143729 0.026

SPRT 4223 5.5502333 5.5502197 -0.122 5.5502303 -0.027

SPRT 4227 5.4509708 5.4509734 0.024 5.4509667 -0.038

Average 0.063 0.030

Table 1. Temperature consistency testing of the re-entrant wells.

The purity of the argon used in the argon system is 99.9999 %. The
estimated uncertainties (k = 2) are 0.26 mK for the triple point of argon
plateau.
4. Modeling of Heat Transfer of the Argon System Parts
In order to study the heat transfer process during realization of the
triple point of argon and to investigate the influence of the experimental
conditions on the thermal equilibrium of the argon system, a finite
element analysis (FEA) modeling of the heat transfer of the argon
system was carried out by using the FEA program ABAQUS. Figure
4 shows the mesh of the simulated central assembly of the argon
system. The simulated central assembly includes the four re-entrant
wells, the vacuum can, the heater shield, and the central argon cell
(also see Fig. 1). The elements of the central assembly are immersed
in liquid nitrogen.
The heat transfer of the argon system is a complex process. It includes
heat power provided from the heater shield, the heat conductivity of
all of the shields and the four re-entrant wells, convection between
the shields, liquid nitrogen outside of the simulated central parts and
the condensed argon inside the central argon cell, and radiation of the
shields surfaces. All of these sources of heat transfer will influence
the thermal equilibrium and the temperature distribution of the central
parts. In the simulation, the liquid nitrogen temperature is assumed to
be -196.5 C. The initial temperature of the whole central assembly

58 | NCSLI Measure www.ncsli.org

 TECHNICAL PAPERS

Uncertainties Value (mK)

Resistance reading (A) 0.020

Reproducibility (A) 0.100
Total A 0.102

Impurities (B) 0.022

Hydrostatic correction (B) 0.030
Pressure correction (B) 0.050
Immersion (B) 0.010
Re-entrant wells consistencey (B) 0.040
SPRT self heating (B) 0.020
Propagated from TPW (B) 0.020
Bridge non-linearity (B) 0.010
Total B 0.081

Total standard uncertainty 0.130

Expanded uncertainty (k = 2) 0.260

Table 2. Estimated uncertainties of the triple point of Figure 4. FEA mesh of the simulated central assembly of
argon system. the argon system.

is assigned to be -196.5 C. It is also assumed that in the beginning
of the simulation, the central argon cell is full of solid argon with
the temperature of -196.5 C, which is the same as that of the liquid
nitrogen outside after overnight freezing.
Two analysis steps are used in the simulation. The first step is the
heating process which is used to quickly raise the temperature of the
whole central assembly to the temperature of the triple point of argon
by using a heating power of 12 W at the outside surface of the heater
shield. The second step is to maintain the temperature of the heater
shield at -189.324 C (20 mK above the triple point of argon) during
the plateau of the triple point of argon to keep the thermal equilibrium
in the three processes: cooling from the outside liquid nitrogen,
heating from the heater shield, and the phase transition from solid and
liquid argon to argon gas occurring inside the central argon cell.
Figure 5 shows the simulated temperature distribution of the heater
shield and the central argon cell when the argon system reaches thermal
equilibrium at the triple point of argon. Figure 6 is the simulated
temperature of the four re-entrant wells. It can be seen from Fig. 5 that
the temperature of the four re-entrant wells is uniform except for the
top area close to the top lid. Figure 6 shows that the temperature of
four re-entrant wells is consistent because of the symmetric structure
of the four re-entrant wells. The simulation results agree well with the
experimental results. If the heater shields temperature is controlled
at a higher value (e.g. 100 mK above the triple point of argon), the
simulated temperature of the four re-entrant wells is the same. The
only difference is that the plateau duration becomes shorter because
more heat is transferred to the central argon cell, inducing a faster
phase transition of argon.
According to the experimental and simulation results, the
maintenance temperature has a great influence on the duration of the
argon triple point plateau. The closer the maintenance temperature
is to the triple point of argon, the longer the plateau lasts. The
recommended maintenance temperature is 20 to 60 mK above the
triple point of argon in order to obtain a stable and flat plateau that can
last for a reasonably long duration.
5. Conclusions
A new triple-point of argon system was developed to realize the triple
point of argon for the calibration of long-stem SPRTs. The triple-
point of argon plateau can be easily realized with the high-accuracy
temperature controller. This system was tested at different experimental
conditions. The testing results show that the triple point of argon
plateau can be as long as 100 hours with the temperature change less
than 0.05 mK when the maintenance temperature is 20 mK above the
triple point of argon. The temperature is consistent among the multiple
re-entrant wells. The estimated uncertainty of the argon system is
0.26 mK (k = 2). The FEA simulation of the heat transfer of the argon
system shows that the temperature is consistent in the four re-entrant
wells. The temperature of the re-entrant wells is the same with different
maintenance temperatures, but the duration of the triple point of argon
plateau becomes shorter at a higher maintenance temperature. In order
to obtain a stable and reasonably long plateau, the optimal maintenance
temperature is 20 to 60 mK above the triple point of argon.

Vol. 7 No. 1 March 2012 NCSLI Measure | 59

TECHNICAL PAPERS

Figure 5. Simulated temperature of the heater shield and Figure 6. Simulated temperature of the four re-entrant wells.
the central argon cell.

6. References
[1] G. T. Furukawa, in Temperature: Its Measurement and Control
in Science and Industry, vol. 6, edited by J.F. Schooley, Springer:
New York, pp. 265-269, 1992.
[2] M. G. Ahmed, Y. Hermier, M.R. Moussa, and G. Bonnier, in
Temperature: Its Measurement and Control in Science and
Industry, vol. 7, edited by D. C. Ripple, Springer: New York,
pp. 197-202, 2003
[3] S. L. Pond, in Temperature: Its Measurement and Control in
Science and Industry, vol. 7, edited by D. C. Ripple, Springer:
New York, pp. 203-208, 2003.
[4] J. Ancsin and J.M. Philips, Calibration apparatus for long-stem
and capsule-type Pt resistance thermometers between the triple
points of Ar and In, Rev. Sci. Instrum., vol. 55, pp. 1321-1324,
1984.
[5] I. Yang, C.H. Song, K.H. Kang, Y.G. Kim, and K.S. Gam,
Development of the Sealed-Type, Triple-Point-of-Argon
Cell for Long-Stem SPRT Calibration at KRISS, Int. J.
Thermophys., vol. 29, no. 5, pp. 1740-1748, 2008.

60 | NCSLI Measure www.ncsli.org

TECHNICAL PAPERS

Economic Impact of Equivalence of

Measurement Standards
Takashi Usuda and Andy Henson

Abstract: This paper discusses a new method for estimating the economic impact of the equivalence of measurement standards.
The method allows a quantitative calculation of the economic impact, based on a distribution function describing the quality of
the product and information about the agreement of measurement standards. In particular, the proposed method considers loss
parameters (false positive, false negative, and loss function) due to the deviation of measurement standards. The method can be
applied to any industry, any market, and to users at any level of the calibration hierarchy. We illustrate the method with an ex-
ample that demonstrates the economic impact of inconsistencies in a mass measurement standard due to the quality distribution
of weighing instruments. The example shows that the current system of assuring the equivalence of measurement standards is ef-
fective and delivers significant benefits. It also demonstrates the importance of information about the agreement of measurement
standards when assessing their economic impact.

1. Introduction
The benefits of measurement standards to
the economy are unquestionable. However,
assessments of both the cost of maintaining
and developing measurement standards
and the benefits associated with the chosen
level of performance are becoming increas-
ingly important in the dialogue between
National Metrology Institutes (NMIs)
and governmental funding bodies. A wide
range of studies on the economic impact of
measurement have been reported by NMIs,
governmental authorities, and academia.
The benefits of measurement standards to
an economy fall into two main categories, the
first relating to the public good [1], and the
second to competitiveness.
The benefit of establishing measurement
standards and their equivalence between
countries generally relates to the public good
rationale; the assumption of international
equivalence underpins all economic activities,
including manufacturing and trade. The
established measurement standards and their
equivalence ensure that measurements can be
made on a consistent, appropriately accurate,
transparent, and internationally recognized
basis throughout the world. This includes all
activities that rely on measurement data as a
basis for decisions, from the daily purchase
of groceries, gasoline, or water, through
to politics, commerce, industry, science,
engineering, international trade, human
health and safety, and environmental and
resource protection.
In contrast, establishing a new measur-
ement standard, or reducing the measurement
uncertainty, increases a countrys competiti-
veness, because it can lead to new products
and markets and can add value to legacy
products. It is fair to say that no one can make
a product that is demonstrably better than
the available measurement accuracy; thus
reducing the uncertainty of a measurement
standard can enable the development of a
higher quality of products. New measurement
standards might also lead to innovation within
an industry and result in new technological
frontiers for the economy.
Measurement standards for the public
good underpin many aspects of our daily and
working lives. The economic impact of the
public good rationale may be considerable
but it is not easy to assess because it may not
be visible. In general, industrialized countries
are very keen to continually develop new
measurement standards. These standards
encourage competitiveness in traditional
industrial activities and can also stimulate
new or emerging economic activities in areas
such as nanotechnology or biochemistry.
Not surprisingly stakeholders would like to
understand the effectiveness of investing
in measurement standards. Consequently,
there is much activity to assess the economic
impact of new measurement standards in
terms of the competitiveness agenda. Here,
the economic impact may be defined as return
on investment (ROI), where investments
refers to the cost of developing the new
standard, and the return relates to the
growth in market share, the development of
new markets, added value to legacy products,
and so on.
Although it is useful to visualize the
public good and competitiveness rationale
as distinct, the two are often intertwined.
In terms of the public good rationale, or at
least that part related to international trade,
established standards among countries must be
equivalent within agreed upon uncertainties.
Of course, if that were the primary objective
for a given measurement standard, once the
required level of performance was reached
there would be little incentive to invest
further. The need for better or new
measurement standards would be limited to
those situations when there was a need to
meet new domestic regulatory requirements,
or to reach equivalence with in important
trading partners. In practice, however, the
availability of new or improved measurement
standards offers specific manufacturers or
service providers opportunities to innovate
and create new market areas, and also to
reduce their costs and increase the quality of
their existing products and services. This can
provide them with a competitive advantage in
the market place. Thus, at least in the leading

62 | NCSLI Measure www.ncsli.org


economies, NMIs are primarily driven by the
need to increase national competiveness in a
globalized world. As a result, they continually
improve their measurement standards and
develop new ones to satisfy the needs of
cutting edge national stakeholders. This need
to satisfy both trade access requirements
and to provide national economies with a
competitive edge highlights an interesting
paradox for international metrology. The desire
to provide a competitive advantage requires
differentiation in measurement standard
performance, while regulatory, conformity
assessment, or interoperability demands
a common level of performance between
trading nations. The International Bureau of
Weights and Measures (BIPM) is responsible
for providing the basis for a single, coherent
system of measurements throughout the
world, traceable to the International System
of Units (SI). This task relates principally
to the public good rationale. The BIPM also
monitors newly developed measurement
standards, including consideration of their
dissemination and confirmation of their
equivalence at the international level. This
poses the question; how can we assess the
economic impact of a measurement standard
in terms of the public good?
The equivalence of measurement stand-
ards maintained by NMIs has always been
confirmed by international comparisons,
though not necessarily in a systematic way, and
the results have not always been consolidated,
or shown in a common format. In 1999, the
Mutual Recognition Arrangement for national
measurement standards and for calibration
and measurement certificates issued by NMIs
(CIPM MRA) [2] was introduced to establish
the degree of equivalence of measurement
standards maintained by NMIs. As part of
this arrangement, the results of international
comparisons are analyzed in terms of the
Authors
Takashi Usuda
National Metrology Institute
of Japan (NMIJ/AIST)
AIST Central 3, 1-1-1 Umezono,
Tsukuba, Ibaraki 305-8563, Japan
takashi.usuda@aist.go.jp
Andy Henson
International Bureau of Weights and
Measures (BIPM)
andy.henson@bipm.org
Vol. 7 No. 1 March 2012 NCSLI Measure |
degree of equivalence (i.e. the deviation
and uncertainty) of each participant relative
to an agreed reference value, and the results
are published in an open-access database
(the BIPM key comparison database,
KCDB) [3]. Calibration and Measurement
Capabilities (CMCs), declared by NMIs
and validated by international experts on
the basis of results obtained in international
comparisons, are also published in the KCDB
[3]. These published CMCs, in the form
of tables of measurement uncertainties for
various measurement quantities, indicate
the level of metrological performance of
NMIs in response to the demand from
national industries or other stakeholders. The
difference between measurement capabilities
of NMIs in different countries may have an
economic impact, or in other words, lead to
economic gain or risk in manufacturing and
trade.
This paper presents a new methodology
that considers loss parameters due to the
deviation of measurement standards. The
method allows an assessment to be made
of economic impact due to equivalence of
measurement standards among countries. It
can be used to evaluate the economic impact
for any industry, any market and for users
at any level of the calibration hierarchy.
Please note that this paper does not discuss
the intangible benefits of measurement, such
as those relating to safety and health, which
form an essential part of the public good
rationale. However, whenever the impact can
be assigned a monetary value, for example,
the economic risk resulting from medication,
then this new methodology can be employed
to assess the economic impact.
2. Previous Economic Impact Analyses
and Their Limitation
Since the early 1980s, there have been a
number of economic studies on standards
and conformity assessments, including
measurement standards. One of the earliest
studies [4] reported the cost of measurement
and the added value gained through a survey of
U.S. industries. In 1984, it was estimated that
the average value added from measurement-
related activities in the U.S. was 3.5 % of the
gross national product (GNP). Since then, a
variety of methods have been proposed to
assess economic impact. These methods are
classified as top-down, bottom-up, and to the research and development sectors of
case study.
TECHNICAL PAPERS
In economic surveys, statistical techniques
that employ macro data or top-down
approaches are common. They involve specific
measurement technologies that benefit a few
identifiable companies or markets, whereas a
proxy measure is used to determine the public
benefit value to the economy as a whole.
Considering the externality of measurement
technologies, estimation by macro analysis
usually involves broad approximations. To
obtain more detailed information, a bottom-
up approach may be employed using the
following indicators [5]:
Number of calibration certificates
(both in NMIs and calibration/testing
laboratories)
Number of accreditations requesting
traceability of measurement (e.g. ISO
17025) or verification of measurement
equipment (e.g. ISO 9001)
Number of patents related to
measurement
Number of document standards related
to measurement
Number of scientific publications and
their citations
In both top-down and bottom-up appro-
aches, the estimation may partly represent
economic impact in terms of the public good
rationale, as described in the previous section.
Innovations driven by new measurement
standards are an interesting subject for case
studies because estimation of their input to
a specific project is more quantitative. Their
economic impact relates to competitiveness
as discussed in the previous section. For
example, NMIJ/AIST (Japan) reviewed the
impact of an infrared thermometric standard
on the market for tympanic thermometers
[6]. The economic impact was estimated
by assessing the market share of the new
product, which used an infrared technique, as
compared to the other legacy products.
Previous studies have tended to focus
on national or product-specific markets.
The costs considered in these studies have
usually been the sum of investments to the
specified national measurement system or
the specified technology. Thus, the studies
63
 respectively) pass the conformity assessment.
Frequency of the product
Distribution
of the product
TECHNICAL PAPERS D(x) Frequency of the product
Distribution
of the product
do not explicitly illustrate the cost and benefit of an international D(x)
measurement standard. Birch reviewed numerous past economic All products All products All products
studies [7]. He concluded that quantifying the costs and benefits equally bad equally good equally bad
of measurement requires data on the operation of the measurement All products All products All products
system, which in many cases is not available, and which may become equally bad equally good equally bad
Quality characteristics
even more difficult to obtain as trade control is privatized. The interest
LTL UTL
in economic impact studies and strategies for measurement systems Nominal
value Quality characteristics
is still growing and many reports are being published. However,
Figure 1. Conformity
LTLassessment for products
UTL with the nominal-is-bes
these reports are likely to continue to focus on national measurement Nominal
value
characteristic.
systems and the competitiveness of national economies, because this
Figure 1. Conformity assessment for products with the nominal-is-bes
is the subject of most concern at the national level [8].
Following the introduction of the CIPM MRA in 1999, the BIPM characteristic.
If we consider that the quality distribution of the product is given by D(x) , the
commissioned an independent consulting company to review the
number of products accepted by the assessment Npass is given by
expected economic benefits of the CIPM MRA [9]. The review was
If we consider that the quality distribution of the product is given by D(x) , the
expected to show a reduction in non-tariff barriers to trade among Figure 1. Conformity assessment for products with the
the signatories to the CIPM MRA. It was also expected to show a nominal-is-best
number of products characteristic.
accepted by the assessment Npass is given by

reduction in cost for NMIs participating in international comparisons, Npass D( x)dx . (1)

because the results of comparisons are interpreted to show equivalence is given by

between any of the participants in other comparisons of the same Npass D( x)dx . (1) (1)

quantity [2]. The report noted that the previous top-down surveys The fraction defective FR of the number of total products Ntotal is given by
estimate the benefit of up to 10 % to trade from the removal of all non- The fraction defective FR of the number of total products Ntotal is given
byThe fraction defective F of the number of total products
tariff barriers, depending on the measures taken and the economies R Ntotal is given by
Ntotal Npass
involved. Clearly, the CIPM MRA is just one factor and can play only a FR . (2) (2)
modest part in the reduction of technical barriers to trade. Nevertheless Ntotal
the study discusses a possible contribution on the order of billions Traditionally,Nthe Npass
totalcore measurement in conformity assessment relies
FR . (2)
of U. S. dollars, a number that is small in percentage terms of total on finding the fraction of defective products. If defective products are
Ntotal
Traditionally, the core measurement in conformity assessment re
non-tariff barriers but worthwhile and significant in absolute terms. shipped, this creates quality problems. If defective products are not
However, the report did not examine the impact from the equivalence findingthey
shipped, therepresent
fraction aof lossdefective products. Ifbutdefective
to the manufacturer products are shipp
avoid damaging
thecreates Traditionally,
companys the core measurement in conformity assessment
a representre
of measurement standards, data for which have subsequently become quality problems. If defective products are not shipped,ofthey
reputation. To determine the economic viability
available from the KCDB. findingit is
product, theimportant
fraction toofforecast
defective products.
its quality If defective
before shipment. products are shipp
the manufacturer but avoid damaging the companys reputation. To determ
If monetary loss MFR due to the absolute cost of defective product
Based on the growing number of international comparisons, the creates quality problems. If defective products are not shipped, they represent
report noted that participation in the CIPM MRA reduces the cost and economic viability
is proportional to the unitof a price
product, it is
of the important
product to forecast
P, then its quality
MFR is given by before shi
the manufacturer but avoid damaging the companys reputation. To determ
number of international comparisons; while still allowing each NMI to
confirm equivalence of its measurement standards with its key trading economic
M FR viability

D(ofx)adxproduct,
Dit( xis)dimportant
x P. to forecast its quality
If monetary loss M FR due to the absolute cost of defective pro

(3) before shi
partners. The report also noted that the balance between costs and
benefits of the CIPM MRA should be monitored over the long term. proportional
It should to be the unitthat
noted pricetheof the product P,
conformity then M FRdiscussed
assessment is given by
here is Ifa binary
monetarydecision M R due
loss between Goodto ortheNotabsolute
good; costrejected
all of defective pro
Since the introduction of the CIPM MRA, there have been a number It should be noted that the Fconformity assessment discussed here is a binary d
of success stories demonstrating its positive impact. Customers of proportional
products
between
are
Goodto the
or unit
considered
Notprice
equally
good; of the
bad, and P, then Mare
product products
cause
all rejected
the same is given byequally b
monetary
FR considered
KRISS (Republic of Korea), for example, have avoided additional loss per unit, irrespective of how close they lie 7 to the testing limits.
cause the all
Conversely, same monetary
shipped lossare
products perconsidered
unit, irrespective
equallyofgood
how as close
longthey lie to the
costs for recalibration because the calibration capabilities of KRISS
were internationally recognized thanks to the CIPM MRA. In this case as limits.
they fall Conversely,
within the all shipped
testing products
limits, are7 considered
irrespective of how farequally
they aregood as long
from the target value.
the ROI of this operation was between 70 and 360 [10]. However, this fall within the testing limits, irrespective of how far they are from the target va
was a bilateral benefit shared between the two parties concerned so it The fraction defective, FR, is always the result of a trade-off
does not explicitly illustrate the total benefit to be gained through an between The thefraction
production cost andFRcustomer
defective, , is always the result
satisfaction. To of a trade-off
obtain a betw
international measurement standard. A more generic and transportable production
lower fraction cost and customer
defective, a narrowsatisfaction. To obtain
quality distribution a lowerisfraction defe
of product
required and this leads to increased production costs. A wider quality
evaluation method is needed. narrow quality distribution of product is required and this leads to in
distribution of product may allow lower production costs but can lead to
3. A New Methodology
production
a higher fractioncosts. A wider products.
of defective quality distribution
Finally, thisofwill
product
lead tomay allow lower pro
higher
3.1 Background costs
costsin terms
but can of lead
productivity.
to a higher Manufacturers may fix the
fraction of defective testing limit
products. Finally, this will
All industrial products exhibit a distribution in terms of quality. forhigher
their products if it is not fixed legislatively. A wider testing
costs in terms of productivity. Manufacturers may fix the testing limit limit
Inspections of a product to determine whether or not it satisfies a given may result in customer dissatisfaction, while a narrower testing limit
products if it is production
not fixed legislatively. A wider
prices.testing limitcost
may result in c
specification are known as conformity assessments [11]. Considering may lead to higher costs and higher Calibration
the nominal-is-best characteristic, as shown in Fig. 1, products that is dissatisfaction,
also one of thewhile factors a narrower testing limit
for consideration may the
because leaduncertainty
to higher production c
lie within upper and lower testing limits (UTL and LTL respectively) of higher
measurement is a component of the quality distribution
prices. Calibration cost is also one of the factors for of product
consideration bec
pass the conformity assessment. characteristic. A manufacturer will at least partially price a product
uncertainty
based of measurement
on its testing is a component
limit and associated productionofcosts
the toquality distribution of
distribute
If we consider that the quality distribution of the product is given
by D(x), then the number of products accepted by the assessment Npass thecharacteristic.
product. Of course, A manufacturer
the appropriatewill atprice,
leastproduction
partially price
cost, aandproduct based
testing limit and associated production costs to distribute the product. Of cou

64 | NCSLI Measure
appropriate price, production cost, and testing www.ncsli.org
limit will ultimately be dec
market mechanisms. For example, the price will be set to optimize the positio
 TECHNICAL PAPERS

Nfn D( x)dx , (4)

and those as false
positive by
Nfn D( x)dx , (4)

Figure 2. Additional loss due to deviation of measurement standard.

Nfp D( x)dx . (5)
Dpositive
testing limit will ultimately be decided by market mechanisms. For Nfnasfalse
and those ( x)dx , by (4)
(4)
example, the price will be set to optimize the position of the product

among its competitors. and those as false positive by

Here, we assume that the measurement accuracy [12] associated The total
additional
monetary loss due to deviation in the measur
with the assessment is good enough. The measurement properties
and those Nfpasfalse
D( x)dx .by
positive
standard, M fn fp , is then given by
(5) (5)
in the assessment should be traceable to the SI through national The total additional monetary loss due to deviation in the measurement
measurement standards maintained by NMIs, or through calibration standard, Mfn+fp,is then given by
D( x)dxmonetary
Nfp additional . (5)

laboratories. The total loss due to deviation in the measur

M
standard, M fn fp , is
fn fp then given by
D ( x )d x D ( x )d x P . (6) (6)
3.2 Additional Loss on Conformity Assessment by Deviation of The latter term (false positive) would usually result in payment of
Measurement Standard The total and
a penalty additional
may also monetarylead to moreloss seriousduelosses
to deviation
through damage in the measur
Assuming that the conformity assessment measurement is appropriate,
the monetary losses associated with defective products are tolerable
standard,
and
The
to the exporting
MM
may also
minimum
latter

fn fnfpfp

term

lead toofmore
estimate
(false
companys
, is then

D(given
x)dx by
positive)

the economic

serious losses

reputation. Thus Mfn+fp representspayment
would
D( x)dx P .
impact through
usually result
damage tobetween
of a difference
in a of a p
the exporting comp
(6)
costs under given conditions. If the shipped products satisfy the the measurement standards.

lossM

conformity assessment, then no additional loss is incurred. However, reputation.
Monetary Thus fn
represents
fp to deviation
due
a minimum
of measurementestimatestandards
of the economic
is impac
if there is any deviation between the measurement standards of the M
The
evident. For
difference
fn latter
fp
between

example, term D ( x )d
(falsex
positive)
in the early
the measurement
D
1970s,
( x )d x
would P .usually result
Stiefel [13] suggested that
standards.
in payment (6)a p
of
two parties (manufacturer and consumer/exporter and importer), then and
themay also lead
economic impactto more serious
of trade can losses through as
be calculated damage to the exporting
overcharging or comp
additional costs occur. Monetaryasloss
undercharging due toof deviation
a result measuringofinstrument measurement standards is eviden
inaccuracies.
reputation. Thus M fn fp represents a minimum estimate of the economic impac
Here, we consider a simplified relationship between an exporter example, The
Recently, in latter
the term1970s,
early
optimizing (false positive)
costsStiefel
for [13] would usually
suggested
measurement thatresult
and the in payment
economic
conformity of a of
impact p
(manufacturer) and an importer (consumer), as shown in Fig. 2. Any difference
and may
assessment
can between
also
be calculated lead tothe
processes measurement
asmore
have serious
been discussed
overcharging standards.
losses or through damage
[14, 15,
undercharging 16]. to thea exporting
However,
as result of compmeas
deviation e between the measurement standards of the two parties will there have been no reports about the economic impact on trade due
instrumentMonetary
reputation. Thus
inaccuracies. loss due
M fn fp representsto
Recently,deviation
aoptimizing
minimum of measurement
estimate
costs standards
of the economic
for information.
measurement is eviden
and impac
confo
cause additional economic loss in trade due to false negative and to deviation of measurement standards with empirical
example,
assessment in the
processes early 1970s, Stiefel [13] suggested that the economic impact of
thehave been used
discussed [14, 15, 16]. However,
impact inthere have be
false positive results, referring respectively to products previously
difference
There are between
two important measurement
factors standards.
to calculate economic
rejected by the exporter that would in fact have been accepted by the reality. The firsttheis deviation of measurement standards.
can be about
reports calculated
Monetary
as overcharging
economic
loss due to impact
deviation
or
onof trade due The
undercharging
measurement
second
as
to deviation is
a result of meas
standardsofis measur
eviden
importer, and to exported products that will be rejected by the importer. quality distribution of product.
instrument
standards inaccuracies. Recently, optimizing costs for measurement and confo
Mathematically, the number of products recognized as false example, inwith
As mentioned the earlyempirical the information.
in1970s, Stiefel [13] suggested
introduction, There are that
currently, twothe
the important
economic
degree of factors
impact us of
negative due to the deviation e is given by assessment
calculate processes
economic have
impact been discussed
in reality. The [14, 15, 16]. However, there have be
can be calculated
equivalence as overcharging
in measurement standards or first is deviation
undercharging
maintained by as of measurement
NMIs a result
is stan
of meas
reports
publishedaboutin the
The second inaccuracies. theKCDB economic
is quality distribution
[3]. Thisimpactdatabase on trade
contains due
CMCs to deviation
under the of measur
instrument Recently,ofoptimizing
product. costs for measurement and confo
standards with
As mentioned empirical in the information.
introduction, There are two important
currently, offactors us
assessment processes have been discussed [14, 15, 16].the degree there
However, equivalen
have be
calculate
measurement economic impact in reality. by The first is deviation of in
|measurement stan
Vol. 7 No. 1 March 2012 reports about standardsthe economic maintained
impact onNMIs tradeis due
NCSLI published
Measure
to deviation the
65 KCDB
of measur [3].
The second is quality distribution of product.
database contains CMCs under the terms of the CIPM MRA. The CMC tables pr
 TECHNICAL
evaluation, becausePAPERS
it embodies the knowledge of performance in relevant
comparisons together with the ability of the NMI to disseminate the national standards.
terms of the CIPM MRA. The CMC tables provide information on
Asthea result we can obtain
measurement comparable
uncertainties deviation
for various to the magnitude
measurement of the CMC for the
quantities
calculation
and differentof economic impact or monetary
levels of measurement loss.
scope. It should be noted that a
CMC provides the uncertainty of measurement capability. This does
This method might be of particular interest to companies, who hold the
not express the deviation from any other measurement standards.
information
However, iton maythebequality
used asdistribution,
a proxy fortesting limits and
the purposes fraction defective, which is
of evaluation,
often confidential
because it embodiesto the the
company.
knowledge of performance in relevant
comparisons together with the ability of the NMI to disseminate the
national standards. As a result we can obtain comparable deviation to
3.3
theExample
magnitude of of
thetheeconomic
CMC forimpact due to deviation
the calculation of economicof impact
measurement
or standards
onmonetary
conformityloss. assessment of analytical balances supplied from Japan
This method might be of particular interest to companies, who hold
the information on the quality distribution, testing limits and fraction
Here,
defective, we apply
which is oftenthe above method
confidential to the to estimate the possible economic impact
company.
due to deviation of measurement standards on inspection of analytical balances as an
3.3 Example of the Economic Impact Due to Deviation of
example.
Measurement Standards on Conformity Assessment of Figure 3. Relationship of product and MPE in a confor-
We makeBalances
Analytical the following conditions
Supplied and assumptions:
from Japan mity assessment under given conditions.
Here, we apply the above method to estimate the possible economic
impact due to deviation of measurement standards on inspection of for which the MPE is 5 mg, we have = 1.67 mg and s 0.507
Conditions
analytical balances as an example. mg (from the assumption of a normal distribution, for which 99.9 %
The conformity
We make the assessment
following of the analytical
conditions balance was carried out of
and assumptions: in the
accordance
Under thefall
products specified 3.3 s ( additional
within conditions, 1.67 mg)).loss This due to a deviation
relationship is of the
with International Organization of Legal Metrology (OIML) [17, 18]. measurement
shown in
Accordingly, Fig. standard is given by
3.
Conditions Under the specified conditions, additional loss due to a deviation of the
Under the specified conditions, additional loss due to a deviation e
the
Thestandard masses
conformity used forofthe
assessment thetest must not
analytical be in was
balance errorcarried
bymeasurement
moreout thanof one-third
standard
the measurementisofgiven by standard is given by
the
in maximum
accordancepermissible error (MPE) of the instrument
of Legalfor the applied load [17, 18]. 1 x2 1 x2
with International Organization Metrology N fp N fn Ntotal
exp 2
dx Ntotal exp dx . (9)
2
(OIML) [17, 18]. Accordingly, the standard masses used for the
22 2
22 2
1 x 1 x
test must not be in error by more than one-third of the maximum N fp N fn Ntotal exp dx
2
Ntotal exp dx . (9)
2
(9)
Assumptions

2 2
2 2
permissible error (MPE) of the instrument for the applied load [17, 18].
If monetary loss is given by the algebraic product of additional loss and unit
Based on some communications with relevant manufactures [19], we takeIf following monetary loss is given by the algebraic product of additional loss
Assumptions
realistic assumptions.
price, then the total economic impact on sales is also given by the algebraic product of
If monetary
and unit price,loss isthen giventhebytotal
the economic
algebraic productimpact of onadditional
sales is also lossgiven
and unit
by
Based on some communications with relevant manufacturers [19], we additional lossproduct
and theoftotal sales. Employing thetotal
value estimated above and using
price, then the total economic impact on sales is also given by the algebraic product of
the algebraic additional loss and the sales. Employing
o The quality
take the following realisticof the analytical balance has a normal distribution.
assumptions. statistics provided by above
Japanese industry 1
, the economic impact bydue to the deviation of
additional thelossvalue
and the estimated
total sales. and
Employing usingthe statistics
value estimatedprovided above Japanese
and using
o 99.9
o The % of
quality of the
products passbalance
analytical the testhas
with reference to the MPE
a normal condition
1
measurement standard in terms of sales of Japanese analytical balances is given in
industry , the economic impact due to the deviation of measurement
statistics provided by Japanese industry1, the economic impact due to the deviation of
distribution. standard in terms of sales of Japanese analytical balances is given in
[17] (fraction defective is 0.1 % of total product). Table 1 for different deviation of the measurement standard.
o 99.9 % of products pass the test with reference to the measurement MPE Tablestandard in terms deviation
1 for different of sales ofeJapanese analytical balances
of the measurement standard. is given in
condition [17] (fraction defective is 0.1 % of total product). Table 1 for different
The level deviation of the measurement
of fraction defective employed standard. in this estimation
Additional lossproduct)
as Additional loss as [19]. Total additional Economic loss on
Using the first assumption, the distribution of the products can be expressed as (0.1 % of the total
false fail (%)
is realistic
false positive (%) loss (%)
The measurement
the sales of
Using the first assumption, the distribution of the products can be deviations e Japanese
For analytical
Additional loss asconsidered Additionalinlossthisas estimation
Total additional are alsoEconomicplausible. loss on 1
expressed as example,
false fail (%)2 g is comparable
false positiveto(%)the uncertainty
loss (%) of the reference the sales ofvaluebalances
for (M Yen)
0.50 mg 0.049 1.001 1.05 analytical 197.0
Japanese
1 x
2
the international key0.025
comparison of 1 0.049 kg mass standards (CCM.M-K1)
D( x) Ntotal exp , (7)to the0.10 mg (7) 0.074 1 (M Yen) 13.8
balances
where is the standard deviation of 2 quality distribution (related
the accuracy of
2 2 0.50 mg 0.05 mg
[20]. The 0.049
Japanese 0.015
CMC 1.001
for mass 0.02
measurement 1.05 at 1 is197.0
0.035
kg 0.058 mg 6.6
the analytical balance
s is the reading). 0.10 mg as0.01 mg
published 0.025 in the 0.0034
KCDB 0.049
(value 0.0036
approved 0.074
on 24 March 0.0069 13.8
2007), and 1.3
where standard deviation of the quality distribution (related
0.05 mg 0.05 2mg g is0.015 comparable 0.0007to 0.02stability
the 0.0007
of the0.035
international 0.0014 6.6
prototype 0.26
to the accuracy
Combining of the
the analytical balance and
given conditions reading).
assumptions, we can 0.01estimate
mg the number 0.0034 0.0036 0.0069 1.3
of Table
the 1.
kilogram Additional
(IPK) loss The
[21]. due to a deviationrequested
uncertainty of the metrology
by standard,
customers of and the
Combining the given conditions and assumptions, we can estimate 2 g 0.0007
economic impact 0.0007 on the sales of Japanese 0.0014analytical balances. 0.26
ofthe
products, N products,
number ofpass
, that will be accepted by11 the conformity assessment
Npass, that will be accepted by the conformity Table accredited
1. Additional laboratories
loss due tofor the E1 class
a deviation of the of mass
metrology [18] is typically
standard, and80the g to
assessment 100 g
economic [22]. Moreover,
impact on the the
salessmallest
of uncertainties
Japanese analytical are usually
balances. obtained
at 1 kg Theand level of fraction
are increased indefective employed inranges
other measurement this estimation
[22]. Thus, (0.1the% of the total

Npass D( x) dx product) is realistic [19]. The measurement deviations considered in this
economic impact on the Japanese mass balance industry estimated here
The level of fraction defective employed in this estimation (0.1 % of the total
represents the lowest estimate of the impact on international trade due
estimation
product) isto such
realistic are
[19].also plausible.
The For example, 2 g is comparable to the uncertainty of
1 x2 a deviation of themeasurement
measurementdeviations standard. considered in this
Ntotal exp dx = 0.999 Ntotal . (8) the reference (8)valueFor forexample,
the international key comparison of 1 kg mass
2 estimation areJapan benefits
also plausible. from a well-maintained
2 g is comparable national
to themeasurement
uncertainty of standards

2 2
(CCM.M-K1)
standard [20]. The Japanese CMC aformeasurement
mass measurement at 1 kg isof0.058 mg as
the reference value for for mass which
the international achieves
key comparison of 1 kg uncertainty
mass standards
Here is the tolerance, which is equivalent to one-third of the MPE around published0.05 in mg the at KCDB 1 kg.(value
At this approved
level ofonuncertainty24 March the 2007), and 0.05 mg is
potential
(CCM.M-K1) [20]. The Japanese CMC for mass measurement at 1 kg is 0.058 mg as
of
the OIML mass, and s
Here is the tolerance, which is equivalent to one-third of the MPE comparable
can be calculated for any given fraction of the OIML
economic loss for weighing industries in Japan is around
to the stability of the international prototype of the kilogram (IPK) [21]. 6.6 million
published in the KCDB (value approved on 24 March 2007), and 0.05 mg is
defective. Yen (approximately 77 thousand USD for an annual turnover of
mass, and can be calculated for any given fraction defective.comparable to the stability requested
The uncertainty by customers of accredited laboratories for the E1 class of
Taking as an example the OIML F1 class of 1 kg mass standard, 18 746 million of Yenthe@international
220 millionprototype USD). However,of the kilogram if the(IPK)national[21].
Taking as an example the OIML F1 class of 1 kg massThe standard, mass [18]
for which
uncertainty is the
requested typically 80 g toof100
by customers g [22].laboratories
accredited Moreover, for the the
smallest
E1 class uncertainties
of are

MPE is 5 mg, we have = 1.67 mg and 0.507 mg (from

massthe assumption
[18] is
1
typically of
80 ag to 100 g [22]. Moreover, the smallest uncertainties are
66 | NCSLI Measure Total export sales of industrial weighing devices. (includes all instruments used bywww.ncsli.org
industry to measure mass or weight)is
normal distribution, for which 99.9 % of the products fall within 3.3 18 ( 1.67 Yen
746 million mg)).
(Year 2010)
1 Source: Japanese Measuring Instrument Federation, http://www.keikoren.or.jp/material/statistics/iande.html
 TECHNICAL PAPERS

Economic loss on
e Additional loss as Additional loss as Total additional the sales of
false fail (%) false positive (%) loss (%) Japanese analytical
balances1 (M Yen)
0.50 mg 0.049 1.001 1.05 197.0
0.10 mg 0.025 0.049 0.074 13.8
0.05 mg 0.015 0.02 0.035 6.6
0.01 mg 0.0034 0.0036 0.0069 1.3
2 g 0.0007 0.0007 0.0014 0.26

Table 1. Additional loss due to a deviation of the metrology standard, and the economic impact on the sales of Japa-
nese analytical balances.

standard could only be measured with an uncertainty of 0.5 mg at 1 kg, economic impact due to the deviation of a measurement standard in
then the potential economic loss would increase to around 190 million individual transactions requires knowledge of the specific conditions.
Yen (approximately 2 million USD). We conclude that the Japanese The relationship between economic loss and the deviation of a
mass balance industry benefits significantly from the current level of quality characteristic from the target value, m, can be described as
equivalence of mass measurement standards. This example is based (m x). Here, function of economic loss due to deviation m x is
on the assumption that the trading partners achieve a similar level of determined under specific conditions, for example the unit price of the
measurement, which is true of the major developed countries today. product, jurisdictional risks, the reputation of the company, and so on
Economic loss to the Japanese mass balance industry would decrease [24]. Once the function is obtained, the monetary loss, or the economic
if the equivalence of measurement standards among the remaining impact due to the deviation of measurement results, is calculable. The
countries were improved, or if the manufacturer reduces the standard simplest relation will be a linear function. For example, for a 100 g
deviation of their quality distribution. In this regard, the methodology pre-packaged product, if the amount totals 98 g instead of 100 g, this
can be decision-making for investments in both the national and leads to a customer loss and supplier benefit of 2 g. As a comparison,
international measurement infrastructure. Although this analysis is for if the content is 102 g, this will lead to a customer benefit and supplier
the Japanese mass balance industry, the methodology allows analysis loss of 2 g. Because both customer benefit/loss or supplier benefit/loss
of economic impact in other industries and other countries. eventually lead to an economic risk to the supplier, through legal and
It should be noted that the uncertainty of measurement does not quality aspects, the loss is expressed as the absolute value P . |m x|
indicate deviation from the nominal value or from the international where the gradient P is the unit price of the product.
standard. Uncertainty is defined as a parameter which is associated The economic impact also depends on the quality distribution of
with the result of a measurement, and which characterizes the dispersion the measuring equipment used for the transaction. Irrespective of any
of the values that could reasonably be attributed to the measurand [12, deviation of the measurement standard, there must be inherent loss Eloss
23]. Thus, deviations of measurement standard discussed here for the associated with the quality distribution of the measuring equipment.
purpose of these calculations can only be discussed stochastically. We The relationship D( x)
of Eloss, quality distribution of the measurement
E loss Ttotaland
f (m x) dx , (10)
point out that the methodology demonstrated shows that the economic equipment N D(x), D ( x) loss function (m x), for the total amount of
E given T f ( m x ) d x , (10)
total
loss due to a deviation between measurement standards can be reduced transactions loss is total
where Ttotal is the total amount N totalby of transactions.
by reducing uncertainty. D( x)
This where TEtotal
is an inevitablelossis total
loss
the Tthe
due toamount
N total
f (mofdistribution
totalquality x) dx , of the measurement equipment
transactions. (10) (10)
3.4 Additional Loss on Measurement Results of Instruments by within .
Thiswhereis anTinevitable loss due to the quality distribution of the measurement equip
total is the total amount of transactions. This is an inevitable
Deviation of Measurement Standard where
When
Ttotal
there
isisthea
total amount
deviation of the of transactions.
measurement standard, then the economic
The previous discussion considers the economic impact for the within loss due . to the quality distribution of the measurement equipment
loss This
Eloss isisaffected.
within an. inevitable
This can loss due to theasquality distribution of the measurement equip
be expressed
instrument manufacturers, but does not include the economic impact When there is a deviation of the measurement standard, then the econ
of the instruments measurement results (in this case, the masses within . D( x ) When there is a deviation e of the measurement standard, then
lossthe EEloss ( is
economic
) affected. loss EThis isTcan
total fbe
affected.(mexpressed
x)dx .can as
This be expressed (11)as
When thereNistotala deviation of the measurement standard, then the econ
measured by the balance) that may deviate from the nominal value. loss loss

In practice, the economic impact may be more significant if a non-

We interpret the difference D ( x in )Eloss with and without deviation as the
conforming instrument is used in daily transactions. However, this loss E is
E
loss loss (
affected.
) N total total
This can be expressed
T f ( m as
x ) d x . (11)
(11)
situation is difficult to assess analytically, because it depends on the economic impact of the deviation of the measurement standard. Thus the generic
D ( x )
goods to be measured. For example, a given deviation in mass will economic We We
impact Eloss
interpret ) the
E(impact
interpret difference
due the

to the
N total
inTE
difference
deviation
total fwith
lossin (mEthe
of x)measurement
loss
and x . anddeviation
dwith
without without (11)
standardas deviation
as
have a different economic impact in the context of pre-packaged daily the economic impact of the deviation e of the measurement standard.
associated
economic with
thetheimpact quality distribution
ofeconomic
the deviation of the impact
measurement
of theEmeasurement equipment Ddeviation
(x) is given
thestandard. Thus the ge
goods from that of clinical medicines. It is clear that calculation of the Thus We generic
interpret impact
the difference in Eloss impact due to
with and without deviation as
by economic
of the measurementimpact Eimpact standard duee to the deviation
associated with the qualityof thedistribution
measurement standar
1
Total export sales of industrial weighing devices. (includes all instruments economic impact of the
of the measurement deviation
equipment D(x) is ofgiven
the measurement
by standard. Thus the ge
used by industry to measure mass or weight)is 18 746 million Yen (Year associated with the quality distribution of the measurement equipment D(x) is g
2010) Source: Japanese Measuring Instrument Federation, economic impact D ( x E impact
) due to the deviation D( x) of the measurement standar
by impact N total total
http://www.keikoren.or.jp/material/statistics/iande.html E T f (m x)dx Ttotal f (m x)dx . (12)(12)
N
associated with the quality distribution of the total
measurement equipment D(x) is g
Vol. 7 No. 1 March 2012 by NCSLI Measure | 67
Figure 4 shows Eloss
D ) distributions of D(x) under
for( xvarious D( x)the same loss
E T f (m x)dx T f (m x)dx . (1
TECHNICAL PAPERS

(a) Rectangular distribution of instruments.

(b) Normal distribution of instruments.

Figure 4. Relationships of economic impact based on loss function, deviation of measurement standard, and distribu-
tion of instruments. 4(b) shows original distribution, 4(a) shows shifted distribution due to deviation of the measurement
standard.

Figure 4 shows Eloss for various distributions of D(x) under the 3.5 Example of the Economic Impact Due to Deviation of
same loss function P . |m x|, with and without the deviation e of the Measurement Standard on a Platinum Deal in Japan
measurement standard. Employing the generic method described in the previous section,
we can estimate the possible economic impact of a deviation of a

68 | NCSLI Measure www.ncsli.org

 Employing
Employing the generic
the generic methodmethod described
described in the previous
in the previous section,
section, we we can the
can estimate estimate the
possible economic
possible impactimpact
economic of a deviation of a measurement
of a deviation standard standard
of a measurement on a platinum
on a platinum TECHNICAL PAPERS
transaction that relies
transaction that on the on
relies analytical balancebalance
the analytical in question.
in question.
The following
The following assumptions are employedestimation:
measurement assumptions
standard on are employed
a platinum in the
transaction that relies on the
in the estimation:
analytical balance in question.
Assumptions
TheAssumptions
following assumptions are employed in the estimation: Potential impact
o The D(x) used Deviation of
o quality distribution
The quality of analytical
distribution balance balance
of analytical D(xfor this for this
) used by means of
Assumptions measurement
economic loss
transaction is given by equation (8) which is described in section 3.3. standard e
o Thetransaction is given by
quality distribution equation (8)
of analytical whichD(x)
balance is described
used for in section 3.3. Eimpact (USD)
o The loss function due to measurement is proportional to the unit price
othisThe loss function
transaction due
is given byto measurement
equation is proportional
(8) which is described to the unit price
0.50 mg 228 600
(price per unit3.3.
in section mass of platinum).
(price per unit mass of platinum). 0.10 mg 9 800
o The loss function due to measurement is proportional to the
unit price (price per unit mass of platinum). 0.05 mg 2 500
According to equation (10), the economic loss Eloss due to the quality
According to equation (10), the economic loss Eloss due to the qualityTable 2. Economic impact of platinum imports to Japan.
distributionAccording
of the analytical
to equationbalance(10), used
the for the transaction
economic loss Eloss is
duegiven byquality
to the
distribution of
Dof
the analytical balance used for the transaction is given by
distribution ( x)the analytical balance used for the transaction is given by can be successfully applied. We also conclude that the stakeholders
E loss T
D( x)
total P ( m x ) dx
E loss Ntotal
of platinum imports in Japan benefit significantly from the Japanese
N total Ttotal P (m x) dx mass measurement infrastructure (i.e. national measurement standard
1 x 2
Ttotal exp P (2m x) dx ,
(13) for mass and conformity assessments for analytical balances).
2 1 2 x
2
Ttotal exp 2
2
P (m x) dx , (13) (13)

2 The relationships of E , D(x) and P m 4.xSummary and Outlook
where P is the unit price of platinum. loss
where P is the unit price of platinum. The relationships of Eloss, D(x) Although numerous reports on the economic impact of measurement
where
are illustrated P is the unit price of platinum. The relationships of Eloss , D(x) and P m x
and P in . |mFigure x| are 4(b).illustrated in Figure 4(b). standards have been published, there are few reports on the economic
2,3
areUsing
illustrated
Using datadata in associated
Figure with
associated 4(b). withmarketmarket statistics
statistics forfor Japanese
Japanese platinum
platinum importsimpact , of equivalence of measurement standards. There is a need for
imports2,3, the total import volume Ttotal is 22 tons and the unit price a more generic
the total import Using volume Ttotal is 22 with
data associated tons market unit priceforP Japanese
and the statistics is 57 USD /g. Theimports2,3, methodology to illustrate the economic impact. We
platinum
P is 57 USD /g. The inevitable loss due to the distribution of the have proposed a new approach that estimates economic impact based
inevitable loss due
theanalytical
total import to thevolume
balance distribution
is Ttotal of is the 22analytical
tons andbalance the unit is price P is 57 USD /g. The
on measurement capability associated with quality distribution of the
inevitable loss due to 1 the distribution x 2 of the analytical balance is product. The methodology requires two essential inputs: the quality
Eloss Ttotal exp 2 P (m x) dx =507 000 USD, (14) (14)

2 2 distribution of the product, and the deviation of the measurement
1 x 2
totalimports
EAccording toT
equation (12), exp
the
economic P
impact ( m E x ) d x =507
resulting 000
from USD,
the (14)
standards.
1.25USD USD
total
while the whiletotalthe imports
loss
equal 1.25
equal 2billion 2 2(57
billion USD (57/gimpact
for 22
USD /gtons).
for 22 tons). The method was illustrated by two examples related to the
deviation
According of the measurement
to equationstandard (12), the is economic
given by impact Eimpact resulting economic impact of inconsistencies in a mass measurement standard
ation (12), the whilefromthe
economic thetotal
impact
deviationimports
Eimpactof equal
resulting 1.25the
from
the measurement billionstandard USD (57e USD is given /g for
by 22 tons). associated with the quality distribution of analytical balances.
ment standard is given by 1 (x ) 2
1 x 2 Although, the examples contain assumptions and use conservative
2
Total import of
Eimpactvolume Ttotal
platinum toexp in 2010:
Japan
P
22 (m . xSource:
tons
) dx Agency Ttotal
for
exp and
Natural Resources P (m x) dx
2 Energy,
2 2 estimations, they confirm that the current system of assuring the
2

2
2
Japan, http://www.enecho.meti.go.jp/info/statistics/kikinzoku/result-2.htm
3 . (15)
Unit
( x price
) 2 of platinum 57 USD /g (average 1 price for x 22010).
Source: The London Platinum and Palladium equivalence of measurement standards and conformity assessment of
expMarket,2Fixing
2 Total
statistics,
Pimport ) dx of Tplatinum
(m http://www.lppm.com/statistics.aspx
xvolume total to exp
Japan in2 2010: P (m22 tonsx) dx. Source: Agency for Natural
(15) Resources and Energy,
2
2
Japan,http://www.enecho.meti.go.jp/info/statistics/kikinzoku/result-2.htm 2 analytical balances delivers significant benefits to the weighing device
3 Table 2shows
shows the
Unit Tableprice of2platinum thepotential
57 USD
potential economic
/g (average(15)
economic loss for
price in loss
Japanese
2010). platinum
Source:
in Japanese Theimports
London
platinumdue to and
Platinum Palladium and to the users of the analytical balances (i.e. stakeholders
industries
Market,
various Fixing statistics, http://www.lppm.com/statistics.aspx
deviations
imports due to various deviations of the measurement e
18 of the measurement standard of platinum trade). The examples also indicate that the BIPM KCDB
standard calculated from equation (15).
e potential economic When
calculated the deviation
loss from ofequation
in Japanese the mass imports
platinum standard
(15). due at 1tokg isthe
When less deviation
than 0.05 mg,ofas the in themass
current an open-access database for measurement standards is invaluable
the measurement standard standard
situation, at 1calculated
then the
kgadditional
is less from equation
economic
than 0.05 loss (15).Eas
mg, in the current situation, then when assessing economic impact. The examples given here discuss
impact is negligible (around 2 500 USD)
18
the
at 1additional
e mass standardconsidering kg is less economic
than 0.05 mg, loss
as in E
the current is negligible
the total imports (1.25 billion USD). While the loss due to the quality
impact (around 2 500 USD) economic impacts (or potential losses) due to a well maintained
considering
loss Eimpact is
onal economic distribution the total (around
negligible imports2 500 (1.25 USD) billion USD). The loss due to the measurement infrastructure (mass standard and analytical balance used
of the balance Eloss (507 000 USD) is dominant. If the national standard
quality distribution of the balance Eloss (507 000 USD) is dominant. If in valuable metals in an advanced economy). The economic impacts
ports (1.25 billion could USD). While
only be standard the losswith
measured due to uncertainty
the quality of 0.5 mg at 1 kg, then the potential
the national couldanonly be measured with an uncertainty of 0.5 will be much larger if we discuss a new measurement standard, or
ce Eloss (507 000 USD) is loss dominant.
economic
mg at 1 kg, then wouldtheIfpotential
the nationalincrease
drastically standard to around 228 600 USD, which is
economic loss would drastically increase a new product based on the new measurement standard, that might
d with an uncertainty comparable
to around of 0.5 mg
to228 at
the half
600 1 ofkg, Eloss
USD, then the methodology
. This
which potential
is comparable allows ustoto the illustrate
halfwhich
of Epartloss.of have a wider deviation of measurement standards or a wider quality

drastically increase to
This methodology
measurement around 228 600
allowsEus
infrastructures, USD, which is
to illustrate which part of measurement distribution of the product.
loss associated with the quality distribution of the
f Eloss . This methodology
infrastructures, allows usEto lossillustrate
associated which part withof the quality distribution of the Our new methodology allows the quantitative calculation of the
measurement equipment, or Eimpact associated with the deviation of the measurement
measurement equipment, or Eimpact associated with the deviation of economic benefits gained due to the equivalence of measurement
ures, Eloss associated with the quality distribution of the
standard
the measurement , is dominant for the possible
standard e , is economic
dominantimpact. for the possible economic standards in any quantity of any industry, market, and for users, at
, or Eimpact associated with We the deviation
conclude that of
the the measurement
generic approach to assess the economic impact due to their own particular level of measurement standards. It can be used
impact.
or the possible instruments
economic
We conclude measurement
impact. that the results associated
generic with deviation
approach to assessof measurement
the economic standard to help decide whether to invest in improving measurement standards
expressed
t the generic approach
impacttodue in equations (10)
assesstotheinstruments to
economic impact (12) can bedue to
measurement successfully applied. We also
results associated with conclude that or to invest in improving the quality distribution of the instruments.
nt results associateddeviation
the stakeholders of measurement
with deviation of platinum
of measurement standard
imports in Japan
standard expressed in equations
benefit significantly from(10)
thetoJapanese
(12) The authors anticipate that readers will employ this methodology
10) to (12) can be mass measurement
2 successfully applied.infrastructure
We also (i.e.
conclude national
Total import volume of platinum to Japan in 2010: 22 tons . Source: Agency that measurement standard for mass and to evaluate the economic impact in their own particular situation.
Japan
num imports inconformityfor Naturalbenefit Resources
assessments
significantly and
for Energy,
analytical
from Japan, http://www.enecho.meti.go.jp/
balances).
the Japanese One important parameter for this evaluation, the quality distribution
info/statistics/kikinzoku/result-2.htm
astructure (i.e. national measurement standard for mass and of the product, is often proprietary information known only to
3
Unit price of platinum 57 USD /g (average price for 2010). Source: The the manufacturer. The other important parameter, the deviation of
for analytical balances). Potential impact by
London PlatinumDeviation and Palladium of measurementMarket,means Fixing of statistics,
economic loss http://www.lppm.
com/statistics.aspx standard measurement standards, can be obtained from the BIPM key comparison
Eimpact (USD)
Potential impact by 0.50 mg 228 600
tion of measurement means of economic loss
standard Vol. 7 No. 1 March 0.10 mg
2012 9 800 NCSLI Measure | 69
Eimpact (USD) 0.05 mg 2 500
0.50 mg 228 600
Table 2. Economic impact of platinum imports to Japan.
TECHNICAL PAPERS
database (KCDB). The authors expect that this proposed methodology
and evaluation will encourage discussion on the equivalence of
measurement standards and its contribution to the economy.
5. Acknowledgements
This study was conducted from September 2010 to December 2011
while one of the authors, T. Usuda was seconded to the BIPM from
NMIJ/AIST Japan. The authors thank Dr. Mitsuru Tanaka and Dr.
Robert Kaarls, both members of the CIPM, Prof. Michael Khne,
Director of the BIPM, and Dr. Terry J. Quinn CBE FRS, Emeritus
Director of the BIPM, for their valuable suggestions. The authors
also thank Dr. Kensei Ehara (NMIJ, Japan), Dr. Masayoshi Koike
(NMIJ, Japan), Dr. John Wright (NIST, US), Dr. Raghu Kacker (NIST,
US), Mr. David Nettleton (NPL, UK), Prof. Leslie R. Pendrill (SP
Measurement Technology, Sweden), Mr. Hirofumi Oima (OECD) and
staff of the BIPM for their valuable comments and communications.
Finally, the authors express sincere thanks to Prof. Andrew J. Wallard
CBE, Emeritus Director of the BIPM, who invited T. Usuda to the
BIPM, giving him the opportunity to undertake this study.
6. References
[1] G. Tassey, Underinvestment in Public Good Technologies, J.
Technol. Transfer, vol. 30, no. 1, pp. 89-113, 2005.
[2] BIPM, Mutual recognition of national measurement standards
and of calibration and measurement certificates issued by
National Metrology Institutes, October 1999. (http://www.
bipm.org/en/cipm-mra/mra_online.html)
[3] BIPM, The BIPM key comparison database, Key and
supplementary comparisons (Appendix B), Calibration and
Measurement Capabilities-CMCs (Appendix C), available at:
http://kcdb.bipm.org/
[4] P. A. DonVito, Estimates of the Cost of Measurement in
the U.S. Economy, National Bureau of Standards Planning
Report, November 1984.
[5] G. Williams, The assessment of the economic role of
measurements and testing in modern society, European
Measurement Project Report, July 2002. (http://www.bis.
gov.uk/assets/bispartners/nmo/docs/nms/economics_of_
measurement_july2002.pdf )
[6] AIST, Case Studies on Outcomes Produced by National
Institute of Advanced Industrial Science Technology (1) -Thin
Film Silicon Solar Cells-, -Traffic Data Processing-, -Functional
Food-, -Biocompatible Ceramics-, -Temperature Standard-,
AIST-TID-R2003-02, 2004.
(http://www.meti.go.jp/committee/materials/downloadfiles/
g40730c30_3j.pdf, in Japanese)
[7] J. Birch, Benefit of Legal Metrology for the Economy and
Society, International Committee for Legal Metrology
(CIML) Report, 2003. (http://www.oiml.org/publications/E/
birch/E002-e03.pdf)
[8] S. Bowns, I. Bradley, P. Knee, F. Williams, and G. Williams,
Measuring the economic benets from R&D: improvements in
the MMI model of the United Kingdom National Measurement
System, Res. Policy, vol. 32, pp. 991-1002, 2003.
70 | NCSLI Measure
[9] KPMG Consulting, Potential Economic Impact of the CIPM
Mutual Recognition Arrangement, KPMG Report submitted to
BIPM, 2002. (http://www.bipm.org/en/cipm-mra/economic.html)
[10] K. Myungsoo, How NMIs benefit Global Enterprises: Success
story of CIPM MRA in Korea, Presented at symposium to
celebrate the ten-year anniversary of the CIPM MRA, Paris,
France, October 2009. (http://www.bipm.org/en/events/10-
year_symposium/presentations.html)
[11] OIML, International vocabulary of terms in legal metrology
(VIML), OIML V1 Edition, 2000.
[12] ISO, International Vocabulary of Metrology Basic and
General Concepts and Associated Terms, VIM, 3rd edition,
JCGM 200:2008, 2008.
[13] S. W. Stiefel, Management Assistance for Weights and
Measures Progress, Measuring Inaccuracys Economic
Distortion, presented at the 58th National Conference on
Weights and Measures, 1973.
[14] G. Beges, J. Drnovsek, and L. Pendrill, Optimising calibration
and measurement capabilities in terms of economics in
conformity assessment, Accredit. Qual. Assur., vol. 15, no. 3,
pp. 147-154, 2010.
[15] K. Weiensee, O. Khn, G. Lin, and K-D., Sommer, Risk
of erroneously deciding conformity of measuring instruments,
Accredit. Qual. Assur., vol. 13, no. 11, pp. 663-669, 2008.
[16] R. Kacker, N.F. Zhang, and C. Hagwood, Real-time control of a
measurement process, Metrologia, vol. 33, pp. 433-445, 1996.
[17] OIML, Non-automatic weighing instruments, Part 1:
Metrological and technical requirements Tests, OIML R 76-1
Edition 2006 (E), 2006.
[18] OIML, Weights of classes E1, E2, F1, F2, M1, M12, M2, M23 and
M3, Part 1: Metrological and technical requirements, OIML R
111-1 Edition 2004 (E), 2004.
[19] M. Tanaka (CIPM Member), Personal communications between
mass balance industries in Japan.
[20] C. Aupetit, L.O. Becerra, N. Bignell, W. Bich, G.D. Chapman,
J.W. Chung, J. Coarasa, S. Davidson, R. Davis, N.G.
Domostrova, K.M.K. Fen, M. Glaser, W.G. Lee, M. Lecollinet,
Q. Li, A. Ooiwa, R. Spurny, A. Torino, J.C.G.A Verbeek, and
Z.J. Jabbour, Final report on CIPM key comparison of 1 kg
standards in stainless steel (CCM.M-K1), Metrologia, vol. 41,
07002, 2004.
[21] R. S. Davis, Possible new definitions of the kilogram, Phil.
Trans. Royal Soc. A, vol. 363, pp. 2249-2264, 2005.
[22] M. Glser, M. Borys, D. Ratschko, and R. Schwartz,
Redefinition of the kilogram and the impact on its future
dissemination, Metrologia, vol. 47, no. 4, pp. 419-428, 2010.
[23] ISO, Evaluation of measurement data Guide to the expression
of uncertainty in measurement: GUM, JCGM 100:2008, 2008.
[24] G. Taguchi, S. Chowdhury, and Y. Wu, Taguchis Quality
Engineering Handbook, John Wiley & Sons, 2004.
www.ncsli.org
TECHNICAL PAPERS

71 | NCSLI Measure www.ncsli.org

TECHNICAL PAPERS

Traceability and Quality Control in a

Radiation Thermometry Laboratory
Frank Liebmann and Tom Kolat

Abstract: In radiation thermometry, a number of steps have been taken to improve calibration quality at temperatures below
1000 C. These steps involve work done by national metrology institutes and standards bodies. The Fluke Infrared Calibration
Laboratory in American Fork, Utah has benefitted from this progress and has established its own radiation thermometry program.
The calibration range of this laboratory is -15 C to 500 C. This program involves calibrating radiometric transfer standards
(with pyroelectric detectors) that, in turn, are used to calibrate flat-plate radiation sources, sometimes referred to as blackbodies.
The transfer standards are calibrated by a sequence of blackbody cavity radiation sources that cover the entire temperature range
of the laboratory. The radiometric transfer takes place between the cavity sources and the flat-plate sources. The intended use of
the flat-plate sources is infrared thermometer calibration. Both the transfer standard calibrations and the flat-plate calibrations
are accredited by the National Voluntary Accreditation Program (NVLAP). This paper discusses the traceability involved in this
radiometric calibration program. It also discusses numerous quality control measures that have been taken to improve and assure
measurement accuracy for both calibrations.

1. Introduction
In 2005, Fluke Hart Scientific (now know as Fluke Calibration in
American Fork and to be referred to as American Fork or AMF in
this paper) began development of flat-plate IR calibrators that are
calibrated using radiometric calibration. To support calibration of this
product, a series of variable temperature liquid bath blackbodies were
developed. These blackbodies support the calibration of a radiometric
transfer standard that is used to calibrate the flat-plate calibrators. A
number of steps were taken to ensure quality during the development
of the radiometric temperature calibration program. Some of these
steps are the result of research done internally; other steps are a result
of the development of new standards.
2. Traceability
The read-out temperature of the flat-plate IR calibrators is based on
a radiometric calibration, using the Heitronics model KT19II.82 (to
be referred to as a KT19 in this paper) as a transfer standard. This
instrument uses a pyroelectric detector [1]. An outline of the KT19
calibration scheme is discussed later in this paper.
The radiometric calibration was chosen over a contact calibration to
account for factors such as emissivity [2] and heat exchange. The KT19
is calibrated using AMFs liquid bath blackbodies. A diagram of the
blackbody is shown in Fig. 1. The temperature of the bath fluid during
this calibration is monitored by a platinum resistance thermometer
(PRT). The cavities have emissivity greater than 0.999 [2]. This number
was verified by modeling with the STEEP3 software [3, 4, 5]. Newer
methods exist to calculate blackbody emissivity [6], but were not
available for this modeling. The inputs to this modeling were based on
testing of blackbody uniformity [2]. One such result is shown in Fig. 2.
The results in Fig. 2 show the temperature deviation on the cavity walls.
At 150 C the cavity had a 40 mK deviation in uniformity and a 60 mK
deviation at 250 C. We utilized the results of this testing and modeling
in our uncertainty budget for the KT19 calibration.
2.1 Traceability Scheme
The true temperature of the cavity baths is measured with a PRT
located inside each bath. The PRT calibrations are performed in
AMFs primary calibration laboratory and are traceable to the
International System (SI) through the National Institute of Standards
and Technology (NIST). The blackbodies radiometric temperature
is verified radiometrically by measurement with a Heitronics TRTII
[7]. The TRTII is calibrated by NIST [8], and test results have shown
normal equivalence [9]. A schematic of AMFs radiometric traceability
chain is shown in Fig. 3., where the box labeled AMF IR Cavities
represents the liquid bath blackbodies discussed earlier. The AMF
CL1 is the primary calibration laboratory in American Fork.
It would be more desirable to use the TRT transfer from NIST
shown in Figure 3 as a direct radiometric traceability path. However,
this method would result in larger uncertainties. An example of the
difference in uncertainties between the two methods is summarized
in Table 1.
2.2 Uncertainty Budgets
There are four different uncertainty budgets for AMFs radiometric
calibrations. Two of these uncertainty budgets are for the two flat-plate
calibrator models. The other two are for the KT19 calibration and the
blackbody verification using the TRT.
The original uncertainty budget for the KT19 calibration was based
on AMFs existing uncertainty analysis for contact thermometry [2].
Since the original uncertainty budget was developed, the Bureau
International des Poids et Mesures (BIPM) has released a standard
(BIPM CCT-WG5) for radiation thermometry uncertainty budgets

72 | NCSLI Measure www.ncsli.org

 TECHNICAL PAPERS

Figure 1. Liquid bath blackbodies. Figure 2. Blackbody uniformity test results.

Contact Scheme Radiometric Scheme

Calibration Point Uncertainty Uncertainty
(C) (k = 2) (k = 2)
(K) (K)

-15 0.127 0.193

0 0.124 0.189
50 0.122 0.186
100 0.121 0.184
200 0.122 0.186
350 0.226 0.345
500 0.366 0.558

Figure 3. Radiometric traceability Table 1. Differences in uncertainties between contact and radiometric traceability.
chain.

[10], and AMF has reevaluated its uncertainty

analysis. The WG5 standard places more
detail on radiometric uncertainties and less on
the contact uncertainty. Regardless, there was
not a significant change in the uncertainties
between the original evaluation and the new
evaluation following the BIPM model.

3. Quality Control - Blackbody Sources

A diagram of the KT19 calibration is shown
in Fig. 4. The steps taken to assure the

Authors
Frank Liebmann
Figure 4. Transfer standard calibration geometry.
Tom Kolat
Fluke Calibration
quality of these calibrations include cross 3.1 Cross-Checks and Verification
799 E. Utah Valley Dr.
American Fork, Utah
checks with a national metrology institute, One check to verify the radiometric temp-
frank.liebmann@flukecal.com determination of cavity uniformity, and use erature of AMFs cavities used a TRT
of a hot gas purge. calibrated at NIST [9]. The TRT calibrated

Vol. 7 No. 1 March 2012 NCSLI Measure | 73

TECHNICAL PAPERS

NIST AMF
Nominal Temperature
Uncertainty Uncertainty Normal
Blackbody Temperature Difference
(K) (K) Equivalence
(C) (K)
(k = 2) (k = 2)
LT -15 0.074 0.34 0.128 0.20
LT 0 0.014 0.3 0.133 0.04
LT 50 -0.051 0.12 0.170 -0.25
LT 100 -0.125 0.11 0.218 -0.51
MT 100 -0.058 0.11 0.218 -0.24
MT 200 -0.155 0.12 0.335 -0.44
HT 200 -0.114 0.12 0.335 -0.32
HT 300 -0.144 0.13 0.226 -0.55
HT 350 -0.222 0.13 0.260 -0.76
HT 420 -0.253 0.14 0.317 -0.73
HT 500 -0.320 0.16 0.392 -0.76

Table 2. Normal equivalence results of comparison of American Fork blackbodies and NIST.

Figure 5. Cavity bottom uniformity. Figure 6. Blackbody purge flow rate test.

at NIST was used to measure the cavities at American Fork (Fig.
2). The temperature difference shown in Table 2 is the difference
between AMFs measurement and NISTs measurement. Since this is
a verification of the cavities temperature and not the traceability path
of the cavities temperature to the SI, some of AMFs uncertainties are
less than NISTs for the TRT measurement.
3.2 Cavity Uniformity
Along with the Z-axis uniformity testing shown in Fig. 2, testing has
been done to determine cavity bottom uniformity. This testing is an
important part of the KT19 uncertainty budget [10] and was done
using a Heitronics TRT 2, by measuring points on the X-axis (vertical)
and Y-axis (horizontal). The field-of-view of the TRT used for this test
was 5.0 mm diameter (98 %), with a measuring distance of 362 mm.
Fig. 5 is one set of data taken from this testing. The temperature map
shown in this figure is created from this data. Temperature differences
are referenced from the center of the cavity bottom.
3.3 Hot Gas Purge
To decrease the effects of temperature drop between the bath fluid and
the cavity walls and to improve temperature uniformity, a hot gas purge
is applied to the apex of the blackbody cone as shown in Fig. 1. The
air goes through tubing and forms a helix inside the bath fluid. In this
way, it reaches the bath temperature before it exits into the blackbody.
Tests have been done to observe the effects of the purge on radiometric
measurement. The results of one such test are shown in Fig. 6. The
dashed line at 28 / min. represents the flow as indicated in AMFs
calibration procedures. The measured temperature difference is based
on the difference from the measured temperature at a flow rate of 28
/ min. The radiometric temperature of the cavity does not change
significantly above half of the flow rate indicated in the procedure.

74 | NCSLI Measure www.ncsli.org


Figure 7. KT19 size-of-source testing results.
4. Quality Control Measures Transfer Standard Calibrations
A number of steps have been taken to assure trueness of the
measurements in the transfer standard calibration. These steps include
a self-consistency check using a chi-squared check [11] of data taken,
an alignment procedure coupled with calculation of size-of-source
effect uncertainty, an analysis of long term stability history, the use
of cross-checks for verification of blackbody radiometric temperature,
controlling transfer standard operating temperature, consideration of
transfer standard warm-up time, and the use of a hot gas purge with the
blackbody. A selected set of these steps is discussed below. A diagram
of the transfer standard calibration geometry is shown in Fig. 4.
4.1 Size-of-Source Effect
The KT19 calibration uses a 35 mm diameter water cooled aperture.
The aperture temperature is controlled near ambient and monitored
as specified in AMFs calibration procedure. Size-of-source effect
testing [12] on the radiometric transfer standard was done during the
development phase of the project [2]. This testing followed a standard
guideline [13] for testing size of source. Results of this are shown in
Fig. 7 as size-of-source effect data. These data were used to determine
aperture diameter and calculate aperture related uncertainties. In
addition, tests were performed to determine the effects of varying
aperture temperature and the test results were applied to the KT19
uncertainty budget.
4.2 Alignment
During the KT19 calibration process, the unit under test is mounted
on a geared tripod head. The tripod head provides angular adjustment
on two axes. The geared tripod head is mounted on an X-Y-Z carriage
system that provides linear adjustment on three axes.
The angular alignment involves mounting a laser on the geared
tripod head and angularly aligning the laser beam from the apex of the
blackbody cone to the center of the aperture. After this, the KT19 is
mounted on the tripod head. The distance is set between the aperture
and the KT19 lens (Zaxis). Then the KT19 is aligned in the side-to-
side direction (X-axis) and the up-and-down direction (Y-axis). For
this procedure, a method from ASTM was considered [13]. However,
it was found that signal received by the unit under test does not reach
a definite peak during the calibration, but instead, forms a plateau
Vol. 7 No. 1 March 2012 NCSLI Measure |
TECHNICAL PAPERS
Figure 8. KT19 horizontal alignment.
as shown in Fig. 8. Taking this into account, an alternative method
for alignment was devised where the KT19 is moved along one axis
until the displayed temperature drops off by 1 % of the displayed
temperature or by 1 C, whichever is greater. Then, the KT19 is
moved along the same axis to the other side of the aperture center until
another 1 % drop in temperature is observed. The KT19 is then moved
to the center of these two points. This procedure is performed for both
the X and Y-axes. Thus, the KT19 is centered in both directions.
A similar method has been suggested to determine size-of-source
effect [14] by moving a radiation thermometer from side to side and
noting its signal. This method uses a vertical slit as an aperture. AMF
uses a circular aperture, so it may be possible to use a similar method
with the data shown in Fig. 8 to determine size-of-source effect.
4.3 Long Term Stability History
The temporal stability of the reference standard must be considered
when establishing traceability and evaluating uncertainty [15]. The
instrument manufacturers specifications are frequently used as
an estimate for this component. In the case of this calibration, the
stability of the Heitronics KT19 is provided in its specifications [1].
However, it was found that the KT19s stability was much better than
its specifications, so this component of uncertainty was determined
by measurements. A linear drift model was chosen by performing
regression analysis on data obtained from 17 calibrations spanning
25 months.
The regression line confidence interval is a function of the number
of data points and the fitting precision. The uncertainties of the
individual data points were not considered because they are consistent
from point to point and will be introduced into the evaluation
elsewhere. Thus, the uncertainties of the projected line depend on the
confidence interval and expand smoothly as a function of time. The
equation used to determine the confidence interval [16] is shown in Eq.
(1). A graph of one such set of data is shown in Fig. 9. A comparison
of AMFs findings and the manufacturers specification are shown in
Table 3. In all cases, AMFs observed stability is much less than the
manufacturers specification. However, at the higher temperatures,
AMFs observed stability is closer to the manufacturers specification.
75
 TECHNICAL PAPERS

Drift / year
Temperature Stability
-Specification
(C) (mK / year)
(mK / year)

-15 6.1 310.8

0 -6.2 327.8
50 -44.9 387.8
100 -101.3 447.8
200 -84.3 567.8
350 -163.8 747.8
500 -560.0 927.8

Figure 9. KT19 drift. Table 3. KT19 stability summary.

5. Quality Control Measures Flat-Plate Calibrations
AMFs flat-plate calibrators are the models 4180 and 4181. A diagram
of the flat-plate calibration scheme is shown in Fig. 11. A number
of steps have been taken to reduce uncertainties in the flat-plate
calibration. First, the reflected radiation is controlled at near room
temperature. Second, for both the KT19 calibration and the flat-
plate calibration, the radiometric temperature of the optical scatter is
controlled by a water cooled aperture that is maintained at a constant
temperature close to room temperature. Third, the lower temperature
range of the flat-plate calibrators is 15 C. There are two calibration
points below ambient, 15 C and 0 C. Any radiometric calibration
(1)
done between 15 C and the dew point can cause dew or ice to
form on the calibrator surface which in turn can cause variations in
the radiation flux. Precautions, described below, have been taken to
Figure 10. KT19 ambient temperature transient. prevent this problem below the dew point.

5.1 Calibrations Below Ambient

Var yc' sY2Var
'

1 ' X 2 X 1 X X
yc sY x
'
(1) To prevent problems with humidity below ambient, the 4180 calibration
(1) inside a (1)
x
n S xx n S xx is done purged chamber [2] that encloses everything between
where: where: the KT19 and the flat-plate surface. This area is purged with a dry gas
Var(yc') Var(y c') of variance
variance
2
estimate of aestimate
point onofaafitted
pointline
on a fitted line at a positive pressure. Humidity is monitored to ensure that the frost
sYx ssample
Yx sample
variance of variance of the temperature
the temperature data curve fit
data curve fit point inside the chamber is well below the calibration point. To ensure
n nnumber of data
number
pointsof data points
X' time under consideration that no heat stacking or other thermal phenomena takes place on the
X' time under consideration
X Xsample mean sample
of themean of the time data
time data IR calibrator surface, tests have been run to ensure that the thermal
Sxx Svariance
xx of variance of the time data
the time data gradient and radiometric temperature on the surface is the same with
and without the chamber at calibration temperatures above ambient.
4.4 Transfer Standard Warm-up Time
The results of these tests are shown in Fig. 12.
AMFs calibration procedure specifies that the KT19 should be
warmed-up for 30 minutes prior to measuring the liquid bath
5.2 Calibration Quality Control Steps
blackbody temperatures. The reason for this warm-up time is based
Many of the quality control steps taken involve the calibration station.
on the accuracy specification from the manufacturer of 15 minutes
The calibration follows the same calibration geometry as the transfer
[1]. Further testing has been done to determine the transient time
standard calibration [2]. On the calibration station, the KT19 is
constant for warm-up. The result of one of these tests is shown in Fig.
mounted with the lens-cap removed. To ensure that foreign particles
10. In this test, the KT19 was enclosed in a temperature controlled
do not become incident on the lens, the area around the lens is entirely
water cooled jacket and its detector temperature was recorded over
enclosed in a box with a shutter. The shutter is only opened when
time. This data were fit to an exponential decay curve [17] with a time
a measurement is being made. In addition, the lens is periodically
constant of approximately 15 minutes.
cleaned using both a contact and a noncontact process.

Figure
Figure 9. KT19 9. KT19 drift.
drift.
76 | NCSLI Measure www.ncsli.org
Temperature
Temperature Stability Stability Drift / yearDrift / year -Specification
-Specification

Figure 11. Flat plate calibration geometry.
6. Conclusions
American Fork has established a quality radiation thermometry
program by building and qualifying a series of blackbodies. The
blackbodies have provided a radiation source for calibration of
radiometric transfer standards. These transfer standards have been
used to calibrate a series of flat-plate infrared sources intended for the
calibration of handheld infrared thermometers. In addition to using
a radiometric calibration for these sources, of other steps have been
taken to ensure the quality of these calibrations.
7. Acknowledgements
The authors thank those who have helped develop the radiation
thermometry capabilities at American Fork, especially Stephen King,
formally of Raytek in Santa Cruz, California, who provided many ideas
as this program developed. We also thank staff members of the Optical
Technology Division of NIST who provided knowledge of great value,
including Benjamin Tsai, Howard Yoon, Leonard Hanssen, Sergey
Mekhonsev, and Charles Gibson. Finally we thank the management at
American Fork for the opportunity to establish this program.
8. References
[1] Operational Instructions, Infrared Radiation Pyrometer KT 19
II, Heitronics, Wiesbaden, Germany, 2005.
[2] F. Liebmann, Infrared Thermometer Calibrator Development
at Fluke Corporation Hart Scientific Division, P. Soc. Photo-
opt. Ins., vol. 6939, no. 5, pp. 693906-1-693906-11, 2008.
[3] A. V. Prokhorov and L. M. Hannsen, Effective Emissivity of a
Cylindrical Cavity with an Inclined Bottom, Metrologia, vol.
41, pp. 421-431, 2004.
[4] V. I. Sapritsky and A.V. Prokhorov, Spectral Effective
Emissivities of Nonisothermal Cavities Calculated by the
Monte Carlo Method, Appl. Optics, vol. 34, no. 25, pp. 5645-
5652, 1995.
[5] A.V. Prokhorov, Monte Carlo Method in Optical Radiometry,
Metrologia, vol. 35, pp. 465-471, 1998.
[6] A. V. Prokhorov, S. N. Mekhontsev, L. M. Hanssen, Radiative
Properties of Blackbody Calibration Sources: Recent Advances
in Computer Modeling, Int. J. Thermophys., vol. 28, no. 6, pp.
2128-2144, 2007.
Vol. 7 No. 1 March 2012 NCSLI Measure |
TECHNICAL PAPERS
Figure 12. Radiometric measurements with and
without purge chamber.
[7] O. Stru, Transfer Radiation Thermometer Covering the
Temperature Range from -50 C to 1000 C, AIP Conf. Proc.,
vol. 684, pp. 565-570, 2003.
[8] C. E. Gibson, B. K. Tsai, A. C. Parr, Radiance Temperature
Calibrations, NIST Special Publication 250-43, pp. 9-36, 49-
52, 1998.
[9] F. E. Liebmann, T. Kolat, M. J. Coleman, and T. J. Wiandt,
Radiometic Comparison Between a National Laboratory and
an Industrial Laboratory, Int. J. Thermophys., vol. 32, no. 11,
pp. 2533-2543, 2011.
[10] J. Fischer, P. Saunders, M. Sadli, M. Battuello, C. W. Park,
Y. Zundong, H. Yoon, W. Li, E. van der Ham, F. Sakuma,
Y. Yamada, M. Ballico, G. Machin, N. Fox, J. Hollandt, M.
Matveyev, P. Bloembergen and S. Ugur, Uncertainty Budgets
for Calibration of Radiation Thermometers below the Silver
Point, CCT-WG5 on Radiation Thermometry, BIPM, Svres,
2008.
[11] C. F. Dietrich, Uncertainty, Calibration and Probability: The
Statistics of Scientific and Industrial Measurement, 2nd Ed.,
Adam Hilger, Philadelphia, pp. 282-283, 1991.
[12] IEC, Industrial process control devices - Radiation
thermometers - Part 1: Technical data for radiation
thermometers, IEC 62492-1 T, Geneva, p. 5, 2007.
[13] ASTM, Standard Test Methods for Radiation Thermometers
(Single Waveband Type), Annual Book of ASTM Standards,
vol. 14.03, ASTM International, pp. 4-5, 2010.
[14] M. Bart, E. W. M. van der Ham, P. Saunders, A New Method
to Determine the Size-of-Source Effect, Int. J. Thermophys.,
vol. 28, no. 6, pp. 2111-2117, 2007.
[15] C. D. Ehrlich, S. D. Rasberry, Metrological Timelines in
Traceability, J. Res. Natl. Inst. Stan., vol. 103, no. 93, pp. 93-
105, 1998.
[16] M. G. Natrella, Characterizing Linear Relationships between
Two Variables, in NBS Special Publication 300, Vol. 1, edited
by H. Ku, NBS, Washington, D. C., pp. 204-227, 1969.
[17] J. V. Nicholas, D. R. White, Traceable Temperatures, Wiley,
Chichester, UK, pp. 140-143, 2001.
77
New Products and Services
Guildline Instruments Receives US Patent
for New High Current Measurement System
Guildline was awarded US Patent No. 8,106,669 on January 31,
2012 for its unique and modular 6623A series of range extenders
and precision current sources. According to Guildline president
Richard Timmons The 6623A is truly a unique modular precision
instrument capable of sourcing direct currents from 3 Amps to
10,000 Amps and more in any increment of 150 Amps. It contains
an extremely stable, low noise, uni-polar, high speed, electronic
switched current source. The 6623A has an available stand-alone
programmable controller for use as a precision source. Guildline
New Cloud-based Service Offering
from AssetSmart
has also extended the range of its industry leading precision
9230A DC shunts to 10kA. AssetSmart is excited to announce that its SMART|CMS,
corporate calibration/maintenance system is now
For more information on the 6623A or other available as a Software-as-a-Service (SaaS) offering.
Guildline products, visit: www.guildline.com This new service will allow both in-house and
commercial labs to move their technology into a hosted
secure service environment, enabling substantial IT
savings. This new SaaS offering by AssetSmart provides
a secure, scalable and affordable solution with reduced
implementation and IT infrastructure costs.
BetaProbe TI/TI+ High Precision Digital
Thermometers
For more information contact, Hilary Wisen,
A single BetaProbe TI high precision digital thermometer can
Marketing Manager, 310-450-2566 or email
replace many liquid-in-glass (l-i-g) thermometers. It can also
sales@assetsmart.com
serve as a reference standard for other types of digital or analog
temperature indicators. Plus, its intrinsically safe certification
means it can be used virtually anywhere.

The BetaProbe TIs integral 3/16 (5 mm) diameter probe houses

Vaisala HMW90 Series Humidity
a quick response thin film sensor for quickly acquiring highly
and Temperature Transmitters for
accurate readings using minimal insertion depth. The BetaProbe
Demanding HVAC Applications
TI+ uses a (6 mm) diameter probe and a wire wound sensing
The wall-mounted Vaisala HMW90 Series HUMICAP
element. The integral probe also rotates through 90 for any angle
Humidity and Temperature Transmitters measure relative
reading of the display.
humidity and temperature in indoor environments
where high accuracy, stability, and reliable operation
For more information email: bill@martelcorp.com
are required. The HMW90 series transmitters are quick
and easy to install, calibrate, and adjust. Models for both
current and voltage output are available. Transmitters
include a display and a sliding cover with either a
display opening or a solid front. Output options include
both special scalings and calculated dew point and
mixing ratio parameters.

Visit www.vaisala.com and search keyword

HMW90

78 | NCSLI Measure www.ncsli.org

TECHNICAL PAPERS NEW PRODUCTS AND SERVICES

Light Lab
Essco Calibration Laboratory is one
of the few third-party laboratories
in the nation to offer photometry
calibration. The Hoffman
Engineering PCS-600 is capable
of calibrating photometers used
for incident light measurement
(foot-candle or Lux meters).
Housed in our newly designed
dark room, Esscos self-contained
Illuminance Projector and detector
can accurately calibrate your
photometer at a correlated color
temperature of 2856 Kelvin from
0 to 2000 foot candles or 0 to
21,520 lux.

For more information,

contact Joe Morse
at: jmorse@esscolab.com

Fluke Acquires Martel Electronics Corp.

Fluke Corp. has acquired Martel Electronics Corporation, a Derry,
NH manufacturer of electronic test tools and original equipment
manufacturer (OEM) products. Martel will continue to operate
from its current location and be managed by current president
Tom Fatur. Martel Electronics offers handheld and bench
calibrators, process instruments, process indicators, and power
supplies.
Do you have a new product or service announcement
For more information on Martel, you would like featured in NCSLI Measure?
visit: www.martelcorp.com
Send it to: lstone@ncsli.org

Vol.
79 |7 No. 1 Measure
NCSLI March 2012
NCSLI Measure | 79
www.ncsli.org
 to the Real Asset World.

CALIBRATION AND LAB MANAGEMENT...

EQUIPMENT MANAGEMENT...

YOUR CUSTOMERS...

YOUR END USERS...

YOUR ERP/FINANCIAL SYSTEM...

WEB BASED ACCESS ANYWHERE IN THE WORLD...

THE CLOUD...

Advertisers Index
Integrated Solutions. One Smart Connection.
Asset Management 360, a true asset management solution,
elevates asset management to a whole new class of enterprise
integration and visibility. Connecting your physical operations
and end users with your ERP/Financial Systems. AM360 ultimately
improves asset utilization and operational efficiencies.

SMART/CMS Calibration and Laboratory Management Software

is the only system powerful enough to manage all of your instrument

Agilent Technologies calibration, repair, maintenance, asset tracking and lab management
needs in a single solution. SMART/ENCATS Enterprise Cataloging

www.agilent.com Back Cover

provides a common tracking language across the enterprise.

Save time and money. Now Thats SMART.

AssetSmart
www.assetsmart.com Inside Back Cover By PMSC

2800 28th Street, Santa Monica, California 90405 USA 310.450.2566 info@assetsmart.com www.assetsmart.com

Under Pressure To Make Connections Fast,

BIPM Calibrate Faster!

www.worldmetrologyday.org Page 37
Calibrate Faster? For transmitters, gauges and pressure
sensing instruments
No wrenches or thread sealants
Eliminates thread damage
Reduces contamination risks

CPEM 2012
Twist-to-connect, sleeve and handle actuated
connectors save time

http://icpem.org/2012 Page 61
Leak-tight sealing from vacuum to 10,000 psi
1/8" to 1/2" NPT, AN37 flare, sanitary style
flange and metric with additional sizes
available

Essco Calibration Laboratory Improve your lab efficiency

www.esscolab.com Page 15 calibration connection manifold

for calibration of up to four
instruments simultaneously!

FasTest www.fastestinc.com
800.444.2373
www.fastestinc.com Page 9

Fluke Calibration
www.fluke.com Page 6
Certified to ISO 9001:2008 and accredited to ISO/IEC 17025:2005
Over 30 metrology technicians on staff.

Guildline GUILDLIN
Calibration
Cert No. 1388.01

www.guildline.com Page 12 6625T T EMPERATU

Isotech North America
www.isotechna.com Page 29

Martel Electronics
www.martelcalibrators.com Page 12 B
Measurements International P
www.mintl.com Page 11 BetaGauge P
High
PIR PRO P
Mensor Corporation
Accuracy
Reference
Class
.. A
www.mensor.com Page 15
.. 17D
Morehouse Instrument Company
www.mhforce.com Page 13
. InD
Single
Handed
.. Ren
NCSLI Expand Your Reach Process Pn
www.ncsli.org Page 71 Validation

NCSLI Joe D. Simmons Scholarship

www.ncsli.org Page 17

NCSLI Training Center

www.ncsli.org Page 2

NCSLI Workshop & Symposium 2012

www.ncsli.org Page 18

Northrop Grumman
www.northropgrumman.com Inside Front Cover

Ohm-Labs, Inc.
www.ohm-labs.com Page 29

Vaisala
www.vaisala.com Page 17

80 | NCSLI Measure www.ncsli.org

ASSET MANAGEMENT 360.

C NNECT to the Real Asset World.

CALIBRATION AND LAB MANAGEMENT...

EQUIPMENT MANAGEMENT...

YOUR CUSTOMERS...

YOUR END USERS...

YOUR ERP/FINANCIAL SYSTEM...

WEB BASED ACCESS ANYWHERE IN THE WORLD...

THE CLOUD...

Integrated Solutions. One Smart Connection.

Asset Management 360, a true asset management solution,
elevates asset management to a whole new class of enterprise
integration and visibility. Connecting your physical operations
and end users with your ERP/Financial Systems. AM360 ultimately
improves asset utilization and operational efficiencies.

SMART/CMS Calibration and Laboratory Management Software

is the only system powerful enough to manage all of your instrument
calibration, repair, maintenance, asset tracking and lab management
needs in a single solution. SMART/ENCATS Enterprise Cataloging
provides a common tracking language across the enterprise.

Save time and money. Now Thats SMART.

By PMSC

2800 28th Street, Santa Monica, California 90405 USA 310.450.2566 info@assetsmart.com www.assetsmart.com
 If youre here, OEM cal/repair service
is closer than you might think.

You dont need to give up local convenience to enjoy

L cal Agilent Reepa
Lo p ir
i & Calib
ibra
braati
tion Opt
tion ptio
ions
io nss
OEM-quality calibration and repair services. Agilent
provides a host of local-access options to t your Local Servicee Ceentter
es FFaast
s picck-
k-up
upp anndd del
eliv
iver
ivery
er
ry
specic needs. Together with our delivery partner, Moobi
bile
le Cal
al Labbs Ouur Se
Serv
rvic
vicce Ce
C nt
nter
e com
er o ess to yo
you
we service Agilent and non-Agilent products to
keep you up and running wherever you are. System Upt
p im
ime O r Syyssttem
Ou e Eng
n in
inee
eerss com
o e to you
o
Resident Professionaal Dedi
Dediccaate
ted Seervvicee Centterr on yo
y ur site
Thats thinking ahead. Thats Agilent.

Agilent Preferred Service Partner

Plan for local service and much more
Discover how Agilent can help
www.agilent.com/find/planlocalservice

2012 Agilent Technologies, Inc. u.s. 1-800-829-4444 canada: 1-877-894-4414 mexico: 1-800-254-2440