Case Study

More Data,
More Problems?
“I
n God we trust; all others bring data,” W.
Advice on Edwards Deming once said. And do we ever

ensuring bring data:

It’s been estimated that we create about 2.5 quin-
data quality tillion bytes of data per day, VcloudNews reported

and integrity
a few years back. That’s the equivalent of 10 million
Blu-ray discs that, if piled high, would almost reach
in the age of the height of four Eiffel Towers stacked one on top
of another.¹ At the same time, it was estimated that
big data about 90% of the world’s data have been created
in the last two years, according to multiple sources,
Topic including IBM.²
Data quality How many rows did you say Minitab could
handle?
Author Recently, Six Sigma Forum Magazine editor Jim
Gary S. Netherton
Bossert wrote that one of the many challenges the
Email Six Sigma community faces is handling big data.³ As
gary.netherton@live.com a Six Sigma Black Belt who has pursued semi-formal
(Udemy.com and Johns Hopkins University) and
personal training in data science (DataCamp.com as
well as many book purchases from Amazon), I have
attempted to use these data science techniques in
my profession as a quality manager and senior pro-
cess engineer. What I’ve learned is that the basics
of Six Sigma’s define, measure, analyze, improve
and control (DMAIC) measure phase are applicable,
regardless of data size.

Are the data good?

The first question I’ve learned to ask is, “Do I trust
the data?” The following is an embarrassing exam-
ple that explains why.
Earlier in my career, my employer was involved
in a dispute with a component supplier regarding
its parts that we used to manufacture our finished
product. Our service history showed an abnormal fre-
quency of failure. This weighed heavily on our service
department, as well as on our warranty costs.
I had been provided a database full of service
information that was coded for failures related to
this component. During a negotiation with the ven-
dor, the vendor asked about a field in our database
that was a code providing summary information
about the finished product—including information
regarding whether it used that vendor’s component.
After the vendor asked a couple of questions, we
quickly realized that our data were not completely
accurate. As a matter of fact, 5% of the product we
were accusing the vendor of causing to fail not only
didn’t contain its component, but it also couldn’t

10 |
November 2018   LEAN & SIX SIGMA REVIEW
contain its component because it did not use that
kind of device. The situation was analogous to
saying that an ink toner cartridge was causing a car
to stall.
This led me to begin questioning any data that
I analyzed. Later, I found several service jobs that
reported replacing components on products that
didn’t use those components, just like the earlier
example. What started as confidence in the data
was quickly proven to be 95% accuracy. Over sub-
sequent months, 95% became 90%, and eventually
the team crossed its fingers and prayed the data
were even 80% to 85% accurate.
With the significant increase in data and its easy
availability, it becomes more critical that quality and
process improvement professionals understand the
quality of the data. I am a firm believer that having
no data is better than using bad data. At least if you
have no data, you can begin the search for data,
ensuring its integrity.
Six Sigma instruction teaches several data review
methods such as measurement system analysis,
control charting and gauge calibration. These meth-
ods ensure that gauges are useful, the measurement
system works and users understand what their
data are doing while attempting to explain natural
variation versus assignable cause.
When businesses create huge databases (for
example, Hadoop or MongoDB) that serve as data
stores for organization information, there is often a
curtain separating the analyst and the data source.
Users may access a data warehouse through a front
end—such as IBM Cognos or another customer
user interface—but have no background on where
exactly the data came from—something that is
important to know.
Where did the data come from?
Thanks to the internet making the process easier,
quality and process improvement professionals are
often tasked with working on data from a multitude
of sources that may or may not be in their building,
their city or even their country. A shortcut on a
computer starts an interface that the professional
uses to access the data warehouse or database. It is
possible that the data may be identified in a manner
to tell the end user where it came from geographi-
cally or what facility (for multifacility organizations)
provided the data, but there are no guarantees.
Sometimes, the facility providing the data on
the product or component is not the same facility
that manufactured it. Consider field service data,
for instance. According to the Certified Reliability
Engineer Handbook, “This information is often
impacted by differences in installation, environment,
operator procedures and similar factors that make
analysis difficult.”⁴
In instances where data are not field service data,
it may be real-time data from equipment running in
a facility and stored continuously in a database. In
these instances, it is nevertheless important that the
gauges or devices used are accurate—something
that’s accomplished through regular calibration.
Another embarrassing episode in my career
involved temperature measurements needed for the
appropriate application of a secondary sealant. The
sealant was applied in conjunction with a primary
sealant that required a minimum temperature to
ensure a hermetic seal. The vendor of both sealants
had specific guidelines for the application of both to
create a successful airtight unit.
During a visit to one of my facilities, I noted an
operator applying sealant at a temperature (per the
digital readout) 20 degrees Fahrenheit below the
desired minimum. When maintenance was called
to evaluate the situation, it discovered that the
Thanks to the readings on the display were 40 degrees Fahrenheit
internet making lower than what was being displayed, meaning that
the process easier, the actual temperature was 60 degrees Fahrenheit
quality and process lower than the desired minimum.
improvement When I confronted the plant leadership about
professionals are the need to regularly calibrate the equipment, the
often tasked with response was, “We didn’t have time.” While this
working on data is more often an excuse given in today’s factory
from a multitude of settings, it is a reality in today’s world of manufac-
sources that may turing. Management often wants more from less.
or may not be in Nevertheless, such situations remind us to ques-
their building, their tion the collected data used for quality or process
city or even their improvements. Using bad data to make changes
country. can lead to wrong decisions that can cause more
problems than if nothing had been done.
How are the data arranged?
Data are often presented in a dirty (that is, not
clearly formatted) and inconsistent manner. I once
used a data warehouse in which the manufacture
date was formatted as 200905 (May 2009), but
the service requested data was formatted 20130412
(April 12, 2013). Midway through my tenure at this
firm, the latter was changed to 2013-04-12. For the
data-wrangling reader, all data were formatted as
text and not as a date, making date-related calcula-
tions impossible without data-type conversions.
Matters are complicated further when data are
presented in a multipage format in a text file. Every
page has multiple lines of header information that
may or may not be consistent in its layout—and with
the meat of the data in the bottom two-thirds of
|
LEAN & SIX SIGMA REVIEW   asq.org/pub/sixsigma 11
Case Study > More Data, More Problems?
each page. Compiling this data into a format that
would be accepted by Minitab or even Microsoft
Excel is not intuitive, and typically is not something
many users are used to doing on a regular basis.
While there are several tools available that make
cleaning this data less of a burden than even a
couple of years ago, there is no easy way to simply
import the data into Minitab or other Six Sigma
software, at least for the average user. With formal
training or self instruction, there are several power-
ful tools available from Microsoft that enable data
wrangling in an effective manner. I only bring up
Microsoft because the Office Suite is typically what
an employee is provided by his or her organization.
Some organizations provide Minitab, Python or R.
In most cases, however, there are several obstacles
to getting these nonstandard software packages
approved by corporate IT for installation.
Understanding how to handle the data is no less
important than the other concerns mentioned so far.
Proper data arrangement helps users and analysts
go the extra mile to understand the data records
and fields within each. Anand Tamboli, the director
of emerging technologies at Knewron Technologies,
an electronics manufacturer in India, reiterates this
when addressing the importance of data mining to
Six Sigma:
“Since quality of results is as good as the quality
and treatment of data, it is highly recommended to
follow the data mining approach religiously while
working on the measure and analyze phases.
“While Six Sigma itself contains some of the data
mining steps, it does not provide detailed know-
how of these steps.”⁵
Tamboli continues to address specific technical
steps for data mining, but not before first stating
that preprocessing (cleaning) the data is a critical
first step.
“Cleaning can remove data with noise and
missing information points. It is also necessary to
validate integrity of data points in a set. These are
essential steps to obtain sanity in the results.”⁶
If only I’d read this earlier in my career, I could
have helped save my employer some embarrass-
ment. Expending the extra energy to ensure data
quality and integrity is now a given in my work.
What can you do?
REVIEW EXPECTATIONS. What are you trying to
collect data for? Is it a critical-to-quality character-
istic of the process or a dimension of a component?
Why do you want that data? Implementing a
measurement system typically requires some capital
or, in some cases, some expense funding. Given the
12 |
November 2018   LEAN & SIX SIGMA REVIEW
needs of the organization, it is important to ensure
that the data collected are reliably stored in an
accessible manner so that quality and processes can
be monitored and analyzed.
After you know why you want data, it is import-
ant to understand whether you are collecting the
right data. Don’t assume the folks running a process
were the ones who established the process. Perhaps
they inherited a machine or a collection of equip-
ment with preestablished data collection. That data
may or may not be worthwhile based on the needs
of the organization or department. If you’re running
a process in which the pressure and temperature
are important, for example, more than likely it won’t
make any sense to measure voltage.
What are the units involved? Degrees Fahrenheit
or Celsius? Meters per minute or feet per min-
ute? This may seem like a simple enough issue to
address. Identify the units needed and ensure that
those are the units used. Purchase a new piece of
equipment that goes in your Richmond, VA, plant
from a German organization, and all the unit dis-
plays on the machine will be in imperial units, right?
This scenario is not only common, but it also
often causes headaches because a thorough check
is not completed before commissioning the equip-
ment. And it is not unheard of to have equipment
that has metric and imperial units, depending on the
measure displayed.
Confirming all readings and metrics are correct
and in the correct units is vital to successfully
running a machine, completing an airplane flight⁷ or
landing on Mars.⁸
GO TO THE GEMBA, AND REVIEW CALIBRATION
AND PREVENTIVE MAINTENANCE SCHEDULES.
Only by properly calibrating measuring devices can
you be confident in the correctness of measurements While there are several
taken by trained personnel using those devices. If a tools available that
machine setup requires certain speeds and pressures make cleaning this
to properly form a part, it behooves the equipment data less of a burden
owner to confirm that readouts and displays on the than even a couple of
equipment are accurate. Calibration is the only way years ago, there is no
to truly confirm this. Calibration isn’t just for hand easy way to simply
tools such as calipers, micrometers or measuring import the data into
tapes. Thermocouples and pressure sensors on Minitab or other Six
automated equipment require the same amount of Sigma software, at
checking to always ensure they display as accurate least for the average
a value as possible. A regularly scheduled calibration user.
programs helps ensure that.
REVIEW OPERATOR TRAINING AND UNDER-
STANDING. For labs, “ISO/IEC 17025, subclause
4.1.5 d, requires that your management system ‘have
policies and procedures to avoid involvement in
any activities that would diminish confidence in its
competence, impartiality, judgment, or operational
integrity.’”⁹ This should apply to all operations, as
well as certified labs.
A colleague recently pointed out that he had
caught an operator on the floor falsifying data.
Certain checks were to be completed hourly and
the time of sample collection written on the part.
The operator had not collected samples regularly.
When he finally collected the number of samples,
he did so at one time and backdated some samples,
giving the impression that he’d been doing his job.
This deception was recognized when my colleague’s
quality department found nonconformances in
the hourly samples, but none in the product that
had allegedly been made when the samples would
have been cut. Putting two and two together, the
team determined that the sample submission was
fraudulent.
The importance of providing operators with
required training and needed resources cannot
be understated. Additionally, a third leg to that
stool would be holding personnel accountable for
compliance with company policies regarding data
integrity. While my colleague did not work in a
regulated industry, his scenario could have cost his
organization significantly with a quality spill and the
associated costs (financial and reputational).
REVIEW TOOLS’ ADEQUACY FOR THE JOB.
Data collection devices may need to be updated
to handle the measurements involved or even for
the ability to communicate with a cloud versus a
local server. In one instance, a company needed
to upgrade a device because it could not generate
sufficient heat to melt beads in preparation for mea-
surement, as well as being unable to measure those
higher temperatures. This testing and measurement
was critical to quality, but the existing equipment
could not handle the requirements, a fact that one
of the engineering directors discovered when dig-
ging into the details of testing a new product.
Never hesitate to question
Above and beyond these recommendations, don’t
be afraid to question the data. As an analyst, a
Six Sigma belt, or even simply a quality or pro-
cess improvement professional, do not shy away
from becoming intimately familiar with the data
presented. Only by having more than a passing
familiarity with the data can you increase confi-
dence that following a data analysis process will
yield accurate results.
There can be no doubt that developing this
increased familiarity is time consuming. Asking ques-
tions, doing research and spending more time on
the floor with operations can be frustrating, exhausting
and certainly appear to be a real drag on resources.
Nevertheless, trying to solve a problem with bad data
is a recipe for disaster. It can cause problems that, in
many unfortunate cases, will not be found before the
customer finds them or, in a worst-case scenario, a
catastrophe occurs.
With all due respect to Deming, I would modify his
quote to the following: “In God we trust, all others
bring good data.”
REFERENCES
Data collection 1. Ben Walker, “Every Day Big Statistics—2.5 Quintillion
devices may need to Bytes of Data Created Daily,” VCloud News, April 5, 2015,
be updated to handle https://tinyurl.com/vcloud-big-stats.
2. Ralph Jacobson, “2.5 Quintillion Bytes of Data Created
the measurements
Every Day. How Does CPG and Retail Manage It?” IBM,
involved or even April 24, 2013, https://tinyurl.com/ibm-jacobson-data.
for the ability to 3. James L. Bossert, “Is Quality 4.0 the End of Six Sigma?”
communicate with a Six Sigma Forum Magazine, Vol. 27, No. 3, May 2016, p. 4.
4. Donald W. Benbow and Hugh W. Broome, The Certified
cloud versus a local
Reliability Engineer Handbook, second edition, ASQ
server. Quality Press, 2013.
5. Anand Tamboli, “An Introduction to Data Mining,”
Process Excellence Network, July 19, 2010, https://tinyurl.
com/intro-data-mining.
6. Ibid.
7. Richard Witkin, “Jet’s Fuel Ran Out after Metric
Conversion Errors,” New York Times, July 30, 1983,
https://tinyurl.com/nyt-jet-fuel.
8. Robert L. Hotz, “Mars Probe Lost Due to Simple Math
Error,” Los Angeles Times, Oct. 1, 1999, https://tinyurl.
com/la-times-mars-errors.
9. Jason Stine, “ISO/IEC 17025: Data Integrity Begins With
Employee Integrity,” Quality Digest, Aug. 30, 2010,
https://tinyurl.com/qd-data-integrity.
GARY S. NETHERTON is a qual-
ity manager at Northwest Door in
Puyallup, WA. He holds a bachelor’s
degree in electrical engineering from
Kettering University (formerly named the GMI
Engineering and Management Institute) in Flint,
MI. Netherton is an ASQ senior member and an
ASQ-certified quality engineer, manager and
Black Belt.
|
LEAN & SIX SIGMA REVIEW   asq.org/pub/sixsigma 13