STRATEGY EXECUTION
Data Quality: You Don't Just
Need a Dashboard!
By Peter Graham
Peter would liiie to thank Brian Swarbrick of Palladium
Group for iois contribution to tiois column.
T
hroughout this series, I have discussed how or-
ganizations can become information empowered,
with an inherent assumption that it's accurate
information. When this assumption is incorrect,
the results can be devastating: decisions made
with poor or incomplete information can often
impact a company's bottom line more negatively than if the
information were not presented at all. This month, I'll discuss
how data quality issues occur and how your organization
can begin to identify and address them.
Data quality issues arise for a variety of reasons, includ-
ing acquisitions of new data sources, poorly designed and
developed applications, and ironically business intelligence
(BI) implementations, where there can be focus on data pres-
entation to the detriment of understanding information accuracy.
The popular adage "you just need a dashboard" doesn't apply
anymore. A well-architected BI solution that enforces data in-
tegrity and implements robust data quality procedures is a
requirement for any enterprise solution but should not be seen
as a complete solution for the organization's data quality issues.
Data quality is a business issue that transcends the entire organ-
ization, including operational systems.
Gartner predicted that more than 50 percent of BI proj-
ects through 2007 would fail or receive lack of acceptance
because of data quality issues.' TDWI estimated that customer
data quality issues alone cost U.S. businesses over $600 bil-
lion dollars a year.^ With the enactment of the Sarbanes-Oxley
(SOX) Act, quality of information is an ever-increasing pri-
ority. A clear initiative that focuses on the data quality
challenge is required to address this situation.
Data quality is a business issue, and business managers
should drive improvement. The first step in identifying data
quality issues is to interview business users across the enterprise,
assessing their use of data and potential opportunities to improve
data quality. This assessment can be done using external re-
sources or leveraging roles from the business intelligence
competency center. The key to this assessment is to understand
business uses of data, therefore this requires an individual with
business expertise. Once this assessment is complete, the data should
be traced to its source systems to investigate the cause of poten-
tial issues. From a BICC perspective, the data architect is often
most qualified to lead this investigation. While this investigation
could be done through human evaluation of data and code,
there are now many data discovery tools that evaluate data. Once
documented, data quality opportunities should be prioritized based
on the perceived financial and operational impact, the resources
required to address them and any project interdependencies to
36 August 2008 I DM Review
Figure 1: Gartner's Magic Quadrant for Data
Quality Tools, 2007
Challengers Leaders
Trillium Software
• • Business Objects
• DataFlux
Group 1 Software • IBM
, FUZZY! Informatik • Informática
O
II I ^ •Human Inference
% uniserv ^ Innovative Systems
•DataMentors ,
Netrics • DataLever
Datanomic*
Datactics •
Niche Players Visionaries
HCompleteness of Vision f
As of June 2007
Source: Gartner Inc., June 2007
create a roadmap for enterprise data quality improvement.
Depending on organization size, the data steward(s)
should have complete responsibility for data quality gover-
nance in his or her area. These gatekeepers must understand
their respective systems, the quality of information across those
systems and the implications of poor data quality. They
must be subject matter experts that provide direction anywhere
their information is used and be the sole point of contact for
all decisions related to the data they own. This data gover-
nance role should fall within the BICC if one exists.
The rr organization should support the data quality improve-
ment initiative, ensuring effective implementation of processes
and procedures and making supporting technologies available.
To assist IT in this endeavor, many vendors now offer soft-
ware solutions that support enterprise data quality initiatives.
Gartner now categorizes these data quality products as a sepa-
rate commodity, although vendors offer several of the underlying
components as standalone offerings (see Figure 1).
These tools help to identify, discover and improve data
quality anomalies. Typically required for implementing qual-
ity BI applications, these products should also be used to
support data quality initiatives beyond a BI engagement.
Typical functionalities available in these products include:
' Data discovery: Quickly analyzes large volumes of data
across different applications and provides statistics regard-
ing the data content (unique values, missing content, data
formats, etc.). Analyses can be used to drive change back
through the operational systems and BI applications.
' Data cleansing: Integrates and cleanses customer data
GRAHAM continued on page 37
www.dmreview.com
 SCEALES and MORRIS continued from page 21
flexibility to continue migrating while this is done.
Rule four: If you can't count it, it doesn't
count. Another challenge is how to meas-
ure data quality in order to assess the
state of your legacy data and determine
the level of quality your business users
require. To make matters worse, data quality
is not static; it erodes and improves over time. It is
important that the measures you use make sense to
business users and not just to technologists. This
enables you to measure deliverabies, perform gap
analyses and monitor and improve ongoing data qual-
ity. It also ensures that you are concentrating your
efforts on where business users see value and can
quantify the benefits.
Reconciliation of the data migrated from
source(s) to target(s) is always a critical activity -
how do you know when you're done? When dealing
with dynamic environments where you can't freeze
the data, this becomes even more challenging;
you need to be able to handle a shifting scope. A
flexible data model and a closely coupled
reporting capability are key to understanding and
driving this process.
Achieving a Business-Driven Migration
Having put a business-driven migration project in place,
the trick is to select methodologies and technologies that
can deliver against these requirements. A business-driven
migration involves decoupling the technical problem of
moving the data from the business processes that use it.
This requires a migration solution that enables you to
easily encapsulate the business problems you face while
being flexible enough to cope when those requirements
change. This ensures that ROI from new applications is
maximized and operations are enhanced rather than
adversely affected.
The critical importance of getting the migration
right from a business perspective was articulated at a
recent British Computer Society meeting by BT's Phil
Dance: "Increasingly, our business case is going to
depend on how good we are at getting our data
across. A bad data migration ultimately means a bad
customer migration, and in a competitive market,
that's very bad news." O
References:
1. Phil Howard and Carl Potter. "Data Migration in the Global 2000 -
Research, Forcasts and Survey Results." Bloor Research, September
2007.
2. Phil Dance. "BT PIC's CIO for Technology." Data Migration
Matters, November 2007.
Tony Sceales is CTO of third-generation migration technology vendor,
Celona Technologies. He may be reached at tony.sceates@celona.com.
John Morris is director of iergo ltd and author of Practicat Data
Migration (British Computer Society, 2006). He may be reached at
imorris@iergo.co.uk.
www.dmreview.com
GRAHAM continued from page 36
(primarily name and address data) from multiple sources and geo-
graphic locations for individuals and organizations.
• Matching or householding: Groups individuals or organizations
together for deduplication or to understand relationships, a task
impossible to achieve without a robust data cleansing strategy.
' Ongoing data quality monitoring: Tracks data quality improvements
over time.
• Master data management: Defines and maintains the data critical
to an organization's business processes.
' Integration with Bl and operational systems: Defines rules for improv-
ing data quality that existing applications can integrate. Changes to
rules are automatically available to subscribers of that information.
To be truly successful, a data quality initiative must be adopted
across the organization, and there must be no barriers. With IT and
business management working together, improvements in the qual-
ity of data can be adopted across the operational systems. Bl
applications will also benefit from improvements in quality. Business
decisions will become reliable, and in turn, impact on the bottom line
will be positive. O
References:
1. "Gartner Identifies the 'Fatal Flaws' of Business Intelligence and Advises Organisations
on How to Avoid Them." Press Release from Gartnei; Inc., February 3,2005.
2. "The Data Warehousing Institute's Recent Study Finds High Quality Data Is Critical To
The Success Of Businesses Worldwide." Press Release from TDWI, February 1,2002.
Peter Graham is the vice president ofthe tnformation Strategy and Architecture practice
at Palladium Group, Inc. He can be reached at pgraham@thepalladiumgroup.com.
AddressObject API
Prevent errors before they occur to
ensure the integrity of your database.
Use AddressObject API to verify,
cieanse and format customer data at
the point of entry or in batch. Address
Object easiiy integrates with Perl, PHP
Java and .NET. - offering a better way
to build-in data vaiidation.
The ability to maintain clean, reliable
marketing data across multiple capture
mediums is essential for everything we do
- Kevin Parker, Digital Evergreen
DM Review I August 2008 37