MEASURE Evaluation

Data Quality Assurance Workshop

Session 1:
What is Data Quality?
 Topics to Cover
 Understand IMF Data Quality Framework
 Conceptual Framework for Data Quality
 Cycle of Information
 Identify Seven Dimensions of Quality
 Understand the Importance of Data Quality
 Key Factors in Ensuring Data Quality
 What is Data Quality?
 Users have different expectations of data and statistics:
- accurate and timely
- comprehensive and cost effective
- locally relevant and also comparable with other countries

 Weight of different criteria depends on user needs – "fit-for-purpose“

 No unique quality ranking possible
 Potentially conflicting requirements: trade-offs, e.g. accuracy versus timeliness
 Perfection is unattainable
 In the absence of a strong proactive effort, quality decreases over time
 Aim for a progressive improvement of data quality with constant vigilance and
regular evaluation to maintain quality
 Why Data Quality Matters

Policy & budget

decisions impact health
outcomes

Data collection Aggregation & Analysis Impact on Health

Data Quality Assessment Framework
(DQAF)
 DQAF in Detail

Prerequisites of Quality

• Legal & institutional environment

• Resources
• Relevant
 DQAF in Detail
Prerequisite of Quality

Assurance of Integrity

• Professionalism
• Integrity
• Ethical Standards
 DQAF in Detail
Prerequisite of Quality

Assurance of Integrity

Methodological
Soundness

• Scope
• Classification
• Basis for recording
 DQAF in Detail
Prerequisites of Quality

Assurance of Integrity

Methodology Soundness

Accuracy & Reliability • Data sources

• Statistical techniques
• Assessment and validation
data
• Revision studies
 DQAF in Details
Prerequisites of Quality

Accuracy of Integrity

Methodology Soundness

Accuracy & Reliability

• Periodicity &
Serviceability
timeliness
• Consistency
• Revision policy &
practice
 DQAF in Details
Prerequisites of Quality

Accuracy of Integrity

Methodology Soundness

Accuracy & Reliability

Serviceability
• Data accessibility
• Metadata
Accessibility
accessibility
• Assistance to users
 Purpose of DQAF
Purpose: Improve the quality of data used by countries for reviews
of progress and performance

Principles
• Adhere to international standards
• Promote transparency of data and methods
• Enhance country capacity including user friendly tools
• Support to institutional mechanisms for data quality
assessment
Data & the Information Cycle

Interpret
 Data Quality Matters
 Strengthens effectiveness in fight against the “Three
Ones” – HIV/AIDS, TB, Malaria
 Helps make efficient and effective use of resources
 Helps improve accountability
 Helps improve program results
 Increases trust in data and in their use for decision-making
 Helps be prepared for audits
Conceptual Framework for Data
Quality
Dimensions of Quality
Accuracy - they measure what they are
intended to measure
Completeness - collected
comprehensively
Reliability - repeated measurements
using the same procedures get the same
results
Timeliness - up-to-date & available on
time
Confidentiality - clients’ data are not
disseminated
Precision - have sufficient detail to use
for decision-making
Integrity - protected from bias or
manipulation
Dimensions of Data Quality in RDQA
 Validity
• considered accurate -- they measure what they are intended to measure
 Timeless
• means that data are sufficiently current and frequent to inform
management’s decision-making
• They are received by the established deadline
 Completeness
• Comprehensive data collection
• percent of all fields on data collection form filled in
• percent of all expected reports actually received
How Good does the Data Need to Be?
 No data are perfect!
 Use professional judgment
 Data should be good enough to document performance and support
decision-making
 Document decisions and supporting information
 Different objectives/indicators may require different levels of
measurement quality
 The expected change being measured should be greater than the
margin of error
Functional Components of an M&E
System to ensure Data Quality
 Key Factors in Ensuring Data Quality
 Standard data collection tools and reporting forms
These 8 factors affect data quality
at all levels of information system

 Steps addressing quality challenges

 Specific reporting timelines
 Description of roles and responsibilities
 Storage policy that allows to retrieve data
 Functioning information systems
 Clear definitions of indicators
 Document data review procedures
Questions & Answers
MEASURE Evaluation is funded by the U.S. Agency for
International Development (USAID) and implemented by the
Carolina Population Center at the University of North Carolina at
Chapel Hill in partnership with Futures Group, ICF International,
John Snow, Inc., Management Sciences for Health, and Tulane
University. Views expressed in this presentation do not necessarily
reflect the views of USAID or the U.S. government.

MEASURE Evaluation is the USAID Global Health Bureau's

primary vehicle for supporting improvements in monitoring and
evaluation in population, health and nutrition worldwide.
www.measureevaluation.org