Monitoring and Evaluation:

FAMILY PLANNING PROGRAMS

Session Objectives
Be able to apply basic M&E concepts
(frameworks, indicators, etc.) to family-planning
programs
Be able to summarize the main issues in M&E
of family-planning programs from a post-Cairo
perspective.
Be able to summarize the emerging issues for
M&E of family-planning programs in high HIV
prevalence countries.

Session Overview

Family-planning frameworks
M&E implications of the Cairo agenda
Contraceptive prevalence and unmet need
Monitoring quality of care
Evaluating the impact of quality
Family planning and HIV

Family Planning Frameworks

Conceptual Framework for FP Demand

and Program Impact on Fertility
Other
intermediate
variables
Societal &
individual
factors

Value &
demand for
children

Fertility

FP demand

Wanted

Spacing
Limiting

Contraceptive
practice

Unwanted

Service outputs:

Development
programs

Access

FP supply
factors

Service Utilization

Quality
Acceptability

Source: Bertrand, Magnani, and Rutenberg, 1996.

Other health &

social
improvements

Conceptual framework of family

planning supply factors
External
Development
Assistance

FP Organizational
Structure
Service
infrastructure

Political and
Administrative
System

Sectoral integration

Political
support

Public-private
partnerships

Delivery strategies

Management &
supervision

Service Outputs

Training

Access

Commodity
acquisition &
distribution

Quality
Acceptability

IEC
Research &
evaluation

Resource
allocations
Legal code /
regulations

Operations

Larger
societal &
political
factors

Source: Bertrand, Magnani, and

Rutenberg 1996

Applying the frameworks for FP

M&E
Inputs, e.g.
Types and levels of resources
Qualified personnel
Unit and total costs of program resources

Outputs functional areas, e.g.

People trained
Performance of people trained
Cost per person trained

Applying the frameworks for FP

M&E
Outputs Service outputs, e.g.,

Service delivery points providing FP services

Quality of FP services
Cost of increasing access/quality of FP services

Outputs Service utilization

New FP acceptors, Couple Years of Protection
(CYP)
Returning clients
Cost of increasing CYP, etc.

Applying the frameworks for FP

M&E
Outcome intermediate outcomes
Contraceptive prevalence rate (CPR)
Unmet need
Costs associated with increased CPR

Outcome long term outcome

Fertility rates
Unintended pregnancy
Costs of changes in fertility, unintended
pregnancy

Indicators for FP programs

See Bertrand and Escudero, 2002,

Compendium of Indicators for Evaluating
Reproductive Health Programs, 2 volumes
Indicators that crosscut program areas
Indicators for specific program areas

What is different about M&E of FP

programs?
Basic principles are the same as in other
health programs
Outcomes relatively well-defined, focused, and
measurable
Long history of data collection on FP outcomes
through WFS, DHS document global trends
Attempts to link outcomes to program outputs evidence of program effects

Programme of Action adopted at

ICPD, Cairo 1994

Traditional (pre-Cairo) focus of FP

program M&E

Demographic impact
Focus on married women
Availability of services
Contraceptive adoption (new users)
Characteristics of women
Cross-sectional measurement

Cairo: Objectives of FP Programs

To help couples and individuals meet their
reproductive goals
To prevent unwanted and high-risk pregnancies
To make quality FP services affordable, acceptable,
and accessible
To improve the quality of family planning IEC,
counseling, and services
To increase the participation and sharing of
responsibility of men in FP
To promote breastfeeding to enhance birth spacing

Exercise 1
Discuss the implications of the Cairo programme
of action for M&E of FP programs. Identify 3 or
more ways in which the traditional focus of FP
programs listed on the earlier slide should change
to respond to the Cairo agenda. What are the
implications of these changes for M&E?

Contraceptive Prevalence Rate

(CPR)
Percentage of (married) women of reproductive
age (15-49) who are currently using a
contraceptive method.

Unmet Need for Family Planning

Percentage of fecund women exposed to the
risk of pregnancy who say they want to wait at
least two years for another birth (spacing) or do
not want any more children (limiting), but are not
currently using a method of contraception.

Related Indicators
Demand for FP = % (married) women using FP +
% (married) women with unmet need for FP
Percentage of demand satisfied = % (married)
women using FP / % (married) women with
demand for FP

Unmet Need Exercise

CPR vs Unmet Need

CPR
Relatively simple to
define
Uni-dimensional
Consistency over time
Does not capture
concept of meeting
needs

Unmet Need
Relatively complex to
define
Multi-dimensional
demand & use
Definition has evolved
Captures concept of
meeting need

Monitoring Quality of Care

What is Quality of Care in FP?

General, loosely-defined concept
Different people define quality in different
ways
Multi-dimensional
Appropriate standards against which to
measure quality vary

Bruce-Jain Framework
Choice of contraceptive methods
Information given to users
Provider competence
client/provider relations
re-contact and follow-up mechanisms
appropriate constellation of services

Indicators for QOC

No single indicator can capture the
different components of QOC
Indicators need to be adapted to specific
program context and priorities
Shortlist of 24 QOC indicators (see
Bertrand and Sullivan, Evaluation Bulletin
No. 1, Table 1 page 2).

Facility Surveys for QOC

Indicators
Situation Analysis
MEASURE Evaluation Quick
Investigation of Quality (QIQ)
MEASURE DHS+ Service Provision
Assessments (SPA)
DHS service availability modules and
community surveys (SAM)

Some Data Collection Issues

Small sample sizes for FP clients,
especially in low prevalence countries
Observation in clinics that use a client
flow approach
Sampling
Courtesy bias and hawthorn effects
Unit of analysis (client, provider, facility)

Case Study: QOC in Turkey

Turkeys Strategic Framework

S t r a t e g ic O b je c t iv e
I n c r e a s e d u t iliz a t io n o f F P / R H s e r v ic e s
I n t e r m e d ia t e R e s u lt 1
S t r e n g t h e n e d s u s t a in a b ilit y
o f F P /R H p ro g ra m

I n t e r m e d ia t e R e s u lt 2
E x p a n s io n o f h ig h q u a lit y F P / R H s e r v ic e s
in t h e p u b lic a n d p r iv a t e s e c t o r s

The Quality Index

Method availability
Availability of trained personnel
Perceived quality of FP counseling
Adequate infection-prevention measures
Availability of IEC materials
Physical access to FP services

Data Source
Istanbul Quality Surveys
Facility inventory
Client exit interviews

Based on MEASURE Evaluation QIQ

The Quality Index

Sum of scores from
the 6 components
(range 0-6)

Method Availability
Proportion of facilities
that distribute or
prescribe 3 or more
modern FP methods

Perceived Quality of FP
Counseling
Proportion of clients who
report
they were seated
had sufficient time with
the provider
clearly understood the
information provided

Adequate Infection Prevention

Measures
Proportion of facilities that
meet the following
standards :
Plastic bucket for CL
solution
Unused IUD kits kept
sterile
Medical waste kept in
leak-proof containers with
lids
Appropriate containers for
sharp objects

Evaluating the impact of quality

of care

Framework for links between quality of

family planning services and fertility
Quality of
services
Choice
Information to
users
Provider
competence
Client-provider
relations
Follow-up
Appropriate
constellation of
services
Other factors

Acceptance
Contraceptive
prevalence

Continuation

Fertility
Other
proximate
determinants

Known effects
Hypothesized effects

Source: Jain, 1989

Outcomes of interest
Intention to use
Contraceptive adoption
Contraceptive discontinuation
Failure
Switching
Stopping

Current contraceptive use

Contraceptive choice

Unwanted pregnancy

Examples of impact studies

Peru (Mensch, et al., 1996)
Morocco (Steele, et al., 1999)
Bangladesh (Koenig et al., 1997)

Morocco Study Design (1)

To explore whether the service environment in
which a woman resides affects adoption and
continuation of the pill
Linkage of 1995 Demographic and Health Survey
calendar with 1992 DHS Service Availability Module
Multi-level hazards models with contraceptive
adoption and discontinuation as outcomes
862 births and 775 episodes of pill use in 107
clusters

Morocco Study Design (2)

Explanatory factors - Individual and
Community
age, education, residence, community drinking
water & toilet facilities, principle economic activity
Contraceptive intention (discontinuation)
Breastfeeding status, last child wanted (adoption)

Explanatory factors Program

Public health center <10km, pharmacy <5km,
outreach services, 3+ methods available at clinic
Source of pills (discontinuation)

Predicted percentage of women

adopting a modern contraceptive
method within 12 months of giving
birth by service factors

Health center <10km

No. methods offered at

closest set of facilities

Predicted 12-month pill discontinuation

rate by reason and service factors,
Morocco

Source

Health center within 10KM

Pharmacy within 5KM

Main Findings: Morocco

Relatively strong service effects on postpartum adoption
Service availability associated with both
adoption and discontinuation
Number of methods available only
associated with adoption
Users of government sources have lower
discontinuation

Limitations of Impact Studies

Measures of quality inadequate (often
limited to access and method choice)
Cross-sectional designs (endogenous
inputs)
Linking individual and program data
(geographic boundaries, service
environment vs. individual service
experience)

Emerging areas: FP/HIV linkages

and integration

Context
Considerable progress in preventing unwanted
pregnancy but unmet need remains substantial
Rapid increases in HIV in many countries
Changing funding focus to HIV from FP
Integrated vs. vertical programs

Synergies between FP and HIV

programs
Both are central to reproductive health
ABC messages in HIV programs also
relevant to FP programs
Youth programs that encourage responsible
sexual behavior prevent both HIV and teen
pregnancy
Strong RH policies support both HIV and FP
programs

Dual Protection
Abstinence
Monogamous couples using effective
contraception
Correct and consistent condom use

FP in high HIV-prevalence
countries
Relationship between HIV and fertility desires
FP/RH needs differ for:

HIV- concordant monogamous couple

HIV- concordant non-monogamous couples
HIV discordant couples
HIV+ concordant couples

HIV counseling in FP services

FP and VCT
FP counseling opportunity for VCT or general HIV
counseling and VCT referral
VCT services could include FP services or FP
counseling and referral
Concern over unintended consequences of
integration
Provider burn-out
Discourage FP clients
Quality of integrated vs. vertical FP & VCT services

FP and PMTCT
Averts child infections by preventing
unintended pregnancies among HIV+ women
PMTCT programs provide opportunity for
prenatal FP counseling and post-partum
contraceptive use
Reduced breastfeeding by HIV+ mothers will
lead to shorter birth intervals in the absence
of FP

FP Counseling of PMTCT clients,

Zambia

Source: Rutenberg & Baek, 2004

PMTCT-Client FP Use 6 Months Postpartum, Zambia

Source: Rutenberg & Baek, 2004

HIV Counseling in FP Sessions,

Uganda

Source: Rutenberg & Baek, 2004

Exercise 3
Select an area of FP/HIV integration (e.g.
PMTCT, VCT, HIV counseling in FP etc.).
Develop a basic input-output-outcome-impact
framework for a simple program in this area.
Suggest 3-6 indicators to monitor your program.
What data sources would you propose to collect
these indicators?