Are we making a difference?

A guide to monitoring and evaluating dog

population management interventions
Dr Elly Hiby
Scientific Coordinator

Background
ICAM Coalition formed in 2006
Discuss, share and learn
But what is really making a difference?
Indicators project
To develop guidance on monitoring and evaluation
of DPM that supports us to track progress, learn
and subsequently improve impact through the use
of measurable indicators

Sharing our language

Impact: (aka
goal or aim) the
change we hope
to contribute
towards through
our interventions

Example: an intervention
that catches, neuters and
returns a number of stray
dogs every month in an
Asian city. The desired
impact of this intervention
is to reduce the density of
dogs and to improve the
welfare of the stray dogs

Sharing our language

Indicators: (aka
metrics) are
measurable
signs of impacts;
they are the
things we would
see or hear if our
desired impact
was occurring.

Example: In our Asian city

intervention, for the impact
of reducing dog density a
suitable indicator may be
the number of dogs seen on
a set of standard routes
along public roads

Sharing our language

Methods of
measurement:
this is how data
relating to the
indicators was
collected

Example: The method of

measurement for the
indicator of dog density may
be a street survey
observing dogs on public
property, conducted once
every 6 months, always
following the same protocol
(e.g. same routes, same
time of day)

Sharing our language

Monitoring: systematic and routine data
collection
Intervention progress and indicators

Evaluation: uses data collected through

monitoring to answer questions about what
difference did this intervention make?
Conducted regularly but not frequently
aka Impact assessment

Why should I care?

To inform donors of the impacts created through
their funding
To inform the public of any impacts on them and/or
their dogs as beneficiaries of the intervention
To provide evidence for lobbying to sustain or
replicate an intervention
To compare interventions and their relative impacts
To learn and improve our impact

Why should I care?

Develop concept
of dog population
situation and
intervention
Initial assessment
and analysis

Intervention
planning

Implementation

Why should I care?

Test assumptions,
refine concept and
evolve the
intervention

Develop concept of dog

population situation
and intervention
Initial assessment and
analysis

Intervention planning
Set impacts and
identify indicators

Evaluation
and learning

Implementation
Monitor progress
of intervention and
indicators

Develop monitoring
and evaluation plan

Why should I care?

Success depends on knowing what works
Bill Gates
Centre for Global Development 2006 report
When will we ever learn? Improving lives
through impact evaluation (Savedoff et al.,
2006)

Rules. Gold standard.

Best yet. Enjoy. Be inspired.
Try it. Innovate. Share.

8 Impacts
1.
2.
3.
4.
5.
6.
7.
8.

Improve dog welfare

Improve care provided to dogs
Reduce dog density/Stabilise turnover
Reduce risks to public health
Improve public perception
Improve rehoming centre performance
Reduce negative impacts of dogs on wildlife
Reduce negative impacts of dogs on
livestock

Indicators
Each impact has a list of indicators
Recommended
Suggested

Indicators
Each impact has a list of indicators
Recommended
Suggested

e.g. Body condition score

recommended

Indicators
Each impact has a list of indicators
Recommended
Suggested

e.g. Dog-dog interactions suggested

Image from www.flickr.com, available under creative

commons licence, credited to Nazly Ahmed

Image from www.flickr.com, available under creative

commons licence, credited to Godo Godjo

Method of measurement
Each indicator is accompanied by at least
one suggested method of measurement

Method of measurement
Each indicator is accompanied by at least
one suggested method of measurement

7 Methods of Measurement
1.
2.
3.
4.
5.
6.
7.

Questionnaire surveys
Participatory research methods
Street surveys
Secondary sources of information
Clinic records
Behavioural observation
Street surveys and questionnaires for
vaccination coverage

e.g. Street surveys

You choose

Impacts
Indicators
Method of measurement

You choose
Background
Impacts
Indicators
Method of measurement

Making your impact assessment robust

Accessing the guidance

On USB stick in your conference pack
Download the full document at www.icam-coalition.org

Navigating the guidance

Navigating the guidance

Use our online
impact assessment
tool

Sign up as a user
Choose your impacts
Series of questions
creates
guidance.pdf incl
indicators and MoM
tailored to your
intervention

Share
info@icam-coalition.org
Representatives of
coalition members
Follow-up survey, and
more

Thanks to
Collaborating partners
Boyd Orr Centre, University
of Glasgow
JMICAWE, University of
Edinburgh
Animal Welfare and
Behaviour group, University
of Bristol
Department of Veterinary
Tropical Diseases,
University of Pretoria

Subject experts and DPM

implementers

8 Impacts
1.
2.
3.
4.
5.
6.
7.
8.

Improve dog welfare

Improve care provided to dogs
Reduce dog density/Stabilise turnover
Reduce risks to public health
Improve public perception
Improve rehoming centre performance
Reduce negative impacts of dogs on wildlife
Reduce negative impacts of dogs on
livestock