Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Toronto, Ontario
Benchmarking
Mini-Workbook
Charlene Beynon
Monique Stewart
OPHA 2004
Dear Colleague:
The enclosed resources represent “work in progress” and are based on our “lived experiences”
from a number of public health benchmarking projects –If only we knew then what we know
now. We wish to express our sincere thanks and appreciation to those colleagues who have
participated in the benchmarking projects and have assisted us in articulating many “lessons
learned” and developing these resources.
Our intent is to offer resources and practical tools which will make the implementation of
benchmarking in public health more meaningful, effective, efficient and successful in identifying
“best practices”.
We welcome your ongoing comments and feedback as we move closer to identifying what are
those “best practices” in using benchmarking to enhance the practice of public health. We
invite you to share your experiences. May we move closer to realizing the full potential of this
quality improvement tool.
Happy benchmarking!
Table of Contents
Presentation: If We Knew Then What We Know Now: Benchmarking in the Real World...1
Recommended Resources.....................................................................................................15
Tools ........................................................................................................................................19
Benchmarking Work Sheets ..............................................................................................21
Selecting Our Options/Increasing the Odds.......................................................................25
Assessing Feasibility of Success .......................................................................................27
Draft Project Discussion Points..........................................................................................29
Southwest Benchmarking Feasibility Assessment.............................................................31
Benchmarking
If Only We Knew Then,
The process of identifying, sharing,
What We Know Now: understanding, learning from and adapting
outstanding or best practices from
Benchmarking In The organizations anywhere in the world in the
quest for continuous improvement and
Real World breakthroughs.
OPHA 2004
Learning Objectives Another Definition
Thinking Smart The process of consistently researching for new
Critically reflect on current practice, and ideas for methods, practices, processes
incorporate research and evaluation findings to
improve clients services Either adopting the practices or adapting the good
features and implementing them to become the
Ends in View best of the best.
Promote benchmarking
Explore common pitfalls and Critical Success Balm, 1992
Factors
1
Lessons Learned from Public Health Benchmarking
Pilot Projects Web Site
Requires many steps, patience and
commitment www.benchmarking-publichealth.on.ca
Data collection not standardized Web-Based
Use data that are available and easily Health Units enter own data
retrievable
Select comparator Health Units
Keep the indicators “simple”
Beware of seasonal realities Select basis for comparison
Remember context Program picks three best
Resource intensive Can browse through practices related to
Determining best practices is challenging! different indicators
Survey of Pilot
Project Participants
Overall, participants were very positive
Acknowledged role of PHRED and OPHBP in
A case study:
providing support, expertise & coordination
Benchmarking process created networking Dental screening
opportunities
Need to keep benchmarking process simple benchmarking
Some specific program changes were made or
a conscious decision was made not to change investigation
2 areas of concern: anonymity & comparability
Participating in projects had a positive influence
on participants practice
Our Report
Benchmarking Projects
9 benchmarking projects completed or Dental Benchmarking Project: Report 1:
in progress Descriptive Characteristics of Dental
– 3 Pilot Projects Screening Programs in 10 Ontario Public
– Breastfeeding Supports Health Units
– Heart Health Coalitions
– School Health www.benchmarking-publichealth.on.ca
– Universal Influenza Immunization www.phred-redsp.on.ca
– Dental Screening
– West Nile Virus
10 Dental sites
Participation of Health Units Haliburton-Kawartha-Pine Ridge
Benchmarking Project Development Hamilton
32 Health Units have participated in at least Hastings & Prince Edward
one, and 24 have participated in more than Leeds-Grenville-Lanark
one Middlesex-London
Benchmarking Website Niagara
34 Health Units have completed at least Peel
one on-line survey, and 28 have completed Simcoe
more than one on-line survey Waterloo
Wellington-Dufferin-Guelph
2
Our Middlesex-
Middlesex-London Strategic Questions
project team
Charlene Beynon Advancing
Joan Carrothers Necessary benchmarking
prerequisites in public health
Meizi He
Bernie Lueske
Ruth Sanderson
Consultant:
Monique Stewart, Ottawa PHRED Program
- GO for it!
identify a few key indicators
critically examine the data information
systems
Recruit partners to ensure comparability learning about the process and the
program
Assess data availability/quality
3
The Question:
Best Practices Is there a better way?
Definition collect the indicator data first?
processes that yield better outcomes
effectiveness then collect the practice data based on
indicator results?
less resources not more
efficiency
need to know vs nice to know
Question: Why are they successful?
Key Messages
Best Practices
Benchmarking is a quality improvement tool
Caveats
that can promote exemplary performance
misnomer – unlikely one best practice and demonstrate accountability
prerequisite practices Success is dependant on many factors
importance of comparators
Comparable quality data is a prerequisite!
customization
Potential is tremendous
time limited
cbeynon@uwo.ca monique.stewart@ottawa.ca
Best Practices
For next time:
identify at the outset how intend to define
best practices
will the indicators and data support the
definition?
4
ONTARIO PUBLIC HEALTH BENCHMARKING PARTNERSHIP
(OPHBP)
The Ontario Public Health Benchmarking Partnership is a collaborative initiative of the Public
Health Research, Education & Development (PHRED) Programs, the Association of Local
Public Health Agencies (alPHa), and the Ontario Council of Community Health Accreditation
(OCCHA).
CODE OF CONDUCT
PURPOSE: To address the appropriate behaviour for all participants involved in
benchmarking in Ontario public health units through the Ontario Public Health
Benchmarking Partnership (OPHBP).
To contribute to efficient, effective and ethical benchmarking, public health units and all other
individuals /partners agree to abide by the following principles for benchmarking:
5
5. Principle of Confidentiality:
Treat any benchmarking interchange as something confidential to the individuals and
organizations involved. Information obtained must not be communicated outside the
Ontario Public Health Benchmarking Partnership without prior consent of participating
benchmarking partners. An organization’s participation in a benchmarking project should
not be communicated to any third party without their permission.
BENCHMARKING PROTOCOL
• Know and abide by the Benchmarking Code of Conduct.
• Have basic knowledge of benchmarking and follow the benchmarking process.
• Have determined what to benchmark, identified key performance variables, recognized
superior performing partners and completed a rigorous self-assessment.
• Have developed a questionnaire and will share this in advance if requested.
• Have the authority to share information.
• Work through a specified host and mutually agree on scheduling and meeting
arrangements.
Face-to-face meeting guidelines:
• Provide meeting agenda in advance.
• Be professional, honest, courteous and prompt.
• Introduce all attendees and explain why they are present.
• Adhere to the agenda; maintain focus on benchmarking issues.
• Use language that is universal.
• Do not share proprietary information without prior approval from the proper authority of
all participants.
• Share information about your process(es) if asked, and consider sharing study results.
6
ONTARIO PUBLIC HEALTH BENCHMARKING PARTNERSHIP
BENCHMARKING AT A GLANCE
Benchmarking:
the process of consistently researching for new ideas for methods, practices, processes; and of either
adopting the practices or adapting the good features and implementing them to become the best of the best.
(Balm 1992, p. 16)
7
• OR do you need to initiate a more comprehensive benchmarking investigation?
The number of partners will depend on your benchmarking question, timeline and resources available.
References:
Balm, G. J. (1992). Benchmarking: A practitioner’s guide for becoming and staying best of the best.
Schaumburg, Illinois: QPMA Press
Sales, P. D., & Stewart, P. J. (1998). Benchmarking Tool Kit: a blueprint for public health practice.
PHRED Program: Middlesex-London Health Unit, Ottawa-Carleton Health Department.
Source:
Beynon, C. (1999). Benchmarking. From Regional Workshops, Public Health Needs, Effective
Interventions, Benchmarking: Implications for Public Health Units
Recommended Readings:
Ammons, D. N. (2001). Municipal benchmarks: Assessing local performance and establishing
community standards. California: Sage Publication Inc.
Keehley, P., Medlin, S., MacBride, S., and Longmire, L. (1997). Benchmarking for best practices in the
public health sector: Achieving performance breakthroughs in federal, state and local agencies. San
Francisco: Jossey-Bass.
8
Deciding What to Benchmark?
be meaningful
be highly visible
be resource intensive
have a history of problems
have the opportunity to improve
have an environment conducive to change
be understood
support the agency mission, vision and strategic directions
need ideas from other sources to be improved
9
10
A Benchmarking Checklist
Before embarking on a benchmarking investigation assess the following and determine the
likelihood of a successful outcome.
12. There are sufficient resources, e.g. staffing, budget, etc to sustain the
project?
1.
S = Specific M = Measurable A = Achievable R = Realistic T = Timeline
This Checklist has not been validated. It has been developed from experiences gained from
benchmarking public health programs and is intended to stimulate strategic dialogue and
decision-making with an overall goal of increasing the likelihood of a successful outcome.
11
12
Prioritizing Program Components
for Benchmarking
Data collection tool Drafting and piloting of + data collection tool(s) available
measurement tools has potential 0 minor tool creation required
to be resource and time - substantial effort likely required
consuming to develop tools
13
14
RECOMMENDED RESOURCES
www.benchmarking-publichealth.on.ca
BENCHMARKING:
Ammons, D.N. (2001). Municipal benchmarks: Assessing local performance and establishing
community standards. Thousand Oaks: Sage Publication Inc.
Ammons, D.N. (1996). Municipal benchmarks: Assessing local performance and establishing
community standards. Thousand Oaks: Sage Publication Inc.
Balm, G.J. (1992). Benchmarking: A practitioner’s guide for becoming and staying best of the
best. Schaumburg, Illinois: QPMA Press.
Beynon, C. & Wilson, V. (1998). Benchmarking in Public Health: An idea whose time has
come. Public Health & Epidemiology Report Ontario, 9(7), 162-163.
Camp, R. C. (1989). Benchmarking: The search for best practices that lead to superior
performance. Part I: Benchmarking defined. Quality Progress, January, 61-68.
Camp, R. C. (1989). Benchmarking: The search for best practices that lead to superior
performance. Part II: Key process steps. Quality Progress, February, 70-75.
Camp, R. C. (1989). Benchmarking: The search for best practices that lead to superior
performance. Part III: Why benchmark?. Quality Progress, March, 61-68.
Davies, A. J. & Kochhar, A. K. (2000). A framework for the selection of best practices.
International Journal of Operations and Production Management, 20(10), 1203.
Doebbeling, B.N., Vaughn, T.E., Woolson, R.F., Peloso, P.M., Ward, M.M., Letuchy, E., Boots-
Miller, B.J., Tripp-Reimer, T., & Branch, L.G. (2002). Benchmarking veterans affairs medical
centers in the delivery of preventive health services. Medical Care, 40(6), 540-554.
Dunkley, G., Stewart, M., Basrur, S., Beynon, C., Finlay, M., Reynolds, D., Sanderson, R., &
Wilson, V. (2001). Benchmarking in public health. Public Health & Epidemiology Report
Ontario, 12(6), 211-215.
Dunkley, G., Wilson, V., & Stewart, M. (2000). Benchmarking Pilot Project: Testing the
concept in public health. Public Health & Epidemiology Report Ontario, 11(1), 14-21.
15
Ellis, J., Cooper, A., Davies, D., Hadfield, J., Oliver, P., Onions, J. & Walmsley, E. (2000)
Making a difference to practice: Clinical benchmarking part 2. Nursing Standard, 14(33), 32-
35.
Fitz-Enz, J. (1992). Benchmarking best practices. Canadian Business Review, 19(4), 28-31.
Herman, R. C. & Provost, S. (2003). Interpreting measurement data for quality improvement:
Standards, means, norms, and benchmarks. Psychiatric Services, 54(5), 655-657.
Johnson, B., & Chambers, J. (2000). Food service benchmarking: Practices, attitudes and
beliefs of foodservice directors. The American Dietetic Association, 100(2), 175-180.
Keehley, P., Medlin, S., MacBride, S., & Longmire, L. (1997). Benchmarking for best practices
in the public health sector: Achieving performance breakthroughs in federal, state and local
agencies. San Francisco: Jossey-Bass.
Ossip-Klein, D.J., Karuza, J., Tweet, A., Howard, J., Obermiller-Powers, M., Howard, L., Katz,
P., Griffin-Roth, S., & Swift, M. (2002). Benchmarking implementation of a computerized
system for long-term care. American Journal of Medical Quality, 17(3), 94-102.
Sales, P.D., & Stewart, P.J. (1998). Benchmarking Tool Kit: A blueprint for public health
practice. Middlesex-London and Ottawa-Carleton Public Health Research, Education and
Development (PHRED) Programs.
Wilson, B. & Beynon, C. (1998). Introducing benchmarking to Ontario Health Units. Public
Health & Epidemiology Report Ontario, 9(8), 183-186.
Witt, M. J. (2002). Practice re-engineering through the use of benchmarks: Part II. Medical
Practice Management, March/April, 237-242.
Zairi, M. & Leondard, P. (1994). Practical Benchmarking: The Complete Guide. London:
Chapman & Hall.
16
PROGRAM LOGIC MODELS:
Dwyer, J. (1996). Applying program logic model in planning and evaluation. Public Health &
Epidemiology Report Ontario, 17(2), 38-46.
Rush, B. & Ogborne, A. (1991). Program logic models: Expanding their role and structure for
program planning and evaluation. The Canadian Journal of Program Evaluation. 6(2), 95-106.
McLaughlin, J.A., & Jordan, G.B. (1999). Logic models: A tool for telling your program’s
performance story. Evaluation and Program Planning, 22(1), 1-14.
17
18
Tools
The following resources were developed from “lessons learned” from the Dental Screening
Benchmarking Investigation to assist future project teams in :
• determining if benchmarking is the tool of choice, and
• mapping out pre-requisites for a successful benchmarking investigation.
These resources were used to facilitate discussion with colleagues in the Southwest to
determine the feasibility of conducting a benchmarking investigation focusing on postpartum
depression.
We look forward to improving these resources and welcome your questions and comments.
19
20
Benchmarking Worksheets
2. What component do you want to benchmark? i.e. Where do you need to improve performance? And where can you
improve performance?
We want to benchmark:
21
4. What is the benchmark? (i.e. What is the gold standard? What will you compare your results to in order to identify “Best
Practices?)
Component
Activities
Target Audience
1.
2.
3.
4.
An indicator:
a quantifiable measure of an outcome or objective
23
24
Benchmarking Worksheet
Selecting Our Options/Increasing the Odds
Program
Component1.
What is the
benchmarking
question?
What is the
benchmark?
Other
Post Note:
The goal is to focus on only one program component. Other components can be addressed in subsequent benchmarking
investigations. If there is debate about which component should be addressed, this Work Sheet can assist in articulating which
focus is more likely to yield a successful benchmarking outcome.
25
26
Benchmarking
Assessing Feasibility of Success
Worksheet
Resource Commitment
27
28
Postpartum Depression
Benchmarking Investigation
Draft Project Discussion Points
1. Identify major components for a Program Logic Model.
2. What component do you want to benchmark?
Principle: Select only 1
3. What benchmarking question do you want to answer?
Principle: the question should be SMARTa
Question: is there a benchmark?
4. Initiate discussion about how intend to identify “Best Practices”.
* Usually indicator and practice data are collected simultaneously. This option requires further
study. It is being presented as one way to keep the project manageable, time limited and is
based on experiences from other projects.
29
30
Postpartum Depression
Southwest Benchmarking Feasibility Assessment
The Question: is benchmarking the right tool to answer the question(s) that needs to be
answered
When to Benchmark*:
* see the “Benchmarking Checklist” and “Priority Program Components for Benchmarking”
Key Phrases:
• Who is doing better at. . . (identifying women at risk for postpartum depression)?
• What practices are most effective and efficient. . .(identifying women at risk for postpartum
depression)?
• type of contact
• with what tool?
• when?
November 2003
31
32
Personal Notes
33
34
35
36