Practice: Evaluate implementation to document

what you are doing

Key Action: Document implementation based on your logic
model

TOOL: Questions, Methods, and Indicators for Implementation Evaluation

Purpose: The following table can help you and your evaluator
determine appropriate evaluation questions, methods
for answering those questions, and some good
indicators of the extent and quality of implementation.

Instructions: 1. Review the sample evaluation questions and

consider the value of the data collection method used
to answer these questions.

2. Review the sample indicators and generate ideas

for other indicators you might use to document your
magnet program’s implementation.

1

Questions, Methods, and Indicators for Implementation Evaluation
EVALUATION
RESEARCH QUESTION
To what extent was the
magnet program
implemented as
designed?
To what degree has the
magnet program helped
reduce minority group
isolation?
What adaptations,
additions, and omissions
were made when the
magnet program was
implemented?
To what extent was staff
trained to implement the
magnet curriculum at a
professional level?
To what extent are
stakeholders (parents,
students, staff,
community members)
informed and
knowledgeable about the
magnet program?
Adapted from U.S. Department of Education, Office of Safe and Drug-Free Schools. (2007). Mobilizing for evidence-based character
education (p. 18). Washington, DC: Author. The entire guide can be downloaded at www.ed.gov/programs/charactered/mobilizing.pdf (last
accessed December 10, 2008).
Practice:
Key Action:
METHODS AND VALUES OF
EVALUATION
Compare the amount and
range of activities done in the
magnet program with those
prescribed by the program
developers. The comparison
gives an indication of the
fidelity of the implementation to
the planned program and the
frequency and intensity of the
magnet program activities.
Compare student
demographics before the
magnet program is
implemented to those after
program implementation for
any indication of changes in
racial and ethnic composition.
R School-sponsored activities
ecord, describe, and count any
modifications, since
adaptations, additions, and
omissions affect analyses of
outcome evaluation data.
Compare the current
professional development
activities against the standards
for optimal training as planned
by the program developers.
This gives an indication of the
potential strength and
weakness of the magnet
program.
Maintain records of meetings
and presentations to
stakeholders, as well as their
questionnaire responses, in
order to discover a range of
their knowledge.
Evaluate implementation to document
what you are doing
Document implementation based on your logic
model
IDEAS FOR YOUR
SAMPLE INDICATORS
INDICATORS
Magnet program
implementation plans
School staff interviews
School-site observations
Review lesson plans and
curriculum documents
Outreach and recruitment
logs (recruitment fairs and
parent nights)
.
Annual analysis of student
application database and
class rosters
logs
Survey stakeholders
Teacher interviews
Professional Development
schedules, activity logs, and
feedback forms
Classroom observation visits
Staff surveys in the spring
Survey stakeholders
Interviews/Focus Groups
with stakeholders
Agendas and logs of events
where stakeholders were
present