Evaluation Proposal

By Shanu Jindal, Zoe Jones, & Casey Thomas

Section 1: Conceptualization of Evaluation

What purpose(s) will be served by the evaluation?

o
This evaluation will inform North Carolinas public, Public Health professionals, legislators and other public servants
about the successes and challenges of the Eat Smart, Move More NC Obesity Prevention Plan. It will also serve as an internal
guide for the programs leaders to determine where they might need to focus attention, and assist them in touting successes to
those to whom they may need to justify the program. It will help ESMMNC identify what motivates sites to participate in the
program and where they might focus their efforts in order to garner more participation.

What broad questions will be addressed by the evaluation?

o
How broadly are the strategies suggested in Eat Smart, Move More being implemented?
o
What mechanisms does the Eat Smart, Move More team use to advance the implementation of these strategies and how
effective are they?

Who are the key stakeholders?

o
Stakeholders include the Eat Smart, Move More, NC Leadership team, local governments and state government; the
food and beverage industry; health care; childcare; health insurance companies, schools, colleges and universities; work sites;
faith based and other community organizations, and North Carolinas general public.

What assumptions are being made?

o
The assumption that the program is built upon improving the strategies put forth in the 2007-2012 version of the plan is
central to the evaluation.
o
This evaluation also assumes that Eat Smart, Move More, NC is working to forward the adaptation of their suggested
behaviors in a variety of settings with a variety of actors.
o
As evaluators, we are also assuming that although this program has been implemented for almost a decade, because it is
so difficult to move the needle on rates of obesity and attribute changes in the biometrics of a state specifically to one

intervention, that it would be more appropriate to evaluate the adoption of the suggested measures themselves, and the
mechanisms for that adoption.
o
While the Eat Smart, Move More NC Leadership Team, made up of 80 organizations, supports the Plans goals and
objectives, the Plan belongs to every North Carolinian. It will take all of us working together to make our state a place where
healthy eating and active living are the norm, rather than the exception.

What are the major contextual factors that need to be considered?

This plan calls for implementation of strategies that require funds in a time of tight public funding for the state of North
Carolina, and in looking at progress it might be apt to include a review of similar efforts in states with similar political
climates.
o
This is a guide that is relatively new, but is building on work that has already been done from 2007 to 2012 and has
expanded the amount of settings in which the plan is to be implemented, and increased the specificity of objectives and
strategies.
o

Section 2: Program Description

The purpose of Eat Smart, Move More NC is to reverse the rising tide of obesity and chronic disease among North Carolinians by
helping them to eat smart, move more and achieve a healthy weight. The new NC Obesity Prevention Plan unites all stakeholders
interested in obesity prevention across the state around one goal making the healthy choice the easy choice when it comes to good
nutrition and physical activity.
1.

The first goal of ESMM NC is to Increase healthy eating and physical activity opportunities for all North Carolinians by
fostering supportive policies and environments. The objectives of this goal is by December 31, 2012 to increase the yearly number
of policies and incentives
1. By December 31, 2012, increase yearly the number of policies and incentives to promote healthy eating and physical
activity wherever North Carolinians live, learn, work, play, and pray.
2. By December 31, 2012, increase yearly the number of facilities/environments to promote healthy eating and physical
activity where North Carolinians live, learn, work, play, and pray.
b. Increase the percentage of North Carolinians who are at a healthy weight.
1. By December 31, 2012, there will be no increase in the percentage of North Carolina adults, youth, and children who
are classified as overweight or obese.
b. Increase the percentage of North Carolinians who consume a healthy diet.
1.
By December 31, 2012, 14 percent more North Carolina adults, youth and children will consume five or more servings
of fruits and vegetables each day

2.

By December 31, 2012, the proportion of North Carolina infants who are breastfed will increase to 75 percent and the
proportion of infants who are breastfed for at least six months will increase to 50 percent
3.
By December 31, 2012, when eating out, more North Carolina adults and children will choose foods and beverages
generally considered to be healthier. Healthier will be defined by: lower in fat, sugar, calories; fast-food meals once per week
or less often and labeled as healthy
4.
By December 31, 2012, 25 percent fewer North Carolina children ages 2-17 will eat fast food three or more times per
week.
5.
By December 31, 2012, at least 70 percent of North Carolinians will prepare and eat their main meal at home at least
five times per week.
6.
By December 31, 2012, the percentage of North Carolina adults, youth and children who typically consume more than
one 12-ounce serving of sugar-sweetened beverages per day will not exceed 50 percent.
b. Increase the percentage of North Carolina adults, youth, and children ages 2 and up who participate in the recommended amounts
of physical activity.
1.
By December 31, 2012, at least 46 percent of adults will get recommended amounts of physical activity each week and
fewer than 15 percent will report no leisure time physical activity
2.
By December 31, 2012, at least 52 percent of youth and children will participate in at least 60 minutes of physical
activity every day
5. Who is the target audience?
a. Institutions such as schools, health-care providers, public service agencies, municipal governments (especially planning
departments), childcare providers, and farmers market-initiatives.
b. Employers and work-sites, colleges and universities, faith-based organizations and community organizations that are in a
position to implement the strategies.
6. Where is the program intended to be implemented?
a. In health care settings, child care settings, schools, colleges and universities, work-sites, faith-based organizations (like a
church walking group), local government and even in the food and beverage industry.
7. Has the program undergone previous evaluations? If so, what were the results?
1.
An evaluation was conducted from October 1, 2008-May 31, 2009 by the North Carolina Childhood Obesity
Prevention Demonstration Project.
2.
In four months residents in the counties reported improvements in eating behavior or physical activity, such as choosing
low fat or low calorie foods or drinks, eating more fruits and vegetables, eating smaller portions or getting more exercise.

Inputs
Funds

Activities

Support of
Stakeholders

Program
materials

Participants

Staff

grants
entitlements
cooperative
agreements

Policy
makers
program
planners
program
implementers

Outputs

Implement
social marketing
campaign to raise
public awareness
and promote
healthy physical
activity and
nutrition
behaviors and
environments

Engaging
the faith
communities.
Training of
the community
leaders

Increase the
availability of
obesity
screening and
counseling and
require state
health plans
coverage of

Expand
existing
community
grants program
to promote
physical activity

Short term & intermediate

outcomes

By January 1, 2020,at least

66% of public middle &
high schools in NC will offer
opportunities for students to
participate in intramural
activities or physical activity
clubs.
joint-use

76% of public middle &

agreements to
high schools in NC will allow
expand use of
children and/or adolescents to
school and
use some or all of the indoor
community
physical activity or athletic
recreational
facilities for community
facilities
sponsored physical activity
classes or lessons outside of
# of program
school hours or when school is
marketing
not in session
materials

the %age of parents in NC

produced/
who report that one or more
distributed
schools in their community
Promote
allow people to use the school
menu labeling to
playing fields, playgrounds, or
make nutrition
athletic facilities for their own
information
personal exercise or recreation
available to
will increase by 3.5% points
consumers
from 2012 baseline.

51% of NC work sites will

Build active
use motivational signs or pointliving
of-decision prompts to
communities
encourage people to be more
Expanding
physically active.
member

9% of public schools in NC
representation
will receive a HealthierUS
by including
School Challenge award,
new and needed
indicating that the school has
partners.
created a healthier environment
that promotes nutrition &
Establish a
physical activity.
full time

32 maternity centers in NC
Healthful Living
will be recognized as awardees
Coordinator in
by the North Carolina Maternity
each local
Center Breastfeeding Friendly
education
Designation Program.
agency

the %age of NC adults who

Long term
outcomes

effective
in decreasing
average BMI
Cost
Effective in
achieving
improvements
Decrease
prevalence of

obesity &
Increase
physical
activity.
Improve
dietary
behaviors
related to the
population
burden of
obesity and
chronic
diseases

Impact
Children
and their
families
participating in
the program are
racially,
ethnically, and
geographically
diverse.
Adoption
of the program
by other
organizations
Implement
ation and
maintenance of
the program to
the longest with
the availability
of support and
funds.

Section 3: Evaluation Plan

What the Evaluation Is Asking
Our evaluation will explore the relationships between the efforts to forward adoption of the health strategies outlined in ESMMNC
and the motivations of potential sites in adopting those strategies. The evaluation asks, How do the efforts to gain more participants
in ESMMNC strategies align with the factors that motivate sites to participate? In answering that question, the evaluation also asks
What needs to occur in order for the strategies to be implemented, and which actors have the leverage to make that happen? It also
asks what mechanisms ESMMNC has in place to promote implementation, and looks for gaps. It will explore the role of potential
stakeholders in developing the ESMMNC plan and their level of investment in implementing the plan. It will also look at how the
number of sites implementing strategies, and the types and amounts of these strategies sites are implementing changes, and explore
what led them to make those decisions. This evaluation will illustrate where ESMMNC is investing their effort and help them decide
how to adapt armed with the knowledge of what motivates the sites they are trying to reach.
The Evaluation Design
This will be a process evaluation that explores the implications and utility of ESMMNCs outreach processes and the decision-making
processes they use to determine how they will attempt to get sites to adopt their strategies. In examining this process, the evaluation
will identify leverage points, and evaluate the utility of those leverage points, and how they match up with the leverage points that
potential and participating sites identify.
To accomplish this, the evaluation will involve interviewing staff and board members of ESMMNC to find out how they see their
process and to obtain and overview of their decision-making process and operations around building more site participation. The
evaluation will also involve observing their meetings and decision-making process by shadowing their staff and board and making
notes on the action items they propose and where they focus most of their effort.
This evaluation will also examine several sites three categories, schools, hospitals, and work-sites. The evaluators will distribute a
questionnaire by mail among a random sample of 300 of each type of site chosen from North Carolinas school, hospital, and business
registries with a gift-card incentive included with each questionnaire to increase the response rate. This random sample is aimed at
finding sites that have and have not adopted the strategies. As a fallback plan, if not enough sites who have adopted the strategies
respond, the evaluators will use the snowball method for selecting sites for the questionnaire. Because many sites implementing these

strategies may not be doing so as part of ESMMNC, or identify these practices as participating in a state-wide initiative, the
questionnaires will ask if the sites implement specific practices rather than if they affiliate with ESMMNC. The questionnaire will also
ask sites why they chose not to implement other strategies. Additionally, the questionnaire will collect demographic and financial data
about these sites to see if there are any traits that make a place more likely to try to implement their strategies.
Following the questionnaire, the evaluators will choose a pool of randomly selected sites until a sample of 10 per category for
both sites that have adopted and have not adopted ESMMNC strategies, have agreed to be interviewed and follow-up with more indepth, qualitative interviews. These interviews will focus on the contexts in which the sites operate, how decisions to implement these
strategies are made, and go further in-depth about what persuades or dissuades them from pursuing a particular strategy.
The first variable the evaluation will measure is the sum of the outreach tactics and strategies chosen by the ESMMNC team to
increase implementation of their site-specific strategies. This is not an outcome evaluation in terms of the efficacy of ESMMNC, but a
process evaluation that focuses on the outcome these tactics have on the adoption of their strategies. With that in mind, the
intermediate outcomes that this variable impacts are the amount of sites that adopt the ESMMNC strategies and the amount of
strategies the sites adopt. The long-term outcome is the efficacy of the program and its ability to reach its goals in reducing obesity.
The second set of variables the evaluation will measure are the motivations and leverage points that motivate sites to adopt ESMMNC
strategies, and barriers in adopting them. The intermediate outcome of this is understanding what will increase the adoption of
ESMMNC strategies. The long-term outcomes are that ESMMNCs strategies are as closely aligned as possible with the motivations
of potential sites, and that they are able to use these leverage points to further the reach of the program and reduce obesity in North
Carolina.
Crosswalk
Instrument
Evaluation questions

How significant are the changes in strategy

implementation/adoption over time?

Interview Observe
staff &
their
board
meetings
members & decision
& sites
making
process

Question
naire
among
those
impleme
nt some
strategies
X

Questionnaire
among those
did not
implement
some
strategies

Interviews
w/ partner
organizati
ons

List of
Schools/W
orksites/
Hospitals

How does Eat Smart Move More, NC measure the

impact of the initiative/count how much more
implementation is taking place?

What efforts were made to ensure that all potential

stakeholders had a role to play in writing the new
Plan?/How invested are stakeholders in
implementing the new plan?

Whats new or different about this Plan compared

to older plans?

What needs to occur in order for the strategies to be X

implemented and who has the leverage to make that
happen?
What mechanisms does Eat Smart Move More have X
in place to promote implementation? Are there
gaps?

What traits do the sites that have implemented the

ESMM strategies share?

What motivated sites that implemented ESMM

strategies to participate?

What barriers existed for ESMM non-participant

sites that decided not to participate?

What was attractive about the specific strategies

that the ESMM sites chose to implement?

What was unattractive about the specific strategies

that ESMM-participating sites did not implement?

Section 4: Reporting Plan

The data will be reported to the ESMMNC team in a presentation that discuss patterns found in the data about what sites that chose to
implement the strategies had in common, what motivated sites to adopt the strategies or dissuaded them from doing so, and how wellaligned their efforts as a team are with the motivations of the sites. This will be mapped visually and presented in summaries with
easily captured upshots. We will also go over the full-report with charts of all relevant data that we collected in appendices. We choose
to include all of the data because a good portion of the planning and reviewing team are academics and may be interested in the
information for future use in ESMMNC, or in using it to conduct their own analyses of our data. We will provide 2 printed copies of
the report and an electronic copy for their internal dissemination. Additionally, we will meet with the ESMMNC leadership to answer
questions after they have had an opportunity to hear the presentation, reflect and to look through the report that we have provided.

Section 5: Detailed Budget

Direct Costs

Per Unit Units

Amount Requested Totals Notes

Key Personnel
Statistician

$7,500.00 Per Evaluator at 2 evaluators

15000

Evaluator

$9,000.00

9000

Fringe Benefits (45%)

Per Evaluator

10800
34800

Data Collection

Recorders
Transcription Labor

150

$2.00

15 per hour at 15 hours

$300.00
225

Envelopes

$0.50

900

$450

Posting /mailings

$0.57

900

$513

10

900

9000

Gift Cards

10488
Travel
Domestic-In State/Per Mile

0.56 per mile at 800

448

448

Other Indirect Costs/Overhead

General Office Supplies
Copier Contract

$2,400.00
$0.60 per unit at 3000

$1,800

Phone Bill

18 per month at 12

216

*20% of bill, $90 per month

Office Space

60 per month at 12

720

*20% of bill, $300 per month

5136

Subtotal Direct Costs

Indirect Cost

45736

5136

Total Direct and Indirect

50872

Section 6: Detailed Timeline

Gantt Chart
Activities/ Time

May July,
October,
&
August,
November
June September

Planning Purchasing gift cards; designing questionnaire; testing and revising

December, February,
January
March,
April

questionnaire and interview questions; setting appointments to shadow and interview

ESMMNC; hiring transcriber; statistician decides which qualitative criteria she is
looking for and how it will be organized; requesting lists of schools, work-sites, and
hospitals

Collecting Data Selecting sample through random sample of sites from registries;

identifying addressees at sites; sending surveys with gift cards to sites; calling sites that
have not responded to follow-up; setting up interviews with interview-sites;
interviewing sites; interviewing ESMMNC; shadowing ESMMNC; transcriber
transcribes for notes

Analyzing Data Statistician enters data for questionnaires as they come in; when

data collection is complete, analyzes information from questionnaires to determine

commonalities in sites which adopt and do not adopt strategies; statistician codes words
from transcriptions and analyzes interviews for patterns; evaluators do qualitative
analysis of interviews and questionnaires highlighting themes, leverage points,
motivations for participation, and contexts

Developing Report Synthesizing data into easily accessible summaries; highlighting

patterns, leverage points, motivations for participation and contexts; developing a visual
representation of where ESMMNCs efforts and tactics for increasing implementation
are and how they line up with site motivations; making presentation to ESMMNCs
team; meeting with leaders to answer questions; creating printed copy of report for
ESMMNC

References

Institute of Medicine. 6 national obesity evaluation plans, evaluating obesity prevention efforts: a plan for measuring
progress. Washington, DC: The National Academies Press, 2013. doi:10.17226/18334
Eat Smart, Move More North Carolina. North Carolinas plan to address obesity: healthy weight and healthy communities
2013-2020. http://www.eatsmartmovemorenc.com/ESMMPlan/Texts/NC%20Obesity%20Prevention%20Plan%2020132020_LowRes_FINAL.pdf