Week 11 Assignment

Chapter Summaries
Part 2: Chapter Summaries
Chapter 1: What is Evaluation?
Chapter 1 served the purpose of introducing readers to the basics of evaluation. It begins with a
brief comparison between evaluation and research. The material then progresses to a discussion
about two definitions for evaluation. The first definition views evaluation as a systematic process
of gathering and analyzing data for the purpose of determining the degree to which objectives are
being met. The second definition also sees evaluation as a systematic process; however, this one
sees evaluation as that which attempts to collect and analyze data for the purpose of making
decisions. From there, the chapter moves into a discussion about efficiency, effectiveness, and
impact.
Another key aspect of the first chapter is the discussion about evaluation as a design format. The
book provides a table that evaluators can use in this effort. The table includes topics like
evaluation questions, data sources, population samples, audience, and few other important
components to evaluation. The authors conclude the chapter with a brief explanation as to the
difference between formative and summative evaluations. Formative evaluations are those that
occur as the processes happen, and summative evaluations focus on the end results of a program
cycle.
Chapter 2: Why Evaluate?
Chapter 2 examines several areas that relate to the purpose of evaluation. The chapter begins
with a discussion about the benefits of evaluation. These benefits include, but are not limited to,
helping staff by identifying and promote program objectives, fostering collaborative
conversations amongst workers, and highlighting areas of concern that can be used as a way of
improving the program or product. That said, the authors point out the limitations of evaluation
as there is no guarantee of action based on the evaluation. They also note the concern that
evaluations can bring out criticisms from staff and fears of retribution, causing workers to act
differently than they normally would have.
The remainder of the chapter is dedicated to describing the steps that one can take to help ensure
a successful evaluation and the ways the evaluation can be used. Objectivity and openness are
key to ensuring a successful evaluation. There also needs to be a willingness on behalf of key
stakeholders to actually use the results for making improvements. The evaluation can be used as
a way of clarifying the goals and mission of the organization and determining which aspects of
the program to keep and/or scale back.
Chapter 3: Decision Making: Whom to Involve, How, and Why?
This chapter takes a look at evaluation as it relates to a program cycle. Ideally, the evaluator
would be involved in the project cycle from the onset, but this is not a requirement. The book

then draws a comparison between the life cycle of a program (philosophy/goals, needs
assessment, program planning, formative evaluation, and summative evaluation) and the
individuals who would need to be involved in each phase. It is important not to overlook
individuals just because they will not be recipients of the report itself. The authors eventually
discuss each of these phases toward the end of the chapter.
The other major section of this chapter is dedicated to discussing the differences between
monitoring and evaluation. This was an important portion of the chapter because I would have
thought of them as one in the same; however, the authors draw clear distinctions between the
two. Monitoring is about observing the actions and activities of the parties involved in the
program that is being evaluated. This collection of notes and data can then be used by the
evaluator in his or her evaluation, especially as it relates to understanding the impact of the
program.
Chapter 4: Starting Point: The Evaluators Program Description
This chapter centers on the Evaluators Program Description (EPD), which basically serves as
the starting point for the evaluation. The EPD provides an insight as to the programs goals and
objectives, the activities that will accomplish them, and measurements tools that are in place to
determine attainment. The EPD begins with the evaluator sharing his or her understanding of the
program, to which the stakeholders will often add or clarify information. Eventually, the EPD
will allow the program evaluator to choose an evaluation model, construct the evaluation
questions, and determine the evaluation design format that will guide the way forward. If done
correctly, the EPD process will help the evaluator build rapport with the individuals who will be
involved in the evaluation. It is important to note that the authors suggest gearing the EPD
toward the recipient from the organization.
For me, this was one of the most important chapters in the entire textbook as it gave me a much
more concrete understanding of evaluation. The initial chapters focused more on understanding
what evaluation is and why it is important. This was the first time that I felt comfortable with the
tasks that I would be taking on in this course.
Chapter 5: Choosing an Evaluation Model
Chapter 5 provides readers with information that can be used in determining the most appropriate
evaluation model given the task at hand. This requires the evaluator to identify the purpose of the
evaluation. The book lists four main models: Discrepancy, Goal-Free, Transaction, and
Decision-Making. The authors then go on to list what they call other models including systems
analysis, art criticism, and adversary. The final, and most commonly selected method of all, is
the goal-based model. The book then moves into the individual components that one would
expect to consider in their evaluation design format.
This chapter was somewhat troublesome for me, not because the information was difficult to
understand or use; rather, I found it tough to fit my evaluation into a single evaluation model. I
waffled between the Decision-Making and Goal-Based models as both seemed to encompass
various aspects of my evaluation project. After additional reflection, I went back and did what

the authors suggested; I asked myself what the purpose of my evaluation is. In my case, the
purpose of the project is to determine if the program is attaining the goals it has established. This
allowed me to settle on the idea of using the Goal-Based model as the foundation for my
evaluation.
Chapter 6: Data Sources
The initial portion of this chapter is dedicated to discussing the differences between qualitative
and quantitative data. Qualitative data is collected through interviews, surveys, and other forms
of data. The key is that it does not involve numerical ratings of any kind. It is presented in a
narrative form. Quantitative data can be collected in a number of ways, but it always involves the
use of numbers in one form or another. The authors extend the discussion on data by
categorizing them into four levels: nominal, ordinal, interval, and ratio. Qualitative data lands in
the first two levels, and quantitative data falls in the latter two. Each level builds upon the
previous one. The remainder of the chapter is dedicated to discussing methods for collecting
each type of data, including interviews, scales, sentence completion, tests, and observational
analysis (unobtrusive and obtrusive).
This topic was one that I somewhat dreaded coming into the course. I have very little experience
with statistical data analysis beyond examining standardized testing scores. Thus, I was not sure
what to expect in terms of the necessary math skills for completing such tasks. That said, the
course text provided an excellent overview of the two types of data, leaving me much more
confident and open to the content in the next chapter: data analysis. All of these skills are
essential components of a successful evaluation.
Chapter 7: Data Analysis
This chapter continued the information presented in Chapter 6, but it moved the conversation
forward to actually begin the discussion on data analysis. Data analysis is basically the process of
working through the data that have been collected from the various data sources identified in the
evaluation. Going into the chapter, I saw data as purely numbers; however, the authors quickly
pointed out that data also includes responses to interview questions, surveys, and observations.
They also point out that a revisit to the objectives of the evaluation will often tell the evaluator a
great deal about who, how, and when to collect data. From there, the authors introduce readers to
some of the common terms that are associated with data analysis. Measures of central tendency
include mean, media, and mode. Measures of variability include ranges, quartiles, and
percentiles. This took me back to my elementary years when everyone wanted to know which
percentile their test scores fell under.
This topic was not nearly as daunting as I had expected it to be. While I have not yet begun the
process of processing the data from my evaluation, I am confident that I will be able to do so
when the time arrives.

Chapter 8: Is it Evaluation or is it Research?

While the authors briefly discussed this in the first chapter of this book, they use this chapter to
dive deeper into the comparison of evaluation and research. In essence, one can determine
whether something is evaluation or research in two ways. The first is to look at the intent of the
actions. If the goal is to use the work as a means of helping program staff improve the product,
then one is completing evaluation. Likewise, if the goal is to advance a theory within a particular
field of study, then one is participating in research. The other way of determining the scope of
the matter is by examining the methods by which one is conducting his or her studies. If the data
are collected in controlled conditions with attempts to account for variables, then it is likely
research. Research also focuses on cause and effect relationships. There are some common
characteristics between the two, but they are not the same. The chapter concludes by examining
the topic of populations and sampling.
This chapter was an excellent read because it cleared up a great deal of confusion that I had
coming out of the first chapter. I took EdTech 504 last semester (which involved heavy reading
on research), so my brain was trying to distinguish between the two. This chapter helped me to
understand how they were different, but that they also involve some common traits. The other
part of the chapter that I found extremely helpful was the suggestions about sample sizes. Given
that my population size is less than 100 individuals, the authors suggest not using sampling. This
confirmed my initial inclination to use them all and helped me to move forward in my project.
Chapter 9: Writing the Evaluation Report
This chapter brought all of the other readings into the perspective as it lays out the writing of the
evaluation report. This was perhaps the chapter that most directly relates to the final project that
we are completing for this course. The authors point out several key components as they relate to
the final report. First, the report should be addressed to the individual or entity that hired you to
evaluate the program. Second, the report should focus on the questions that served as the basis
for the evaluation. This makes the report meaningful. The remainder of the chapter is set aside
for describing each section of the evaluation report. This includes an evaluation report template
and a sample evaluation that can be found in Appendix B of the course text.
This chapter was key as it provided me with more information about what we will actually be
doing with all of the materials we have worked on up to this point. I look forward to taking this
and moving my evaluation report forward as we near the deadline.
Part 3
The internet reading that stands out the most is the one entitled Developing Questionnaires,
Interview Schedules, and Sociometric Instruments. I elected to read this chapter because I knew
my evaluation project was going to include multiple questionnaires, and I hoped that the contents
of the reading would provide assistance with writing successful ones. The author points out that
questionnaires that attempt to identify opinions (as mine will) should focus initially on who and
what the evaluator wants opinions about. Mine will likely focus on opinions from students
regarding their experiences with Cyber High and Studies; moreover, I will attempt to ascertain

the teachers opinions about the very same topics. From there, the author shifts the focus to
writing the questions.
Like the B&D text, this article lists several different types of questions that can be used in a
questionnaire including checklists and scales, but extends the list to include open-ended
responses, fill-in-the-blanks, and choice types. I had not considered these latter options as they
were not mentioned in the course text, at least not from what I remembered. The author also took
the time to discuss several question types (leading, comparative, feelings, should, and many
other). This list has greatly broadened the scope of questions that I had initially planned for my
surveys. According to this author, the third step is to arrange the questions in one of four
possible arrangements (general to specific, noncommittal to sensitive, topical, and mixed).
From there he discussion moves to examining the ways to respond and how to write the
instructions. I had not considered the instructions that I would have to write for my survey, so
this was very informative. For example, the author points out that all instructions should include
a rationale to tell the participants why they should answer the question in the first place. I figured
I would simply need to tell them how to take the survey. The author concludes this section of the
chapter by noting important considerations that should be paid to reproducing the questionnaire.
This includes leaving ample room for responses and making sure copies are easy to read. The
remainder of the article is dedicated to topics that I felt did not apply much to my situation;
however, the impact of the first section made this stand out in my mind as one of the most
impactful in preparing me to complete the data collection and analysis phases of my evaluation
project.