Goals-Based Evaluation

Procedures: How
Students Perceive
What Teachers Intend
Kathleen M. Bailey, Donald Freeman, and Andy Curtis

Why Goals-Based
Evaluation?
MOST CLASSROOM EVALUATION
procedureswhether they are standardized
tests or informal assessmentsseek to
gather information about learners experiences and their increasing expertise against
some rubric of curriculum and intended
outcomes. This formulation appears to be
relatively straightforward: Students
perform, their performance is measured,
and those measures are taken as indications
of what they know and are able to do.
However, this approach, we would argue,
misses out on the link between instructionthe specific activities and learning
experiences provided for studentsand the
goals or expected outcomes of those
activities. So most evaluation procedures
tend to gather information about what
learners have supposedly learned (e.g., via
standardized tests) or about their opinions
of course activities (e.g., via student
surveys). But seldom do such procedures
truly illuminate the connection between
what was done in the course or seminar

Winter 2001

and why it was done. The procedure we

call goals-based evaluation focuses on this
critical connection.

Isnt All Evaluation

Goals-Based?
In some senses, the notion of goals-based
evaluation might be an oxymoron. After
all, you could say that any form of classroom evaluation aims to assess participants learning against some version of
what the teacher intended. Furthermore,
we can assume that teacher intention
carries with iteither explicitly or
implicitlysome commitment to goals or
outcomes. This is true; however, often
missing in this equation are the students
perceptions of how what was done in a
course or seminar connected (or did not
connect) to the ostensible purposes. Therefore, as we argue in this article, evaluation
that focuses explicitly on this relationship
between the teachers intentions (or goals)
and the students perception of learning in
terms of those goals is distinct.

Background
Like many ideas that develop through
collaboration, the specific origins of what
we now refer to as the goals-based
evaluation procedure are hazy. Donald
Freeman recalls his dismay, as a novice
EFL teacher in Japan, in trying to get his
adult students to complete the mandatory
course evaluation form. Often students
seemed not to remember what they had
done during the month-long intensive
course, so their evaluations contained little
useful information. Out of frustration he
began asking his students, before they
completed the mandatory form, to collectively recall what they had done in the
course. Soon he discovered that these
collaborative recollections were more
revealing than the data elicited by the
official form itself because the recollections constituted a sort of de facto view of
what had been valuable to them. When he
started asking students what they had
learned from the activities they recalled, he
began to see the outlines of a relationship
between what he had intended as a teacher
and what his students felt they had learned.

TESOL Journal

Much later, Freeman collaborated with

Kathi Bailey in an ongoing series of
professional development seminars for
staff members of U.S. government language programs. In that context, they
revived this initial idea and began to flesh
it out. Over the past 7 years, they have
worked with the concept of goals-based
evaluation, as they came to call it, in their
various language teaching and teacher
training contexts. When the opportunity
arose to write about it, they realized they
needed someone to press them on their
thinking on why they did what they did.
They asked Andy Curtis, a colleague and
friend, to be their interlocutor. What
follows, then, is a conversation among the
three as they examine the reasoning and
the procedures of goals-based evaluation.
(See the sidebar on p. 8 for a list of
procedures.)

A Conversation
Among Colleagues
Kathi: The basic premise of goals-based
evaluation is quite simple: As the teacher,
you are trying to discover the connections,
in the students minds, between the course
goals and what you did together in the
class. The procedure can be used with
either ESL/EFL students in a language
course or with teachers as the participants
in a workshop or seminar. I have found that
this process has profoundly influenced the
way I teach and the way I get my students
to evaluate my courses.
Andy: To me the procedure youve
outlined here seems somewhat teachercentered. Isnt the teacher the one who is
defining all the goals?

Kathi: Or, if you have a fully negotiated

syllabus, the students can contribute to
determining all of the objectives at the
beginning of the course. What matters is
that there is an explicit set of goals
available at the beginning of instruction.
Andy: O.k., so I can see how the negotiated creation of new goals or changing
existing goals can make the process more
student-centered if you want it to be.

Using the Procedure

in Practice
Setting Things Up
Kathi: As I said, the students need to be
able to see the goals at the outset of the
course. For example, these could be
included in a course description or
syllabus, or displayed prominently on a
poster in your classroom.
Andy: If youre working electronically, I
suppose its also possible to either e-mail
the goals to the participants ahead of time
or put them on an electronic bulletin board
or Web page so that participants can read
and possibly respond to them ahead of time.
Donald: Sure. The point is that the
participants need to be able to see where
they are headed in the course. So when you
are finished teaching the course, as part of
the assessment process, you give students a
worksheet with each goal written on it.
Their task is to come up with a couple of
activities or procedures that were used,
either in or outside of class, to address
each goal. As the teacher, you can specify
the number of activities they should try to
list, thus
giving the
task a finite
focus and
stimulating
their thinking
and recall.

I have found that this process

has profoundly influenced the
way I teach and the way I get my
students to evaluate my courses.
Donald: Not necessarily. You can discuss
the goals with the students at the beginning
of the course or workshop and see if they
want to add any other goals to the statement youve prepared.

TESOL Journal

Kathi: There
are a number
of ways to break down the scope of the
course into manageable components. For
instance, you can use this procedure as an
informal midterm evaluation. Or you can
let students list any number of activities

per goal or cite the same activity for two or

more goals if they choose. In general, I
find that if there are only three or four
goals for a course, I will ask students to
write four or five activities for each goal.
Andy: Is there an assumption that all
activities should be associated with one or
more goals? Its a logical assumption, but I
just want to be clear about it. I suppose the
planning implication is that if I, as the
teacher, cant identify one or more course
goals that possibly relate to an activity, I
may want to set aside that particular
activity.
Donald: Yes, or you could think about
whether there was a way to adapt the
activity so it does match the course goals
more closely. Or you could see if the
activity suggests that a goal is missing or
maybe that an existing goal needs to be
articulated further. So this harmonizing of
goals and activities serves both a proactive
function in lesson planning and an
evaluative one.
Andy: I used something similar to goalsbased evaluation recently in a workshop for
EFL teachers in Brazil. I wanted to know to
what extent the workshop goal had been
achieved for each participant, so I asked the
teachers to rate the goals as fully, partially,
or not at all. But I sometimes dont include
this rating question because I dont want to
make the form too cumbersome and timeconsuming to complete. What do you
think?
Donald: Actually, Andy, I think that what
youre describing is a bit different. In
asking participants to rate goals or stated
activities, you have already posited the
connection between the activities and the
goals. In goals-based evaluation, we ask
the participants to tell us which learning
activitiesin their viewsare linked to
the course goals. We dont supply the
connection for them; we want to see what
they think it was.
Kathi: But once the students have identified
the activities, the kind of rating system
Andy describes can easily fit into this procedure. For example, Donald and I recently
used a single-page goals-based evaluation
sheet on which wed printed each of the six
goals, left room for the students to list the

Winter 2001

activities that supported each goal, and then

asked them your question: To what extent
was this goal accomplished? Each goal and
the accompanying list of activities generated by the students was followed by a
Likert scale of 1 to 5, where 1 represented

between instructional activities and goals.

Some students remember events or
activities that others did not recall, so the
room is filled with comments like Oh,
yeah! I remember that!
Donald: Ive
noticed that some
students will work
entirely from
memory, whereas
others will leaf
through their class
notes in doing the
task. You can even tell the students to
review their notes in doing the task. Or,
going back to my experience in Japan, you
can have the students do a group brainstorm
and generate a list of activities that you can
write on the blackboard or a flip chart. They
can then individually choose specific
activities from the public list to attach to
particular goals on their own worksheet.

We actually see the procedure

not simply as data collection
but in pedagogical terms.
not at all and 5 represented completely
accomplished. The combination of qualitative information (the activities-goal
connection) and quantitative rankings was
quite valuable to us as workshop leaders.

Taking the Time

Andy: You know were always concerned
about having enough time in the classroom.
One possible criticism, it seems to me, is
that goals-based evaluation could take
more time than the traditional form-filling
evaluation process, even though it does
seem more useful both for the students and
the teacher. Is that a fair comment?
Kathi: Yes, doing the goals-based evaluation in class does take valuable class time,
and it may result in the students writing
hurried or incomplete thoughts. But doing
the procedure in class also assures that
everyone present will be working in the
same time frame and will have the materials at hand for the rest of the process.
Andy: Right. Allowing any form to leave
the classroom reduces the likelihood that
youll ever see it again! Still, I suppose that
taking it home gives the students more
time to reflect and therefore to write more
thoughtful answers. Because they are
thoughtful, those responses might be more
useful to the teacher.

Comparing Collective
Perceptions
Kathi: We actually see the procedure not
simply as data collection but in pedagogical
terms. After the students have completed
the goals-based evaluation form individually, they compare their responses in
groups or pairs. This task (Step 5) becomes
a very interesting process of sharing
memories and debating relationships

Winter 2001

Kathi: One other thing Donald and I have

done lately is to provide an inventory of the
activities we have used during the course or
workshop. After the students have talked
together, we give them a paper copy of that
inventory and ask them to see which items
on the inventory did not arise in their own
discussions. If there are any, we then
discuss why those activities didnt appear
on their own list. Did they happen too long
ago? Were they simply not memorable?
Did they blend into other activities?
Donald: This issue of activities running
together in participants minds is an
interesting one. We find the inventory/
recall procedure especially useful in a
training-of-trainers situation, in which the
goals often refer to learning about training
from the training processes as well as from
the course content.
Andy: I suppose another benefit of compiling an inventory is that it can help the
teacher or presenter to remember activities
that may have been forgotten in the natural
business of managing group interactions.
Kathi: Thats right. If I make the effort to
create the activities inventory as the course
progresses, it helps me see which goals
Ive been addressing adequately and which
ones need more attention. In fact, thats
one way in which using this process has

influenced my teaching, as I mentioned

earlier: I find it keeps me focused on the
goals throughout the course. But mainly,
the inventory serves as an interesting
memory-jogger for the students. As they
are talking, you hear comments like Oh,
yeah! I remember that day; we used those
colored plastic pieces to solve the problem. And so on.
Donald: Its also interesting that some
students remember the same events but
associate them with different, or fewer, or
more goals: Theyll say, Yeah, I remember that activity, but I thought it was more
closely related to Goal No. 3 because. . . .
This discussion becomes a very rich
examination of why things were done in
the course or workshop. For me, it works
as a powerful stimulus to a discussion of
pedagogical means and ends.
Andy: Let me play the devils advocate
again. I can see the clear benefits of these
steps, especially in reviewing notes and
notions. But isnt there a danger that by
doing this youll get consensus responses
based on what a group or pair agrees on?
The sharing process could mask some of
the individual evaluation responses.
Donald: You do have a point. If your aim
is to hear from each student individually,
based on what he or she recalls about the
instructional activities, then the group procedure may not be appropriate. But goalsbased evaluation is not intended to be a
memory test. The key issue is the connection that students make between any given
activity and the course goals, not whether
they remember the activities. One modification Ive tried is to have students work
alone first. They either draw a line on their
papers when they have finished working
independently, or I ask them to use a
different colored pen. Then they can add
any of the information stimulated by the
group discussion to their own list if they
choose to.
Andy: I like this idea. Its a clear but
simple way of not only enabling group and
individual meaning-making, but also
showing on the page which one is which.
Kathi: If I have time, I like to do one more
iteration of this part of the procedure by
having the students rotate groups. If you

TESOL Journal

take care to mix the students well, the new

group should have people who represent
the ideas that emerged in the previous
round of discussion.
Donald: The discourse that emerges in this
second round is quite interesting because
students are not only presenting their own
ideas but those of their former partner(s) as
well. Youll hear comments like, Well, in
my other group, Janna was just saying
exactly the opposite. She thought that . . .
and so on. The point here is that students
weave together a view of class activities in
relation to the course goals.
Andy: One of the things I really like about
these kinds of repairing arrangements is
watching the difference between individuals knowledge recall and the way that
knowledge is viewed and reconstructed by
considering it from others perspectives.
That old clich about the whole being
more than the sum of the parts is true.

Returning to the Final

Whole-Class Discussion
Kathi: The wind-up plenary discussion is
usually very important and should be
allotted adequate time. This part of the
process often spurs questions or comments
to the teacher like, Well, Jos thought that
Activity X addressed Goal 5, but Mika and
I didnt agree. So why exactly did you have
us do that activity? The ensuing discussion
becomes another opportunity for teaching.
Donald: This plenary interaction plays
another function often missing in other
evaluation processes. It lets students
knowand even debatewhat their peers
learning experiences have been. This
metalevel reflection is very important to
learning.
Andy: This point raises another question:
What does the teacher do if the participants interpretations of which activities
addressed which goals differ dramatically
from one another? Or what if they differ
from what the teacher had intended? Is one
connection between goal and activity
somehow more correct than another? In
other words, how do you judge among
conflicting views of the same activity?
Kathi: To me there isnt any right or
wrong answer. We are simply asking

TESOL Journal

Goals-Based Evaluation Procedures

Step 1: For a course or a workshop, articulate the goalswhat the students or
participants will know and be able to do at the end of the program.
Step 2: You may want to discuss the goals with the students at the beginning of the
course or workshop to see if they want to amend any or add others.
Step 3: At the end of the course, give the students a sheet with each goal written on
it, along with some version of the following instructions: Listed below are the goals
for our course (workshop). For each goal, try to think of at least three activities or
procedures we used to address that goal. These can be assignments, class
activities, discussions, guest speakersany process we have used that you think
addresses the particular goal.
Step 4: Have the students do this task (matching activities with goals) individually in
writing.
Step 5: In class, have the students compare their ideas with one another about the
relationship between the instructional processes and the course or workshop goals.
Step 6: If there is time, you can do one more iteration by having the students rotate
groups and repeating Step 5: Have each student share with the others in this new
group which classroom activity he or she associated with which goal.
Step 7: Have a plenary discussion with the entire group or class. Review each goal
and ask the students to enumerate the activities that were used to accomplish that
particular goal.
Step 8: While one group is reporting, ask the other groups to listen to see whether
the first group thought of everything their groups came up with, or if the reporting
group articulates an activity they did not identify.
Step 9: You may also want to provide the students or participants with an inventory
of the activities that were used during the course or workshop to remind them of
those they may have forgotten.
Step 10: Have the students give you their written comments on the goals-based
evaluation form. Review the students ideas to see which activities they connected
with the various goals of the course.

students to report their own perceptions of

how the activities matched the goals. If
there are some surprising matches, I ask
students to explain their thinking. Often I
learn something about my teaching from
this input. For instance, I am now more
explicit during the course or seminar in
articulating my own thinking about how a
particular activity relates to a goal
especially when the students are themselves teachers or future teachers.
Andy: When you first mentioned goalsbased evaluation, I first thought of the
usual kind of student-evaluation-ofteaching format, but I realize now that
goals-based evaluation serves quite
different purposes and provides different
feedback from the more typical institutionally required (and often institutionally
designed) feedback forms. To some extent,
those forms answer the question How
well has this teacher done on this course in
the opinion of the students? As we said
earlier, the question is usually posed in the

form of Likert scale statements with

quantitative or categorical rating options,
and sometimes there is space for openended comments. However, students are
not usually required to question their own
perceptions of the activities they learned
from, maybe because its assumed that
their grades for the course will provide
evidence of that link.
Donald: You make a good point. We
started developing goals-based evaluation
as a lens through which to view participants perceptions of what we had done in
our courses. However, from a theoretical
point of view, we began to see that, in fact,
we were only capturing half of the conversation. In other words, we knew what our
intentions had been in doing what we did,
but we didnt know how our students had
experienced the results of those intentions.
Andy: Goals-based evaluation is another
way of addressing what I see as an
endemic problem in traditional student or

Winter 2001

Andy: At the beginning, Kathi, you said

that this process has profoundly influenced
the way you teach and the way you get
your students to evaluate your courses. At
first I was wondering why that might be,
but having read through your description of
what you do, thinking about what I do that
is similar, I can see why you said that. As
you can see from my responses, these ideas
have certainly tapped into some of my core
beliefs about teaching, learning, and
assessment of outcomes.
Donald: To me, the significance of this
procedure lies in the focus on the participants as learners. By capturing their
perceptions of what taught them in the
course, we are looking at the classroom
experience from their points of view. And
as you both say, this process can be
humbling to the teacher. When you see
how what you had intended to teach
doesor does notmap on to what
participants perceive as instrumental in
their learning, you are subordinating your
work to theirs. It is a reorientation, but one
that carries endless fascination, I think.

Winter 2001

Authors
Kathleen M. Bailey is professor of applied
linguistics at the Monterey Institute of
International Studies in California, in the
United States, and a former president of
TESOL (19981999).
Donald Freeman is a professor at the
School for International Training in
Brattleboro, Vermont, in the United States,
and a former president of TESOL (1993
1994).

* * AD * * *
Fulbright Teacher
Exchange Program

Kathi: True. But lest we create an either/or

choice between goals-based evaluation and
the traditional student-evaluation-ofteaching process, they can certainly be
used in tandem. At the Monterey Institute
of International Studies, where I work, I
have used goals-based evaluation in
conjunction with the usual studentevaluation-of-teaching format. My students
do the goals-based evaluation procedure
first, as a way of recalling what we did in
class and trying to figure out why we did it.
Then they complete the institutional form.
Since I began combining the two procedures, Ive found that the comments I get
on the institutional form are much more
precise and informative.

Caleb Gattegno used to say in his seminars

that communication is a miracle, meaning
that you never really know what another
person is understanding. The same is true
of the relationship between teaching and
learning: It is indeed a miracleif, when,
and how they connect to each other. In the
goals-based evaluation procedure, we are
trying to make a bit of that miraculous
connection visible.

participant evaluations: the overemphasis

on how well the presenter has done in a
particular session in the opinion of the
participants. Instead, goals-based evaluation asks participants, What activities
have we done that mean something to you
as individuals and as a group in making
sense of these goals? Its also a way for
participants to build on each others
understandings, particularly in terms of
different perceived purposes for the same
activities.

Andy Curtis is an associate professor at

the School for International Training in
Brattleboro, Vermont, in the United States.

* AD * *
Lane Press

TESOL Journal