Center Based Learning Program Evaluation

EME 640

Introduction to Program Evaluation

Soniya Virani and Ivin Oommen

The University of Tampa

Background

The Tampa Religious Education Center is one of the 5 major REC’s in the state of
Florida. The center follows the pattern of a regular grade school with lower
primary consisting of Pre-K and KG, primary section consisting of grade 1 to 6 and
the secondary section includes grades 7 to 12. The focus of this evaluation will be
on the pre-primary section where CBL has been introduced and implemented in
class. Pre-K and KG both are managed by 2 teachers in each class and the lower
primary is taken care by the coordinator. The REC’s function under the Ismaili
Tariqah and Religious Education Board (ITREB) which comes under the Aga
Khan Education Services. The Aga Khan Education Services is a branch of the
globally operating not for profit organization called the Aga Khan Development
Network which is headed by the Aga Khan. In a recent revamping of the education
system the Tampa REC was introduced with the Center Based Learning program
which is an emerging learning experiment designed specifically for institutions
which encourage learning by doing.

REC is a volunteer organization and teachers and the management team strive their
best to give their time and knowledge to kids and their future. With changing times
and technology CBL was introduced and implemented for kids in the pre-primary
section to form a base as they move forward with their classes.

Program and Stakeholders:

CBL is when teachers turn the dull classroom to an educational space. As the name
suggests, the classroom is setup with different centers like drama, science, reading,
art keeping in mind the topic for the day. The CBL approach is considered as a
new age innovative method which does not focus on direct instruction but gives the
learners an opportunity to construct and discover new knowledge and concept.
Children who actively participate in the CBL program learn by experimenting,
investigating, experiencing and exploring.

The teacher here becomes the catalysts of knowledge rather than being considered
the sole provider of information. They provide the necessary materials required for
the day, explain the tasks and specifies the objectives of the lesson.
When CBL was introduced at the Tampa REC, teachers went through training and
mock classroom setups. Once the training was completed it was up to the teachers
to implement the CBL program in classroom.
While designing the program, the regional team came up with lesson plans which
are to be provided to the teachers at least 3 days before the class to prepare them
for what they have to teach, the specific objectives for the class for that particular
day and the goals which will be achieved at the end of the class.

The following is one example of how the children go through their day for the
class:

Table 1. Class Schedule

12:00 p.m. 12:00- 1:30-1:45 1:45- 3:00-3:30 p.m.
1:30 p.m. p.m. 3:00
p.m.
Children arrive and Class Break and Class Lunch and assembly
settle in classroom snacks with primary section

Along with helping the children learn by doing the CBL program also encourages
the children in to think of various ways to complete the goals.

To keep the evaluation team on track with the requirements of the stakeholder the
evaluation team decided to use the Kellogg Evaluation Logic Model. The Kellogg
Logic Model has the following steps:

Resources→Activities→Output→Outcomes→Impact
The following breakdown will give a clear picture about how we can provide a
roadmap about the process of evaluation that will be conducted:

Kellogg Logic Model for CBL Program:

Resources: Classes are conducted every Saturday for 3 hrs. The concept of CBL is
applied to the class every day. Teachers need different resources for their classes.
For example, for drama center, the teacher might require costumes related to the
topic to be taught.

Activities: The activities conducted in class are related to the topic or the theme
that is to be taught. The children use the provided resources and learn by doing the
activities set out for them in the classroom.
Output: For maximum retention for the child, he/she must go around all the centers
in the classroom. For this purpose, the teachers keep a track record with the name
of the child and which centers has he/she been to. Along with this they also come
up with a lesson plan for the day.

Outcomes: Once the child has explored all the centers he/she has a certain amount
of knowledge about the topic or the theme of the day. At the end of the class the
teachers do a 10 mins review with the kids to determine whether they have learned
the topic.

Impact: Long term impact for the topic taught to the child can be determined when
he/she goes home and shares the same with their parents or uses the knowledge
and teachings in their day to day activity. For example, if the topic taught to the
child id honesty, he/she when does something wrong at home and is aware that
lying is bad accepts his/her mistakes in front of their parents.

Stakeholder:

There are three types of stakeholders for the Centre based learning program.

Upstream Stakeholders:
Our upstream stakeholders include the headmistress of the Tampa REC and the
coordinator of the pre-primary section.

Direct Impactees:
Students of the pre-k and KG section along with their teachers would be included in
our direct impactees. Since they are currently affected by the program directly.

Indirect Impactees:
Under our indirect impactees we would include the parents of the participating
children and the past participants along with the future participants of the CBL
program.

Evaluation Methodology:
Evaluation Purpose and Type:
Based on the initial information and the requirements of the client, the program
evaluation team will conduct a formative based evaluation to determine whether
the program is effective for the target audience.
The formative evaluation will focus on assessing the CBL program and how
impactful is the program for the children and much they are able to retain the
knowledge and information taught to them in class.
The evaluation methodology will consist of the following steps:

1. Developing data collection instruments

2. Data collection
3. Analyze data
4. Conclusion

Evaluation Dimension, Question and Importance Weighting:

Upon our initial discussion with the client, the evaluation team proposed the
dimensions for the evaluation which will form the base of the evaluation. On
discussing further, the team and the client came up with 5 dimensions:

Dimension and Data Instrument Rationale

Dimensional Question Collection Developed
Method

1. Review/Feedback: Interviews 2 separate Interviews with

Do children apply the with teachers questionnaires teachers will give
knowledge they gain from Observation for teachers an insight on how
CBL at home? of meetings involved in the children perform
conducted program and for in class and what
with parents parents do they learn at
the end of the day
Observation of
meetings will
give the
evaluators an idea
if the CBL
program is
impacting the
children at home.
 2. Events: How Direct Checklist Direct
do the activities in class Observation observation of
affect the children? activities in class
will enable the
evaluator to
determine how
the CBL program
is implemented in
class and what
activities are
helping children
learn more about
the topic.
3. Design & Structure: Pull the design The data from the
Does the content and design Extant Data and structure of teachers will give
of the program meet the Review the program a clear picture of
requirement of our direct from teachers what the
impactees/stakeholders? who have objectives and
attended expected
training outcomes of the
stakeholders for
the program are
4. Attendance: Are the Review Checklist with Comparing
children motivated or attendance names of the before and after
excited to come to the sheets student attendance rates
class? of the CBL
program will help
us analyze if the
program is
effective
 5. Managerial Support: Interviews Checklist Direct one on one
Does the management Observation Questionnaire interview with the
effectively support the teachers,
program? coordinator will
enable the
evaluator to
determine if lack
of managerial
support affects
the functioning of
the program

Data Collection Procedure and Methods:

The evaluation team will use multiple types of data collection method which
includes,

 Interview - One on one interview with the teachers and questionnaires for
parents.
 Direct observation - Involves keenly observing the students and making a
checklist.
 Extant data review - Review the design and structure of the program
 Review attendance sheets - Comparing before and after participation will be
useful.

Below is a tentative timeline for the implementation phase to be completed in 2018:

 Data collection instrument development - March 6th through March 12th
 Data collection - March 13th through April 1st
 Data analysis - April 2nd through April 16th
 Preparation of a final report - April 17th through April 29th
 Delivery of the final report to the client - April 30th

Assessments of Feasibility and Risk Factors:

The following is a summary of the evaluation team’s assessment on project

feasibility:
 1. Maturity: The program is already in use at high school level which gives us
an idea how it works and what have been the benefits and challenges of the
program.

2. Scope: It is medium in scope and can be well completed in the time frame.

3. Support: The evaluation team has full support of the upstream stakeholders
and has
access to any information they require to successfully complete their
evaluation.

4. Ethical Concerns: No sign of ethical concerns. However, the client has

requested to remain anonymous.

5. Resources: Everything that is required for the evaluation is available.

The following risk factors have been noted and the evaluation team will work
closely with the stakeholders to see that its effect is minimal.

1. Access to classrooms when class is in progress

2. Access to data and resource available in premises

Table 3. Risk Assessment Matrix

Damage to Minor Moderate Substantial Detrimental

project

Likelihood

- - Lack of data Immediate

Unlikely access impactee

participants
 (Children aged

4-5 yrs.)

Maybe - - - -

Likely Lack of awareness - - -

of the evaluation

process

(Teachers and

management team)

References

Chyung, S.Y (2018). Ten - step evaluation for training and performance
improvement. Thousand Oaks, CA : SAGE

Kirkpatrick, D (1996). Evaluating training programs : The four levels. San

Francisco: Berrett - Koehler