Distance Learning in Elementary Schools In Cyprus: The Evaluation, Methodology And Results

by Panagiotes S. Anastasiades
A Paper Analysis by Hawwa Shiuna Musthafa
University of Cyprus

Abstract

This report attempts to reevaluate and analyze the evaluation findings of a collective
endeavor, undertaken by the Ministry of Education and Culture of Cyprus together with the
Department of Computer Science of the University of Cyprus, under the sponsorship of the Bank
of Cyprus, on a pilot program on DL via teleconferencing, in a project named ‘‘ODYSSEUS’’ 1,
in two elementary schools of Cyprus. The aim of this analysis is to provide useful insights,
critically analyze and offer explicit constructive suggestions by critically commenting on the
evaluation efforts undertaken for the DL system. The analysis and comments made in this paper
is specific to the evaluation carried out by the author 2 regarding DL as an effective teaching
strategy in Cyprus elementary schools, but can have wide-ranging implications to similar
program evaluations. At each stage of the methodology and evaluation, the reviewer 2 has
attempted to critically scrutinize, offer alternative constructive suggestions and pass critical
judgment. Through this analysis, the reviewer tries to optimize on evaluation theories, strengthen
arguments, and enhance work processes through useful suggestions aimed at improving the
program.

Keywords: Distance learning, Evaluation, Elementary level, Effective

Introduction

Research on distance modes of delivery and its effectiveness has been a focus for
investigation for several decades. It is important to recognize the rationale behind every DL plan
is fulfilled when the program objectives are effectively met as Belanger (2000) rightly puts, "The
key consideration for the success of DL is whether or not, and to what degree, learning
objectives can be met using different delivery technologies" (p. 14).

1
The ODYSSEUS program is the pivot of a collective endeavor, in which the students and teachers of the two
elementary schools that participated in the program were the main component parts of an open learning
environment. The elementary schools that participated in the pilot program (Helenio Elementary School of Nicosia
and Elementary School of Limassol) have already considerable experience in the area of videoconference through
18 P.S. Anastasiades / Computers & Education 40 (2003) 17–40 OIKADE program (www.oikade.gr). The main
objective of this program is to form a cultural and intellectual bridge over schoolchildren from Cyprus, Greece and
all around the world (Anastasiades, Eleftheriou, & Xambiaouris, 2001).

2
In this paper, reference to “Author” indicates the original author of paper under analysis i.e. Panagiotes S.
Anastasiades
Reference to “Reviewer” indicates the critic of the original paper i.e. Hawwa Shiuna
 In this context, the pilot program on DL via teleconferencing was put into feat and then
evaluated in two elementary schools of Cyprus. Good evaluation requires that evaluation in itself
to be subjected to appraisal. Even with the best evaluation procedures in place, there is the
possibility of technical errors, imprecise judgments or political and administrative sets backs.
Through evaluation and meta-evaluation, one can gather information about programs’ success
and failures in order to make informed decisions about the advancement of a program.

Objectives – An Analysis.

As Holmberg (1981) rightly states, "A statement of detailed objectives is the most
important step which will guide the construction of the whole program" (p. 44). Adhere to the
aforementioned indication; the study was carried out to realize the following objectives
(Anastasiade, 2003).

Development of complete pedagogic methodology as a prototype model for the

introduction of DL.
Development of evaluation methodology with regard to learning effectiveness and social
implications.
Discussion, reflections (results).

In this era of digital technology, incorporation of technological applications in the

educational process is vital. Before embarking on this study, the author should have considered
questions like what effective DL practices supplements traditional learning. Or what learning
conditions influence the effectiveness of DL in order to create the prototype model? Much of the
author’s focus is concentrated on the final result i.e. the model in itself rather than the process
and its effectiveness. This was a study on DL undertaken to supplement teaching effectiveness
not to overcome challenges of distance/deficiency in the school infrastructure. Also, key terms
used could have been defined like teleconferencing, tele-cooperation etc.
The author also could have been clearer about the specific evaluation objectives: was it
test effectiveness? To present critical points f DL? To check level of formation of cultural
intellectual bridge over school children?

Evaluation Processes and Practices- An Analysis.

On the initial perusal of the paper, stages of the program evaluation carried out are
unclear. A better approach could have been to separate the DL plan from the evaluation. A
diagrammatic representation of the whole process broken down to its components and concept
mapping would have been very helpful instead of just the broader areas. (Fig 1.1 Appended).

1
 Over all the stages of the implementation seem to be in accordance with literature on DL.
A very good practice noted was on the phase B2 and B33 : to allow not just teachers and students
to be evaluators of the program but also provide ways of solving the problems faced”.

Detailing on components for instance A1.1 “Familiarization of educators with basic traits
of new learning environment”, questions such as, “how were they familiarized, what /for how
long the techniques were used, were they tested for level of familiarity?” needs to be answered
which is of importance.

The study does not specify long term implications for implementing DL. The author’s
claim that DL at the elementary level is “a decisive factor for the preparation of tomorrow’s
citizens” is not sufficient.

Methodology of The Program4 - An Analysis.

3
Phase B2: creation of the virtual classroom
2.1 Description of the action: The creation of a complete learning environment is attempted in this phase, where
the whole lesson period will be covered by teleteaching and where questions from both classrooms will be
encouraged (discussion—exchange of ideas—other activities). This phase is aiming at the creation of the necessary
conditions in order to unfold all the activities that take place in a conventional classroom. Teachers and students
familiarize themselves with the idea of the virtual classroom. 2.2, Evaluation. 2.2.1, Students’ evaluation
(evaluation of knowledge, experience from the teaching etc.) by the teachers. 2.2.2, Teachers’ evaluation (noting
down problems, ways of solving them, alternate approaches etc.) by the teachers themselves in cooperation
with other teachers, specialists etc. 2.2.3, Evaluation of the procedure as a whole. 2.3, Recording of the evaluation
results. 2.4.4. Phase B3: tele-cooperation: development of joint activities between teleclassrooms

Phase B3 3.1 tele-cooperation: development of joint activities between teleclassrooms

Description of the action: In this phase we have an attempt to create an open cooperative environment between
the active students of the two classes. Basically, we have an attempt to create a work group that will be in
cooperation from a distance, in order to carry through a joint activity. During this phase, the students are the
leading actors in the new cooperative environment, while the teachers play a rather supervising-guiding role,
interfering whenever they find it necessary. 3.2, Evaluation. 3.2.1, Self-evaluation of the students (evaluation of
the procedure by the students themselves). 3.2.2, Students’ evaluation by the teachers (noting down problems,
ways of solving them, alternate approaches etc.) by the students themselves in cooperation with other students,
specialists etc. 3.2.3, Evaluation of the procedure as a whole.

4
METHODOLOGY
3.2.1. Evaluation target -Main target of the evaluation is to detect elements and data that will contribute to a
continuous improvement of the teaching methodology in real time. 3.2.2. Object of the evaluation The object of
the evaluation is divided into two main levels: teaching-pedagogic and social. The objective of the proposed
methodology at a teaching level is the evaluation, on the part of the students and instructors, of the teaching
methodology that was used. Pointing out the weak points in the planning and implementation of the various
subjects to be taught, spotting good practices that should be promoted further and, finally, estimating the
friendliness, efficiency and effectiveness of the teaching methodology were the key points of the evaluation. At a
social level, we were concerned about whether the distance learning process changed—and in what direction—the
impression and idea of the students about their teacher, schoolmates and the area of their school as an institution
of socialization. 3.2.3. Subject of the evaluation-The evaluation methodology is targeting the students, the
instructors and the components of the new learning environment (technological equipment, educational
technologies, subjects etc.). 3.2.4. Evaluation performers Evaluation is carried out by the students themselves, the
teachers, the support team (university students and scientific collaborators), and, finally, the research team of the
program. (Anastasiades, 2003).
2
 The notion of DL at elementary level necessitates the development of a complete
methodology focusing on special needs and requirements of such an endeavor and justifications
of why and how the methodology adopted is appropriate and benefits, which the author has failed
to do so.

The evaluation data collection methods are also questionable. For instance, testing of
students, teachers’ technological knowledge by means of a questionnaire is not enough. A better
approach would have been hands on assessment in which knowledge tested have actual effect on
the program. Similarly, evaluation of learning effectiveness on the part of the students was tested
through a questionnaire to find out whether “students found it pleasant to attend the course”. A
more appropriate approach would have been through observations. Question asked should have
been indicative to more of the criteria’s indicating effectiveness not how “pleasant” it is to attend
a course.

Evaluation of the results, discussions, and conclusions.

The evaluation used was dynamic and utilization focused, in the sense that evaluation
results of one stage influenced the implementation and decisions of the next. The author claims
the main purpose of evaluation result is to see the impact and improvement of the teaching
system on a long-term basis but the program impact5 assessment is one the most demanding

.
5
Evaluation of the social impact from the students’ standpoint: A close look into the social impact and the
relations between the students and their school environment in its new form.
_ Description: Based on a number of structured questions, there was an effort to outline the impression of the
students regarding the schoolmates of their own class and also the schoolmates of the other class they contacted
through teleconference, their teachers etc.
_ Target: To outline the impression of the students about their school as a place of incorporation into the society,
before, during and after the end of the teleconference sessions.
_ Method One: Self-evaluation of students. Implementation: Planning of easy and comprehensible questions with
scales of answers: I totally agree (5), I agree (4), I don’t know (3), I disagree (2), I totally disagree (1).
28 P.S. Anastasiades / Computers & Education 40 (2003) 17–40
_ Method Two: Evaluation of students by their teachers.
_ Implementation: Verbal questions and discussion between teachers and students during the lesson period.
Evaluation of the social impact from the teacher’s standpoint: A close look into the social impact and particularly
into the relationship of the students with their teachers and the relationship of
the teachers with each other.
_ Description: Recording any changes in the attitude of the students during the conventional
courses, as well as during teleconference sessions, level of relations between teachers and students etc.
_ Target: To outline the teachers’ idea of the students and their school as a place of incorporation into the society,
before, during and after the end of the teleconference sessions.
_ Method: Self-evaluation of teachers.
_ Implementation: Development of reports.

3
evaluating research tasks (Rossi, 2004). In this case, impact could be determined by predicting
what the outcome could have been without the program. Exante evaluation showed the pilot
programs impact prior to implementation. The ongoing evaluation also showed a positive change
in student’s attitude towards teleconferencing. However, it is incorrect to conclude based on the
quantitative analysis alone, that it is rational to implement, by comparing the exante and expost
evaluation scores alone when the most important factors has been highlighted by statements such
as “the traditional school environment (with the courtyard games) is a factor of significant
importance with regard to socialization of children as absolutely necessary component part of
student’s everyday school lives. Or “They seem to have reservations in regards to attending their
courses through teleconferencing from now on”. The author justifies this reaction being
“natural”; since it is the first time they were taught “this way”. The author cannot make
assumptions of this sort without any research support.

Concluding remarks.

In many ways, reading the data analysis/conclusion first made more sense of the program
and evaluation. Reference to page 33, 22 indicate lack of fluency and formality in English 6
language seen in sections of paper, For instance “……., movements that made by the small
students on their desks ….disappeared almost noise.”
The authors evaluation showed that the majority (80%) of students declaration that they
are PC users. There is no indication to show how this was measured. Was students’ declaration
enough? What criteria were used to determine and define PC users? The questions asked should
have been: is 80% enough? How were the DL instructions differentiated to cater to the whole
class? Another claim made was “all students had the essential knowledge to participate in DL
program” There is no indication to show how this claim was tested.

A comprehensive need assessment should have been done to determine whether there
was a need for DL. Process evaluation should have specific goals to answer questions like: did it
actually do what it was supposed to do? The program’s performance, perceived by students,
faculty should have been taken into consideration. Findings of this evaluation have immense
benefit especially for evaluators, teachers and parents to ascertain that DL alone at the
elementary level is not sufficient. Research must focus on placing effectiveness on the pinnacle

6
4.4. Evaluation of technological infrastructure Data analysis: The quality of picture is good enough. The camera
comes with a remote control, so we didn’t have the facility of automatic focusing. We had many problems with the
sound system. This problem was realised at the first videoconference. The main cause of this problem was
technical; the transmitted sound was not clean. This problem was reported immediately to the technical team of
program, which supplied us with a new advanced technology microphone, which isolates the noise and transmits
completely clean speech. A second factor, which influenced the quality of sound, was the various movements that
made by the small students on their desks (the locomotion of pencils etc.); they created important interjections in
the quality of sound. The initial constitution of schoolteachers was not effective. The solution was found when we
placed on the desks special buckram, which disappeared almost the noise. As a result of these two suggestions was
the exceptional quality of sound that we had at the duration of program. Results: The system of videoconferencing
did not support automatic focus techniques. The three ISDN lines ensured sufficiently good image quality; the
quality of sound broadcast was problematic, causing considerable difficulties in the process of videoconference.
Corrective interventions as they described above help us to solve this problem .
4
 of any DL programme rather than diminish it to a placard that seems to project affluences rather
than quality.

Figure 1 - represents certain important components of DL evaluation should focus on to

make it more comprehensive and encompassing:

LEVEL OF WORKABILITY OF
LEARNING INSTRUCTIONAL
LEVEL OF
EFFECTIVE DESIGN
INSTITUTIONAL
NESS
READINESS

LEVEL OF
STUDENT
DISTANCE EDUCATION PROGRAMME READINESS
FACULT EVALUATION
Y
SUPPORT

LEVEL OF
TEACHER
READINESS

LEARNING
STUDENT OUTCOMES
SUPPORT TEACHING
METHODOLOGIES

REFERENCES

Anastasiades, P. S., & Retalis, S. (2001). The Educational Process in the Emerging
Information Society: Conditions for the Reversal of the Linear Model of Education and the
Development of an Open Type Hybrid Learning Environment.
Anastasiades, P. S. (2003). Distance learning in elementary schools in Cyprus: the
evaluation methodology and results. Computers & Education, 40(1), 17.
5
 Anastasiades, P. S., & Vitalaki, E. (2011). Promoting Internet Safety in Greek Primary
Schools: the Teacher's Role. Journal Of Educational Technology & Society, 14(2), 71-80.
Belanger, F., & Jordan, D. H. (2000). Evaluation and Implementation of Distance
Learning : Technologies, Tools, and Techniques. Idea Group.
Clark, R. (1983). Reconsidering research on learning from media. Review of Educational
Research, 53(4), 445–459.
Holmberg, B. 1981. Status and trends of distance education. New York: Nichols
Publishing.
Lockhart, M., & Lacy, K. (2002). An assessment model and methods for evaluating
distance education programmes. Perspectives: Policy & Practice In Higher Education, 6(4), 98-
104. doi:10.1080/136031002320634998
Panagiotis, A., & Chrysoula, I. (2010). Communication between Tutors--Students in DL:
A Case Study of the Hellenic Open University. European Journal Of Open, Distance And E-
Learning, (2)
Patton,M.,Q. (2008). Utilization Focused Evaluation: The new century text (4 th
ed).London: SAGE publications .
Rossi,P.H.,Lipsey,M.W., & Freeman,H.E. (2004). Evaluation –A systemic Approach (7th
ed) USA :SAGE publications.
Tricker, T., Rangecroft, M., Long, P., & Gilroy, P. (2001). Evaluating Distance
Education Courses: the student perception. Assessment & Evaluation In Higher Education,
26(2), 165-177. doi:10.1080/02602930020022002