CHAPTER 7

EVALUATING EDUCATIONAL TECHNOLOGY

Educational technology and software evaluation are essential in the process of technology integration in education. The
purpose and processes of evaluating software and educational
technology are discussed in this chapter in which in the end you will have a knowledge and ability in evaluating such
tool to make teaching, assessing, and learning more effective and functional.

EDUCATIONAL TECHNOLOGY: EVALUATING ITS EFFECTIVENESS IN CLASSROOM USE

Evaluation is an important part of any process or any entity
or item consumed by an individual, to know how well it works,
whether it is effective or not, and to determine whether it benefits
or brings complication to person. Performance of students in class,
work performance of employees, teaching performance of teachers,
performance task done by students, seminars, and workshops are
some of the examples where evaluation occurs.
Evaluating the use of education technology in class is as
important as evaluating the processes done in any project. To
evaluate a process or item is to determine its importance and
usefulness. Determining the appropriateness of educational
technology in classroom instruction and to gauge how much it
enhances teaching and learning are some areas that we take to look
at to arrive at a final decision about its use and implementation.
Further, educational technology must:
1. be suitable for the teaching and learning situation
2. be motivational
3. promote learning
4. be at the right level or age of the learners
5. address the needs of the curriculum
help meet the learning outcomes
Educational technology is being evaluated before instruction, during instruction, and after instruction (Figure 7.1).
Evaluating Before Instruction
Before acquisition or before using any educational technology tool and software, the teacher should be asking himself/
herself the following question:
1. Will it match the needs of the curriculum?
2. Is it appropriate for the age of my students?
3. Is it appropriate for the teaching and learning situation?
4. Is it updated?
5. Will it match the learning capacity of my students?
6. Will it help me convey the message of the lesson?
7. Will it help demonstrate the processes?
Reading various resources about the particular educational technology that teacher wants to apply in his/her to
determine if this highly recommended by other users. Reviews about the technology, asking recommendations from
colleagues, attending conferences to make the teacher updated about the different educational technology would
likewise help in updating one’s knowledge about educational technology.
Evaluating During Instruction
During instruction, the teacher should continue to observe and assess the effectiveness of the educational technology
being implemented in classroom use. The teacher may take note of the benefits and problems encountered during
implementation.
This then, will help him/her in adjusting to the use and implementation of educational technology. The teacher should
be asking himself/herself the following question.
1. Is educational technology attractive to the students?
2. Is it innovative and exiting?
3. Is it being used appropriately?
4. Is it easy to use?
5. Does it help the student understand the lesson better?
6. Does it help me explain the lesson?
7. Does it help students to develop their higher order thinking skills?
8. Does it help student to stay in focus?
9. Does it improve learner’s behavior when it comes to learning?
10. Are students more participative and more engaged in learning?
11. Are there problems encountered during the use od educational technology?
Evaluating After Instruction
This is the stage where teacher will finally decide whether the educational technology used in instructional is of help in
the teaching and learning process. Before putting your thumbs up or down, you might want to ask yourself a few more
questions to help you determine if there is a need to change the educational technology, or you just need a little more
adjustment and practice in using the technology.
1. Did it help my students achieve the intended learning outcome?
2. Did it improve the performance of the students in the subject matter?
3. Did it enhance my students’ learning?
4. Did it help me determine the strong and weak points of my students in learning the subject matter
5. Are students looking forward to learn the next lesson?
Evaluating Software Programs
Evaluating the software particularly the instructional software to be used in class is likewise important as evaluating
technologies. The following are the areas that need to be evaluated (Table 7.1).
1. Content – The teacher has to determine if the content relates to the curriculum, school standards, and
instructional objectives. The teacher has to determine also the validity of the content , whether it is up-to-date or
not.
2. Documentation and Technical Support – When thinking of acquiring or using a certain instruction software, look
if there is proper documentation and technical support about the software. Documentation refers to information
that may assist the teacher in the installation of the software. Technical support refers to the service that
hardware, software and the company provide whenever the user encounters technical problems upon
installation, to repair and other assistance needed whenever problems arise during the use of the software.
3. Ability Levels and Assessment – the teacher has to evaluate if the software matches the ability level – student’s
current competency/skill level – in achieving a particular learning outcome/
4. Technical Quality and Ease of Use – Teacher have to consider the following when it comes to technical quality:
clarity of text, design, and multimedia elements; correctness of prompts and feedback; appropriateness of
graphics, audio, sound effects, animation, and video. When it comes to ease of use, the software should be user-
friendly for both the teacher and students yet sustains student’s interest and attitude of being challenged.
Use the following system to rate the software
1 = strongly Disagree; 2 = Disagree, 3 = Agree, 4 = Strongly Agree; NA = Not Applicable
Content
1. The content is accurate and factual. 1 2 3 4 NA
2. The content is educationally appropriate. 1 2 3 4 NA
3. The content is free for errors. 1 2 3 4 NA
4. The content meets your learning goals and objective. 1 2 3 4 NA
5. The content is free of stereotypes and cultural bias 1 2 3 4 NA
6. The content is meets the schools standards. 1 2 3 4 NA
Comments:
Documentation and Support
1. The teacher/ instruction manual is clear and thorough. 1 2 3 4 NA
2. The software has a support number. 1 2 3 4 NA
3. Online technical support is available. 1 2 3 4 NA
4. Help and tutorials are clear and easy to use. 1 2 3 4 NA
Comments:

Ability Levels
1. The ability level can be set by teacher. 1 2 3 4 NA
2. The ability level automatically advances. 1 2 3 4 NA
3. The software covers a variety of ability/skills levels. 1 2 3 4 NA
Comments:

Assessment

1. Software has built – in assessment and reporting tools 1 2 3 4 NA

2. Assessment methods are appropriate and suited to learning objectives. 1 2 3 4 NA
3. Software documents and records student progress. 1 2 3 4 NA
4. Teachers can assess student’s progress easily by evaluating progress reports. 1 2 3 4 NA
Comments:

Technical Quality
1. Animation and graphics are used well. 1 2 3 4 NA
2. Audio (voice input/output) is used well. 1 2 3 4 NA
3. Feedback and prompts are appropriate. 1 2 3 4 NA
4. The application allows branching and chunking. 1 2 3 4 NA
 Comments:

Ease of Use
1. Direction are clear. 1 2 3 4 NA
2. Students can exit the program at any time. 1 2 3 4 NA
3. Student can restart program where they stopped. 1 2 3 4 NA
4. The software is reliable and free of disruption system errors. 1 2 3 4 NA

Comments:

Educational Technology Integration: Evaluating its Effectiveness

Evaluating educational technology’s effective is no easy task for this is linked with the student’s learning. Thus
evaluating the effectiveness of educational technology is connected with the assessment given to students. We are aware
that giving of standardized test is one way to measure how much the student has learned. However, this does not totally
measure the total academic performance of the students. The skills and students’ ability have to be taken in consideration
when assessing the totality of students’ learning. It is therefore important that teachers have to develop indicators that
would measure not only the cognitive skills of the students but more importantly the psychomotor and effective domains.
The teacher should see to it that the learning objectives or outcomes specified at the beginning of the lesson will come
into fruition through the assessment to be taken by the students. Assessment in the field of education, when defined, is
any method used to measure how much the student has learned – knowledge and performance of students. The following
are the different assessment that may given to students:
1. Traditional Assessment - Includes multiple choice, essay, short answer, true or false, fill – in – the blank, and
matching types.
2. Alternative Assessment – Is a nontraditional method of measuring students’ mastery and skill level. Authentic
assessment, project – based assessment, and portfolio assessment are under alternative assessment.
3. Authentic Assessment – This is based on authentic learning method which allows students to do hands – on
experiments or activities, do research papers, and create or produce an output that is based on intended learning
outcomes/objectives. Authentic assessment helps students to develop a real-word skill that makes them life-ready
which can be applied outside school.
Wiggins(1998) list six characteristics of an authentic assessment:
1. The assessment is realistic; it reflects the way the information or skills would be used in the “real-word”.
2. The assessment requires judgment and innovation; it is based on solving unstructured problems that could easily
have more than one answer and, as such requires the learner to make informed choices.
3. The assessment asks the student to “do” the subject that is, to go through the procedures that typical to the
discipline under study.
4. The assessment is done in the situation as similar as possible to the context in which the related skills are
performed.
5. The assessment requires the student to demonstrate a wide range of skills that are related to the complex
problem, including some that involve judgment.
6. The assessment allows for practice, feedback, and second chances to solve problem being addressed.
Project-based Assesment – This is based on authentic learning which is project based learning(PBL) which engages
students in real-word projects where they are given a significant task of fulfill. Students are likewise given an opportunity
to construct their own knowledge , solve problems and create realistic projects.
Portfolio Assessment – this is also known as embedded assessment. Portfolio assessment is considered as a long-term
assessment assignment in which students can see and discover their own strength and weaknesses which they can
improve as they go along the process of learning. This may take student a great deal time in completing their portfolio and
teacher long period of time in evaluating the portfolio but it is worth for doing for both teacher and teachers and students
can see the progress in students’ accomplishment and learning. Portfolio can be done electronically called as e-folio,
eFolio, ePortfolio, or e-portfolio.
Educational technology plays an important role in the assessment mentioned above. Thus, completion of assessment
is aided with educational technology that helps students achieve the learning goals. When students succeed, the
technology then is beneficial and suitable in the learning process, of it is otherwise, the educational technology may not
be the appropriate tool for meeting the ends of education – that is learning.
Checklists, Rating Scales, and Rubrics
 After the students submit their requirements, projects, and other assessments, the teacher evaluate all these and
their learning process using checklist, rating scales, and rubrics. These evaluation tools are likewise helpful in evaluating
the integration of educational technology in education.
Checklist – An evaluation tool that indicates specific criteria that allow students to keep track of the things they

Tips for Developing Checklists, Rating Scales, and Rubrics:

1. Use checklist, rating scales, and rubrics in relation to outcomes and standards.
2. Use simple formats that can be understood by students.
3. Ensure that the various skills that students need to demonstrate are clear, specific, and observable.
4. Ensure that checklist, rating scales, and rubrics are properly dated to track student’s progress history.
5. Leave appropriate space remarks and comments.

need to complete and perform. This is also being used by teachers to judge or evaluate the knowledge and skills
performed by the students which are related to pre-determined learning outcomes. This is usually in a YES/NO
format that used to record the performance of students, individually, by group, or whole class, see Table 7.2
Table 7.2 Example of Checklist Evaluation Tool Performance
Assessment Evaluation Checklist
Criteria YES NO Remarks
The educational technology is impressive
The educational technology is effective in conveying the message.
The educational technology is free from formatting errors.
The educational technology was used effectively during discussion.
The text are clear and readable.
There are no errors on grammar and spelling of words.
The graphics are engaging.
The project was submitted on time.
Rating Scales – An evaluation tool that allows teacher to indicate the degree point/ observation equivalent to the
behavior or skills displayed by the learner. Rating scale indicates the criteria which are graded through numerical
value (e.g. 1,2,3,4) or descriptive word (e.g. exemplary, superior, satisfactory, unsatisfactory), see Table 7.3.
Rubrics – If you can recall, rubrics are introduced in the early part of this chapter. Rubrics then are defined as
scoring guides which consist of specific pre-determined performance criteria and used in evaluating student work
and performance. Rubrics allow teacher to grade the performance of the their students in a more precise manner.
This helps students to determine the areas they need to improve on and be better the next assessment; it allows
teachers to save time in grading the work performance of their students; it also help teachers to refine their
teaching skills. Rubrics may be holistic or analytic, see Table 7.4.

Criteria Exemplar Superior Satisfactor Unsatisfactor Remarks

y 3 y y
4 2 1
The educational technology is impressive.
The educational technology is effective in conveying
the message.
The educational technology is free from formatting
errors.
The educational technology was used effectively
during discussion.
The texts are clear and readable.
There are no errors in grammar and spelling of words.
The graphics are engaging.
 Criteria Excellent Very Good Good Poor Score Remarks
4 3 2 1
Contend/ The content The content The content The content lacks
Organization includes an includes a highly includes a a central theme,
outstanding clarity satisfactory somewhat clear clear point of view
of statement of clarity statement of and logical
theme; Message statement of theme; Message sequence of
are presented in a theme; message are somewhat information.
highly are presented in presented in a
Logical order a satisfactory logical manner.
logical manner.
Focus/Level Project is deeply Project is Project has Project’s focus
of focused and very focused and problems in and information
Information informative; it informative; it staying focused on are in question;
highly promote the promotes the topic; it provides the use of
use of computer use of computer some information; computer
technology in technology in the use of technology is
creating a video to creating a video computer ineffective in
deliver to deliver technology is delivering
information. information. somewhat information.
effective in
creating a video to
deliver
information.
Quality of All shots are of high There are There are several The entire vedio is
shots quality; Shots and minimal shots poor shots; Shots of poor quality;
scenes flow that are of low and scenes have The shots and
seamlessly. quality. Shot many unnatural scenes are full of
and scense flow breaks and/ or unnatural breaks
in good pacing early cuts. and/ or early cuts.
and timing.
Clarity of The audio is very The audio is The audio in The audio is cut-
audio clear and has an clear and has a inconsistent in off and has an
outstanding level of highly clarity at times unsatisfactory
effectiveness in satisfactory and has a level of
communicating the level of satisfactory level effectiveness in
main idea. effectiveness in of effectiveness in communicating
communicating communicating the main idea.
the main idea. the main idea.
Running The running time The running The running time The running time
Time and falls within the time falls within falls a little longer falls short of the
Video assigned 1-2 1-2 minutes and than 1-2 minutes; required 1-2
Effectiveness minutes. The video the video is the video has minutes; the
is effective in somewhat lapses in video fails to
communicating the effective in communicating communicate the
message to the communicating the message to message to the
audience. the message to the audience. audience.
the audience