Challenges in the Evaluation of Educational
Computer Games
Hasiah Mohamed @ Omar
Universiti Teknologi MARA (UiTM)
Faculty of Computer and Mathematical Sciences
Dungun Campus, 23000 Dungun, Terengganu, Malaysia
hasia980@tganu.uitm.edu.my
Abstract—Computer games have been used for educational
purposes nowadays. The implementations of educational
computer games bring some challenges and issues that need to
be highlighted. There are three challenges in evaluating
educational computer games which are the evaluation criteria,
the evaluators and the evaluation process. Evaluation criteria
have been identified and it consists of five issues which are
Interface, Educational, Content, Playability and Multimedia.
The evaluators are real and surrogate users. The third
challenge is regarding evaluation process and one of the
suitable processes is by implementing formative evaluation in
order to develop a comprehensive and practical educational
computer games.
Keywords-component; educational computer games;
challenge; issues; evaluation crtiteria
I. INTRODUCTION AND MOTIVATION FOR RESEARCH
Computer games have been popular nowadays and it has
been proposed as a potential learning tool by both
educational researchers [1, 2] and game developers [3, 4].
There are many arguments and debate regarding the
implementation of computer games in classroom and among
them are; a) computer games able to raise active engagement
in learners [1] b) computer games able to encourage active
participation of learners, learning by doing and more
motivation [5, 6, 7, 8, 9] and c) accommodates multiple
learning styles and skills [10, 11, 12, 13, 14, 15, 16].
Implementation of computer game in classroom
highlighted the importance of design and integration of
pedagogical elements in order to develop effective
educational computer games and brought along the
importance of evaluation of the developed educational games
[3, 4, 7, 8, 9, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,
29, 30]. This is important to make sure that fun and serious
elements can be integrates very well to achieve predefined
goal.
Effective usability evaluation for computer games that
being used in teaching and learning process or sometime
known as Educational Computer Games (ECG) is necessary
in order to ensure that games adequately meet the
requirements of fun and educational elements. A range of
approaches have been used in the evaluation of computer
games that relate to educational purposes. Most of the
evaluation focuses on summative evaluation process with the
objective of accessing how well completed games meet the
Azizah Jaafar
Faculty of Information Science and Technology
Universiti Kebangsaan Malaysia
43600 Bangi, Selangor, Malaysia
aj@ftsm.ukm.my
pre-defined goals regarding the implementation of the
developed games in teaching and learning purposes [6, 9, 29,
30, 31, 32, 33, 34, 35, 36].
In spite of the variety evaluation of ECG, most of the
evaluation focus on effectiveness of the ECG and self
generated questionnaire were used in their evaluation
process. There were very little study has being reported
regarding the usability evaluation of ECG especially that
focused on interface of ECG [37]. Failure of focus to design
usable game interfaces can interfere the goal of creating a
convincing experience for users and may have a negative
effects on the overall quality and success of developed
games [37]. Specific methods are needed to identify
usability problems in both early design and in more complete
prototype [38, 39, 40, 41].
In evaluating ECG, there are few aspects need to be
focused in order to achieve a good ECG that combine
elements of fun and serious. This is important to attract
users in using it and able to enhance their knowledge in
specific subject. Details evaluation of ECG that integrates
all of the mentioned elements needs to be done at the early
stage of development and should involve various users either
real or surrogate. Combination of real and surrogate users
able to detect as many as possible usability problem of the
evaluate computer system [40, 42, 43].
This paper is organized as followed: Section 1 provides
the background of this study; Section 2 illustrates some
frameworks and methods being used to evaluate educational
computer game; Section 3 presents some ideas and elements
of interface evaluation that an educational game should
contain; Last section of this paper concludes and provides
future directions of this project.
II. BACKGROUND AND RELATED WORK
A. Educational Computer Game (ECG)
People play computer games for many reasons, because
of the enjoyment and entertaining attributes [44], for leisure
and educational purposes [45]. Adolescence prefer playing
computer games as a leisure time activity, to avoid stress, to
challenge their skills and to enjoy the attractive
characteristics of the games [46]. Computer games can be
effective educational tool [47, 48] and they can provide
enjoyment during learning process to learners [46], allow
users to engage in education while they enjoying themselves
[6, 23, 49].
The aim of educational games is to facilitate the player’s
experience [50, 51], desired objective [6], allow uses to
engage in education while they are enjoying themselves [52,
53]. Educational games seem to put learners in the role of
decision maker [2, 44], received immediate feedback of their
action and decisions, inviting exploration and
experimentation [32].
B. Evaluation of Educational Computer Game
Educational computer games has been used in many
areas such as in engineering [54, 55], medical [56, 57],
literacy [58] and mathematics [59]. Once the ECG being
developed, the evaluation process normally take part for a
group of students who use the game in their learning process
and the effectiveness of the game being evaluated [9, 10, 55,
58, 59, 60, 61, 62, 63, 64, 65].
Most of the past evaluation of educational computer
games use questionnaire [9, 26, 54, 66, 67], interview, log-
files and observation [50]. Other than questionnaire,
interview, log-files and observation, there were few attempt
in developing framework to evaluate educational games [68].
Figure 1 shows the developed framework by [68].
Figure 1. A Framework for Evaluating games-and simulation-based
education (Source:[68])
Heuristic Evaluation (HE) also being used in evaluating
educational computer game. HE shows a huge potentail to
be used in evaluating educational computer game [69]. HE
has being used in evaluating educational game since 1982 by
[70], in 1998 [71] developed a set of game design principles.
In 2002, [21] studied on three areas of computer games;
interface, mechanics and playability. Later, in 2004 [38]
developed Heuristic for Evaluating Playability and in 2006,
[41] developed playability heuristic for mobile games.
Song and Lee in 2007 [72], compiled key factors of heuristic
evaluation for game design and categorized it into four areas:
interface, play, narrative and mechanic. In 2008, [73]
developed heuristic evaluation for video game design that
adapt usability inspection for games
III. THE CHALLENGE
The challenge of evaluating educational games is divided
into three categories, the evaluation criteria, evaluators and
the suitable evaluation process. Evaluation criteria discuss
about the necessary elements that need to be evaluated and
why it is important. Second category discuss about the
suitable evaluators and why they are needed as an
evaluators, how they can contribute towards the
enhancement of develop ECG. The last category is about
the suitable evaluation process, either summative or
formative and why it is important. These three challenges
are important to be highlighted because proper selection of
evaluation criteria, evalutors and evaluation process able to
help developers to produce ECG that practical to be used,
consists of fun elements (game) and at the same time able to
help users in their learning process.
A. Evaluation Criteria
There are many issues involves in evaluating ECG. In
any system, interface plays an importance role in order to
provide appealing and attractive interface for the users. It
implies the same concept for ECG, interface is the main
issue that needs to be taken into consideration in evaluating
ECG. Issues of consistency, interactivity, navigation,
screen design and pleasant to use are the important elements
that need to be evaluated. Since ECG integrates fun and
education, pedagogical issue is another important element
that needs to be evaluated. Elements of pedagogical such as
clear goal and objective, motivation, learners control,
learning values, immediate feedback and challenge need to
be taken seriously in evaluating ECG.
The third issue is multimedia such as the usage of all
multimedia elements should be suitable according to it
functions, combination of multimedia elements in each
screen and suitability of multimedia elements are among the
focus of evaluation. Other then interface, pedagogical and
multimedia, content is another issue that need to be
highlighted in evaluating ECG. The reliability of the
content, clear and understandable structure and interesting
and engaging materials are among the importance elements
of content issue.
The fifth element is playability. Since it is a game, the
playability plays important elements in order to provide
game that is fun and enjoyable and at the same time able to
attract users to play and enhance their knowledge. Elements
in playability that need to be evaluated are challenge,
strategy, pace in balance, players able to control and able to
know the progress of the game. Figure 2 shows the five
evaluation issues and criteria that need to be evaluated in
order to develop a comprehensive and practical ECG.
Framework in Figure 1 shows the evaluation of ECG that
already being used by the users as compared to Figure 2,
these are the evaluation issues that need to be addressed in
formative evaluation of ECG.

Figure 2. Educational Computer Games Evaluation Issue
B. Evaluators
Evaluation of educational computer games is not limited
to evaluation criteria only. Issues of who are the suitable
evaluators and why they are importance need to be taken
into consideration. In usability evaluations, there are two
types of users normally involve in evaluation process which
are real users and surrogate users [74]. The importance of
evaluators for ECG need to be chosen based on their
expertise. After thorough review and discussion, the
involvement of the two types of evaluators is importance in
the study because of their knowledge. Normally, surrogate
users can be either experts in HCI or other related domain
based on definition by the researchers. In ECG, after
consideration of the highlighted issues that involve in
evaluating ECG, the need to addresss various evaluators
(surrogate and real users) is necessary.
Surrogate users that are involved in the evaluation
divided into four categories – HCI, educational, content and
game developers’ experts. They are needed because it is
difficult to find an expert evaluator that has knowledge in all
the five issues highlighted previously. HCI experts need to
evaluate interface and multimedia issues based on
developed heuristic as a guideline. The second surrogate
evaluators are educational expert that need to evaluate
issues of pedagogical and multimedia. The third surrogate
evaluators are content expert need to evaluate content and
multimedia issues. The last surrogate users are games
developers that need to evaluate issues of playability and
multimedia. Figure 3 shows the surrogate and real
evaluators.
In this study, multimedia issues are considered as
universal issues and they can be evaluated by all of the
surrogate users. It is believed that with the knowledge of
computer technology and familiarization in such field and
it’s applications, the surrogate users able to accomplish the
evaluation on the multimedia issues as well as detecting
usability problems found in the educational computer
games.
Surrogate
Evaluators –
HCI expert
Surrogate
Real Evaluators Evaluators –
– Potentail Educational
users expert
ECG
Evaluators
Surrogate
Surrogate
Evaluators –
Evaluators –
Game
Content expert
Developers
Figure 3. Evaluators
Real evaluators are the real users and they are the target
group of the developed ECG. Their involvement in
evaluation is important in order to detect usability problem
from users’ perspective. In order to guide real users in
evaluation process, the heuristic that being developed for
experts evaluators will be designed in form of questionnaire
so that it is much easier for real users during the evaluation
process.
C. Evaluation Process
Once the evaluation criteria are highlighted and the
evaluators have been identified, the next step is to plan for
the evaluation process. In term of evaluation process, the
debate either to use formative or summative evaluation is
depend on the technique of evaluation. Since the focus in on
the development of comprehensive and practical ECG,
formative evaluation process is suitable to be used in this
project. In order to cater for the formative evaluation and
after comprehensive review of issues in ECG, Playability
Heuristic Evaluation for Educational Games (PHEG)
technique has been developed derived from Heuristic
Evaluation [75, 76] and Heuristic Evaluation Playability [38,
39]. In order to simplify the process, online evaluation
system will be developed. Online evaluation process may
help interm of data collection and able to be used by various
numbers of users [77].
PHEG will be used by surrogate users as a guideline
during evaluation process. PHEG has been divided into five
sections; Interface, Pedagogical, Content, Multimedia and
Playability. These sections are used to accommodate
different surrogate users based on their expertise. Table II
shows the section for Interface of ECG that will be used by
HCI expert. Table III shows the pedagogical section that
will be used by education experts and they are required as
not all of the HCI or content experts have the knowledge in
educational theories.
Table IV shows content section. This section will be
used by the content experts The experts may be lecturers or
teachers who have great experience in teaching particular
subjects and the comprehensiveness of the developed ECG
in accommodating necessary syllabus will be identified and
commented by them.
TABLE I. PHEG - INTERFACE
 Usability Educational Game Usability Heuristic – Interface Usability Educational Game Usability Heuristic - Multimedia
UI1 Uses aesthetic and minimalist design MM1 Usage of multimedia elements are acceptable
UI2 Maximizes consistency and matches standards MM2 Combination of multimedia elements are adequate
UI3 The uses of space, color and text are according to the MM3 The presentation of multimedia elements are well manage
principles of screen design MM4 Suitability of multimedia elements for specific use
UI4 The uses of text, color and font follow the principles of MM5 Not too many multimedia element in one screen
readability MM6 The use of multimedia elements support meaningfully the
UI5 Ensure appropriate use of standard and propriety control text provided.
UI6 The interactivity of the game is suitable to learners level MM7 The quality of multimedia elements (text, image,
UI7 The integration of presentation means is well-coordinated animation, video and sound) used is acceptable.
UI8 Quality of user interface is acceptable MM8 The uses of multimedia elements enhance the
UI9 Provide specific and self- identified key for specific task presentation of information.
(exit, glossary, main, objective)
UI10 Overall interface of the game is appealing
As for the real users, they will be using questionnaire that
TABLE II. PHEG – PEDAGOGICAL / EDUCATIONAL has been modified based on PHEG and known as PAEG. It
has additional criteria of usability that are effectiveness,
Usability Educational Game Usability Heuristic – efficiency and subjective satisfaction. Details of PAEG will
Educational/Pedagogical
ED1 Clear goal and learning objectives
be discuses in another paper. Real and surrogate evaluators
ED2 The activities are interesting and engaging will be using same ECG (prototype) for the evaluation
ED3 The design and the contents are reliable and proven. process and the combination of both evaluators are expected
ED4 Can be used as self- directed learning tools. to uncover as many usability problems as possible.
ED5 Support for self- learning skills.
ED6 Medium for learning by doing. IV. RESULT AND DISUCSSION
ED7 Considers the individual differences.
ED8 Performance should be an outcome-based. Evaluation process is very important in developing ECG.
ED9 Offers the ability to select the level of difficulty in games The comprehensiveness in following syllabus and suitability
ED10 Ability to work in their own pace of ECG contents to be used in teaching and learning process
are identified. This paper highlighted the challenges in
TABLE III. PHEG – CONTENT evaluating ECG that comprise of three which are evaluation
Usability Educational Game Usability Heuristic – Content
criteria, the evaluators and evaluation process. These three
CO1 Reliable content with correct flow. challenges have been highlighted and potential solutions
CO2 Clear and understandable structure of contents. were discussed. Evaluation criteria can be resolved using
CO3 Navigation of content is easy and accurate. five evaluation issues that are Interface, Educational,
CO4 Supporting materials are sufficient and relevant (quiz). Content, Multimedia and Playability.
CO5 Materials are interesting and engaging.
CO6 Players able to understand the learning goal.
Evaluators’ issues can be established by combining real
CO7 The content is chunk based on topic and subtopic and surrogate evaluators that able to accommodate all of the
CO8 Major and minor topic is differentiate clearly highlighted evaluation criteria. The third issue, evaluation
process will be using formative evaluation with the help of
PHEG technique for surrogate users and list of questionnaire
Playability is another section in PHEG that will be used (PAEG) for real users that will be discussed details in
for game developers as a guideline in evaluation process. another paper.
Table V shows the section of playability.
V. CONCLUSION
TABLE IV. PHEG – PLAYABILITY Three challenges in evaluating ECG have been
Usability Educational Game Usability Heuristic - Playability highlighted. Evaluation criteria, the evaluators and
PL1 Challenge provided are up to the users standard/level evaluation process are the three challenges that need to be
PL2 Users able to strategies overcome in the evaluation process of ECG. It is believed
PL3 The pace of the game are in balance that combination of the three highlighted challenges able to
PL4 Players able to control the game detect usability problem of ECG at early stage of
PL5 Progress of the game can be seen at anytime development process. Improvement of ECG based on the
PL6 Players able to perform to their best ability usability problems highlighted able to produce
PL7 Challenge is adequate – not too easy and not too difficult comprehensive and practical ECG that can be used to help
learners in their learning process.
The last section in PHEG is multimedia and it will be
used by all of the surrogate evaluators in evaluation process. ACKNOWLEDGMENT
Table VI shows the section of multimedia. This reseacrh is conducted using e-Science Funds Grant
(01-01-02-SF0089), Universiti Kebangsaan Malaysia.

REFERENCES

TABLE V. PHEG - MULTIMEDIA

[1] T. W. Malone, What makes things fun to learn? A study of
intrinsically motivating computer games. Palo Alto, CA: Xerox Palo
Alto Research Center, 1980.
[2] J. P. Gee, What video games have to teach us about learning and
literacy. New York: Palgrave Macmillan, 2003.
[3] M. Prensky, Digital game-based learning. New York: McGraw-Hill.,
2001.
[4] D. Gibson, C. Aldrich, and M. Prensky, Games and simulations in
online learning: research and development frameworks: Information
Science Publishing, 2007.
[5] F. Ke, "A case study of computer gaming for math: Engaged learning
from gameplay?," Computers & Education, vol. 51, pp. 1609-1620,
2008.
[6] P. Moreno-Ger, D. Burgos, I. Martínez-Ortiz, J. L. Sierra, and B.
Fernández-Manjón, "Educational game design for online education,"
Computers in Human Behavior, vol. 24, pp. 2530-2540, 2008.
[7] Y.-T. Sung, K.-E. Chang, and M.-D. Lee, "Designing multimedia
games for young children's taxonomic concept development,"
Computers & Education, vol. 50, pp. 1037-1051, 2008.
[8] J. Wall and V. Ahmed, "Use of a simulation game in delivering
blended lifelong learning in the construction industry - Opportunities
and Challenges," Computers & Education, vol. 50, pp. 1383-1393,
2008.
[9] M. Papastergiou, "Digital Game-Based Learning in high school
Computer Science education: Impact on educational effectiveness and
student motivation," Computers & Education, vol. 52, pp. 1-12, 2009.
[10] D. Burgos, C. Tattersall, and R. Koper, "Re-purposing existing
generic games and simulations for e-learning," Computers in Human
Behavior, vol. 23, pp. 2656-2667, 2007.
[11] C.-L. Hsu and H.-P. Lu, "Consumer behavior in online game
communities: A motivational factor perspective," Computers in
Human Behavior, vol. 23, pp. 1642-1659, 2007.
[12] L. M. Sherlock, "When social networking meets online games: the
activity system of grouping in world of warcraft," in ACM 25th
International Conference on Design of Communication, 2007, pp. 14-
20.
[13] J. Bardzell, S. Bardzell, T. Pace, and J. Karnell, "Making user
engagement visible: a multimodal strategy for interactive media
experience research," in Proceedings of ACM CHI 2008 Conference
on Human Factors in Computing Systems, 2008, pp. 3663-3668.
[14] H. Desurvire and C. Wiberg, "Master of the game: assessing
approachability in future game design," in Proceedings of ACM CHI
2008 Conference on Human Factors in Computing Systems, 2008, pp.
3177-3182.
[15] J. B. Ellis, K. Luther, K. Bessiere, and W. A. Kellogg, "Games for
virtual team building," in Proceedings of DIS'08: Designing
Interactive Systems, 2008, pp. 295-304.
[16] D. Grammenos, "Game over: learning by dying," in Proceedings of
ACM CHI 2008 Conference on Human Factors in Computing
Systems, 2008, pp. 1443-1452.
[17] H. Ellington, E. Addinall, and F. Percival, A handbook of game
design: Kogan Page London, 1982.
[18] C. Crawford, The art of computer game design: Osborne/McGraw-
Hill, 1984.
[19] C. S. Greenblat, Designing games and simulations: An illustrated
handbook: Sage Publications, Inc, 1988.
[20] J. M. Randel, Morris, B. A., Wetzel, C. D., & Whitehill, B. V., "The
effectiveness of games for educational purposes: A review of recent
research," Simulation & Gaming, vol. 23, pp. 261-276, 1992.
[21] M. A. Federoff, "Heuristic And Usability Guidelines For The
Creation And Evaluation Of Fun In Video Games," Indiana
University, 2002.
[22] R. J. Pagulayan, Keeker, K., Wixon, D., Romero, R. L., & Fuller, T.,
User-centered design in games. NJ: Lawrence Erlbaum Associates,
2003.
[23] M. Sharon O’Leary, Lisa Diepenhorst, MD, Ruth Churley-Strom,
MS, RN, Diane Magrane, MD, "Educational games in an obstetrics
and gynecology core curriculum," American Journal of Obstetrics
and Gynecology, vol. 193, p. 51, 2005.
[24] M. S. El-Nasr and J. B. Gross, "Fun & games: on the process of game
design," in Proceedings of DIS'06: Designing Interactive Systems:
Processes, Practices, Methods, & Techniques, 2006, pp. 369-370.
[25] G. K. Akilli, "Games and Simulations: A new approach in
Education," Games and simulations in online learning: Research and
Development Frameworks, pp. 1-20, 2007.
[26] P. Barr, J. Noble, and R. Biddle, "Video game values: Human-
computer interaction and games," Interacting with Computers, vol.
19, pp. 180-195, 2007.
[27] P. O. Kristensson and S. Zhai, "Learning shape writing by game
playing," in Proceedings of ACM CHI 2007 Conference on Human
Factors in Computing Systems, 2007, pp. 1971-1976.
[28] E. Mazzone, D. Xu, and J. Read, "Design in Evaluation: Reflections
on Designing for Children's Technology," in Proceedings of the
HCI'07 Conference on People and Computers XXI, 2007, p. 38.
[29] R. Raisamo, Patom, auml, S. ki, M. Hasu, and V. Pasto, "Design and
evaluation of a tactile memory game for visually impaired children,"
Interacting with Computers, vol. 19, pp. 196-205, 2007.
[30] J. W. Stefan Marks, Burkhard W¨unsche, "Evaluation of Game
Engines for Simulated Surgical Training," in GRAPHITE 2007, Perth,
Western Australia, 2007.
[31] F. F. Mueller and M. R. Gibbs, "Evaluating a distributed physical
leisure game for three players," in Proceedings of OZCHI'07, the
CHISIG Annual Conference on Human-Computer Interaction, 2007,
pp. 143-150.
[32] J. N. Pippin Barr, Robert Biddle, "Video game values: Human–
computer interaction and games," Interacting with Computers, vol.
19, pp. 180-195, 2007.
[33] S. E. Walter, K. Forssell, B. Barron, and C. Martin, "Continuing
motivation for game design," in Proceedings of ACM CHI 2007
Conference on Human Factors in Computing Systems, 2007, pp.
2735-2740.
[34] R. Bernhaupt, W. Ijsselsteijn, F. F. Mueller, M. Tscheligi, and D.
Wixon, "Evaluating user experiences in games," in Proceedings of
ACM CHI 2008 Conference on Human Factors in Computing
Systems, 2008, pp. 3905-3908.
[35] M. J. Taylor, D. C. Pountney, and M. Baskett, "Using animation to
support the teaching of computer game development techniques,"
Computers & Education, vol. 50, pp. 1258-1268, 2008.
[36] T.-H. Wang, "Web-based quiz-game-like formative assessment:
Development and evaluation," Computers & Education, vol. 51, pp.
1247-1263, 2008.
[37] D. Pinelle, N. Wong, and T. Stach, "Using genres to customize
usability evaluations of video games," 2008, pp. 129-136.
[38] H. Desurvire, Caplan, M., Toth, J.A, "Using heuristic to evaluate the
playability of games," in In: Proceedings of the Conference on CHI
2004, 2004, pp. pp. 1509-1512.
[39] D. Heather, C. Martin, and A. T. Jozsef, "Using heuristic to evaluate
the playability of games," in CHI '04 extended abstracts on Human
factors in computing systems Vienna, Austria: ACM, 2004.
[40] H. M. Omar and A. Jaafar, "Playability Heuristic Evaluation (PHE)
approach for Malaysian educational games," in International
Symposium on Information Teachnology (ITSIM), Kuala Lumpur,
Malaysia, 2008.
[41] K. Hannu and M. I. K. Elina, "Playability heuristic for mobile
games," in Proceedings of the 8th conference on Human-computer
interaction with mobile devices and services, Helsinki, Finland, 2006.
[42] A. Jaafar, "Cognitive Walkthrough: Improving Software Usability,"
2001, pp. 2002-11.
[43] A. Jaafar and N. Cassaigne, "Cognitive walkthrough: Improving
educational software usability," 2001, pp. 472-480.
[44] G. James Paul, What Video Games Have to Teach Us About Learning
and Literacy: Palgrave Macmillan, 2004.
[45] C. Chou and M.-J. Tsai, "Gender differences in Taiwan high school
students' computer game playing," Computers in Human Behavior,
vol. 23, pp. 812-824, 2007.
[46] T. Karakus, Y. Inal, and K. Cagiltay, "A descriptive study of Turkish
high school students' game-playing characteristics and their
considerations concerning the effects of games," Computers in
Human Behavior, vol. 24, pp. 2520-2529, 2008.
[47] D. J. V and R. K. L. B, "Instructional gaming: implications for
instructional technology," in Annual Meeting of the Association for
Educational Communications and Technology, Nashville, TN, 1994.
[48] J. V. Dempsey, B. A. Lucassen, L. L. Haynes, and M. S. Casey, "An
exploratory study of forty computer games (COE Technical Report
No 97-2)," Mobile, Al. University of South Alabama, 1997.
[49] K. Sedig, "Toward operationalization of `flow' in mathematics
learnware," Computers in Human Behavior, vol. 23, pp. 2064-2092,
2007.
[50] K. Kiili, "Content creation challenges and flow experience in
educational games: The IT-Emperor case," The Internet and Higher
Education, vol. 8, pp. 183-198, 2005.
[51] K. Kiili, "Digital game-based learning: Towards an experiential
gaming model," The Internet and Higher Education, vol. 8, pp. 13-
24, 2005.
[52] R. Hamalainen, "Designing and evaluating collaboration in a virtual
game environment for vocational learning," Computers & Education,
vol. 50, pp. 98-109, 2008.
[53] G. J. Leach and T. S. Sugarman, "Play to win! Using games in library
instruction to enhance student learning," Research Strategies, vol. 20,
pp. 191-203, 2005.
[54] B. D. Coller and M. J. Scott, "Effectiveness of using a video game to
teach a course in mechanical engineering," Computers & Education,
vol. 53, pp. 900-912, 2009.
[55] M. Ebner and A. Holzinger, "Successful implementation of user-
centered game based learning in higher education: An example from
civil engineering," Computers & Education, vol. 49, pp. 873-890,
2007.
[56] B. D. Mann, B. M. Eidelson, S. G. Fukuchi, S. A. Nissman, S.
Robertson, and L. Jardines, "The development of an interactive game-
based tool for learning surgical management algorithms via
computer," The American Journal of Surgery, vol. 183, pp. 305-308,
2002.
[57] S. O'Leary, L. Diepenhorst, R. Churley-Strom, and D. Magrane,
"Educational games in an obstetrics and gynecology core
curriculum," American Journal of Obstetrics and Gynecology, vol.
193, pp. 1848-1851, 2005.
[58] R. Owston, H. Wideman, N. S. Ronda, and C. Brown, "Computer
game development as a literacy activity," Computers & Education,
vol. 53, pp. 977-989, 2009.
[59] S. Costu, S. AydIn, and M. Filiz, "Students' conceptions about
browser-game-based learning in mathematics education:
TTNetvitamin case," Procedia - Social and Behavioral Sciences, vol.
1, pp. 1848-1852, 2009.
[60] S. Çankaya and A. Karamete, "The effects of educational computer
games on students' attitudes towards mathematics course and
educational computer games," Procedia - Social and Behavioral
Sciences, vol. 1, pp. 145-149, 2009.
[61] J. Robertson and C. Howells, "Computer game design: Opportunities
for successful learning," Computers & Education, vol. 50, pp. 559-
578, 2008.
[62] D. Robertson and D. Miller, "Learning gains from using games
consoles in primary classrooms: a randomized controlled study,"
Procedia - Social and Behavioral Sciences, vol. 1, pp. 1641-1644,
2009.
[63] R. Shultz Colby and R. Colby, "A Pedagogy of Play: Integrating
Computer Games into the Writing Classroom," Computers and
Composition, vol. 25, pp. 300-312, 2008.
[64] Y. Turgut and P. Irgin, "Young learners' language learning via
computer games," Procedia - Social and Behavioral Sciences, vol. 1,
pp. 760-764, 2009.
[65] H. Tüzün, M. Yilmaz-Soylu, T. Karakus, Y. Inal, and G. KizIlkaya,
"The effects of computer games on primary school students'
achievement and motivation in geography learning," Computers &
Education, vol. 52, pp. 68-77, 2009.
[66] L. A. Annetta, J. Minogue, S. Y. Holmes, and M.-T. Cheng,
"Investigating the impact of video games on high school students'
engagement and learning about genetics," Computers & Education,
vol. 53, pp. 74-85, 2009.
[67] P. Lacasa, L. Méndez, and R. Martínez, "Bringing Commercial
Games into the Classroom," Computers and Composition, vol. 25, pp.
341-358, 2008.
[68] S. de Freitas and M. Oliver, "How can exploratory learning with
games and simulations within the curriculum be most effectively
evaluated?," Computers & Education, vol. 46, pp. 249-264, 2006.
[69] H. M. Omar and A. Jaafar, "Heuristic Evaluation in Computer
Games," in International Conference on Information Retrieval and
Knowledge Management (CAMP10), Shah Alam, Malaysia, 2010.
[70] T. W. Malone, Heuristic for designing enjoyable user interfaces:
Lessons from computer games. Norwood, NJ: Ablex Publishing
Corporation, . 1982.
[71] C. Clanton, "An interpreted demonstration of computer game design,"
in CHI 98 summary: Human Factors in Computing Systems. Chi 98,
1998, pp. 1-2.
[72] S. Song and J. Lee, "RETRACTED: Key factors of heuristic
evaluation for game design: Towards massively multi-player online
role-playing game," International Journal of Human-Computer
Studies, vol. 65, pp. 709-723, 2007.
[73] D. Pinelle and N. Wong, "Heuristic evaluation for games: usability
principles for video game design," in Proceedings of ACM CHI 2008
Conference on Human Factors in Computing Systems, 2008, pp.
1453-1462.
[74] H. M. Omar and A. Jaafar., "Conceptual Framework for a Heuristic
Based Methodology for Interface Evaluation of Educational Games.,"
in The 2009 International Conference on Computer Technology and
Development (ICCTD2009). Kota Kinabalu, Malaysia, 2009.
[75] J. Nielsen, Heuristic evaluation. New York: John Wiley & Sons,
1994.
[76] J. Nielsen and R. Molich, "Heuristic Evaluation of User Interfaces,"
in In Proceedings of Human Computer Interaction, 1994, pp. 249-
256.
[77] R. Ramli, A. Jaafar, and H. M. Omar, "Remote Usability Evaluation
System (e-RUE). ," in The 2nd International Conference On
Computer and Electrical Engineering (ICCEE2009), Dubai, UAE.,
2009.