Sei sulla pagina 1di 49

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/321063416

The Effect of Gamification on Motivation and Engagement

Article · November 2017


DOI: 10.1108/IJILT-02-2017-0009

CITATIONS READS
21 10,131

1 author:

Raed Alsawaier
Washington State University
4 PUBLICATIONS   21 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The Effect of Gamification on Motivation and Engagement View project

Research Design Bias in the Study of Gamification View project

All content following this page was uploaded by Raed Alsawaier on 03 April 2018.

The user has requested enhancement of the downloaded file.


International Journal of Information and Learning Technology
ti o
na
lJ
ou
rn
The Effect of Gamification on Motivation and Engagment
al
Journal: International Journal of Information and Learning Technology
of
Manuscript ID IJILT-02-2017-0009

Manuscript Type: Research Paper


Inf
Keywords: Gamification, Game-based learning
or
m
at
ion
an
dL
ea
rn
ing
Te
ch

http://mc.manuscriptcentral.com/cwis
no
Page 1 of 47 International Journal of Information and Learning Technology
ti o
Running head: GAMIFICATION: MOTIVATION AND ENGAGEMENT 1
1
2
na
3
4
5
6
lJ
7
8
ou
9
10 Abstract
11
12
rn
Gamification is the application of game features, mainly video game elements, into non-game
13
14
context for the purpose of promoting motivation and engagement in learning. The application of
al
15
16
17 gamification in a pedagogical context provides some remedy for many students who find
of
18
19 themselves alienated by traditional methods of instruction. There is a direct link between
20
21
Inf

22 increased motivation and higher levels of engagement when the gamification intervention is
23
24 introduced. This is a literature review on gamification, motivation, and engagement. In the
or

25
26 literature regarding how gamification promotes students’ motivation and task engagement in a
27
m

28
college environment, some important topics are: conceptualizing gamification, advantages of
29
at

30
31 gamification over game-based learning, theoretical connections to gamification, motivation and
32
ion

33 engagement, connecting gamification to motivation and engagement, emotions and fun in


34
35 gamification, player types and gamification features, gamification in action, and implementation
36
37
an

38 guidelines.
39
40
dL

41 Keywords: gamification, game based learning (GBL).


42
43
44
ea

45
46
47
rn

48
49
50
ing

51
52
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 2 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 2
1
2
na
3
4
5
6
lJ
7
8
ou
9
10
11
12
rn
Gamification is a newly coined term that reflects a social phenomenon arising with a
13
14
generation of digitally literate population. Gamification has been defined as the use of “game-
al
15
16
17 based mechanics, aesthetics, and game thinking to engage people, motivate action, promote
of
18
19 learning, and solve problems” (Kapp, 2013, p. 125). This operational definition, which will be
20
21
Inf
used in this literature review, incorporates important pedagogical components. First, the
22
23
24 pedagogical application of gamification to promote learning is emphasized as contrasted to
or

25
26 business applications. Second, digital game mechanics, which includes, but is not limited to,
27
m

28 avatars, badges, points, levels, leaderboard, virtual rewards, and storyline or quests, are
29
at

30
31 highlighted. Third, there is reference to game dynamics, which is focused on game elements that
32
ion

33 allow for social interaction between players. In addition, motivation and engagement are
34
35 included in this definition as possible effects of gamification. The fourth pedagogical component
36
37
an

38 in Kapp’s definition is the emphasis on critical thinking skills, which are essential in learning and
39
40 could be partially promoted through gamification.
dL

41
42 Although non-digital game components have wide applications in educational contexts,
43
44
ea

the focus of this paper is on the digital video game elements that are used in pedagogical context
45
46
47 to promote task engagement, increase motivation, and enforce desirable learning behaviors. The
rn

48
49 rationale behind deploying video game elements in an educational context is that they have
50
ing

51 already captured the attention of millions of loyal players all over the world. In the United States,
52
53
54 about 58% of the population played video games (Folmer, 2015). Unlike the general assumption,
55
Te

56 the average age of video game players is thirty with almost a balanced percentage of males and
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 3 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 3
1
2
na
3 females: “There are more females over the age of thirty playing video games than boys under
4
5
6 eighteen, and one-third of parents play video games with their kids regularly” (p. 2).
lJ
7
8 A significant problem that many schools and educators are facing today, as Zickermann
ou
9
10 and Cunningham (2011) posited, is that many students are lacking the motivation and interest to
11
12
rn
13
learn. If given a choice, many of them would prefer to play video games rather than reading a
14
al
15 book or completing a homework assignment. The solution is not, as many educators, policy
16
17 makers, and politicians suggest, resolved by creating additional educational standards or adding
of
18
19
more standardized tests in an endless cycle of trial and error. Prensky (2001) enthusiastically
20
21
Inf

22 presented the solution to learners’ disengagement through the marriage of education and
23
24 entertainment; thus, the term “edutainment:” “Great education is edutainment that has gotten the
or

25
26 mix right. . . . No rational person condemns the genre; we speak of good books, good plays, and
27
m

28
29 good movies as those that stand the test of time” (p. 380). Some proponents of gamification, such
at

30
31 as Zickermann (2010), are for more –tainment than –edu. Prensky (2001) confirmed that there is
32
ion

33 no magical recipe, but that depends on the context and is left to teachers to design the perfect
34
35
36
mix of education and entertainment depending on their students’ needs.
37
an

38 According to Prensky (2011), game features can provide the –tainment part of the
39
40 educational design needed to engage learners. Borrowing game elements, he argued, and
dL

41
42
incorporating them into the classroom environment can facilitate engagement. Little research has
43
44
ea

45 been done on the effect of gamification on motivation and engagement of the learners. The
46
47 research literature connected to gamification is limited on multiple levels and there is a need to
rn

48
49 explore the long-term effect of gamification in promoting and sustaining learners’ motivation
50
ing

51
52 and engagement. This literature review will include studies done on the effect of gamification on
53
54 learners’ engagement and motivation.
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 4 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 4
1
2
na
3 In this literature review, research connecting gamification to motivation and engagement
4
5
6 will be explored under these themes: conceptualizing gamification, advantages of gamification
lJ
7
8 over game-based learning, theoretical connections to gamification, motivation, and engagement,
ou
9
10 connecting gamification to motivation and engagement, emotions and fun in gamification, player
11
12
rn
13
types and gamification features, gamification in action, and implementation guidelines.
14
al
15 Conceptualizing Gamification
16
17 There is no consensus on the definition of gamification among researchers, nor is there an
of
18
19
agreement on the difference between game-based learning (GBL) and gamification. Kapp (2013)
20
21
Inf

22 discussed the definition of gamification in a pedagogical context contrasting it to game-based


23
24 learning (GBL). Based on Kapp’s view, the instructional strategy is changed to accommodate
or

25
26 game elements where, instead of the learning objectives, the teacher in a gamified classroom will
27
m

28
29 present a challenge or quest that the players will need to undertake leading them to the learning
at

30
31 experience.
32
ion

33 Some researchers define gamification as the use of game elements, mechanics, features,
34
35
36
design, and structure in a non-game environment or context (Attali & Arieli, 2015; Bruder, 2015;
37
an

38 Dale, 2014; Davis, 2014; Deterring, 2012; Gonzales et al., 2016; Hanus & Fox, 2015; Issacs,
39
40 2015; Kapp, 2013; Powers et al., 2013; Keeler, 2015; Koivisto & Hamari, 2014; Seaborn & Fels,
dL

41
42
2015; Sheldon, 2011; Whitton, 2012; Zickermann, 2011). This “non-game environment” is broad
43
44
ea

45 enough to cover the wide application of gamification in business, thus Zickermann and
46
47 Cunningham’s (2011) definition: “The process of game-thinking and game mechanics to engage
rn

48
49 users and solve problems” (p. xiv). Gamification, according to Simões, Redondo, and Vilas
50
ing

51
52 (2013), is the utilization of game mechanics and game dynamics in “non-game applications” (p.
53
54 348). The definition of Simões et al. (2013) is focused on the social aspect of gamification, such
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 5 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 5
1
2
na
3 as collaboration. Leaning’s (2015) definition of gamification as an experience outside of gaming
4
5
6 context is as broad as the previous definitons.
lJ
7
8 Other definitions are more focused on critical thinking skills that are usually used in
ou
9
10 games and can be activated in non-game context (Farber, 2013). There are some defintions
11
12
rn
13
where the pedagogical applications of gamification are emphasized. Kingsley and Grabners
14
al
15 (2015) posited that gamification should be understood as a combination of “ content area
16
17 instruction, literacy, and 21st century learning skills in a highly-engaging learning environment”
of
18
19
(p. 51). In the eyes of Hamari, Koivisto, and Sarsa (2014), gamification becomes more complex
20
21
Inf

22 with specific focus on “motivational affordances” and change in behavior as an outcome (p.
23
24 3026).
or

25
26 Folmar’s (2015) definition captured an important idea in the application of gamification
27
m

28
29 in learning and other fields: game thinking. Folmer defined gamification as “the use of game
at

30
31 thinking and game mechanics to meet non-game ends” (p. 2). The lack of game thinking when
32
ion

33 using gamification in an educational context is what many gamification proponets, such as


34
35
36
Zickermann (2010), considered the chief reason for its occasional failure in different contexts.
37
an

38 Game thinking mandates rethinking teaching practices, not just adding game elements without
39
40 considering how gamification works: “Gamification is not just making a game, which imparts a
dL

41
42
lesson; it is applying game thinking to how we impart that lesson and continuing to develop it
43
44
ea

45 based on the feedback from the players” (Folmar, 2015, p. 5).


46
47 In traditional instructional methods, the students earn their grades based on a
rn

48
49 performance of a task as they demonstarte achievement, whereas in gamification “the effort is
50
ing

51
52 rewarded, with badges or points even when the objective is not completed: “that [is] what
53
54 gamification does, it rewards the effort [emphasis added], not the winning” (p.7). In other words,
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 6 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 6
1
2
na
3 in a gamified environment the students are encouraged to engage in the process and reasoning
4
5
6 and are evaluated accordingly regardless of whether they succeeded in their endeavor or not. Of
lJ
7
8 course, this is not to undermine the importance of task but rather to motivate the learners to exert
ou
9
10 effort in tackling different learning challenges. Brewer et al.’s (2013) study on children (five to
11
12
rn
13
seven) has shown that the use of gamification increased the percentage of task completion from
14
al
15 73% to 97%.
16
17 To illuminate the different elements that comprise the meaning of gamification, it is
of
18
19
important to discuss the difference between gamification and game-based learning (GBL).
20
21
Inf

22 Advantages of Gamification over Game-Based Learning


23
24 Gamification has a lot of advantages over educational games. Before highlighting the
or

25
26 characteristics that make gamification superior to other digital games used in education and
27
m

28
29 otherwise, it is important to understand some game terminolgy. Game-based learning is intended
at

30
31 solely for education and relies on a learning game that has a beginning and an end. Serious
32
ion

33 gaming is a broader term used to describe games intended for education, industry, training and
34
35
36
stimulation (Connolly, Boyle, MacArthur, Hainey & Boyle, 2012). Table 1 shows the low
37
an

38 frequency in the search results of the term gamification as compared to game-based learning,
39
40 serious games, and educational games. The search in the databases, Eric, Sage, ScienceDirect,
dL

41
42
and GoogleScholar, was conducted using keywords: gamification, game-based learning,
43
44
ea

45 educational games, and serious games. Other than gamification, all the search included the word
46
47 “game” to narrow down the results. GoogleScholar had the largest number of results on
rn

48
49 gamification because it included conference proceedings, websites, and online documents. All in
50
ing

51
52 all, the results reflected the recency of the topic of gamification as compared to other game-based
53
54 topics.
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 7 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 7
1
2
na
3 Gamification and game-based learning are an area of confusion and misunderstanding for
4
5
6 many educators and game designers. The two concepts: gamification, and game-based learning,
lJ
7
8 are distinctively blurred for Nah, Telaprolu, Rallapalli, and Venkata (2013). They stated: “By
ou
9
10 turning an activity or a process into a computer game, i.e. through various game design elements
11
12
rn
13
such as rewards for achievement, desirable behavioral change can be induced” (p. 99).
14
al
15 Gamification is not when learning is changed into a computer game but rather when adding a
16
17 design layer of game elements to enhance learning, increase engagement, and encourage positive
of
18
19
behavior.
20
21
Inf

22 Table 1
23
24 Database Search Results
or

25
26 Database Gamification Game-based Educational Serious
27
m

28 Learning Games Games


29
at

30 Eric 60 16,266 11,149 6,507


31
32
ion

33
Sage 276 92,557 41,472 37,834
34
35 ScienceDirect 648 213,648 42,072 55,270
36
37
an

GoogleScholar 29,000 3,340,000 2,260,000 2,030,000


38
39
40
dL

41 Keeler (2014) posited major differences between gamification and game-based learning.
42
43
44 Game-based learning is “when students play games to learn content,” whereas gamification is the
ea

45
46 “application of game-based elements to non-game situations” (p. 1). Issacs (2015) also made a
47
rn

48 clear distintion between the two approaches: “[G]amification has been integrated in a more
49
50
ing

51
authentic manner as some classrooms have become a living breathing game.. . . . Unlike
52
53 gamification, game-based learning relates to the use of games to enhance the learning
54
55
Te

experience” (pp. 1-2).


56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 8 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 8
1
2
na
3 Simões et al. (2013) elaborated upon how gamification works differently from game-
4
5
6 based learning in the classroom: “The gamification of education approach has the advantage of
lJ
7
8 introducing what really matters from the world of video games – increasing the level of
ou
9
10 engagement of students – without using any specific game” (p. 347). Gamification in education
11
12
rn
13
is an ongoing process that harvests the most engaging game components and applies them to
14
al
15 increase motivation and engagement among the learners. The results are realized on the long
16
17 term as compared to game-based learning where engagement is short-lived, usually during the
of
18
19
duration of the game. Once the game is completed, many learners or players do not have any
20
21
Inf

22 more interest in a game they have mastered and completed. Folmar (2015) described how serious
23
24 games work in contrast to gamification: “[Serious gaming] is a game with a completion stage as
or

25
26 compared to gamification which is meant to create ongoing and prolonged engagement” (p. 5).
27
m

28
29 On one hand, gamification works by adding elements inspired from games to the classroom
at

30
31 environment mainly to increase motivation, engagment, and promote desired learning behaviors.
32
ion

33 Whereas game-based learning, on the other hand, relies on using games to meet learning
34
35
36
outcomes. The learning is facilitated through playing games whether digital or non-digital.
37
an

38 Zickermann (2010), in his support of gamification over game-based learning, emphasized


39
40 the idea that the educational games industry had produced few successful games. The last one, he
dL

41
42
claimed to be so, was Where in the World is Carmen Sandiego which came out in 1985. Since
43
44
ea

45 then, very few games have achieved similar record success. McGonigal (2011) supported
46
47 Zickermann’s argument in advocating gamification and further elaborated on why educational
rn

48
49 games are short-lived and do not meet the needs of the schools and educational systems:
50
ing

51
52 “Educational games are at best a temporary solution. The engagement gap is getting too wide for
53
54 a handful of educational games to make a significant and lasting difference” (p.128). McGonigal
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 9 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 9
1
2
na
3 (2011) joined Prensky (2001) in recommending the gamified path in teaching and learning that
4
5
6 allows the students to engage in from the beginning to end.
lJ
7
8 It is important to explore the theoretical foundations to this research paper before
ou
9
10 discussing in detail the connection between gamification, motivation, and task engagement.
11
12
rn
13
Theoretical Connections to Gamification
14
al
15 The knowledge base connecting gamification to theoretical principles is thin and the
16
17 empirical research on gamification founded on theoretical principles is scarce. For example,
of
18
19
Seaborn and Fels (2015) did a review of peer-reviewed literature on gamification and
20
21
Inf

22 engagement of 32 studies. Only ten of them were founded on theories (five of which by the same
23
24 author) and the rest had no connection to theoretical foundations. Furthermore, there is “a gap
or

25
26 between theory and practice- where theory is empirically unexamined [in the context of
27
m

28
29 gamification] and applied work lacks reference to theory- which serves to limit the growth of the
at

30
31 field” (Seaborn & Fels, 2015, p. 27). This highlights the need of research on gamification with
32
ion

33 strong theoretical links that bridge the gap between theory and practice. In understanding the
34
35
36
connection between gamification, motivation, and engagement, the three theories of self-
37
an

38 determination, new literacies study (NLS), and behaviorism will be deployed.


39
40 Self Determination Theory
dL

41
42
Motivation and engagement, as a major focus of this literature review, are at the heart of
43
44
ea

45 self-determination theory of human motivation. Self-determination theory (SDT) rests on the


46
47 three principles of autonomy, competence, and relatedness (Deci & Ryan, 2008; Seaborn & Fels,
rn

48
49 2015). According to Baard, Deci, and Ryan (2004), competence is connected to the motivation to
50
ing

51
52 overcome challenges and achieve success. The need for autonomy, they add, relates to volition
53
54 and choice-making in pursuing and being responsible for one’s actions. The need for relatedness,
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 10 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 10
1
2
na
3 they elaborate, is about social status and a connection with others based on mutual respect and
4
5
6 interdependence. The three elements of SDT constitute human psychological needs to make
lJ
7
8 choices, to compete and collaborate with others; all of which can be afforded in the gamified
ou
9
10 environment.
11
12
rn
13
Many players in a gamified environment, according to Gee (2003), choose their own
14
al
15 avatars, choose to play the game competitively, or by working with others in affinity groups
16
17 (autonomy and volition). Many feel satisfied as the results are displayed on leaderboards of the
of
18
19
gamified environment they engage in highlighting the social element of relatedness. Studies have
20
21
Inf

22 shown that the elements of SDT positively affect intrinsic motivation: “considerable research has
23
24 found interpersonal contexts that facilitate satisfaction of the basic psychological needs for
or

25
26 competence, autonomy, and relatedness to enhance autonomous motivation, which comprises
27
m

28
29 intrinsic motivation and well-internalized extrinsic motivation” (Deci & Ryan, 2008, p. 14).
at

30
31 Researchers have established a connection between video game elements and motivation,
32
ion

33 on one hand, and self-determination theory (SDT), on the other hand. When players engage the
34
35
36
gamified environment, they willingly immerse themselves in virtual challenges for the purpose
37
an

38 of achieving fun and play; elements deeply rooted in human beings: “Intrinsically motivated
39
40 activities are those that the individual finds interesting and performs without any kind of
dL

41
42
conditioning, just by the mere pleasure of carrying them out (Francisco-Aparicio, Guti'errez-
43
44
ea

45 Vela, Isla-Montes, & Sanches, 2013, p. 114).


46
47 The framework that can be used to research gamification is the one that best reflects the
rn

48
49 concept of gamification and its different components. Robson et al. (2015) proposed a
50
ing

51
52 framework that relies on three gamification principles of mechanics, dynamics, and emotions
53
54 which is adapted from game design literature. According to the mechanics, dynamics, and
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 11 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 11
1
2
na
3 emotions (MDE) framework, mechanics refer to goals, rules, rewards; dynamics refer to how
4
5
6 players act and apply the mechanics; emotions, as the name suggests, refers to players’ feelings
lJ
7
8 during the gamified experience.
ou
9
10 New Literacies Study (Theory)
11
12
rn
13
New literacies study (NLS) is an extension of the new literacy theory with a special focus
14
al
15 on the digital environment as a semiotic domain for taking and processing meaning. In the NLS
16
17 theory, meaning is considered outside of language and beyond the digital tools as it “involve [s],
of
18
19
as well, ways of acting, interacting, valuing, believing, and knowing as well as often using other
20
21
Inf

22 sorts of tools and technologies” (Gee, 1997, p.10). The new literacies study is an umbrella term
23
24 for all kinds of digital literacies which include taking in and processing meaning (Gee, 1997).
or

25
26 Gamification is a form of digital literacy where many layers of meaning making and
27
m

28
29 processing take place. One of these modes of learning is through the affinity groups and the
at

30
31 many forms of social interaction in the digital game environment (Gee, 2003). Consequently,
32
ion

33 behaviorism, with the individual learner at its center (Gee, 2005), conflicts with new literacies
34
35
36
study that seeks to find meaning outside of the individual through and in the social environment.
37
an

38 This seemingly conflicting premise of the two theories of behaviorism and new literacy studies
39
40 will be reconciled by drawing from elements of both theories as they pertain to the learning
dL

41
42
atmosphere afforded by gamification. Chalco et al. (2015) argued that gamification allows for
43
44
ea

45 learning to happen individually as the learners feel extrinsically and intrinsically motivated
46
47 through gaining points and winning awards. At the same time, the social aspect of gamification
rn

48
49 through collaboration and competition, they added, is very important. Thus, the use of both
50
ing

51
52 behaviorism and new literacies study is justified.
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 12 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 12
1
2
na
3
4
5
6 Behaviorism: Relating Conditioning to Gamification
lJ
7
8 Gamification also has a strong connection to theories in human psychology, specifically
ou
9
10 behaviorism. According to Gonzales et al. (2016), gamification can produce significant
11
12
rn
13
behavioral change “from an early age using the dynamics of games” (p. 549). Some researchers
14
al
15 explained this connection between gamification, on one hand, and human psychology and
16
17 behavioral science, on the other hand, gamification “rests on three primary factors: motivation,
of
18
19
ability level, and triggers” (Dale, 2014, p. 85). Some of the basic principles of behaviorism, such
20
21
Inf

22 as enforcing a certain behavior by rewards and correcting a misbehavior by lack of rewards or a


23
24 form of a penalty, are parallel to gamification elements such as rewarding and penalizing through
or

25
26 points and badges, or upgrading and demoting in a game setting.
27
m

28
29 Skinner (1984), one of the fathers of behaviorism after Watson in the 1920s, realized the
at

30
31 connection between behaviorism principles and some of the elements in the simple video games:
32
ion

33 “No one really cares whether Pac-Man gobbles up all those little spots on the screen . . . What is
34
35
36
reinforcing is successful play, and in a well-designed instructional program students gobble up
37
an

38 their assignments” (p.952). Skinner (1937) coined the term “operant conditioning” when
39
40 discussing the premises of behaviorist principles which is the “study of reversible behavior
dL

41
42
maintained by reinforcement schedules” (Staddon, 2003, p. 115). There are two types of
43
44
ea

45 reinforcement schedules: fixed and variable. Chou (2013) posits that both fixed and variable
46
47 reinforcement schedules are used in the gamification design. However, the fixed reward, which
rn

48
49 he named “earned lunch,” is less engaging than the variable reward schedule, which he calls
50
ing

51
52 “mystery box” (p. 1). Whereas, fixed reinforcement schedule, in the context of gamification, has
53
54 resulted in low engagement levels immediately following the reward or penalty, variable
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 13 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 13
1
2
na
3 reinforcement schedule, as the element of surprise is activated, has produced higher engagement
4
5
6 levels in the gamified context (Raymer, 2011).
lJ
7
8 Folmar (2015) considered the real power of gamification its ability to produce desirable
ou
9
10 behavior change. Some researchers have defined gamification from a behaviorist-scientific
11
12
rn
13
perspective: “Gamification is a designed-behavior shift through playful experiences” (Reiners &
14
al
15 Wood, 2015, p. vi). Based on the variable schedule of the operant conditioning, not every
16
17 positive behavior is rewarded. Consider for example how casinos function where players lose
of
18
19
many times for the hope of winning once, yet they still come back to gamble (Nicholson, 2015).
20
21
Inf

22 In the next section, motivation and engagement are explored further to see how
23
24 gamification can foster them.
or

25
26 Motivation and Engagement
27
m

28
29 Motivation and engagement are two closely related concepts which often overlap in areas
at

30
31 of intrinsic motivation and cognitive engagement (Dornyei & Ushido, 2011; Guthrie, Willinger
32
ion

33 & You, 2012). Despite this strong link between motivation and engagement, the two terms are
34
35
36
not synonymous and the presence of one does not necessarily dictate the presence of the other
37
an

38 one. According to Brooks, Brooks, and Goldstein (2012), motivation is linked to psychological
39
40 elements that drive behavior and choice-making. Engagement, in the view of Russell, Ainley,
dL

41
42
and Frydenberg (2005), is an “energy” linked to different actions and tasks (p. 1). Appleton,
43
44
ea

45 Christenson, Kim, and Reschly (2006) highlighted the importance of both motivation and
46
47 engagement in learning, but emphasized their separation as independent constructs.
rn

48
49 Although the separation of motivation and engagement is an “ongoing issue” (Brooks et
50
ing

51
52 al., 2012, p. 548), there are some areas where the relationship between the two is nuanced.
53
54 Griffiths, Lilles, Furlong, and Sidhwa (2012) stated that engagement has evolved to include the
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 14 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 14
1
2
na
3 psychological inner processes and the manifestation of that in human behavior in the form of
4
5
6 task engagement, affective, and cognitive engagement. Willms (2003) emphasized the
lJ
7
8 connection between psychological attitudes and the participation in school activities when
ou
9
10 presenting an operational definition for engagement. Other scholars focused on the observable
11
12
rn
13
aspects of engagement such as the learners’ behaviors, their effort and dedication in performing
14
al
15 schoolwork, and their levels of participation and attendance (Ryan, 2000).
16
17 Motivation and engagement are sometimes distinguished chronically in occurrence.
of
18
19
Intrinsic motivation and prior attitudes about learning can be a precursor to task engagement and
20
21
Inf

22 increased participation. Participation could work in the opposite direction changing negative
23
24 prior attitudes. The combination of strong motivation and high task engagement facilitate
or

25
26 successful learning experience (Davis & McPartland, 2012). Engagement as an observable
27
m

28
29 positive behavior (i.e. involvement in school activities) is driven by prior attitudes or as Ryan
at

30
31 (2000) called them “beliefs” (p. 102).
32
ion

33 Dornyei, and Ottó, (1998) presented a comprehensive definition of motivation as “the


34
35
36
dynamically changing cumulative arousal in a person that initiates, directs, coordinates,
37
an

38 amplifies, terminates, and evaluates the cognitive and motor processes whereby initial wishes
39
40 and desires are selected, prioritised, operationalised, and (successfully or unsuccessfully) acted
dL

41
42
out” (p. 64). Motivation is divided by some researchers into five components: intrinsic and
43
44
ea

45 extrinsic motivation, task value, ability belief, and expectancies for success (Hsieh, 2014).
46
47 Intrinsic motivation is triggered by human needs for mastery, curiosity, and overcoming
rn

48
49 challenges. Extrinsic motivation is relevant to elements not related to the task value such as
50
ing

51
52 rewards, grades, “performance and competition or evaluation by others” (p. 419). Task value is
53
54 the perception and the value of the task by the learners and whether it is beneficial for them or
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 15 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 15
1
2
na
3 not. Finally, expectancy for success is how the learners expect to perform in the future as they
4
5
6 engage in a specific task (Wigfield, Byrnes, & Eccles, 2006).
lJ
7
8 Engagement indicates the passion and emotional involvement in participating and
ou
9
10 completing learning activities (Skinner & Belmont, 1993). Kuh (2009) tracked the evolution of
11
12
rn
13
the engagement construct throughout history from meaning the time learners spend on task, to
14
al
15 the outcome and achievement of learning, the quality of students’ effort, student interaction and
16
17 immersion in the learning experience, and finally, his own definition, the quality and effort
of
18
19
learners invest in an authentic activity. Notice that the common theme among all definitions Kuh
20
21
Inf

22 discussed is the visible aspect of engagement as it is manifested in the learners’ behavior towards
23
24 the learning experience and the quality and time they invest in the learning task. However, to
or

25
26 equal engagement to time on task is unfair in capturing the full scope of this term. Schlechty
27
m

28
29 (2001) argued that engagement is not simply synonymous with time on task, but it is “the
at

30
31 enthusiasm and diligence” in doing the task that makes the engagement a reality (p. 64).
32
ion

33 Csikszentmihalyi (1997) emphasized this connection between engagement in a task and the
34
35
36
overwhelming deep involvement of the learners that transcends time and space.
37
an

38 Connecting Gamification to Motivation and Engagement


39
40 In many studies, students’ levels of engagement increased significantly following the
dL

41
42
introduction of game elements (Table 2). The following inclusion criteria were followed in the
43
44
ea

45 summary of literature review on gamification as illustrated in Table 2. Literature included


46
47 articles and conferences proceedings published after 2012. All of the studies selected employed
rn

48
49 elements of video games and any study that did not employ them was excluded. The studies were
50
ing

51
52 obtained through Washington State University (WSU) library data system and some of them
53
54 were made available through interlibrary loan network. In addition, some conference
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 16 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 16
1
2
na
3 proceedings were obtained through GoogleScholar. All the studies were examined carefully for
4
5
6 theoretical foundations utilized in the study of gamification, the gamified content, the number
lJ
7
8 and age of the participants, and the results.
ou
9
10 Researchers who conducted empirical studies on the utilization of gamification elements
11
12
rn
13
agree on the positive effect on students’ engagement, motivation, and overall performance
14
al
15 through instant feedback and collaboration (Kingsley & Grabner, 2015; Leaning, 2015;
16
17 Papastergiou, 2009; Seaborn & Fels, 2015; Koivisto & Hamari, 2014). Some authors proposed
of
18
19
positive results showing higher likeability ratings when gamification features are introduced
20
21
Inf

22 (Attali & Arieli, 2015). These results refer to how students felt toward the introduction of the
23
24 game elements to the learning environment. Some studies showed no connection between
or

25
26 students’ engagement and motivation and the introduction of gamification features to the
27
m

28
29 learning environment (Hanus & Fox, 2015).
at

30
31 In a review of peer-reviewed empirical studies on gamification, Hamari, Koivisto and
32
ion

33 Sarsa (2014) reviewed 24 studies, and stated that most of the studies yielded positive results of
34
35
36
the relationship between gamification and learners’ engagement. Seaborn and Fels (2015)
37
an

38 reviewed 32 studies on the utilization of digital gamification elements pedagogically. Out of the
39
40 32 studies, 20 yielded positive results connecting gamification to increased levels of motivation
dL

41
42
and engagement. The remaining 12 studies yielded negative results showing no correlation
43
44
ea

45 between students’ engagement and introduction of game elements. Some empirical studies
46
47 (Leaning, 2015; Berkling & Thomas, 2013) which produced negative or mixed results, focused
rn

48
49 on limited features of gamification, or forced the students to work with the game options
50
ing

51
52 available and failed to give them choice.
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 17 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 17
1
2
na
3 Table 2
4
5 Summary of Literature Review on Gamification
6
lJ
7 Reference Gamification Results Gamified Theoretical n Age
8 Features Used Content Connections
ou
9
10 Barta, Gama, Badges, points, Positive Computer N/A 77 18-21
11 Jorge, challenges, engineering
12
rn
Gonclaves leaderboards, course
13
14 (2013) levels
al
15
16 Berkling & Levels, Negative Software Motivational theory 90 20-21
17 Thomas leaderboards, engineering
of
18 (2013) points course
19
20
21
Betts, Bal & Levels, choice Positive Web-based N/A 33 N/A
Inf

22 Betts elements collaborative


23 (2013) learning
24 tool called
or

25 Curatr
26 Brewer et al. Points and positive Lab experiment N/A 14 5-7
27
m

(2013) rewards on children


28
29
at

30 de Freitas & de positive Computer N/A 17 20-23


31 Freitas Rewards, science
32 (2013) points, levels class
ion

33
34 Eleftheria Points, badges, positive Augmented N/A N/A 10-12
35
et al. (2013) challenges, reality Following
36
37 virtual goods science book Bartler’s player-type
an

38 model.
39 Gibson et al. positive General use of Intrinsic and extrinsic N/A N/A
40 (2013) Badges badges as a motivation.
dL

41 motivating tool
42 Goehle Levels and positive Web-based N/A 60 N/A
43
44
(2013) points homework
ea

45 Program called
46 WebWork
47 Hanus & Leaderboard positive Two college Cognitive evaluation 80 18-24
rn

48 Fox (2015) and badges courses theory and motivational


49 theory
50
ing

Kingsley & Badges, points, Positive 3D GameLab New literacies studies 47 12


51
52 Grabner- quests. software.
53 Hagen (2015)
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 18 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 18
1
2
na
3
4
5
6
lJ
7
8 Reference Gamification Results Gamified Theoretical n Age
ou
9 Features Used Content Connections
10 Kumar & positive Teaching N/A 207 25-30
11
12
Khurana Badges, points, programming
rn
13 (2012) levels languages
14
al
15 O’Byrne Badges positive Youth program New literacies studies 1 12
16 et al. (2015) called MOUSE
17
of
18
Leaning Leaderboard Mixed Undergraduate Situated motivational 125 18-23
19
20 (2015) and points. media-studies affordance
21 non-digital course.
Inf

22 games
23
24 Todor & Rewards, positive E-learning N/A N/A N/A
or

25 Pitic (2013) points, badges platform for a


26
course in
27
m

28 electronics
29 Raymer Rewards and positive e-Learning Behaviorism N/A N/A
at

30 (2013) progress bars software


31
32 Thom, Badges, points *Negative Social network Intrinsic and extrinsic 3486 18+
ion

33 Millen & And status service motivation


34
35
DiMicco
36 (2012)
37 Note. *The removal of gamification elements resulted in significant decline in participants’
an

38 participation.
39
40 It is important when designing a gamified course fully or partially to create a challenge,
dL

41
42 when possible, that is appropriate to the level of the students to maintain their engagement
43
44
ea

45 (Nicholson, 2015). A complicated challenge can have reverse effect on engagement and cause a
46
47 lack of interest, and even “anxiety and frustration.” (p. 13). Nicholson (2015) divided
rn

48
49 engagement in the context of gamification into two categories: First, engagement in the form of
50
ing

51
52
interaction, cooperation, and altruism between the players (in a social manner). Second,
53
54 engagement between players achieved through the utilization of game mechanics.
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 19 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 19
1
2
na
3 Francisco-Aparicio et al. (2013) argued that higher levels of extrinsic motivation when
4
5
6 using gamification are not enough criteria to consider its benefits. The positive effects are
lJ
7
8 usually temporary if not combined with principles from self -determination theory: autonomy,
ou
9
10 competence, and relatedness (Nicholson, 2015). According to Zickermann and Cunningham
11
12
rn
13
(2011), one of the roles of educators is to help create circumstances that would allow for intrinsic
14
al
15 motivation to be born. They posited that extrinsic elements, such as points and badges for
16
17 example, could be used to lead to this outcome and critiqued not targeting intrinsic motivation in
of
18
19
gamification design. If no permanent positive behavior change is created in the learners, the
20
21
Inf

22 long-term effects of gamification cannot be fully evaluated. This can only be done using a
23
24 longitudinal study that captures the long-term effects of the relationship between gamification
or

25
26 and learners’ motivation and task engagement. One of the steps in understanding the long-term
27
m

28
29 effects of the gamification implementation in a pedagogical context is to combine, if data is
at

30
31 available, the quantitative and qualitative design.
32
ion

33 Fun is one of the elements that attracts video game players to engage in playing activities
34
35
36
and keep coming back for more. Gamification borrows from video games the element of fun not
37
an

38 only to gain the learners’ engagement, but also to positively increase their motivation. In the next
39
40 section, this connection between the component of fun in gamification and its effect on
dL

41
42
motivation and engagement is explored further.
43
44
ea

45 Emotions and Fun in Gamification


46
47 Gamification provides the component of fun that helps in transforming the students’
rn

48
49 attitudes towards learning. Fun can allow for better learning, a concept that Prensky (2001)
50
ing

51
52 explained as he discussed the transformation in the learners’ attitude towards learning: “It
53
54 appears that the role of fun in the learning process is to create relaxation and motivation.
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 20 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 20
1
2
na
3 Relaxation enables a learner to take things in more easily, and motivation enables them to put
4
5
6 forth effort without resentment” (p. 111). McGonigal (2011) discussed the essential role of fun
lJ
7
8 historically in the human experience. She refers to the story of the ancient Lydians who managed
ou
9
10 to live through famine on very limited rations distracting themselves from hunger through
11
12
rn
13
elaborate games on a state level.
14
al
15 Video games, for many players, produce an emotional state induced by several factors,
16
17 most important of which is fun. This feeling of fun is created in the players through their feeling
of
18
19
of achievement, a sense of exploration, the reward of completing a level, or simply winning a
20
21
Inf

22 game (Zickermann, 2010). If this element of play is incorporated into the learning experience, an
23
24 intrinsic interest in learning can follow (Liebermann, 2006). For many learners, the fun part in a
or

25
26 gamified environment is the product of solving problems and overcoming challenges as they
27
m

28
29 engage critical thinking skills. “Desirable difficulties,” as Yue, Bjork, and Bjork (2013) called
at

30
31 them (p. 266), are important qualities, according to Liebermann (2006), in the process of
32
ion

33 learning. As learners exert their best effort and become mindful of these challenges, “close
34
35
36
attention and intense mental effort lead to deeper understanding [and] learning” (p. 386).
37
an

38 Gamification cannot be understood holistically without the essential components of video


39
40 games that can be incorporated into learning environment. In the next section, player types and
dL

41
42
game components will be discussed.
43
44
ea

45 Players Types and Gamification Features


46
47 Player types
rn

48
49 The research about players’ types inform us about a proper gamified design for delivering
50
ing

51
52 pedagogical content. In 1996, Bartle developed a test called Bartle’s Test of Game Psychology.
53
54 The test was adopted, recreated, and maintained by many game-design websites, such as
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 21 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 21
1
2
na
3 GameDNA, and users have taken the test more than 800 thousand times (everthing2com, 2015).
4
5
6 Based on this test, there are four types of gamers or players:
lJ
7
8 1- Killers: those who compete and play against other gamers.
ou
9
10 2- Achievers: those who achieve status due to a high level of performance.
11
12
rn
13
3- Explorers: those who collect virtual goods and discover things.
14
al
15 4- Socializers: those who are good team players and collaborate with others in the
16
17 game environment.
of
18
19
Folmar (2015) commented in detail on the four types of players and how their needs should be
20
21
Inf

22 considered in an effective pedagogical gamification design. He noted that killers rely on badges
23
24 and points displayed on a leaderboard to gain public recognition in the game environment.
or

25
26 Achievers track their achievements through badges and points and are keen to know the status of
27
m

28
29 their progress. Scoializers interact with others through mutual support. Finally, explorers are
at

30
31 independent and are more interested in pursuing a quest rather than impressing others.
32
ion

33 Bartle’s test was modified in 2010 to become pedagogically friendly (Farber, 2013). Kim
34
35
36
(2014) changed the semantics to describe different learners and added verbs to reflect diffrerent
37
an

38 methods of student interactions in the gamified envrionmnet which is silmilar to Bloom’s (1956)
39
40 Taxonomy. These verbs can be used by teachers in the creation of lesson plans and activities.
dL

41
42
Some of these verbs created by Kim (2014) are: build, design, customize , challenge, explore,
43
44
ea

45 comment, and share. The meaning embedded within these verbs is intended to facilitate learning
46
47 in the gamified environment.
rn

48
49 In the following section, some digital gamification features will be discussed including:
50
ing

51
52 avatars, quests and challeneges, badges, points and levels.
53
54 Gamification features
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 22 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 22
1
2
na
3 avatars.
4
5
6 Avatars are representative of players in the sense that they reflect their aspirations,
lJ
7
8 vulnerabilities, and the different roles they play in life. Players need to choose or create their
ou
9
10 avatars as manifestations of their autonomy needs: “Avatar spaces indisputably involve choice in
11
12
rn
13
the creation of one’s avatar; there is substantial scope in which to exercise choice and create
14
al
15 meaning” (Wilson, 2003, p. 2). Gee (2003) indicated that avatars mimic human identities such as
16
17 parents, workers in different professions, religious or areligious people, and different social
of
18
19
classes. He referred to how gaming software allows the players to design their avatars with
20
21
Inf

22 different costumes, powers, complexions, and in some settings, heroes or leaders as opposed to
23
24 foot soldiers or workers. He added that avatar creation goes beyond segregation, racism, sexism,
or

25
26 and many other social illnesses as it introduces virtual and fictional characters. Between the real
27
m

28
29 and the avatar, lies a third “projective identity” where the real character projects its own
at

30
31 aspirations and desires unto the virtual character (Gee, 2003, p. 55). Waggoner (2009) discussed
32
ion

33 the philosophy of representation in the creation of avatars. He cited findings showing that the
34
35
36
real-world identities “continually informed” the virtual ones (p. 1).
37
an

38 Wilson (2003) created an elaborate definition of an avatar as a: “Virtual, surrogate self


39
40 that acts as a stand in for our real-space selves that represents the user. The cyberspace avatar
dL

41
42
functions as a locus that is multifarious and polymorphous displaced from the facility of our real-
43
44
ea

45 space selves” (p. 2). Another definition that adds the real-life element to avatars as “models
46
47 driven by humans in real time” (Bailenson, Yee, Blascovich, & Guadagno, 2008, p. 78). Avatars
rn

48
49 represent an opportunity for players to venture into a risk-free world (Boss, 2009). The freedom
50
ing

51
52 to choose or design their own avatars creates an atmosphere where students can “find their own
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 23 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 23
1
2
na
3 voice” (p. 4). After choosing an avatar, the learners will be faced with the next challenge: to
4
5
6 pursue a quest.
lJ
7
8 quests and challenges.
ou
9
10 Quests are a series of challenges that require players to solve mystery engaging critical
11
12
rn
13
thinking skills (Whitton, 2010). When students embark on a quest or accept a challenge, they
14
al
15 engage in a story line that usually embeds a time-sensitive pattern. Quests and challenges give
16
17 players a sense of direction or a purpose in a gamified environment (Zickermann &
of
18
19
Cunningham, 2011). Adding story components or beginning a course with a form of a challenge
20
21
Inf

22 are more engaging than a list of course objectives; both strategies recommended and applied by
23
24 gamification proponents (Sheldon, 2011; Kapp, 2011). Employing quests in this manner
or

25
26 “provides context or activities that are used within games and adds them to the content being
27
m

28
29 taught” (Kapp, 2011, p. 126). Quests and challenges support the sense of adventure and activate
at

30
31 critical thinking skills by setting the exploration and discovery elements ( Dale, 2014; Powers,
32
ion

33 Brooks, Aldrich, Palladino & Alfieri, 2013).


34
35
36
There are several learning elements unique to the gamified environment as it pertains to
37
an

38 the quest or challenge story line (Kiang, 2014). In the virtual world of gamification, failure does
39
40 not have the same negative connotations as in the real world. Consequently, the failure or death
dL

41
42
of an avatar character is a chance to contemplate, learn from mistakes, and restart again as the
43
44
ea

45 concept of failure is fragmented to small failed attempts. Kiang (2014) recommended that the
46
47 teacher “try providing ways for the students to ‘fail’ frequently in many small ways, rather than
rn

48
49 in one big high-stakes test” (p. 2). In a gamified context, there is no single way to achieve
50
ing

51
52 success or accomplish a goal and students are empowered by this diversification or “flexibility
53
54 dynamic [to] take a personalized path to success” (p. 2).
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 24 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 24
1
2
na
3 Quests can offer students the opportunity to work cooperatively and develop teamwork or
4
5
6 they can choose to work individually where their achievements will “roll up to a group”
lJ
7
8 (Zickermann & Cunningham, 2011, p. 65). This sense of unity is one of Gee’s (2003) 36
ou
9
10 pedagogical principles called “affinity group” principle that the gamified environment fosters.
11
12
rn
13
He explained: “learners constitute an ‘affinity group,’ that is, a group bonded primarily through
14
al
15 shared endeavor, goals, and practices and not shared race, gender, nation, ethnicity, or culture”
16
17 (p. 212). This affinity group principle is manifested in many types of video games including
of
18
19
massively multiplayer online games (MMOG). These games typically support hundreds or even
20
21
Inf

22 thousands of players coming from different geographical, social, and racial backgrounds contrary
23
24 to the stereotypical image of a video game player as White, reclusive teenage male who is
or

25
26 typically overweight (Bergstrom, Fisher, & Jenson, 2014). The power of this virtual social
27
m

28
29 environment can be employed allowing for learners to interact and collaborate instead of
at

30
31 competing against other.
32
ion

33 As players advance in the stages of their epic quests, they start gaining badges; one of the
34
35
36
oldest game elements used to boost motivation and engagement.
37
an

38 badges.
39
40 Badges have a long history in many fields outside of gamification. Antin (2012) dated the
dL

41
42
history of badges back to 1911, where the Boy Scouts of America “understood the motivational
43
44
ea

45 power of goals, mastery seeking, reputation, and identity signaling with valued
46
47 accomplishments” (Social Mediator, p. 3). Digital badges are a “validated indicator of an
rn

48
49 accomplishment, skill, quality, or interest that can be earned in various learning environments”
50
ing

51
52 (Grant, 2013, p. 1). In the context of education, badges are chosen in a gamified environment to
53
54 accommodate different learners considering their motivation levels and capabilities
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 25 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 25
1
2
na
3 (Abramovich, Schunn, & Higashi, 2013, p. 217). As students advance through different levels
4
5
6 and accumulate badges associated with different achievements, badges then serve as an “online
lJ
7
8 record of a learner’s achievement” (Devedžić & Jovanović, 2014, p. 603).
ou
9
10 According to Nicholson (2015), players who earned badges feel inner satisfaction as
11
12
rn
13
their status is announced publicly in the gamified environment. He added that badges not only
14
al
15 serve as “signposts” signaling the players progress, but also indications of past achievements.
16
17 Richter, Raban and Rafaeli (2015) argued that badges serve as a record of an individual’s past
of
18
19
and present successes. On a social level, they added, badges establish the reputation of an
20
21
Inf

22 individual in the game environment. Whether individually or socially, badges help enhance
23
24 qualities such as “self-competence and self-efficacy” (p. 34).
or

25
26 Not only will badges represent the learners’ achievement, but also points and levels are
27
m

28
29 important signs of their progress. (p. 10).
at

30
31 points and levels.
32
ion

33 Points have a significant place in a gamified environment. Some game proponents


34
35
36
consider points as an essential part of a gamified world or an “absolute requirement for all
37
an

38 gamified systems” (Zickermann & Cunningham, 2011, p. 36). Attali and Arieli (2015) used
39
40 points as the main gamification component that they included in their study of performance
dL

41
42
measuring fluency and understanding of math concepts. Gamification opponent Bogost (2011),
43
44
ea

45 critiqued the dependency on and exclusiveness of many gamified designs to points, which he
46
47 thought were the least significant part of video games.
rn

48
49 Rewards in the form of points are not necessarily permanent, contrary to what
50
ing

51
52 Zickermann and Cunningham (2011) stated: “Once you start giving someone a reward, you have
53
54 to keep her in that reward loop forever” (p. 27). Rewards reinforce desirable learning behaviors
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 26 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 26
1
2
na
3 and once these learning behaviors are established, rewards have no purpose. They have little
4
5
6 effect on highly motivated learners and sometimes that effect could be negative. Nicholson
lJ
7
8 (2015) stated: “As the subject will continue to use the skill for the real-world value, the rewards
ou
9
10 are no longer needed” (p. 3). He elaborated further on the idea that rewards do not work with
11
12
rn
13
everyone, certainly not for ones with strong intrinsic motivation: “This use of incentives to
14
al
15 motivate someone to do something when they have no other reason to do so is a very common
16
17 use of rewards and for tasks that do not require creative thinking, incentive program can improve
of
18
19
performance” (p. 3).
20
21
Inf

22 Modern games usually have scaffolding techniques where players are introduced to
23
24 simple levels to start with to encourage their progress. The players cannot advance to the next
or

25
26 level until they have achieved mastery in the previous level (Kiang, 2014; Kolb, 2015). Players
27
m

28
29 usually need “frequent rewards not penalties” as rewards signal skill mastery (Prensky, 2001, p.
at

30
31 135), although critics might think of this idea as limited to learners or players who are only
32
ion

33 driven by extrinsic motivation symbolized by virtual rewards. According to Zickermann and


34
35
36
Cunningham (2011), we should not wait for the birth of intrinsic motivation. By creating
37
an

38 conditions which are extrinsically motivating, “we shift the focus of responsibility from hoping it
39
40 happens to a structure and process for making it happen” (p. 28). According to Hanus and Fox
dL

41
42
(2015), the groups of students who seem to be most engaged in a gamified classroom
43
44
ea

45 environment are low achievers or learners with weak intrinsic motivation. For those types of
46
47 learners, “rewards and incentives might increase intrinsic motivation” (p. 160).
rn

48
49 Sheldon’s case study and Quest to Learn school are some of the many examples of fully
50
ing

51
52 gamified learning experiences that will give help explore the gamification implementation..
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 27 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 27
1
2
na
3 Gamification in Action
4
5
6 The Sheldon Case Study
lJ
7
8 Sheldon (2011) gamified a college course by, among other strategies, assigning points to
ou
9
10 reward the students’ academic work who would advance in level as they accumulate enough
11
12
rn
13
points. Sheldon (2011) considered his experiment as successful in promoting the students’
14
al
15 engagment and improve their retension but with mosdest changes in their “overall performance”
16
17 (p. 178). Sheldon focused on the non-digital aspects of gamification as he announced this in the
of
18
19
introduction to his book (p. xiv). In Sheldon’s design, indiviual points carried less importance
20
21
Inf

22 than scores. Sheldon(2011) assigned hundreds of points total for the gamified course activities he
23
24 created. Kolb (2015) suggested that teachers engaging gamification points should be generous in
or

25
26 using them where 100 points is no longer the equivlent of grade A. Sheldon tried to foster
27
m

28
29 intrinsic motivation by incorporating gamification elements other than points and levels which
at

30
31 are both symbols of extrinsic motivation. He establsiohed guilds where students can collaborate
32
ion

33 and manifest altruism in helping “guild mates” (p. 119). He expanded the concept of rewards to
34
35
36
include knnowledge and practice which he called “intrinsic rewards” (p. 166).
37
an

38 Stott and Neustaedter (2013) critiqued Sheldon’s gamified design for “implementing
39
40 game components by simply trading out the partance of pedagogy for that of game culture” (p.
dL

41
42
1). Sheldon (2011) overly focused on points and levels in his experiment as manifested in the
43
44
ea

45 multiple syllabi he presented for his courses. Lawley (2012) stated that going beyond surface
46
47 characteristics of gamification, in reference to points and levels, is essential in the game design.
rn

48
49 She advocated creating an interactive game design which is aligned with pedagocial principles
50
ing

51
52 and conductive to collaboration. She added that a faulty or superficial gamified design can
53
54 “damage existing interest or engagment” (p. 16).
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 28 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 28
1
2
na
3 In the next section, a more immersive gamified experience will be explored as in the
4
5
6 Quest to Learn example.
lJ
7
8 Quest to Learn
ou
9
10 Established in June of 2008 and funded by MacArthur and Bill and Melinda Gates
11
12
rn
13
foundations, Quest to Learn is a New York-based school that has adopted gamification and
14
al
15 game-based learning throughout its entire curriculum from grades six to twelve (Quest to Learn,
16
17 2008). It is the result of collaboration between game designers from the Institute of Play (2007)
of
18
19
and educators for the hope that this school will work as a model for other schools around the
20
21
Inf

22 world. McGonical (2011) commended the process of learning in this unique and pioneer school:
23
24 “It is how they learn that is different: students are engaged in gameful activities from the moment
or

25
26 they wake up to the moment they finish up their final homework assignment at night” (p. 129).
27
m

28
29 O’Keefe (2012) explained how the curriculum is designed as a result of coordination between the
at

30
31 teachers and the game designers based the needs of the students. The school assignments are far
32
ion

33 from traditional and involve students facing different challenges and engaging in quests that start
34
35
36
before the beginning of a school day. The creators of Quest to Learn commented on its mission:
37
an

38 “Each trimester students encounter a series of increasingly complex, narrative challenges, games
39
40 or quests, where learning, knowledge sharing, feedback, reflection and next steps emerge as a
dL

41
42
natural function of play” (Quest to Learn, 2008, p. 1).
43
44
ea

45 The founders of Quest to Learn describe the learning experience in their school as
46
47 “immersive” and “narrative-based” (Quest to Learn, 2008, p. 1). Craven (2015) emphasized the
rn

48
49 nature of gamification as an immersive experience which creates engagment for the user. This
50
ing

51
52 implies approaching the implementation of gamification holistically in a manner that
53
54 incorporates effective elements such as quests and challenges. Wood and Reiners (2015)
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 29 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 29
1
2
na
3 connected the immersive experience to story-telling elements in gamification where the “virtual
4
5
6 environment for visualization” is activated (p. 316).
lJ
7
8 The examples of Sheldon’s case study and Quest to Learn, in addition to many other
ou
9
10 gamification models and studies in business and education, are unique but not in the sense that
11
12
rn
13
they cannot be adaptable. In the next section, some of the guidelines for implementing
14
al
15 gamification will be discussed.
16
17 Implementation Guidelines
of
18
19
Some proponents of gamification suggested using 20% of class time for gamification
20
21
Inf

22 following Mclaughlin’s (2011) Google model where employees were given 20% of work time to
23
24 do anything they were passionate about (Bruder, 2015). Between full gamification of the
or

25
26 classroom and 20% quota suggestion, gamification is a tool to aid in achieving learning
27
m

28
29 objectives and not vice versa. Employing game features, or any kind of educational technology,
at

30
31 is not the goal at the expense of students’ learning.
32
ion

33 Simões et al. (2013) suggested utilizing “distinctive characteristics of good games,


34
35
36
particularly social games in order to understand what makes sense to apply to teaching
37
an

38 processes” (p. 346). The principles to incorporate in gaming are intended to be “equally relevant
39
40 to learning in video games and learning in content areas in classrooms” (Gee, 2003, p. 41).
dL

41
42
Kolb (2015) outlined four principles necessary in any gamified design with learning
43
44
ea

45 content. First, she recommended that the learning objectives be well-stated and explicitly-
46
47 presented prior to engaging gamification elements, and preferebly embedded within the gamified
rn

48
49 design. Second, she highlighted considering practical steps when applying gamification design in
50
ing

51
52 the learning environment, such as the use of reliable gamification software that allows for full
53
54 deployment of gamification features. Few companies produced that kind of software such as
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 30 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 30
1
2
na
3 3DGamelab, Gradecraft, and Classcraft. Third, game designers and educators. According to Kolb
4
5
6 (2015), need to prepare quests to increase motivation and engagement. Fourth, the gamified
lJ
7
8 design should allow for modding (shortened from modification). Modding, in a gamified context,
ou
9
10 means to allow users to make their own game choices. Students should be empowered to choose
11
12
rn
13
their own avatars, create quests, and decide to engage the gamified content individually or by
14
al
15 working in teams.
16
17 Research has shown that intrinsically motivated students experience gradual
of
18
19
disengagement and loss of motivation when forced to use game features (Hanus & Fox, 2015).
20
21
Inf

22 Gisbson and Jakl (2015) connected choice in the gamified experience to the autonomy of self-
23
24 directed learning. Autonomy as one of the three principles of SDT is connected to intrinsic
or

25
26 motivation (Nicholson, 2015). He added that allowing the learner to make choices in the
27
m

28
29 gamified experience will allow for meaningful gamification to happen.
at

30
31 According to Gee (2003), the game design elements should allow for collaboration and
32
ion

33 team work through the use of leaderboards or “guilds.” Zhang and Clear (2015) emphasized that
34
35
36
any successful gamified design need to support collaboration among the students. They added
37
an

38 that successful gamified designs which incorporate collaboration help the emerging of positive
39
40 learning behaviors. Kim, Glassman, and Williams (2015) stated that as the players collaborate,
dL

41
42
they “engage in a shared, relevant, goal oriented activity” (p. 333).
43
44
ea

45 Successful game design allows the players to try multiple times to achieve success. In the
46
47 gamified environment, failure is redifined where it is no longer a set back but rather “an
rn

48
49 opportunity to learn from mistakes and correct them” (Hanus & Fox, 2015, p. 3). Research
50
ing

51
52 showed that players who received constructive feedback following failure in the gamified
53
54 environmnet expressed positive emotions about their experience (Herzig , Ameling, Wolf &
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 31 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 31
1
2
na
3 Schill, 2015). Gee (2014) considered failure in the gamified world as a “path to success” (p.
4
5
6 186).
lJ
7
8 Researchers suggested studying the deployment of full game features over a longer
ou
9
10 period to evaluate its full potential (e.g. Attali & Arieli, 2015; Hanus & Fox, 2015; Gonzales et
11
12
rn
13
al., 2016). Employing limited game features in isolation to an existing course may not produce
14
al
15 desirable or measurable effect (Whitton, 2010). The limited use of gamification in the form of
16
17 employing some features of games into the learning experience produced mixed or adverse
of
18
19
results on performance and motivation (Attali & Arieli, 2015).
20
21
Inf

22 Researchers who engaged the learners with limited game features reported the
23
24 participants asking for more elements of gamification (Papastergiou, 2009). There is srong
or

25
26 evidence to suggest a direct link between the effective use of gamification elements and meeting
27
m

28
29 basic psychological human needs. Many of these needs are connected to self-determination
at

30
31 theory. Francisco-Aparicio et al. (2013) highlighted the importance of a gamification design that
32
ion

33 meets the inner psychological needs of the learners. These inner psychological needs intersect
34
35
36
with principles of self-determination theory of relatedness, competence, and automomy. For
37
an

38 example, relatedness, according to Francisco-Aparicio et al. (2013), could be supported by


39
40 online interaction and collaboration. Competence needs, in their view, can be supported through
dL

41
42
“practice activities” (p. 183). They posit that offering learners choices and allowing them to take
43
44
ea

45 control in the gamified environment fulfills the autonomy needs.


46
47 When presenting her proposed guideline for a full gamified experiece, Kolb (2015)
rn

48
49 suggested allocating 10-25 minutes for each traditional assignment to be transformed into a quest
50
ing

51
52 in the gamified environment. This length of time for quest development, among other
53
54 gamification components, might not be practical for many teachers. For example, Matera (2010)
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 32 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 32
1
2
na
3 spent, based on his own testimony, more than 100 hours just to prepare a leaderboard table for
4
5
6 his six-grade gamified class with outstanding results (https://twitter.com/mrmatera).
lJ
7
8 Although many teachers, as Gee (2003) suggested, are gamers themselves, they are often
ou
9
10 reluctant to use methods not directly related to their professional practice. Many teachers have no
11
12
rn
13
difficulty introducing game elements into the classroom with the help of supporting and useful
14
al
15 software such as Classcraft, for example. Watching the software tutorials and experimenting with
16
17 it is not beyond the skills and abilities of many teachers. Lee and Hammer (2011) argued that
of
18
19
gamification can give teachers the necessary tools to direct and motivate their students
20
21
Inf

22 transforming the learning experience into a joyful one.


23
24 Conclusion
or

25
26 Between the zealots in support of full gamification of the curriculum and the educational
27
m

28
29 system and opponents who think gamification is a distraction from the learning objectives, there
at

30
31 remains a need for further exploration of the full impact of gamification on engagement and
32
ion

33 motivation. Further empirical research need to be conducted to examine the potentials of


34
35
36
gamification. In addition to deploying full gamification features to study its impact, longitudinal
37
an

38 studies need to be conducted in order to develop a full nderstanding of the effect of gamification
39
40 on the learners’ engagment and motivation. There is a need to research the most effective
dL

41
42
components of game elements that could create proper conditions for the birth of intrinsic
43
44
ea

45 motivation. Furthermore, the learners’ perceptions of the gamification intervention need to be


46
47 researched using mixed method design to help understand the relationship between gamification,
rn

48
49 engagement, and motivation in a holistic way.
50
ing

51
52
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 33 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 33
1
2
na
3
4
5
6
References
lJ
7
8
ou
9 Abramovich, S., Schunn, C., & Higashi, R. M. (2013). Are badges useful in education? It
10
11 depends upon the type of badge and expertise of learner. Educational Technology
12
rn
13 Research and Development: A Bi-monthly Publication of the Association for Educational
14
al
15
16
Communications & Technology, 61(2), 217-232. doi:10.1007/s11423-013-9289-2
17
of
18
19
Andre, S. (2013). Tips and techniques for incorporating team-based learning (TBL) methods into
20
21 a college classroom. The Exchange, 2(2). Retrieved from
Inf

22
23 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2395502
24
or

25
26 Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and
27
m

28 psychological engagement: Validation of the Student Engagement Instrument. Journal of


29
at

30
31 School Psychology, 44(5), 427–445. https://doi.org/10.1016/j.jsp.2006.04.002
32
ion

33 Attali, Y. &.-A. (2015). Gamification in assessment: Do points affect test performance?


34
35 Computers and Education, 83, 57-63. Retrieved from
36
37
an

38 http://doi.org/10.1016/j.compedu.2014.12.012
39
40
dL

41
Bailenson, J. N., Yee, N., Blascovich, J., & Guadagno, R. E. (2008). Transformed social
42
43 interaction in mediated interpersonal communication. In K. Elly A, U. Sonja, T. Martin,
44
ea

45 & B. Susan B, Mediated interpersonal communication (p. 78). N.Y.: Routledge.


46
47
rn

48 Baard, P. P., Deci, E. L., & Ryan, R. M. (2004). Intrinsic Need Satisfaction: A Motivational
49
50
ing

Basis of Performance and Weil-Being in Two Work Settings1. Journal of Applied Social
51
52
53 Psychology, 34(10), 2045–2068.
54
55
Te

Barata, G., Gama, S., Jorge, J., Goncalves, D. (2013). Engaging engineering students with
56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 34 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 34
1
2
na
3 gamification. 5th International Conference on Games and Virtual Worlds for Serious
4
5
6 Applications, Greece (pp. 1–8).
lJ
7
8 Bergstrom, K., Fisher, S., & Jenson, J. (2016). Disavowing that guy: Identity construction and
ou
9
10 massively multiplayer online game players. Convergence: The International Journal of
11
12
rn
13
Research into New Media Technologies, 22(3), 233–249.
14
al
15 https://doi.org/10.1177/1354856514560314
16
17 Berkling, K., Thomas, C. (2013). Gamification of a software engineering course. International
of
18
19
Conference on Interactive Collaborative Learning, United Kingdom (pp. 525–530).
20
21
Inf

22 Betts, B.W., Bal, J., Betts, A.W. (2013). Gamification as a tool for increasing the depth of
23
24 student understanding using a collaborative e-learning environment. International
or

25
26 Journal of Continuing Engineering Education and Life-Long Learning 23(3-4), 213–228
27
m

28
29 Bartle, R. (1996, August 28). Hearts, clubs, diamonds, spades: Players who suit MUDS.
at

30
31
32 Retrieved from http://mud.co.uk/richard/hcds: http://mud.co.uk/richard/hcds.htm
ion

33
34
35
Biopac. (2009, April 30). Biopac. Retrieved from Biopac.com:
36
37 http://www.biopac.com/application/fnir-functional-near-infrared-optical-brain-imaging/
an

38
39
40 Bloom, B. (1956). Taxonomy of educational objectives; the classification of educational
dL

41
42 goals (1st ed.]. ed.). New York: Longmans, Green.
43
44
ea

45 Bogost, I. (2011, May 3). Persuasive games: Exploitionware. Retrieved from Gamustura:
46
47
rn

http://www.gamasutra.com/view/feature/134735/persuasive_games_exploitationware.php
48
49
50
ing

Boss, S. (2009, May 27). Edutopia. Retrieved from Avatars teach teens about self-image:
51
52
53 https://www.edutopia.org/avatars
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 35 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 35
1
2
na
3 Boss, S. (2012). Bringing innovation to school: Empowering students to thrive in a changing
4
5
6 world. Bloomington, IN: Solution Tree.
lJ
7
8
ou
9 Brewer, R., Anthony, L., Brown, Q., Irwin, G., Nias, J., Tate, B. (2013). Using gamification to
10
11 motivate children to complete empirical studies in lab environments. 12th International
12
rn
13 Conference on Interaction Design and Children, New York (pp. 388–391).
14
al
15
16
Brooks, R., Brooks, S., & Goldstein, S. (2012). The Power of Mindsets: Nurturing Engagement,
17
of
18 Motivation, and Resilience in Students. In S. L. Christenson, A. L. Reschly, & C. Wylie
19
20 (Eds.), Handbook of Research on Student Engagement (pp. 541–562). Boston, MA:
21
Inf

22 Springer US. Retrieved from http://dx.doi.org/10.1007/978-1-4614-2018-7_26Bruder, P.


23
24
or

25 (2015). Game on: Gamification in the classroom. Education Digest, 80(7), 56.
26
27
m

28 Challco, C. G., Moreira, A.D., Bittencourt, I., Mizoguchi, R., & Isotani, S. (2015).
29
at

30 Personalization of Gamification in Collaborative Learning Contexts using Ontologies.


31
32
ion

Latin America Transactions, IEEE (Revista IEEE America Latina), 13(6).


33
34
35 Chou, Y.-K. (2015, November 11). yukaichou.com. Retrieved from Yu Kai Chou: Gamification
36
37
an

38
and Behavioral Design: http://yukaichou.com/marketing-gamification/six-context-types-
39
40 rewards-gamification/
dL

41
42
43 Clements, J. (2015). Gamification: Freshman English can be a game. Edutopia, 1-3. Retrieved
44
ea

45 from http://www.edutopia.org/discussion/gamification-freshman-english-can-be-game
46
47
rn

48 Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic
49
50
ing

literature review of empirical evidence on computer games and serious games.


51
52
53 Computers & Education, 59(2), 661–686. https://doi.org/10.1016/j.compedu.2012.03.004
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 36 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 36
1
2
na
3 Craven, D. (2015). Gamification in virtual worlds for learning: A case study of PIERSiM for
4
5
6 business. In T. Reiners, & L. C. Wood (eds), Gamification in education and business (pp.
lJ
7
8 385-401). Switzerland: Springer.
ou
9
10
11 Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York, NY:
12
rn
13
14 Harper & Row.
al
15
16
17 Dale, S. (2014). Gamification: Making work fun, or making fun of work? Business Information
of
18
19 Review, 31(2), 82–90. Retrieved from http://doi.org/10.1177/0266382114538350
20
21
Inf

22 Davis, V. (2014, March 20). Gamification in education. Edutopia, 1-4. Retrieved from
23
24
or

http://www.edutopia.org/blog/gamification-in-education-vicki-davis
25
26
27
m

Davis, M. H., & McPartland, J. M. (2012). High School Reform and Student Engagement. In S.
28
29
L. Christenson, A. L. Rashly, & C. Wylie (Eds.), Handbook of Research on Student
at

30
31
32 Engagement (pp. 515–539). Boston, MA: Springer US. Retrieved from
ion

33
34 http://dx.doi.org/10.1007/978-1-4614-2018-7_25
35
36
37
de Freitas, A.A., de Freitas, M.M. (2013). Classroom live: A software-assisted gamification tool.
an

38
39 Computer Science Education 23(2), 186–206
40
dL

41 Deci, E. L. (1976). Notes on the theory and metatheory of intrinsic motivation. Organizational
42
43
Behavior and Human Performance, 15(1), 130–145.
44
ea

45
46 Deci, E. L., & Ryan, R. M. (2008). Facilitating optimal motivation and psychological well-being
47
rn

48 across life’s domains. Canadian Psychology/Psychologie Canadienne, 49(1), 14–23.


49
50
ing

https://doi.org/10.1037/0708-5591.49.1.14
51
52
53 Deterding, S. (2012, July). Social Mediator. Retrieved from Forum:
54
55
Te

56 https://www.cs.auckland.ac.nz/courses/compsci747s2c/lectures/paul/p14-deterding.pdf
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 37 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 37
1
2
na
3 Devedžić, V. &. Jovanović, J. (2015). Developing open badges: A comprehensive approach.
4
5
6 Educational Technology Research and Development, 63(4), 603-620. Retrieved from
lJ
7
8 http://doi.org/10.1007/s11423-015-9388-3
ou
9
10
11 Dornyei, Z., & Ottó, I. (1998). Motivation in action: A process model of L2 motivation.
12
rn
13 Retrieved from http://eprints.nottingham.ac.uk/39
14
al
15
16
Dörnyei, Z., & Ushioda, E. (2011). Teaching and researching motivation (2nd ed.). Pearson
17
of
18 Education Limited.
19
20 Eleftheria, C.A., Charikleia, P., Iason, C.G., Athanasios, T., Dimitrios, T. (2013). An Innovative
21
Inf

22 Augmented Reality Educational Platform using Gamification to Enhance Lifelong


23
24
or

25 Learning and Cultural Education. 4th International Conference on Information,


26
27 Intelligence, Systems and Applications, Greece (pp. 1–5).
m

28
29 everthing2com. (2015, November 27). everything2. Retrieved from everything2:
at

30
31
32 http://everything2.com/title/Bartle+Test
ion

33
34
35
Farber, M. (2013). Beyond badges: Why gamify? Edutopia, 1-4. Retrieved from
36
37 http://www.edutopia.org/blog/beyond-badges-why-gamify-matthew-farber
an

38
39
40 Folmar, D. (2015) Game it up: Using gamification to incentivize your library. Maryland:
dL

41
42 Rowman & Littlefield.
43
44
ea

Francisco-Aparicio, A., Guti'errez-Vela, F., Isla-Montes, J., & Sanches, J. (2013). Gamification:
45
46
47 Analysis and application. In V. Penichet, New trends in Interaction, Virtual Reality and
rn

48
49 Modeling, Human Computer Interaction Series (pp. 113-126). London: Springer-Verlag.
50
ing

51
52
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 38 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 38
1
2
na
3 Gee, J. P. (2014). Games, passion and “higher” education. In Tierney, W.G., Corwin, Z.B.,
4
5
6 Fullerton, T., Ragusa, G. (eds), Postsecondary play: The role of games and social media
lJ
7
8 in higher education (pp. 171-189). Baltimore: John Hopkins University Press.
ou
9
10
11 Gee, J. P. (2005). The new literacy studies: From’socially situated’to the work. Situated
12
rn
13 Literacies: Reading and Writing in Context, 2, 177–194.
14
al
15
16
Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York,
17
of
18 N.Y.: Palgrave Macmillan.
19
20
21 Gee, J. P. (1997). Situated sociocultural mind. In D. Kirshner, & J. Whiton, Situated cognition:
Inf

22
23 Social, semiotic and psychological perspectives (p. 10). New Jersey: Lawrence Erlbaum
24
or

25 Associates.
26
27
m

28 Gibson, D., Jakl, P. (2015). Theoretical considerations for game-based e-Learning analytics. In
29
at

30
31 T. Reiners, & L. C. Wood (eds), Gamification in education and business (pp. 403-416).
32
ion

33 Switzerland: Springer.
34
35
36 Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., Knight, E. (2013). Digital badges in
37
an

38 education. Education and Information Technology. Springer, New York


39
40
dL

Goehle, G. (2013). Gamification and web-based homework. Problems, Resources, and Issues in
41
42
43 Mathematics Undergraduate Studies 23(3), 234–246.
44
ea

45
46 González, C. S., Gómez, N., Navarro, V., Cairós, M., Quirce, C., Toledo, P., & Marrero-
47
rn

48 Gordillo, N. (2016). Learning healthy lifestyles through active videogames, motor games
49
50
ing

and the gamification of educational activities. Computers in Human Behavior, 55, 529–
51
52
53 551. Retrieved from http://doi.org/10.1016/j.chb.2015.08.052
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 39 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 39
1
2
na
3 Grant, S. (2013, March 6). Digital badges. Retrieved from hastac:
4
5
6 https://www.hastac.org/collections/digital-badges
lJ
7
8
ou
9 Griffiths, A.-J., Lilles, E., Furlong, M. J., & Sidhwa, J. (2012). The Relations of Adolescent
10
11 Student Engagement with Troubling and High-Risk Behaviors. In S. L. Christenson, A.
12
rn
13 L. Reschly, & C. Wylie (Eds.), Handbook of Research on Student Engagement (pp. 563–
14
al
15
16
584). Boston, MA: Springer US. Retrieved from http://dx.doi.org/10.1007/978-1-4614-
17
of
18 2018-7_27
19
20 Guthrie, J. T., Wigfield, A., & You, W. (2012). Instructional Contexts for Engagement and
21
Inf

22 Achievement in Reading. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.),


23
24
or

25 Handbook of Research on Student Engagement (pp. 601–634). Boston, MA: Springer


26
27 US. Retrieved from http://link.springer.com/10.1007/978-1-4614-2018-7_29
m

28
29 Hamari, J. K., Koivisto, J., & Sarsa, H. (2014). Does gamification work? A literature review of
at

30
31
32 empirical studies on gamification. IEEE, 3025–3034. Retrieved from
ion

33
34 http://doi.org/10.1109/HICSS.2014.377
35
36
37 Hanus, M. D. (2015). Assessing the effects of gamification in the classroom: A longitudinal
an

38
39 study on intrinsic motivation, social comparison, satisfaction, effort, and academic
40
dL

41
performance. Computers & Education, 80, 152–161. Retrieved from
42
43
44 http://doi.org/10.1016/j.compedu.2014.08.019
ea

45
46
47 Heick, T. (2011, December 15). The gamification of education: What schools can learn from
rn

48
49 video games. Edutopia, 1-4. Retrieved from http://www.edutopia.org/blog/gamification-
50
ing

51 education-terrell-heick
52
53
54 Herzig, P., Ameling, M., Wolf, B., & Schill, A. (2015). Implementing gamification:
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 40 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 40
1
2
na
3 Requirements and gamification platforms. In T. Reiners, & L. C. Wood (eds),
4
5
6 Gamification in education and business (pp. 431-471). Switzerland: Springer.
lJ
7
8 Hsieh, T.-L. (2014). Motivation matters? The relationship among different types of learning
ou
9
10 motivation, engagement behaviors and learning outcomes of undergraduate students in
11
12
rn
13
Taiwan. Higher Education, 68(3), 417–433. https://doi.org/10.1007/s10734-014-9720-6
14
al
15 Issacs, S. (2015). The difference between gamification and game-based learning. Edutopia, 1-8.
16
17 Retrieved from http://inservice.ascd.org/the-difference-between-gamification-and-game-
of
18
19
based-learning/
20
21
Inf

22 Jang, B. G., Conradi, K., McKenna, M. C., & Jones, J. S. (2015). Motivation: Approaching an
23
24
or

25 Elusive Concept Through the Factors That Shape It. The Reading Teacher, 69(2), 239–
26
27 247. https://doi.org/10.1002/trtr.1365
m

28
29 Kapp, K.M. (2012). Games, gamification, and the quest for learner engagement. Training and
at

30
31
32 Development 66(6), 64–68
ion

33
34
35
Kapp, K. (2013). The gamification of learning and instruction fieldbook: Ideas into practice.
36
37 N.Y.: Wiley.
an

38
39
40 Keeler, A. (2014). Beyond the worksheet: Playsheets, GBL, and gamification. Edutopia, 1-3.
dL

41
42 Retrieved from http://www.edutopia.org/blog/beyond-worksheet-playsheets-gbl-
43
44
ea

gamification-alice-keeler
45
46
47
rn

Keeler, A. (2015, April 22). Gamification: Engaging the students with narrative. Edutopia, 1-3.
48
49
50 Retrieved from http://www.edutopia.org/blog/gamification-engaging-students-with-
ing

51
52 narrative-alice-keeler
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 41 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 41
1
2
na
3 Kiang, D. (2014, Oct 14). Edutopia. Retrieved from Using gaming principles to engage students:
4
5
6 https://www.edutopia.org/blog/using-gaming-principles-engage-students-douglas-kiang
lJ
7
8
ou
9 Kim, A. J. (2014, January 15). Innovate with game thinking. Retrieved from Amy Jo Kim:
10
11 http://amyjokim.com/blog/2014/02/28/beyond-player-types-kims-social-action-matrix/
12
rn
13
14 Kim, Y., Glassman, M., & Williams, M. S. (2015). Connecting agents: Engagement and
al
15
16 motivation in online collaboration. Computers in Human Behavior, 49, 333–342.
17
of
18
https://doi.org/10.1016/j.chb.2015.03.015
19
20
21 Kingsley, T. L. & Grabner-Hagen, M.M. (2015). Gamification: Questing to integrate content,
Inf

22
23 knowledge, literacy, and 21st-century learning. Journal of Adolescent & Adult Literacy,
24
or

25 51-61. Retrieved from http://doi.org/10.1002/jaal.426


26
27
m

28 Koivisto, J. &. (2014). Demographic differences in perceived benefits from gamification.


29
at

30
31 Computers in Human Behavior, 35, 179–188. Retrieved from
32
ion

33 http://doi.org/10.1016/j.chb.2014.03.007
34
35
36 Kolb, L. (2015). Epic fail or win? Gamifying learning in my classroom. Edutopia, 1-5. Retrieved
37
an

38 from http://www.edutopia.org/blog/epic-fail-win-gamifying-learning-liz-kolb
39
40
dL

41 Kuh, G. D. (2009). The national survey of student engagement: Conceptual and empirical
42
43
foundations. New Directions for Institutional Research, 2009(141), 5–20.
44
ea

45
46 https://doi.org/10.1002/ir.283
47
rn

48 Kumar, B., & ParulKhurana. (2012, December). Gamification in education: Learn computer
49
50
ing

programming with fun. International Journal of Computers and Distributed Systems,


51
52
53 2(1), 46-53.
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 42 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 42
1
2
na
3 Lawley, E. (2012, July). Social Mediator. Retrieved from Forum:
4
5
6 https://www.cs.auckland.ac.nz/courses/compsci747s2c/lectures/paul/p14-deterding.pdf
lJ
7
8
ou
9 Leaning, M. (2015). A study of the use of games and gamification to enhance student
10
11 engagement, experience, and achievement on a theory-based course of an undergraduate
12
rn
13 media degree. Journal of Media Practice, 16(2), 155-170. Retrieved from
14
al
15
16
http://doi.org/10.1080/14682753.2015.1041807
17
of
18
Lee, J. J. & Hammer, J. (2011). Gamification in education: What, how, why bother? Academic
19
20
21 Exchange Quarterly, 15(2).
Inf

22
23
24 Lieberman, D. A. (2006). What can we learn from playing interactive games? In P. Vorder, & J.
or

25
26 Bryant, Playing video games, responses, and consequences (p. 397). Mahwah, NJ:
27
m

28 Lawrence Erlbaum Associates Inc.


29
at

30
31 Matera, M. (2010, November). Twitter.com. Retrieved from twitter: https://twitter.com/mrmatera
32
ion

33
34 McGonigal, J. (2011). Reality is Broken: Why games make us better and how they can change
35
36
37
the world. New York: Penguins Press.
an

38
39
40
McLaughlin, K. (2011, August 14). Steps in learning and teaching. Retrieved from Kevin
dL

41
42 McLaughlin: http://www.ictsteps.com/2011/08/20-time.html
43
44
ea

45 Nah, F. F.-H., Telaprolu, V. R., Rallapalli, S., & Venkata, P. R. (2013). Gamification in
46
47 education using computer games. In M. J. Smith, Human Interface and the Management
rn

48
49 of Information. Designing Information Environments (pp. 99-107). Las vegas, NV: HCI
50
ing

51
52 International.
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 43 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 43
1
2
na
3 Nicholson, S. (2015). A recipe for meaningful gamification. In T. Reiners, & L. C. Wood (eds),
4
5
6 Gamification in education and business (p. 1). Switzerland: Springer.
lJ
7
8
ou
9 O’Byrne, I., W., Schenke, K., WillisIII, J. E., & Hickey, D. T. (2015). Digital badges
10
11 recognizing, assessing, and motivating learners in and out of school contexts. Journal of
12
rn
13 Adolescent & Adult Literacy, 58(6), 451–454. https://doi.org/10.1002/jaal.381
14
al
15
16
O'keefe, D. (2012). Quest to Learn. School Library Journal, 58(12), 22.
17 Papastergiou, M. (2009). Digital game-based learning in high school computes science
of
18
19
20 education: Impact on educational effectiveness and student motivation. Computers &
21
Inf

22 Education, 52(1), 1-12. Retrieved from http://doi.org/10.1016/j.compedu.2008.06.004


23
24
or

25 Penichet, V. M. (2013). New trends in interaction, virtual reality and modeling. London:
26
27
m

Springer London. Retrieved from http://link.springer.com/10.1007/978-1-4471-5445-7


28
29
at

30 Pensky, M. (2001). Digital game-based learning. New York: McGraw Hill.


31
32
ion

33 Powers, K. L. Brooks, P. J., Aldrich, N. J., Palladino, M. A., & Alfieri, L. (2013). Effects of
34
35
36
video-game play on information processing: A meta-analytic investigation. Psychonomic
37
an

38 Bulletin & Review, 20(6), 1055–1079. Retrieved from http://doi.org/10.3758/s13423-013-


39
40 0418-z
dL

41
42
43 Quest to Learn. (2008, June). Institute of Play. Retrieved from Quest to Learn:
44
ea

45 http://www.instituteofplay.org/work/projects/quest-schools/quest-to-learn/
46
47
rn

48 Raymer, R. (2011, September). Gamification: Using game mechanics to enhance eLearning.


49
50
ing

51 Retrieved from eLearn Magazine: http://elearnmag.acm.org/featured.cfm?aid=2031772


52
53
54 Reeves, C. (2000). Alternative assessment approaches for online learning environments in higher
55
Te

56 education. Journal of Education Computing Research, 1, 101-111.


57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 44 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 44
1
2
na
3 Reiners, T., & Wood, L. C. (Eds.). (2015). Gamification in Education and Business. Cham:
4
5
6 Springer International Publishing. Retrieved from http://link.springer.com/10.1007/978-
lJ
7
8 3-319-10208-5
ou
9
10 Richter, Ganit, Raban , Daphne, & Rafaeli, Sheizaf. (2015). Studying gamification: The effect of
11
12
rn
13
rewards and incentives on motivation. In T. Reiners, & L. C. Wood (eds), Gamification
14
al
15 in education and business (pp. 21-46). Switzerland: Springer.
16
17
of
18 Robson, K. & Plangger, K. & Kietzman, J. H. & Mccarthy, L. & Pitt, L. (2015). Is it all a game?
19
20 Understanding the principles of gamification. Business Horizons, 58(4), 411–420.
21
Inf

22 Retrieved from http://doi.org/10.1016/j.bushor.2015.03.006


23
24
or

25 Russell, J., Ainley, M., & Frydenberg, E. (2005). Issues Digest: Motivation and engagement.
26
27
m

28 Australian Government: Department of Education, Science and Training. Retrieved


29
at

30 from Australian Government: Department of Education, Science, and Trading


31
32
ion

website:
33
34
35
http://www.dest.gov.au/sectors/school_education/publications_resources/schooling_
36
37 90 issues_digest/
an

38
39 Ryan, A. M. (2000). Peer groups as a context for the socialization of adolescents’ motivation,
40
dL

41
engagement, and achievement in school. Educational Psychologist, 35(2), 101–111.
42
43
44 Saeed, S., & Zyngier, D. (2012). How Motivation Influences Student Engagement: A Qualitative
ea

45
46 Case Study. Journal of Education and Learning, 1(2).
47
rn

48 https://doi.org/10.5539/jel.v1n2p252
49
50
ing

51 Schlechty, P. C. (2001). Shaking up the schoolhouse: How to support and sustain educational
52
53
54 innovation. San Francisco: Jossey-Bass.
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 45 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 45
1
2
na
3 Seaborn, K. & Fels. (2015). Gamification in theory and action: A survey. International Journal
4
5
6 of Human-Computer Studies, 74, 14-31. Retrieved from
lJ
7
8 http://doi.org/10.1016/j.ijhcs.2014.09.006
ou
9
10
11 Sheldon, L. (2011). Multiplayer classroom: Designing coursework as a game. Boston, MA:
12
rn
13 Cengage learning.
14
al
15
16 Simões, J., Redondo, R. D. & Vilas, A. F. (2013). A social gamification framework for a K-6
17
of
18
learning platform. Computers in Human Behavior, 29(2), 345-353.
19
20
21
Inf
Skinner, B. F. (1937). Two types of conditional reflex: A reply to Konorski and Miller. Journal
22
23
24 of General Psychology. 16, 272-279.
or

25
26
27 Skinner, B. F. (1984). The shame of American education. American Psychologist, 39(9), 947-
m

28
29 954.
at

30
31
32 Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of
ion

33
34 teacher behavior and student engagement across the school year. Journal of Educational
35
36
37
Psychology, 85(4), 571.
an

38
39 Staddon, J. E. R., & Cerutti, D. T. (2003). Operant Conditioning. Annual Review of Psychology,
40
dL

41 54(1), 115–144. https://doi.org/10.1146/annurev.psych.54.101601.145124


42
43
Stott, A., & Neustaedter, C. (2013). Analysis of gamification in education. Surrey, BC, Canada,
44
ea

45
46 8. Retrieved from http://clab.iat.sfu.ca/pubs/Stott-Gamification.pdf
47
rn

48 Thom, J., Millen, D., DiMicco, J., 2012. Removing gamification from an enterprise SNS. In:
49
50
ing

Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work.


51
52
53 Presented at CSCW'12. ACM, Seattle, WA, pp. 1067–1070.
54
55
Te

56 Todor, V., & Pitică, D. (2013). The gamification of the study of electronics in dedicated e-
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
International Journal of Information and Learning Technology Page 46 of 47
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 46
1
2
na
3 learning platforms. In Proceedings of the 36th International Spring Seminar on
4
5
6 Electronics Technology (pp. 428–431). IEEE. Retrieved from
lJ
7
8 http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6648287
ou
9
10 Waggoner, Z. (2009). My Avatar, my self: Identity in video role-playing games. North Carolina:
11
12
rn
13
McFarland & Company.
14
al
15
16
Whitton, N. & Moseley, Alex. (2010). Using games to enhance learning and teaching: A
17
of
18 beginner's guide. London: Taylor and Francis.
19
20
21 Wigfield, A., Byrnes, J. P., & Eccles, J. S. (2006). Development during early and middle
Inf

22
23 adolescence. In P. Alexander, & P. Winne (Eds.), Handbook of educational psychology
24
or

25 (2nd Ed.). New York: Macmillan Publishing.


26
27
m

28 Willms, J. D. (2003). Student engagement at school: A sense of belonging and participation:


29
at

30 Results from PISA 2000. Publications de l’OCDE.


31
32
ion

Wilson, L. (2003). Interactivity or interpassivity: a question of agency in digital play. In Fine Art
33
34
35
Forum (Vol. 17). Retrieved from
36
37 http://www.academia.edu/download/8517885/wilson.pdf
an

38
39 Yue, C. L., Bjork, E. L., & Bjork, R. A. (2013). Reducing verbal redundancy in multimedia
40
dL

41
learning: An undesired desirable difficulty? Journal of Educational Psychology, 105(2),
42
43
44 266–277. https://doi.org/10.1037/a0031971
ea

45
46 Zhang, D., Clear, T. (2015). Shaping behaviours through space and place in gamified virtual
47
rn

48 learning environments. In T. Reiners, & L. C. Wood (eds), Gamification in education and


49
50
ing

51 business (pp. 331-354). Switzerland: Springer.


52
53
54 Zichemann, G. & Cunningham, Christopher. (2011). Gamification by design: Implementing
55
Te

56 game mechanics in web and mobile apps. CA: O'Reilly Media Inc.
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no
Page 47 of 47 International Journal of Information and Learning Technology
ti o
GAMIFICATION: MOTIVATION AND ENGAGEMENT 47
1
2
na
3 Zichermann, G. (2011, July 6). Mashable. Retrieved from Mashable.com:
4
5
6 http://mashable.com/2011/07/06/7-winning-examples-of-game-mechanics-in-
lJ
7
8 action/#PkEftCunR8qz
ou
9
10
11 Zickermann, G. (2010, October 26). Fun is the future: Mastering gamification. Google Tech
12
rn
13 Talk. San francisco, CA, U.S.A.: Google Tech Talk. Retrieved from
14
al
15
16
https://www.youtube.com/watch?v=6O1gNVeaE4g
17
of
18
19
20
21
Inf

22
23
24
or

25
26
27
m

28
29
at

30
31
32
ion

33
34
35
36
37
an

38
39
40
dL

41
42
43
44
ea

45
46
47
rn

48
49
50
ing

51
52
53
54
55
Te

56
57
58
ch

59
60 http://mc.manuscriptcentral.com/cwis
no

View publication stats

Potrebbero piacerti anche