Computers & Security 96 (2020) 101875

Contents lists available at ScienceDirect

Computers & Security

journal homepage: www.elsevier.com/locate/cose

Cultivating cybersecurity learning: An integration of

self-determination and flow
Hwee-Joo Kam a, Philip Menard b, Dustin Ormond c, Robert E. Crossler d,∗
a
Sykes College of Business, University of Tampa, 401W. Kennedy Blvd., Tampa, FL 33606, United States
b
College of Business, The University of Texas at San Antonio , One UTSA Circle , San Antonio, TX 78249, United States
c
Heider College of Business, Creighton University, 2500 California Plaza, Omaha, NE 68178, United States
d
Carson College of Business, Washington State University, PO Box 644743, Pullman, WA 99164-4743, United States

a r t i c l e i n f o a b s t r a c t

Article history: Because there is a critical shortage of cybersecurity talent, information security professionals and re-
Received 4 December 2019 searchers should cultivate cybersecurity skills by encouraging individuals to pursue cybersecurity learn-
Revised 21 April 2020
ing. However, some aspects of cybersecurity require substantial effort and perseverance for conceptual
Accepted 12 May 2020
understanding to be gained. We propose motivation as the key to ensuring continuous engagement with
Available online 23 May 2020
and successful learning of such cybersecurity concepts. With a lab-based training program that taught
Keywords: participants about SQL injection attacks, we tested a research model that integrated flow theory and self-
Flow theory determination theory. Within the training program, we captured participants’ persistence in attempting
Self determination theory and successfully completing the training exercises while also measuring their perceptions of motivation
Cybersecurity learning and flow. We found that flow facilitated motivation and its key antecedents. Flow and task significance
Motivation had the strongest effects on motivation, while motivation fostered learning persistence and performance.
Cygotobersecurity talent
We recommend training programs that maximize flow and task significance.
Survey
© 2020 Elsevier Ltd. All rights reserved.

1. Introduction ees could be retrained for cyber operations (Cordell, 2018). In a

An (ISC)2 Cybersecurity Workforce Study (Beuran et al., 2018)
revealed that there is a shortage of about three million cyber-
security professionals globally. Since then, the impacts related
to the workforce shortage have only become more pronounced
(Oltsik, 2019). When there is a lack of personnel who are knowl-
edgeable in applying information security (InfoSec) safeguards, or-
ganizational information assets will be vulnerable to cyberattacks
((ISC)2 Cybersecurity Workforce Study, 2018). Due to the critical-
ity of this issue, the President of the United States (U.S.) issued
Executive Order 13,800 to expand the cybersecurity workforce
(U.S. Department of Homeland Security, 2017) and subsequently
signed Executive Order 13,870 (Executive Office of the President,
Abraham and Chengalur-Smith, 2019) to further incentivize the de-
velopment of competent cybersecurity personnel.
One potential pathway to addressing the employee shortage is
to retrain current non-cyber professionals. The U.S. National Se-
curity Council reported an interest in using organizational train-
ing and aptitude examinations to decide which current employ-
∗
Corresponding author.
E-mail addresses: hkam@ut.edu (H.-J. Kam), philip.menard@utsa.edu (P. Menard),
DustinOrmond@creighton.edu (D. Ormond), rob.crossler@wsu.edu (R.E. Crossler).
similar effort, the Office of Management and Budget launched the
Federal Cyber Reskilling Academy, which is a partnership with the
SANS Institute that will train the current federal workforce on an
array of cybersecurity skills in line with Executive Order 13,870
(Heckman, 2018). Such initiatives leverage current employees’ ex-
perience with bureaucracy and governmental policy while also cul-
tivating diverse professional backgrounds (Power, 2019).
The retraining effort will likely also encompass employees
already serving in cybersecurity roles. Seventy percent of cyberse-
curity professionals are open to new opportunities, with fifty-nine
percent reporting that they desire to work for an organization
that trains employees on cybersecurity (Security Magazine, 2018).
Although training and professional development opportunities are
desirable to cybersecurity employees, a separate report found that
most employees are already offered training as an incentive to
stay in their current organizations (Downs, 2019). These findings
indicate that although cybersecurity training may provide an
avenue for organizations to retain or repurpose its existing em-
ployees in the short term, the training offered must move beyond
basic techniques and concepts to maintain a competent and robust
workforce in the long term. Indeed, the U.S. National Initiative
for Cybersecurity Education (NICE) stated that “an integrated
cybersecurity workforce must be capable of designing, developing, im-

https://doi.org/10.1016/j.cose.2020.101875
0167-4048/© 2020 Elsevier Ltd. All rights reserved.
2 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875
plementing, and maintaining defensive and offensive cyber strategies.”
(NIST Special Publication 800–181, 2017, p. iv) These are long term
goals, and the training that teaches employees about cybersecurity
should likewise tap into techniques that have long term impacts.
Cybersecurity requires individuals to understand both the tech-
nological perspective (e.g., database, software, computer program-
ming, and networking) and the organizational and human per-
spective (e.g., administrative controls) (González-Manzano and de
Fuentes, 2019; NIST Special Publication 800–181, 2017). With such
a convoluted subject, motivation in support of learning persistence
(Vansteenkiste et al., 2004) may be an essential building block to-
ward cultivating a pool of cybersecurity talent for addressing the
critical workforce shortage.
Motivation has been shown to increase cognition, attitudes, and
positive behavior in short-term situational training programs and
have lasting effects in the long term (Vallerand, 1997). For em-
ployees who are being retrained to effectively learn cybersecu-
rity in the short term and be driven to continue learning cy-
bersecurity over time, training programs must adequately sat-
isfy the motivations of its employees. A widely used theoreti-
cal model for motivation is self-determination theory (Deci and
Ryan, 1985, 1980; Ryan and Deci, 20 0 0). Self-determination the-
ory proposes a positive change in learning outcomes through the
satisfaction of basic psychological needs (e.g., needs of being au-
tonomous, feeling competent, and experiencing emotional connec-
tion) (Deci and Ryan, 1980). Motivation toward cybersecurity learn-
ing, which is critical for solving cybersecurity workforce shortage,
has largely been ignored by prior research. Integrating this frame-
work into organizational-based cybersecurity training may likewise
show positive results. Therefore, we use self-determination theory
to examine the following research question:
RQ1: How are employees in cyber or cyber-adjacent fields moti-
vated to learn cybersecurity?
While the impacts of motivation as conceptualized through self-
determination theory largely address long term impacts by sat-
isfying psychological needs, employees in a cybersecurity train-
ing program must be enticed to engage with the materials early
in the training. Otherwise, the employee will not have the op-
portunity to continuously experience the critical factors embed-
ded in the training that is designed to bolster self-determination.
Flow theory highlights instant gratification (e.g., hedonistic out-
comes) (Csikszentmihalyi, 1990) and could be an essential com-
plement to self-determination in a cybersecurity training context.
Self-determination theory and flow theory balance each other by
presenting different perspectives of motivation. Thus, we pose the
following research question:
RQ2: How does flow influence employees’ motivation to learn cy-
bersecurity?
This study presents the motivating factors that drive individu-
als to engage in cybersecurity learning and addresses the critical
shortage of cybersecurity talents through some viable suggestions
that promote cybersecurity learning. This paper is organized as fol-
lows. First, we build our theoretical foundation to develop a theo-
retical framework. Next, we detail our research design and present
our data analysis. We then discuss our research findings. Finally,
we share our research implications, limitations, and conclusion.
2. Literature review
2.1. Cybersecurity learning
The U.S. National Initiative for Cybersecurity Education (NICE)
framework, directed by the National Institute of Standards and
Technology (NIST) in the U.S. Department of Commerce, proposes
that, to prepare individuals for the workforce, cybersecurity learn-
ing should include the core knowledge consisting of Securely Pro-
vision (i.e., securing IT infrastructure), Operate and Maintain (i.e.,
secure system administration), Oversee and Govern (i.e., security
leadership and management), Protect and Defend (i.e., mitigating
cyber threats), Analyze (i.e., specialized reviews of IT infrastruc-
ture for cyber intelligence), Collect and Operate (i.e., use deception
to gather cyber intelligence), and Investigate (i.e., investigate cy-
bercrimes through digital forensic) (NIST Special Publication 800–
181, 2017). This suggests that cybersecurity learning requires a high
cognitive capacity to master the complex concept of software and
network security, involves analytical skillsets for big data analy-
sis, and combines managerial learning with technical knowledge
in management decision-making. Overall, cybersecurity learning is
multidisciplinary, relying on computing infrastructure, policies, and
people. With such a complex subject matter, learners may experi-
ence some difficulties to grasp the intricate concepts of cyberse-
curity. Therefore, learners’ motivation is very important to ensure
success in cybersecurity learning.
2.2. Integrated framework
Numerous studies (see Table 1) have examined individuals’ mo-
tivation in a learning environment context (e.g., e-learning or or-
ganizational settings). However, the motivation to learn cyberse-
curity, which is presently an important topic for addressing the
critical shortage of cybersecurity talent, has largely been ignored.
As noted, the challenges in cybersecurity learning may not ease
the critical shortage of cybersecurity talent in both the private and
public sectors. These challenges may act as a deterrent for learn-
ers who may otherwise be interested in the area. Therefore, it is
important to motivate learners to be persistent in cybersecurity
learning.
To study learners’ motivation in cybersecurity learning, we inte-
grate self-determination theory (Deci and Ryan, 1985, 1980; Gagné
and Deci, 2005) and flow theory (Csikszentmihalyi, 1990). This in-
tegrated framework investigates the potential factors that influence
the motivation of cybersecurity learning. Self-determination theory
proposes that intrinsic motivation is stemmed from humans’ basic
needs of feeling competent, autonomous, and emotionally invested
while performing activities (Deci and Ryan, 1985). Meanwhile, flow
theory posits that when individuals engage in a challenging activ-
ity that matches their abilities, they will experience “flow” (i.e.,
become immersed in the activity), leading to intrinsic motivation
(Csikszentmihalyi, 1990).
Both self-determination theory and flow theory present a dif-
ferent perspective of motivation. Self-determination theory focuses
on innate human psychological needs, and flow theory highlights
enjoyment and fun (Eccles and Wigfield, 2002). Conversely, self-
determination theory is designed to address the ultimate outcomes
of cybersecurity learning (e.g., the innate human need of feeling
competent), while flow theory emphasizes more immediate sat-
isfaction (e.g., hedonistic outcomes) (Eccles and Wigfield, 2002).
In this respect, both theories complement each other (Eccles and
Wigfield, 2002) to provide a strong theoretical foundation for this
study. Hence, we integrate both theories to examine how hedonis-
tic factors entice learners to study cybersecurity (i.e., flow theory)
and investigate how to sustain their learning processes based on
innate human needs (i.e., self-determination theory). An integra-
tion of both theories provides multiple dimensions in the study of
self-determined motivation and serves as the comprehensive theo-
retical foundation for the remainder of this paper.
2.2.1. Self-determination theory
Self-determination theory was developed in response to prior
motivational research that classified intrinsic and extrinsic moti-
vation as dichotomous constructs. Self-determination theory of-
fered a more granular perspective of motivation by defining sev-
 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875 3

Table 1
Summary of literature review.

Research study Theoretical framework Research findings

Effects of extrinsic and intrinsic motivation on employee
knowledge sharing intentions (Lin, 2007)
The effects of an organizational learning culture, perceived
job complexity, and proactive personality on organizational
commitment and intrinsic motivation (Joo and Lim, 2009)
Organizational networks and the process of corporate
entrepreneurship: How the motivation, opportunity, and
ability to act affect firm knowledge, learning, and innovation
(Turner and Pennington, 2015)
Motivation in online learning: Testing a model of
self-determination theory (Chen and Jang, 2010)
Simulated computer-mediated/video-interactive distance
learning: A test of motivation, interaction satisfaction,
delivery, learning & perceived effectiveness (Guzley et al.,
2001)
The effects of peer intrinsic and extrinsic motivation on
MMOG game-based collaborative learning (Kong et al., 2012)
Motivation theory & theory of
reasoned action (Fishbein and
Ajzen, 1975)
Organizational theories & motivation
theory
Organizational theories & motivation
theory
Self-determination theory (Deci and
Ryan, 1985)
Theoretical framework related to
learners’ interactions
Motivational theories
Intrinsic and extrinsic motivation drive one to share
knowledge for organizational learning.
Perceived job complexity and proactive personality
engender intrinsic motivation in organizations. Perceived
job complexity also has a mediating effect on intrinsic
motivation.
The long-term orientation of promoting knowledge
sharing motivates employees to share knowledge.
Consequently, this facilitates organizational learning.
There is a mediating effect of need satisfaction between
contextual support and motivation for learning.
A relationship exists between learners’ motivation and
class participation in distance learning. The innovative
instructional design also supports learners’ participation.
One’s peer intrinsic and extrinsic motivations generated a
positive impact on one’s intention to learn.

eral forms of extrinsic motivation. Individuals’ extrinsic moti-
vation differs based on the degree of internalization, or self-
determination, exhibited within the motivational type (Deci and
Ryan, 1980; Gagné and Deci, 2005; Ryan and Deci, 20 0 0). The
specific types of motivation on the self-determined spectrum
are (ordered from least internalized): external regulation, intro-
jected regulation, identified regulation, and integrated regulation
(Vallerand, 1997). External regulation refers to an individual be-
ing motivated purely by rewards or sanctions (Vallerand 1997). In-
trojected regulation is related to an individual’s motivation being
driven by the opinions of important others, such as the attain-
ment of pride associated with verbal praise or the avoidance of
shame associated with verbal reprimands (Vallerand 1997). Iden-
tified regulation is the motivation to perform some behavior that
is not self-determined but may lead to the involvement in some
other internalized behavior (i.e., an externalized means to an in-
ternalized end) (Vallerand 1997). Integrated regulation is the moti-
vation to perform a behavior as an extension of oneself (Vallerand
1997).
We adopt situational motivation (i.e., task-based motivation) in-
cluding four types of motivation along the self-determined spec-
trum: intrinsic motivation, identified regulation, external regula-
tion, and a motivation. The values of these four types of motivation
are computed to yield a single composite index (Vallerand, 1997),
in which a high value tilts to intrinsic motivation via an aggre-
gation of the different degrees of self-determination. While a sin-
gle index showed high reliability and validity (Fortier et al., 1995),
Vallerand (1997) advised applying it appropriately in a study. We
argue that a composite index does not pin motivation down as a
dichotomous construct of intrinsic vs. extrinsic motivation, thus of-
fering a comprehensive view of motivation.
2.2.2. Flow theory
According to flow theory, flow is defined as “the state in which
people are so involved in an activity that nothing else seems to mat-
ter; the experience itself is so enjoyable that people will do it even
at great cost, for the sheer sake of doing it” (Csikszentmihalyi, 1990,
pg. 4). Essentially, a person in a state of flow experiences the fol-
lowing: (1) temporal dissociation: the transformation of time in that
the passage of time is different (either faster or slower) than the
absolute time according to a clock, (2) focused immersion: the com-
plete absorption of attention in that nothing else seems to matter,
(3) heightened enjoyment: the pleasure and enjoyment in that bio-
logical or social expectations have been met (pleasure) and extend
beyond what is expected or imagined (enjoyment), and (4) control:
a sense of control in that worries that normally preoccupy the in-
dividual’s attention seem to vanish (Csikszentmihalyi, 1990).
In a state of flow, an individual’s attention will be consumed
by the object of attention (Csikszentmihalyi, 1990). Flow not
only occurs with physical activity but also with symbolic sys-
tems such as attitudes toward information systems (Agarwal and
Karahanna, 20 0 0) and information systems adoption decisions
(Trevino and Webster, 1992; Zhang et al., 2006). Individuals who
enjoy activities such as ethical hacking will feel strong hedonic
processes that they stop being aware of themselves as separate
from the actions they are performing. For these activities to move
from pleasure to enjoyment, individuals need to dedicate an un-
usual amount of attention and interaction (Csikszentmihalyi, 1990)
with the technology itself. For example, flow motivated novice
hackers to hack (Voiskounsky and Smyslova, 2003). When individ-
uals with ethical hacking knowledge comparable to novice hackers
designate more attention to learn hacking skills, they will quickly
experience flow that leads to greater enjoyment. This enjoyment
could then enhance factors that motivate individuals to engage in
cybersecurity learning.
3. Theoretical framework
Fig. 1 below presents a proposed research model.
3.1. Situational motivational determinants
Drawing on self-determination theory, competence refers to the
innate human need to attain a sufficient understanding of a subject
through overcoming challenges (Ryan and Deci, 20 0 0). Roca and
Gagné (2008) empirically demonstrated that perceived skill com-
petence (i.e., individuals’ perceptions of their competency) intrin-
sically motivated individuals to learn. In a cybersecurity learning
context, perceived skill competence refers to individuals’ percep-
tions about their abilities in acquiring cybersecurity knowledge.
We suggest that individuals who perceive that they have a high
level of competence will be intrinsically motivated to learn cyber-
security. We then propose:
H1: Perceived skill competency positively affects motivation to
learn cybersecurity.
Perceived learning autonomy pertains to individuals’ percep-
tion of the extent of control over preferred topic areas and learn-
ing styles. Self-determination theory posits that greater perceived
learning autonomy raises individuals’ self-determination (Deci and
Ryan, 1980). Specifically, individuals who feel that they have the
4 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875
Fig. 1. Proposed research model.
freedom to learn more about certain topics of interest will in-
ternalize the learning process (Gagné and Deci, 2005), resulting
in greater intrinsic motivation to learn. Abraham and Chengalur-
Smith (2019) also showed perceived learning autonomy produced
positive outcomes in information security training. Thus, individ-
uals will be motivated to learn cybersecurity when they perceive
that they can control their learning process. We then theorize:
H2: Perceived learning autonomy positively affects motivation to
learn cybersecurity.
Another antecedent to motivation as described by self-
determination theory is perceived relatedness. Perceived related-
ness refers to an individual’s perceived emotional connection to
other related individuals or objects (Ryan and Deci, 20 0 0). When
individuals have an elevated perception of relatedness (e.g., an
emotional connection to certain people, things, or tasks), they ex-
hibit a higher degree of internalization (Gagné et al., 1997; Gagné
and Deci, 2005). Prior research has conceptualized relatedness as
the interpersonal connection among trainees in a learning session
(Niemiec and Ryan, 2009), but this notion does not fit particularly
well with the context of cybersecurity programs intended to bol-
ster national or organizational security. Rather, we argue that per-
ceived emotional connection is derived from the desire to protect
one’s nation or organization and is typically manifested through
an elevated perception of the significance of protective behaviors.
Under this premise, we adapt the task significance construct, as
conceptualized from Job Characteristics Theory (JCT) (Hackman and
Oldham, 1976).
JCT posits that when an individual perceives a task to be signif-
icant, the individual will be more motivated to perform the task.
Significance, as perceived by the individual, may materialize in
a variety of ways including personal relevance, organizational ef-
fects, or impact on the community at large. Task significance rep-
resents another theoretical similarity to self-determination theory
that may apply to task-based situations regarding cybersecurity.
For example, based on the emotional connection with an orga-
nization, individuals may internalize behaviors that improve the
well-being of the organization. Similarly, when individuals perceive
their work on certain tasks has positive effects on the commu-
nity, they will be intrinsically motivated to perform these tasks
(Hackman and Oldham, 1976). By exploring motivation related to
cybersecurity tasks, this study offers a research context that is
highly significant in terms of the overall impact on the community.
Cybersecurity is often considered the next frontier in global war-
fare as protecting national information assets is critical to the con-
tinued success and security of the country. In our research, people
will be more motivated to learn about cybersecurity when they
perceive their cybersecurity skills will help protect their country
or community. We then propose:
H3: Perceived task significance positively affects motivation to en-
gage in cybersecurity learning.
3.2. Impact of flow on self-determined motivation
People who are in a state of flow feel intrinsically motivated
to change and do things beyond the present and into the future
(Csikszentmihalyi, 1990). As they find the experience so enjoyable
(Csikszentmihalyi, 1990), individuals who enter a state of flow will
discover that their activities are intrinsically rewarding (Keller and
Bless, 2008), motivating them to engage in the activity even fur-
ther. The reverse is true as well; as an individual experiences anx-
iety and boredom, he or she has a significant decrease in moti-
vation and focus (Murphy et al., 2013). Liao (2006) demonstrated
that flow positively influenced students to participate in distance
learning, and Ghani and Deshpande (1994) determined that flow
motivated individuals to use computers in the workplace.
Prior studies have empirically established that novice hackers
who experience flow will feel motivated to hack (Voiskounsky and
Smyslova, 2003). Since learners have little knowledge of ethical
hacking, their knowledge base is very similar to that of novice
hackers. This suggests that, when these learners experience flow,
they will be more motivated to learn cybersecurity (e.g., ethical
hacking) because they will find learning intrinsically rewarding.
Hence, we propose:
H4: Flow is positively associated with motivation to engage in cy-
bersecurity learning.
3.3. Impact of flow on situational motivational determinants
Compatibility between one’s skill set and the demands of the
task sustains flow (Keller and Bless, 2008). Flow propels individ-
uals to increase their competency (Csikszentmihalyi and Massi-
mini, 1985) to face tougher challenges to alleviate boredom from
 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875
the former and seemingly less difficult challenges. For example,
learners who experienced flow while using an e-learning system
were more likely to explore the system functions and increase
their knowledge of the system (Liao, 2006). Similarly, once cyber-
security learners are “hooked” to learn ethical hacking, they will
expand their hacking skills to face greater challenges. We then pro-
pose:
H5: Flow is positively associated with perceived skill competency
of cybersecurity learning.
Flow also enhances perceptions of autonomy concerning an
individual’s actions (Fullagar and Kelloway, 2009; Kowal and
Fortier, 1999; Mills and Fullagar, 2008). For example, individuals
who are fully immersed in playing video games need a higher level
of autonomy to feel they are in full control of the game. As the
complexity of the game unfolds, immersion leads to the need for
controlling more complex movements of a character and is critical
to perceptions of autonomy. For cybersecurity learning, individu-
als who immerse themselves in a complex topic require a higher
sense of autonomy to decide their own learning. Such autonomy
may include the freedom to decide one’s learning schedule. Hence,
we propose:
H6: Flow is positively associated with perceived learning autonomy
in cybersecurity.
The concept of task significance is rooted in the perceived im-
portance or relevance an individual develops at an emotional level
during an activity. Immersion in a particular activity will result in
an emotional connection with others through shared experiences
as well as with the activity itself and its overall purpose, hope-
fully for some greater good. For example, in an online gaming en-
vironment, gamers communicate and form connections with oth-
ers during gameplay, especially in games featuring team-based ap-
proaches (Teng et al., 2012). Individuals will also form an emo-
tional connection with the game due to the hedonic benefits expe-
rienced during flow, thus increasing the perceived significance of
the given task. Eventually, this flow experience may result in on-
line gamers feeling their gaming activities create a positive impact
on the online gaming community. Similarly, individuals who reach
a state of flow in cybersecurity training programs may gradually
form positive perceptions about the significance and importance of
cybersecurity to a nation’s or an organization’s well-being. Hence,
we propose:
H7: Flow is positively associated with perceived task significance
of cybersecurity learning.
3.4. Actual learning behavior
Motivation affects attitude, cognition, and behavior in a variety
of research contexts and often results in increases in positive affect
and self-esteem (Chan and Ahern, 1999). For example, motivation
in education leads to continued learning (Vallerand, 1997). In con-
gruence with prior studies (Vansteenkiste et al., 2004), individuals
who participate in a cybersecurity learning program may become
more intrinsically motivated and exhibit positive learning behav-
iors (e.g., persistence) that ultimately end the successful comple-
tion of the program. Hence, we propose:
H8: Motivation to engage in cybersecurity learning facilitates
the actual learning persistence.
Motivation also leads to more effective cognitive processing
such that learning efforts are strategically managed (Pintrich and
De Groot, 1990) and performance is improved. This is evident in
fundraising among callers (Grant, 2008) and academic performance
(Fortier et al., 1995; Giesbers et al., 2013; Schmidt, 2005). In cyber-
security learning, motivated individuals may begin acquiring net-
working skills to serve as a foundation and contribute to the mas-
tery of more advanced ethical hacking skills. Thus, we propose:
5
H9: Motivation to engage in cybersecurity learning facilitates
the actual learning performance.
4. Research methodology
To thoroughly examine the efficacy of incorporating motivation
and flow into our theoretical model, we geared our study toward
one of the harder, more technical concepts from the NICE frame-
work (NIST Special Publication 800–181, 2017). Ethical hacking em-
ulates real-life cyberattack scenarios to discover system vulnera-
bilities. However, unlike traditional (or black-hat) hacking, ethical
(or white-hat) hacking prevents rather than exploits vulnerabili-
ties. We argue that framing cybersecurity training with an ethical
hacking approach promotes an understanding of system vulnerabil-
ities (i.e., cyber offense) and on identifying the best security coun-
termeasures to mitigate these vulnerabilities (i.e., cyber defense)
(Beuran et al., 2018). This suggests that ethical hacking incorpo-
rates the key components of cybersecurity learning (NIST Special
Publication 800–181, 2017), making it relevant to our study. Ad-
ditionally, by building our study around more challenging con-
cepts, we should be able to apply our findings toward training built
around less difficult concepts, thereby broadening the study’s gen-
eralizability. By designing our training around an ethical hacking
technique, we observed optimal variance in the impact of motiva-
tion and flow on our cybersecurity training outcomes.
We ran an online laboratory experiment followed by an on-
line survey to decide each subject’s (N = 133) level of motiva-
tion to engage in ethical hacking. Our subjects were undergrad-
uate students who were primarily majoring in management in-
formation systems (MIS) with a good majority declaring a sec-
ond major in the business discipline. Most of them are either ju-
niors or seniors. The average age was 24 years old. There were
104 males and 29 females. Although our sample skews heavily
toward male participants (78%), our respondent pool is reflective
of the current state of the cybersecurity workforce, which is be-
tween 20% (Morgan, 2018) and 24% ((ISC)2 Cybersecurity Work-
force Study, 2018) female. Our subjects were enrolled in a database
course where they were asked to participate in this study after dis-
cussing the importance of data security. They were incentivized
to participate by receiving a nominal amount of extra credit to
their course grade. Additionally, students were encouraged to par-
ticipate in the study to help the researchers with the project re-
gardless of the extra credit. Participation was completely voluntary.
Most students who participated had good grades (A’s or B’s) at the
time of participation. Furthermore, 77% of the students continued
to attempt problems beyond the minimum required number of at-
tempts. Students formed an appropriate sample for this study be-
cause they are majoring in technology-related fields and lack the
skills necessary to perform security-related tasks. This is similar to
people in the workforce that would need to develop cybersecurity
skills to take on security roles within an organization.
This study involved SQL injections (SQLi) where attackers use
malicious Structured Query Language (SQL) to manipulate the
database server to do something other than intended. Three SQLi
exercises were built with PHP scripts that connected to a MySQL
database server. Subjects who agreed to participate were asked to
execute SQLi attacks against these PHP pages. Most of the subjects
had a basic understanding of relational databases and web appli-
cation development, but they had very little knowledge of SQLi.
SQLi attacks require an understanding of the database structure,
SQL, database security, and the frontend web user interface. Be-
cause SQLi attacks demand an increased understanding of IT which
often requires high amounts of motivation to attain, the challenge
to successfully run an SQLi attack can feasibly be used to examine
motivation.
6 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875
Fig. 2. Feedback for an error in SQL injection attempt.
Subjects were invited to participate in this study via email. They
were first presented with an electronic consent form followed by
an introductory survey and an online video that presented the im-
portance of ethical hacking. A subject could either watch or skip
the video. Then, all subjects were presented with a problem and
were asked to run an SQLi. The subjects either successfully ran an
SQLi or tried at least 10 times. After the 10th attempt, an “I Give
Up!” button appeared where the subject could continue with more
attempts or end the exercise. In each attempt, the subjects could
refer to videos on how to launch an SQLi. They were also free to
use any Internet resources. Moreover, subjects received feedback if
they made a mistake when running an attack (see Fig. 2). Upon
a successful attack, the subject was presented with a harder ex-
ercise. This cycle continued for three exercises. If at any time the
subject clicked the button to give up or completed the last SQLi,
they were redirected to a final Qualtrics survey that evaluated the
perceptions of skill competency, task significance, autonomy, moti-
vation, and flow. All instrument items were randomized to reduce
order effects (Podsakoff et al., 2003).
The difficulty of these exercises has been consistently demon-
strated based on prior focus groups (i.e., students) and their rank-
ing of the difficulty level of each SQLi exercise. Hence, we pre-
sented the exercises in order based on the difficulty level. All par-
ticipants had a general understanding of select statements and
a less foundational understanding of update statements. We de-
signed the exercises by referring to the ethical hacking exer-
cises implemented by the Open Web Application Security Project
(OWASP) (OWASP, 2015). The first exercise (i.e., string SQLi) eval-
uated the participants’ general knowledge regarding select state-
ments and their ability to take advantage of an unprotected se-
lect statement. The second exercise (i.e., stacked SQLi) evaluated
the participants’ knowledge of update statements and their abil-
ity to run multiple queries instead of a single query as expected.
The final exercise (i.e., blind SQLi) assessed the participants’ ability
to run multiple queries to guess and check the results and narrow
down to one specific value.
The actual learning persistence (PERS) of the subjects was cap-
tured by summing up the number of attempts in all the exercises.
We argue that the number of SQLi attempts represents the inten-
sity of one’s commitment, which eventually denotes one’s learn-
ing persistence (Meyer et al., 2004). We also assessed the actual
learning performance (PERF) by tracking the number of exercises
completed by the subjects. As the subjects progressed, the next
exercise became harder, so overcoming a harder exercise signifies
learning performance.
4.1. Measurement items
Using an online survey, we measured latent constructs us-
ing a fully anchored 7-point Likert scale (i.e., 1= strongly dis-
agree, 4=neutral, and 7=strongly agree). All measurement items
were adapted from prior studies (Agarwal and Karahanna, 20 0 0;
Hackman and Oldham, 1976; Vallerand, 1997) (see Appendix A).
We first ran a pilot study (N = 37). The results of our pilot study
were not included in this study. Additionally, participants in this
study were different from those in the pilot study. Overall, the
feedback we received from our pilot study enabled us to refine the
questionnaires by removing any ambiguities. The feedback also al-
lowed us to improve our questionnaires by using the best practice
(i.e., wordings and order) suggested by Dillman et al. (2014). Fi-
nally, Perceived task significance (TS), autonomy (AUT), skill com-
petency (COMP), actual learning persistence (PERS), actual learn-
ing performance (PERF), and situational motivation (MOV) were
first-order reflective constructs. FLOW was a second-order reflec-
tive factor defined by four first-order reflective constructs: tem-
poral dissociation (TEMP), focus immersion (IMR), heightened en-
joyment (ENJ), and control (CTL). MOV is measured by first reflec-
 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875 7

Table 2 4.3. Data analysis

Composite reliability (CR) and Cronbach’s alpha (CA).

Construct CA CR Construct CA CR We ran bootstrapping with 10 0 0 re-sampling for hypothe-

AUT 0.832 0.900 IMR 0.915 0.946 ses testing (Hair et al., 2016) because 10 0 0 resampling is ade-
COMP 0.883 0.928 TEMP 0.861 0.915 quate for the estimation of standard error and confidence interval
CTL 0.805 0.885 TS 0.829 0.897 (Hesterberg, 2011). Table 4 and Fig. 3 show the results of hypothe-
ENJ 0.928 0.955 ses testing. Additionally, the R-squared values for all the endoge-
nous variables are shown in Table 5. We also presented the ef-
Table 3
fect size (F2 ) for each hypothesis. Wilkinson (1999) recommended
Discriminant validity.
using effect size along with null-hypothesis significance testing
AUT COMP CTL ENJ IMR TEMP TS (NHST) because NHST is sensitive to sample size (Cohen, 1992).
AUT 0.866 Moreover, Aguirre-Urreta & Rönkkö (2015) urged IS researchers to
COMP 0.607 0.900 use effect size in the Partial Least Square model. Since effect size is
CTL 0.370 0.415 0.848
not sensitive to sample size, it produces an accurate measure of the
ENJ 0.460 0.414 0.721 0.935
IMR 0.303 0.341 0.641 0.728 0.925 magnitude of the effect between two variables (Ferguson, 2009).
TEMP 0.423 0.320 0.552 0.727 0.695 0.885 Effect sizes larger than 0.02, 0.15, and 0.35 signify small, medium,
TS 0.418 0.336 0.299 0.423 0.278 0.285 0.863 and large effect sizes, respectively (Cohen, 1977).
Our results reveal that perceived task significance (β =0.388,
p < 0.001, F2 =0.310) and flow (β =0.381, p < 0.001, F2 =0.267)
tively capturing four types of motivation along the self-determined produce a medium effect on self-determined motivation. Although
spectrum (intrinsic motivation, identified regulation, external regu- perceived learning autonomy (β =0.239, p < 0.001) facilitates mo-
lation, amotivation) and then calculating a single composite index tivation, it creates only a small effect (F2 =0.085). Conversely, per-
(Vallerand, 1997). ceived skill competency shows no significant impact (β =−0.015,
p > 0.05) on motivation. Moreover, flow creates a medium effect
4.2. Construct validity and reliability on perceived skill competency (β =0.450, p < 0.001, F2 =0.254), on
perceived task significance (β =0415, p < 0.001, F2 =0.208), and on
Using SmartPLS 3 (Hair et al., 2016), we assessed our theoret- perceived learning autonomy (β =0.481, p < 0.001, F2 =0.301). Fi-
ical model, and construct validity as well as reliability. The key nally, our results show that self-determined motivation generates
advantages of using PLS-SEM are (1) it relaxes the normal distri- a positive effect on the actual learning persistence (β =0.472, p <
bution assumptions required by the maximum likelihood method, 0.001, F2 =0.287) and performance (β =0.388, p < 0.001, F2 =0.177).
which is often used to estimate covariance-based (CB)-SEM; and In addition, Table 6 below presents average attempts of SQLi at-
(2) PLS-SEM is better in estimating complex models with smaller tack (i.e., learning persistence) for each exercise. Interestingly, the
sample sizes (Gefen et al., 2011; Hair et al., 2019; Khan et al., result of Pearson Correlation (see Table 7) reveals a significant cor-
2019; Shiau et al., 2019). Due to these reasons, PLS-SEM is a bet- relation (0.612, p < 0.001) between learning persistence and learn-
ter methodology for our data analysis that serves the predictive ing performance. This suggests that learning persistence (i.e., mea-
purposes (Gefen et al., 2011; Khan et al., 2019; Hair et al., 2019; sured by the numbers of attempts) correlates to learning perfor-
Shiau et al., 2019). Therefore, we selected PLS-SEM to first assess mance (i.e., measured by the successful completion of the exer-
the validity and the reliability of our measures with a measure- cises).
ment model, and then to evaluate our research model for hypothe-
ses testing. 5. Discussion
Due to the single composite index as discussed, MOV is not as-
sessed for Cronbach’s alpha (CA), composite reliability (CR), and 5.1. Research findings
Average Variance Extracted (AVE). Prior studies have empirically
established the reliability and validity of a single composite index One of the major aims of our research was to determine how
for self-determined motivation (Fortier et al., 1995; Tremblay et al., employees in cyber or cyber-adjacent fields were motivated to
2009). Similarly, PERS and PERF were not assessed for CA, CR, and learn cybersecurity. To do so, we adapted self-determination the-
AVE, as each construct has only one indicator variable. ory to the context of cybersecurity training and tested extant re-
Table 2 below shows that both CA and CR for each construct lationships from this theoretical foundation. We found that the
exceeded 0.7, indicating good construct reliability (Chin, 1998b; key tenets of self-determination theory mostly held except for
Fornell and Larcker, 1981). Moreover, Table 3 below confirms perceived skill competency. Contrary to self-determination theory,
convergent validity as the AVE for all constructs exceeded 0.5 perceived skill competency does not affect self-determined motiva-
(Fornell and Larcker, 1981), and establishes discriminant validity as tion; but in congruence with self-determination theory, perceived
correlations of each construct with any other construct is less than autonomy and perceived task significance do. This suggests that
the square root of the AVE (Chin, 1998a). novices in cybersecurity are more attracted by the autonomy and
We also examined the factor loadings of each measure- task significance associated with training than by the desire to ac-
ment item on its intended construct (see Appendix B). All the quire cybersecurity knowledge. Due to the complexity of cyberse-
items loaded greater than the threshold of 0.70 with p-value < curity topics and the relatively short training period, participants
0.001, and the difference between loadings on the intended con- may not perceive a high level of skill competency, thus limiting
struct and any other constructs was greater than 0.1. Thus, our competency’s impact on trainees’ perceptions of motivation.
data show both convergent and discriminant validity (Gefen and Although perceived autonomy supports self-determined moti-
Straub, 2005). For the second-order reflective construct (FLOW) vation, its effect on motivation is relatively small. With the nov-
(see Table B-3 in Appendix B), each path of the first-order to the elty of the subject matter, participants may not realize whether
second-order latent variable (LV) was larger than 0.70 with t-values the lessons offered fit their learning styles. As a result, they may
larger than 2.0, thus confirming convergent validity (Chin, 1998a). not know how to take charge of their training. In contrast, per-
The R2 values of all the first-order LV exceeded 0.60, signifying ceived task significance generated the most substantial effect on
good reliability (Doll et al., 1994). self-determined motivation. A key factor may be the proliferation
8 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875

Table 4
Results of hypotheses testing.

Hypothesis β Mean Standard deviation t-value p-value F2 (Effect size)

H1: COMP → MOV −0.014 −0.016 0.066 0.210 0.417 0.000

H2: AUT → MOV 0.239 0.233 0.066 3.623 < 0.001 0.085
H3: TS → MOV 0.388 0.380 0.057 6.797 < 0.001 0.310
H4: FLOW → MOV 0.381 0.393 0.077 4.933 < 0.001 0.267
H5: FLOW → COMP 0.450 0.474 0.080 5.623 < 0.001 0.254
H6: FLOW → AUT 0.481 0.510 0.079 6.073 < 0.001 0.301
H7: FLOW → TS 0.415 0.435 0.090 4.630 < 0.001 0.208
H8: MOV→ PERS 0.472 0.468 0.089 5.288 < 0.001 0.287
H9: MOV → PERF 0.388 0.387 0.086 4.485 < 0.001 0.177

Fig. 3. Results of hypotheses testing.

Table 5 of high profile cyberattacks across the news. The impact of these
R-square values. cyberattacks has been felt by private citizens, thereby increasing
Construct R2 values Mean Standard deviation awareness of heightened cybersecurity on their nation’s critical in-
frastructure. Gradually, these citizens may feel they must learn and
AUTO 0.231 0.258 0.082
COMP 0.203 0.225 0.076
apply new cybersecurity skills to help protect their nation’s or or-
MOV 0.628 0.639 0.065 ganization’s well-being.
PERF 0.150 0.152 0.066 The other major goal of our research was to determine the ef-
PERS 0.223 0.225 0.080 fect of flow on motivation within a cybersecurity training program.
TS 0.172 0.191 0.076
To do so, we integrated flow theory with self-determination theory
Table 6
to complete the theoretical foundation of our research. As hypoth-
Participant attempts by exercise. esized, flow creates a positive effect on self-determined motivation
and situational motivational determinants (i.e., innate psychologi-
Exercises completed 0 1 2 3
cal needs) encompassing perceived skill competence, perceived au-
Total participants 34 20 21 2 tonomy, and perceived task significance. Finally, motivation posi-
Minimum attempts 10 9 15 60
tively affects training persistence and performance, suggesting that
Maximum attempts 28 26 37 64
Average attempts 13 16 25 62 self-determination leads to the completion of a cybersecurity pro-
Average attempts per exercise 13 8 8.33 20.66 gram and produces good training outcomes.

Table 7 5.2. Theoretical implications

Correlation between learning persistence and learning per-
formance.
Our findings provide implications for cybersecurity training in
Learning performance an organizational context. In general, our results revealed that per-
Learning persistence Pearson correlation 0.612∗ ∗ ceived flow and perceived task significance generated larger ef-
Sig. (2-tailed) 0.000 fects on self-determination, suggesting that these factors played
N 133 a significant role in motivating learners to engage in cyber-
∗∗
Correlation is significant at the 0.01 level (2-tailed). security training. Traditionally, flow exhibits short-term effects
 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875
(Csikszentmihalyi, 1990), while task significance is shown to have
more lasting effects (Hackman and Oldham, 1976). Conversely, we
found that flow and task significance had a relatively equal im-
pact on motivation despite being measured within the same cross-
section period. Therefore, we propose that instant gratification (i.e.,
flow) and purpose (i.e., perceived task significance) go hand in
hand to motivate learners to participate in cybersecurity training.
Future research may build on this work to empirically demonstrate
the longitudinal effects of both flow and task significance on cyber-
security trainees’ motivation.
We also suggest that cybersecurity training may incorporate
both hedonic values (e.g., the fun of ethical hacking stemmed from
flow) as well as utilitarian values (e.g., obtaining skills that are
both useful toward protecting information assets and highly val-
ued by the government and private firms). Cybersecurity training
may be an ideal scenario under which fun and utility may co-exist
(Okan, 2003), thereby capitalizing on the benefits offered by both
hedonic and utilitarian approaches. For example, the training tech-
niques used to teach ethical hacking are often considered fun by
participants, but the same techniques are critical for implementing
both offensive and defensive security strategies in organizations.
Future studies can further explore how hedonic and utilitarian as-
pects of cybersecurity can be leveraged equally for maximum im-
pact on trainees.
Furthermore, as individuals understand how and where adver-
saries launch ubiquitous cyberattacks, individuals may interpret
the environment and develop perceived task significance as it re-
lates to their nation’s critical infrastructure. Gradually, some indi-
viduals may develop a personal connection to cybersecurity train-
ing based on a sense of duty. In an organizational context, such
a personal connection suggests that employees may be motivated
by a sense of duty of protecting their organizations’ IT infrastruc-
ture. This further suggests that training transcends individuals, ex-
tending to the social context (Crossan et al., 1999). Future research
could explore the interaction between cybersecurity training and
social environment using Social Learning Theory (Bandura, 1977),
which suggests that inner forces (e.g., desire to learn) and social
environment (e.g., communities or organizations under cyberat-
tacks) generate a positive impact on human behaviors (e.g., engag-
ing in cybersecurity training).
As noted, employees’ connection to cybersecurity (i.e., perceived
task significance) motivates individuals to engage in cybersecurity
training. In an organizational context, we further suggest that per-
ceived organizational support promotes self-determined motivation
in cybersecurity training through encouraging organizational com-
mitment among the employees (Shore and Wayne, 1993). As em-
ployees perceive that their organizations care about their well-
being (i.e., perceived organizational support), they will shape a
personal connection to their organizations, and gradually augment
their perceptions about the critically of cybersecurity in their or-
ganizations. As a result, they will be more willing to protect their
organizations’ InfoSec (i.e., organizational commitment) (Shore and
Wayne, 1993) by engaging in cybersecurity training. Accordingly,
future studies can explore the relationships between perceived or-
ganizational support and self-determined motivation in a cyberse-
curity training context.
Finally, self-determined motivation generates a positive impact
on actual training behavior encompassing actual persistence and
actual performance in cybersecurity training. In a prior study that
adopted self-determination theory, Chen and Jang (2010) failed
to establish that self-determined motivation predicted the actual
training behavior involving performance (e.g., final grade) and en-
gagement (e.g., hours spent per week for studying). To the best of
our knowledge, this is one of the few studies to incorporate users’
actual behavior in motivation studies and demonstrates that self-
determined motivation predicts actual training behavior and out-
9
comes. Accordingly, this study makes a theoretical contribution to
the extant literature by providing empirical evidence that indicates
the prediction of actual training behavior.
5.3. Practical implications
To attract cybersecurity talent, organizations should stress the
criticality of cybersecurity (i.e., importance to organizational secu-
rity) rather than its complexity. Organizations may also want to
foster a good working environment to cultivate the significance of
cybersecurity training that will eventually safeguard organizations’
information security. With a supportive and friendly working en-
vironment, employees will value their organizations and will per-
ceive that their organizations are a community that they are part
of. As a result, these employees will be more motivated to engage
in cybersecurity training for protecting their organizations’ infor-
mation security.
These firms can also empower training through autonomy. For
example, organizations can design retraining programs such that
employees work on cybersecurity skills based on personal prefer-
ence. Essentially, employees could customize their niche within the
cybersecurity needs of the organization. Moreover, cybersecurity
training programs should emphasize both the fun of cybersecurity
tasks and the importance of protecting against cyberattacks. For
example, training programs based on ethical hacking could help
employees realize the seriousness of cybersecurity policies and the
broader implications of cyber threats via fun tasks such as vulner-
ability testing.
Moreover, it is noteworthy that perceived skill competency does
not enhance the motivation of cybersecurity training, possibly be-
cause the difficulty in cybersecurity training does not attribute to
high perceived values in skill competency. This suggests that the
complexity of the subject matter does not attract individuals to cy-
bersecurity training. Hence, organizations may want to avoid over-
whelming employees with a high volume of information during
training program deliveries. For example, the instructions of cyber-
security can be delivered in smaller chunks with shorter training
timeframes. This is to sustain employees’ interest in cybersecurity
training.
5.4. Limitations and future research
Since most of our participants showed an interest in choosing
a computer-related major, this study may have introduced some
biases. However, most of our participants had no prior cybersecu-
rity experience, so this study still provides an adequate rationale
for what factors affect the self-determined motivation of cyberse-
curity training and may be generalizable to others (e.g., working
professionals) who are interested in cybersecurity. Moreover, our
cybersecurity training is relatively short, so the participants may
not realize their competency in such a short time. To address this
shortcoming, we suggest a longitudinal study involving in-depth
cybersecurity training be conducted to bolster competency levels
and impact motivation. Additionally, this study presented a cyber
offense (i.e., SQLi). Unlike cyber offense, cyber defense (e.g., net-
work monitoring) is more tedious which suggests that it may be
less “fun” compared to cyber offense (Kam and Shang, 2019). Fo-
cusing on the fun aspects of cybersecurity may elicit more positive
responses which may lead to bias, but we argue that introducing
cyber offense to novices is a good recruiting strategy that helps
address the cybersecurity workforce shortage.
A further limitation is that subjects were motivated to partici-
pate in this study by receiving a nominal amount of extra credit.
Despite this limitation, the results indicate 77% of subjects contin-
ued to attempt problems beyond the minimum required number
of attempts. This may serve as a representation that subjects were
10 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875

motivated to engage in ethical hacking and pass exercises rather
than just get through the experiment to gain extra credit.
Another limitation of our study is the relative complexity of cy-
bersecurity as compared to other fields or topics. Individuals who
try to first learn a subject matter with high cognitive demand may
not be solely motivated to acquire knowledge. The complexity in
the subject may serve as a barrier to training as compared with
easier subjects. For example, individuals may fail to cognitively
process heavy information, which then causes cognitive overload
(Moreno and Mayer, 2007). Cognitive overload reduces motivation
(Paas and Van Merriënboer, 1994), as individuals may lose moti-
vation due to excessive information processing. Although manip-
ulating subject complexity fell outside of the scope of our work,
future motivational research should examine the cognitive capac-
ity to learn a complex subject as a new factor on motivation.
nation’s or an organization’s well-being. Organizations can also in-
corporate “fun” into cybersecurity training and minimize cognitive
overload during course deliveries. Finally, this study contributes to
the InfoSec research by (1) presenting the motivating factors of cy-
bersecurity training; (2) providing empirical evidence to establish
that self-determined motivation predicts actual training behavior;
and (3) addressing the critical shortage of cybersecurity talent by
suggesting how to attract individuals to the cybersecurity work-
force.
Declaration of Competing Interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared to
influence the work reported in this paper.

6. Conclusion
Appendix A
Our findings reveal that perceived skill competency generates
no effect on, and perceived autonomy generates only a small effect Table A-1
on the self-determined motivation of cybersecurity training. Con-
versely, both perceived task significance and perceived flow pro- Appendix B
duce high impacts on motivation. To promote cybersecurity train-
ing, organizations can stress the criticality of cybersecurity on a Table B-1, Table B-2.

Table A-1
Measurement items.

Construct Measurement items Reference

COMP I feel I have been making progress with learning SQLi in this training program. Vallerand (1997)
I feel that I’m doing a good job learning the material in this training.
I feel that I can manage the requirements of this training.
TS Performing penetration tests (e.g., SQLi) is likely to significantly affect the lives of other people. Adapted from
Hackman and
Oldham (1976)
Penetration testing (e.g., SQLi) is very significant and important in the broader scheme of things.
Penetration testing (e.g., SQLi) has a large impact on people outside my group or organization.
AUT The concepts I learn in this training program are compatible with my choices and interests. Vallerand (1997)
I feel that what I’m told to learn in this training program fits perfectly with what I prefer to learn.
I feel that I have the opportunity to make choices with respect to what I learn in this training program.
MOV I am performing penetration testing…
…because I think that this activity is interesting.1 Vallerand (1997)
…because I think that this activity is pleasant.1
…because I think that this activity is fun.1
…because I feel good when doing this activity.1
…because I am doing it for my own good.2
…because I think that this activity is good for me.2
…because I decided that this activity is beneficial.2
…because I believe that this activity is important to me.2
…because I am supposed to do it.3
…because it is something that I have to do.3
…because I don’t have any choice.3
…because I feel that I have to do it.3
…but I am not sure if it is worth it.4
…but I don’t see what the activity brings me.4
…but I am not sure it is a good thing to pursue it.4
…but personally I don’t see any good reasons to do this activity.4
Temporal Time appears to go by very quickly when I engage in penetration testing (e.g., performing SQLi). Agarwal and
Dissociation Karahanna (2000)
(TEMP) Sometimes I lose track of time when I engage in penetration testing (e.g., performing SQLi).
When I engage in penetration testing (e.g., performing SQLi), I end up spending more time than I had planned.
Focused Immersion While engaged in penetration testing (e.g., performing SQLi), I am able to block out most other distractions.
(IMR) While engaged in penetration testing (e.g., performing SQLi), I am absorbed in what I am doing.
While engaged in penetration testing (e.g., performing SQLi), I am immersed in the task I am performing.
Heightened I have fun while engaged in penetration testing (e.g., performing SQLi).
Enjoyment (ENJ) Penetration testing (e.g., performing SQLi) provides me with a lot of enjoyment.
I love the thrill of penetration testing (e.g., performing SQLi).
Control (CTL) When engaged in penetration testing (e.g., performing SQLi), I feel in control.
I feel I have control over my penetration testing attempts (e.g., performing SQLi).
I have complete control over my computer interaction when I engage in penetration testing (e.g., performing SQLi).

1 Intrinsic motivation.
2 Identified regulation.
3 External regulation.
4 Amotivation).
 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875 11

Table B-1 References

Factor loading.

AUTO COMP CTL ENJ IMR TEMP TS Abraham, S., Chengalur-Smith, I., 2019. Evaluating the effectiveness of learner con-
trolled information security training. Comput. Secur. 87, 101586. doi:10.1016/j.
AUTO1 0.882 0.545 0.349 0.417 0.266 0.397 0.341 cose.2019.101586.
AUTO2 0.906 0.521 0.282 0.405 0.203 0.353 0.412 Agarwal, R., Karahanna, E., 20 0 0. Time flies when you’re having fun: cognitive ab-
AUTO3 0.807 0.513 0.335 0.373 0.330 0.350 0.331 sorption and beliefs about information technology usage. MIS Q. 24 (4), 665–
COMP1 0.567 0.908 0.383 0.365 0.296 0.284 0.325 694. doi:10.2307/3250951, JSTOR.
COMP2 0.532 0.904 0.371 0.365 0.276 0.259 0.287 Aguirre-Urreta, M., Rönkkö, M., 2015. Sample size determination and statistical
COMP3 0.540 0.889 0.366 0.388 0.344 0.319 0.294 power analysis in PLS using R: an annotated tutorial. Commun. Assoc. Inf. Syst.
36. doi:10.17705/1CAIS.03603.
CTL1 0.269 0.280 0.878 0.671 0.597 0.520 0.190
Bandura, A., 1977b. Social Learning Theory. Prentice-Hall.
CTL2 0.351 0.416 0.852 0.563 0.482 0.432 0.343
Beuran, R., Tang, D., Pham, C., Chinen, K., Tan, Y., Shinoda, Y., 2018. Integrated frame-
CTL3 0.325 0.363 0.814 0.599 0.549 0.449 0.232 work for hands-on cybersecurity training: cyTrONE. Comput. Secur. 78, 43–59.
ENJ1 0.449 0.378 0.668 0.941 0.653 0.674 0.406 doi:10.1016/j.cose.2018.06.001.
ENJ2 0.413 0.412 0.668 0.941 0.685 0.692 0.356 Chan, T.S., Ahern, T.C., 1999. Targeting Motivation-adapting flow theory to in-
ENJ3 0.430 0.372 0.687 0.923 0.704 0.674 0.424 structional design. J. Educ. Comput. Res. 21 (2), 151–163. doi:10.2190/
IMR1 0.319 0.345 0.630 0.666 0.933 0.614 0.311 UJ04- T5YB- YFXE- 0BG2.
IMR2 0.282 0.278 0.636 0.685 0.920 0.690 0.202 Chen, K.-.C., Jang, S.-.J., 2010. Motivation in Online learning: testing a model of self-
IMR3 0.238 0.322 0.504 0.667 0.921 0.622 0.258 determination theory. Comput. Hum. Behav. 26 (4), 741–752. doi:10.1016/j.chb.
TEMP1 0.427 0.279 0.555 0.696 0.659 0.918 0.320 2010.01.011.
TEMP2 0.313 0.206 0.382 0.504 0.558 0.811 0.121 Chin, W.W., 1998a. Commentary: issues and opinion on structural equation model-
TEMP3 0.374 0.352 0.509 0.708 0.621 0.921 0.290 ing. MIS Q. 22 (1), vii–xvi.
TS1 0.315 0.290 0.194 0.316 0.201 0.230 0.816 Chin, W.W., 1998b. The partial least squares approach to structural equation mod-
eling. In: Marcoulides, G.A. (Ed.), Modern Methods for Business Research.
TS2 0.360 0.296 0.297 0.398 0.249 0.282 0.884
Lawrence Erlbaum Associates Publishers, pp. 295–336.
TS3 0.401 0.285 0.274 0.374 0.265 0.226 0.886
Cohen, J., 1977. Statistical Power Analysis For the Behavioral Sciences. Lawrence Erl-
baum Associates Publishers doi:10.1016/C2013- 0- 10517- X.
Cohen, J., 1992. Statistical power analysis. Curr. Dir. Psychol. Sci. 1 (3), 98–101.
doi:10.1111/1467-8721.ep10768783.
Table B-2 Cordell, C., 2018. Retrained Agency Employees Can Be A Key Source
Factor loading – mean, standard deviation and t-value. of Cybersecurity Talent. Fedscoop https://www.fedscoop.com/
agencies- can- retrain- employees- get- cyber- talent/.
Factors Loading Mean Standard deviation t-value Crossan, M.M., Lane, H.W., White, R.E., 1999. An organizational learning framework:
from intuition to institution. Acad. Manag. Rev. 24 (3), 522–537. doi:10.5465/
CTL1 ← CTL 0.878 0.877 0.027 32.556∗ ∗ ∗ amr.1999.2202135.
CTL2 ← CTL 0.853 0.853 0.041 20.627∗ ∗ ∗ Csikszentmihalyi, M., 1990. Flow: The Psychology of Optimal Experience. Harper &
CTL3 ← CTL 0.813 0.812 0.046 17.878∗ ∗ ∗ Row.
ENJ1 ← ENJ 0.945 0.944 0.010 94.705∗ ∗ ∗ Csikszentmihalyi, M., Massimini, F., 1985. On the psychological selection of
ENJ2 ← ENJ 0.930 0.930 0.015 61.042∗ ∗ ∗ bio-cultural information. New Ideas Psychol. 3 (2), 115–138. doi:10.1016/
ENJ3 ← ENJ 0.889 0.887 0.028 32.076∗ ∗ ∗ 0732-118X(85)90 0 02-9.
IMR1 ← IMR 0.933 0.933 0.014 65.173∗ ∗ ∗ Deci, E.L., Ryan, R.M., 1985. Intrinsic Motivation and Self-Determination in Human
IMR2 ← IMR 0.920 0.919 0.031 30.059∗ ∗ ∗ Behavior. Springer US doi:10.1007/978- 1- 4899- 2271- 7.
Deci, E.L., Ryan, R.M., 1980. The empirical exploration of intrinsic motivational pro-
IMR3 ← IMR 0.920 0.920 0.018 51.937∗ ∗ ∗
cesses. In: Berkowitz, L. (Ed.). In: Advances in Experimental Social Psychology,
TEMP1 ← TEMP 0.918 0.918 0.016 58.903∗ ∗ ∗
13. Academic Press, pp. 39–80. doi:10.1016/S0065-2601(08)60130-6.
TEMP2 ← TEMP 0.811 0.809 0.065 12.454∗ ∗ ∗ Dillman, D.A., Smyth, J.D., Christian, L.M., 2014. Internet, Phone, Mail, and Mixed–
TEMP3 ← TEMP 0.921 0.923 0.014 64.695∗ ∗ ∗ Mode Surveys: The Tailored Design Method. John Wiley & Sons..
AUTO1 ← AUTO 0.883 0.880 0.028 31.794∗ ∗ ∗ Doll, W.J., Xia, W., Torkzadeh, G., 1994. A Confirmatory factor analysis of the end-
AUTO2 ← AUTO 0.907 0.904 0.018 51.166∗ ∗ ∗ user computing satisfaction instrument. MIS Q. 18 (4), 453–461. doi:10.2307/
AUTO3 ← AUTO 0.805 0.798 0.051 15.744∗ ∗ ∗ 249524, JSTOR.
COMP1 ← COMP 0.908 0.906 0.021 43.124∗ ∗ ∗ Downs, F., 2019, March 20. The battle for cybersecurity tal-
COMP2 ← COMP 0.904 0.906 0.021 42.741∗ ∗ ∗ ent must include retention emphasis. Infosecur. Mag.
COMP3 ← COMP 0.889 0.886 0.027 33.028∗ ∗ ∗ https://www.infosecurity-magazine.com:443/blogs/talent-retention-emphasis-1-1/.
TS1 ← TS 0.817 0.808 0.048 17.091∗ ∗ ∗ Eccles, J.S., Wigfield, A., 2002. Motivational beliefs, values, and goals. Annu. Rev. Psy-
TS2 ← TS 0.884 0.879 0.029 30.097∗ ∗ ∗ chol. 53 (1), 109–132. doi:10.1146/annurev.psych.53.100901.135153.
TS3 ← TS 0.886 0.883 0.027 33.327∗ ∗ ∗ Ferguson, C.J., 2009. An effect size primer: a guide for clinicians and researchers.
Prof. Psychol. Res. Pract. 40 (5), 532–538. doi:10.1037/a0015808.
∗∗∗ Fishbein, M., Ajzen, I., 1975. Belief, Attitude, Intention, and Behavior: An Introduc-
p < 0.001.
tion to Theory and Research. Addison-Wesley Publishing Company.
Fornell, C., Larcker, D.F., 1981. Evaluating structural equation models with unob-
servable variables and measurement error. J. Market. Res. 18 (1), 39–50. doi:10.
Table B-3 2307/3151312, JSTOR.
Second-order factor (FLOW). Fortier, M.S., Vallerand, R.J., Guay, F., 1995. Academic motivation and school per-
formance: toward a structural model. Contemp. Educ. Psychol. 20 (3), 257–274.
β (t-value) R2 values (t-value) doi:10.1006/ceps.1995.1017.
Fullagar, C.J., Kelloway, E.K., 2009. Flow at work: an experience sampling approach.
FLOW→ CTL 0.826 (21.504)∗ ∗ ∗ CTL 0.682 (10. 969)∗ ∗ ∗ J. Occup. Organ. Psychol. 82 (3), 595–615. doi:10.1348/096317908357903.
FLOW→ ENJ 0.944 (67.806)∗ ∗ ∗ ENJ 0.891 (34.227)∗ ∗ ∗ Gagné, M., Deci, E.L., 2005. Self-determination theory and work motivation. J. Organ.
FLOW→ IMR 0.838 (18.660)∗ ∗ ∗ IMR 0.701 (9.633)∗ ∗ ∗ Behav. 26 (4), 331–362. doi:10.1002/job.322.
FLOW→ TEMP 0.838 (26.478)∗ ∗ ∗ TEMP 0.702 (13.411)∗ ∗ ∗ Gagné, M., Senécal, C.B., Koestner, R., 1997. Proximal job characteristics, feelings of
empowerment, and intrinsic motivation: a multidimensional model. J. Appl. Soc.
∗∗∗
p < 0.001. Psychol. 27 (14), 1222–1240. doi:10.1111/j.1559-1816.1997.tb01803.x.
Gefen, D., Rigdon, E.E., Straub, D., 2011. Editor’s comments: an update and extension
to SEM guidelines for administrative and social science research. MIS Q. 35 (2),
iii–xiv. doi:10.2307/23044042, JSTOR.
Gefen, D., Straub, D., 2005. A practical guide to factorial validity using PLS-graph:
CRediT authorship contribution statement tutorial and annotated example. Commun. Assoc. Inf. Syst. 16 (1). doi:10.17705/
1CAIS.01605.
Ghani, J.A., Deshpande, S.P., 1994. Task characteristics and the experience of optimal
Hwee-Joo Kam: Writing - original draft, Conceptualization, For- flow in human-computer interaction. J. Psychol. 128 (4), 381–391. doi:10.1080/
mal analysis. Philip Menard: Conceptualization, Writing - review 00223980.1994.9712742.
Giesbers, B., Rienties, B., Tempelaar, D., Gijselaers, W., 2013. Investigating the re-
& editing. Dustin Ormond: Visualization, Writing - review & edit-
lations between motivation, tool use, participation, and performance in an e-
ing. Robert E. Crossler: Writing - original draft, Writing - review learning course using web-videoconferencing. Comput. Hum. Behav. 29 (1),
& editing. 285–292. doi:10.1016/j.chb.2012.09.005.
12 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875
González-Manzano, L., de Fuentes, J.M., 2019. Design recommendations for online
cybersecurity courses. Comput. Secur. 80, 238–256. doi:10.1016/j.cose.2018.09.
009.
Grant, A.M., 2008. Does intrinsic motivation fuel the prosocial fire? Motivational
synergy in predicting persistence, performance, and productivity. J. Appl. Psy-
chol. 93 (1), 48–58. doi:10.1037/0021-9010.93.1.48.
Guzley, R., Avanzino, S., Bor, A., 2001. Simulated computer-mediated/video-
interactive distance learning: a test of motivation, interaction satisfaction, de-
livery, learning & perceived effectiveness. J. Comput. Mediat. Commun. 6 (3).
doi:10.1111/j.1083-6101.2001.tb00122.x.
Hackman, J.R., Oldham, G.R., 1976. Motivation through the design of work: test
of a theory. Organ. Behav. Hum. Perform. 16 (2), 250–279. doi:10.1016/
0 030-5073(76)90 016-7.
Hair, J.F., Risher, J.J., Sarstedt, M., Ringle, C.M., 2019. When to use and how to report
the results of PLS-SEM. Eur. Bus. Rev. doi:10.1108/EBR- 11- 2018- 0203.
Hair, J.F., Hult, G.T.M., Ringle, C., Sarstedt, M., 2016. A Primer on Partial Least Squares
Structural Equation Modeling (PLS-SEM). Sage Publications.
Heckman, J., 2018. Federal Cyber Reskilling Academy to Retrain Fed-
eral Employees as Cyber Defense Analysts. Federal News Network
https://federalnewsnetwork.com/cybersecurity/2018/11/omb-launches-cyber-
reskilling-to-retrain-federal-employees-as-cyber-defense-analysts/.
Hesterberg, T., 2011. Bootstrap. WIREs Comput. Stat. 3 (6), 497–526. doi:10.1002/
wics.182.
(ISC)2 Cybersecurity Workforce Study. (2018). Cybersecurity Professionals Focus on
Developing New Skills as Workforce Gap Widens. (ISC)2 . https://www.isc2.org/-
/media/ISC2/Research/2018-ISC2-Cybersecurity-Workforce-Study.ashx
Joo, B.-.K., Lim, T., 2009. The effects of organizational learning culture, per-
ceived job complexity, and proactive personality on organizational commitment
and intrinsic motivation. J. Leadersh. Organ. Stud. 16 (1), 48–60. doi:10.1177/
1548051809334195.
Kam, H.-.J., Shang, Y., 2019. Improving cybersecurity learning: an integration of cy-
ber offense and cyber defense. In: Proceedings of the Twenty-Third Pacific Asia
Conference on Information Systems.Pacific Asia Conference on Information Sys-
tems. Xi’an, China.
Keller, J., Bless, H., 2008. Flow and regulatory compatibility: an experimental ap-
proach to the flow model of intrinsic motivation. Personal. Soc. Psychol. Bull.
34 (2), 196–209. doi:10.1177/0146167207310026.
Khan, G.F., Sarstedt, M., Shiau, W.-.L., Hair, J.F., Ringle, C.M., Fritze, M.P., 2019.
Methodological research on partial least squares structural equation modeling
(PLS-SEM). Internet Res. doi:10.1108/IntR- 12- 2017- 0509.
Kong, J.S.-L., Kwok, R.C.-W., Fang, Y., 2012. The effects of peer intrinsic and extrinsic
motivation on MMOG game-based collaborative learning. Inf. Manag. 49 (1), 1–
9. doi:10.1016/j.im.2011.10.004.
Kowal, J., Fortier, M.S., 1999. Motivational determinants of flow: contributions
from self-determination theory. J. Soc. Psychol. 139 (3), 355–368. doi:10.1080/
00224549909598391.
Liao, L.-.F., 2006. A flow theory perspective on learner motivation and be-
havior in distance education. Distance Educ. 27 (1), 45–62. doi:10.1080/
01587910600653215.
Lin, H.-.F., 2007. Effects of extrinsic and intrinsic motivation on employee knowledge
sharing intentions. J. Inf. Sci. 33 (2), 135–149. doi:10.1177/0165551506068174.
Meyer, J.P., Becker, T.E., Vandenberghe, C., 2004. Employee commitment and mo-
tivation: a conceptual analysis and integrative model. J. Appl. Psychol. 89 (6),
991–1007. doi:10.1037/0021-9010.89.6.991.
Mills, M.J., Fullagar, C.J., 2008. Motivation and flow: toward an understanding of the
dynamics of the relation in architecture students. J. Psychol. 142 (5), 533–556.
doi:10.3200/JRLP.142.5.533-556.
Moreno, R., Mayer, R., 2007. Interactive multimodal learning environments. Educ.
Psychol. Rev. 19 (3), 309–326. doi:10.1007/s10648- 007- 9047- 2.
Morgan, S. (2018). Women Represent 20 Percent of the Global Cybersecurity Work-
force in 2019. Cybercrime Magazine. https://cybersecurityventures.com/women-
in-cybersecurity/
Murphy, C., Chertoff, D., Guerrero, M., Moffitt, K., 2013. Design better games! Flow,
motivation, & fun. In: Hussain, T.S., Coleman, S.L. (Eds.), Design and Develop-
ment of Training Games: Practical Guidelines from a Multi-Disciplinary Perspec-
tive, 1st ed.. Cambridge University Press.
Niemiec, C.P., Ryan, R.M., 2009. Autonomy, competence, and relatedness in the class-
room: applying self-determination theory to educational practice. Theory Res.
Educ. 7 (2), 133–144. doi:10.1177/1477878509104318.
NIST Special Publication 800-181, 2017. National Initiative for Cybersecurity Educa-
tion (NICE) Cybersecurity Workforce Framework. National Institute of Standards
and Technology (NIST) doi:10.6028/NIST.SP.800-181.
Okan, Z., 2003. Edutainment: is learning at risk? Br. J. Educ. Technol. 34 (3), 255–
264. doi:10.1111/1467-8535.00325.
Oltsik, J. (2019). Is the Cybersecurity Skills Shortage Getting Worse?[Cybersecurity
Snippets]. CSO. https://www.csoonline.com/article/3394876/
is- the- cybersecurity- skills- shortage- getting- worse.html
OWASP. (2015). OWASP Mutillidae 2 Project. https://www.owasp.org/index.php/
OWASP_Mutillidae_2_Project
Paas, F.G.W.C., Van Merriënboer, J.J.G., 1994. Instructional control of cognitive load
in the training of complex cognitive tasks. Educ. Psychol. Rev. 6 (4), 351–371.
doi:10.1007/BF02213420.
Pintrich, P.R., De Groot, E.V., 1990. Motivational and self-regulated learning com-
ponents of classroom academic performance. J. Educ. Psychol. 82 (1), 33–40.
doi:10.1037/0022-0663.82.1.33.
Podsakoff, P.M., MacKenzie, S.B., Lee, J.-.Y., Podsakoff, N.P., 2003. Common method
biases in behavioural research: a critical review of the literature and recom-
mended remedies. J. Appl. Psychol. 88 (5), 879–903. doi:10.1037/0021-9010.88.
5.879.
Power, R., 2019. Here’s How to Retrain the Workforce—And Why We Need
to. Federal Times https://www.federaltimes.com/opinions/2019/04/05/
heres- how- to- retrain- the- workforce- and- why- we- need- to/.
Roca, J.C., Gagné, M., 2008. Understanding e-learning continuance intention in the
workplace: a self-determination theory perspective. Comput. Hum. Behav. 24
(4), 1585–1604. doi:10.1016/j.chb.20 07.06.0 01.
Ryan, R.M., Deci, E.L., 20 0 0. Intrinsic and extrinsic motivations: classic definitions
and new directions. Contemp. Educ. Psychol. 25 (1), 54–67. doi:10.1006/ceps.
1999.1020.
Schmidt, C.P., 2005. Relations among motivation, performance achievement, and
music experience variables in secondary instrumental music students. J. Res.
Music Educ. 53 (2), 134–147. doi:10.1177/0 0224294050530 0204.
Security Magazine, 2018. What Will Improve Cyber Talent Re-
tention. BNP Media. https://www.securitymagazine.com/articles/
88833- what- will- improve- cyber- talent- retention.
Shiau, W.-.L., Sarstedt, M., Hair, J.F., 2019. Internet research using partial least
squares structural equation modeling (PLS-SEM). Internet Res. doi:10.1108/
IntR- 10- 2018- 0447.
Shore, L.M., Wayne, S.J., 1993. Commitment and employee behavior: comparison of
affective commitment and continuance commitment with perceived organiza-
tional support. J. Appl. Psychol. 78 (5), 774–780. doi:10.1037/0021-9010.78.5.
774.
Teng, C.-.I., Chen, M.-.Y., Chen, Y.-.J., Li, Y.-.J., 2012. Loyalty due to others: the rela-
tionships among challenge, interdependence, and online gamer loyalty. J. Com-
put. Mediat. Commun. 17 (4), 489–500. doi:10.1111/j.1083-6101.2012.01586.x.
Tremblay, M.A., Blanchard, C.M., Taylor, S., Pelletier, L.G., Villeneuve, M., 2009. Work
extrinsic and intrinsic motivation scale: its value for organizational psychology
research. Can. J. Behav. Sci. 41 (4), 213–226. doi:10.1037/a0015167.
Trevino, L.K., Webster, J., 1992. Flow in Computer-mediated communication: elec-
tronic mail and voice mail evaluation and impacts. Commun. Res. 19 (5), 539–
573. doi:10.1177/0 093650920190 050 01.
Turner, T., Pennington, W.W., 2015. Organizational networks and the process of cor-
porate entrepreneurship: how the motivation, opportunity, and ability to act af-
fect firm knowledge, learning, and innovation. Small Bus. Econ. 45 (2), 447–463.
U.S. Department of Homeland Security, 2017. A Report to the President on Sup-
porting the Growth and Sustainment of the Nation’s Cybersecurity Workforce:
Building the Foundation for a More Secure American Future. The Secretary of
Commerce and the Secretary of Homeland Security https://www.nist.gov/sites/
default/files/documents/2018/07/24/eo_wf_report_to_potus.pdf.
Vallerand, R.J., 1997. Toward a hierarchical model of intrinsic and extrinsic motiva-
tion. In: Zanna, M.P. (Ed.). In: Advances in Experimental Social Psychology, 29.
Academic Press, pp. 271–360. doi:10.1016/S0065-2601(08)60019-2.
Vansteenkiste, M., Simons, J., Lens, W., Sheldon, K.M., Deci, E.L., 2004. Motivating
learning, performance, and persistence: the synergistic effects of intrinsic goal
contents and autonomy-supportive contexts. J. Personal. Soc. Psychol. 87 (2),
246–260. doi:10.1037/0022-3514.87.2.246.
Voiskounsky, A.E., Smyslova, O.V., 2003. Flow-Based Model of Computer Hackers’
Motivation. Cyberpsychol. Behav. Impact Internet Multimed. Virtual Real. Behav.
Soc. 6 (2), 171–180. doi:10.1089/109493103321640365.
Wilkinson, L., 1999. Statistical methods in psychology journals: guidelines and ex-
planations. Am. Psychol. 54 (8), 594–604. doi:10.1037/0 0 03-066X.54.8.594.
Zhang, P., Li, N., Sun, H., 2006. Affective quality and cognitive absorption: extend-
ing technology acceptance research. In: Proceedings of the 39th Annual Hawaii
International Conference on System Sciences (HICSS’06), 8 doi:10.1109/HICSS.
2006.39.
Hwee-Joo Kam, D.Sc., CISSP is an Assistant Professor at the University of Tampa.
Her concentration is cybersecurity. She has been teaching a variety of cybersecu-
rity courses, including digital forensic, ethical hacking, principles of information
security, secure coding, and security analytic. Dr. Kam has published in the refer-
eed journals and conference proceedings, such as Information Systems Frontiers,
Computers & Education, Journal of Information Technology, Journal of Information
Privacy and Security, Proceedings of European Computer Information Systems, Pro-
ceedings of Information Conference of International Conference etc. Dr. Kam is also
Certified Information Systems Security Professional (CISSP) certified.
Philip Menard is an Assistant Professor of Information Systems and Cyber Security
at the University of Texas at San Antonio. He received his Ph.D. from the Depart-
ment of Management and Information Systems at Mississippi State University. He is
interested in the impacts of security measures on organizational end users and se-
curity education training and awareness (SETA) programs. He has published at JMIS,
JAIS, Computers & Security, Information & Management, Information Systems Fron-
tiers, and JCIS. He has presented his work at several conferences and workshops
and has served as a reviewer for several IS journals and conferences.
Dustin Ormond, Ph.D. is an Assistant Professor at Creighton University. He is teach-
ing cybersecurity subjects including principles of information systems and ethical
hacking. Dr. Ormond published in Computers & Security, Journal of Association of
Information Systems, Journal of Computer Information Systems, and Information
Systems Frontiers.
 H.-J. Kam, P. Menard and D. Ormond et al. / Computers & Security 96 (2020) 101875 13

Robert E. Crossler is the Philip L Kays Distinguished Professor in Information Sys-
tems and an Associate Professor of Information Systems in the Carson College of
Business at Washington State University. His-research focuses on the factors that
affect the security and privacy decisions individuals make. He has published in
leading MIS journals, including MIS Quarterly, Information Systems Research, Jour-
nal of Management Information Systems, Journal of the Association for Information
Systems, European Journal of Information Systems, Information Systems Journal,
and Journal of Strategic Information Systems. He was named an AIS Distinguished
Member–Cum Laude in 2019.