Journal Review Final

Running head: EVALUATING AUTHENTICATION 1
Evaluating authentication options for mobile health applications
in younger and older adults
Amrit Virdee
University of San Diego
MSNC-507-02B-SP18 - Statistics
Dr. Thidarat Tinnakornsrisuphap, PhD
March 12, 2018

JOURNAL ARTICLE REVIEW 2
Evaluating authentication options for mobile health applications in younger and
older adults
It is estimated that approximately 77% of Americans own a smartphone
(Smith, 2017) and approximately 62% of smartphone owners have used their
phone in the past year to look up health condition information (Lee, 2016).
Smartphones are increasingly being used by consumers to manage their health for
example, health and fitness apps, healthcare portals and more recently the
adoption of personal health records with iPhones. Privacy has been cited as one
of the biggest factors that bring down consumer ratings of mobile apps and the
poor attention of privacy and security is holding back the adoption of useful apps
in healthcare (Grindrod et al, n.d). Improving the adoption of health apps on
smartphones provides another means of increasing patient access to health
information.
The primary basis of the study is based around the privacy of mobile
health apps on smartphones and the study tests the reliability and usability of
common user authentication techniques. Dr. Kelly Grindrod is the primary author
and is the assistant professor at the School of Pharmacy, University of Waterloo,
Ontario, Canada. Her expertise is in the role of eHealth technologies particularly
in the use of mobile phones in community-based primary care, pharmacy practice,
patient-engaged research and health professional education. She has authored a
number of publications and is also a licensed pharmacist and a member of the
Ontario Pharmacy Association.

How the research was conducted?
The research was conducted using usability testing in two age groups, 18
to 30 years and 50 years and older. The younger group was recruited through
university undergraduate programs and online through Kijiji.ca, an online buy-
and-sell website, whilst the older group was recruited through public libraries,
senior education sessions, senior computer clubs and community centers. All
eligible participants were required to have had prior experience using a
smartphone or tablet, be able to speak English, complete a Health Literacy
Assessment (HLA) in order to assess health literacy and complete the Montreal
Cognitive Assessment (MoCA) to access cognitive function. Each participant
was also given a $10 honorarium.
Each participant was tested on four common user authentication
techniques, three of which were knowledge-based and one was biometric based.
The three knowledge-based techniques included 1) a four-digit pin where the user
created a sequence of four numbers and then used those four numbers to access
the phone (figure 1(a)), 2) a graphical password where the participant selected a
cell from a 3 by 5 grid superimposed on an image (figure 1(b)), and 3) a pattern-
lock where the participant drew a pattern visualized as a line by connecting dots
in a 3 by 3 grid (figure 1(c)). The biometric technique involved using the
participants fingerprint (figure 1(d)). Each participant entered each authentication
measure 20 times while reading a health message between each entry. The test
app then captured data on whether the participant was successful in authenticating
on a task and the time it took to perform each successful authentication task.
Figure 1: (a) Four-digit PIN, (b) Graphical password (c) Pattern-lock (d) Fingerprint
Data collection involved all participants completing a paper-based
questionnaire which asked about their experience with technology, security
preferences and demographics (Appendix 1: S1 Questionnaire). After testing the
participants completed a System Usability Scale (SUS) test using a 5-item Likert
scale (Appendix 2: S2 Questionnaire) and finally after each participant completed

all four authentication techniques, they rated each technique using a questionnaire
(Appendix 3: S3 Questionnaire).
The design of the study follows a methodological approach where
candidates were required to have used smartphones in the past as well as test
favorably for health literacy and cognition. This helps to reduce bias in the test
set as well as ensure a level of consistency between the participants.
Collected Data, Analysis, and Results
The study used a number of statistical methods such as t-tests, chi-square
tests and Mann-Whitney U tests. For participants that successfully completed the
authentication options a mixed model analysis of variance (ANOVA) was used to
compare the time it took for each authentication method, the number of errors and
the success rate. All statistical analysis was performed using IBM SPSS (Version
24).
A total of 102 participants were recruited of which 43 were in the younger
range whilst 59 were in the older range. Of the 102 participants, 62% were
female and 38% were male.
Table 1: Success rate, time per authentication task, and errors per task PIN, PATTERN,
GRAPHICAL and FINGERPRINT authentication techniques.
Table 1 compares the four different authentication techniques with success
rate, authentication time and errors per attempt. The results show that the
graphical method had the highest login success rate (100%), the pattern method
had the least mean authentication time (3.44s) and hence was the quickest method
for access and the pin and graphical methods had the least number of errors per
attempt (Mean of 0.02).
Table 2: ANOVA comparing the average success rate, time per authentication
task, and errors per for PIN, PATTERN, GRAPHICAL and FINGERPRINT
authentication techniques (N = 86)
The results of the study illustrated that 100% of younger participants were
able to complete all trials of the pin, pattern and graphical methods and 98% of
younger participants were able to complete the fingerprint method. In contrast
only the older participants were only able to complete 100% of the graphical
method, 98% of the pattern and pin methods and 76% of the fingerprint method.
Figure 1: Boxplot of systems usability scale score for PIN, PATTERN,

GRAPHICAL and FINGERPRINT for younger and older adults (N = 56)
A mixed model ANOVA was used to determine the presence of
significant differences in mean SUS scores between authentication techniques and
age groups. Figure 1 displays the results illustrating that for young adults the pin
method had the highest SUS score whilst the graphical method had the lowest
SUS score. For old adults, the pin and pattern methods had the highest SUS
scores and the fingerprint method had the lowest SUS score.
Conclusions
The primary basis of the study is based around the privacy of mobile
health apps on smartphones and the study tests the reliability and usability of
common user authentication techniques. More consumers are using their
smartphones for health apps however, methods to secure stored information on
mobile devices may adversely affect usability (Grindrod et al, 2017). The authors
were interested in testing the reliability and usability of common authentication
techniques in younger and older participants. The study was able to conclude
differences between the younger and older participants, for example older
participants took two to three seconds longer than younger participants to
authenticate using the pin, pattern and graphical methods.
The study also showed that older participants were also less likely to be
successful in authenticating using a fingerprint. The authors stated that it was
unclear as to why the difference in the fingerprint method but could have been
related to dry skin, shaking hands or poor coordination. This is an important
observation as older patients will likely start using health apps on their
smartphones. The paper also concluded that the pin and pattern method was the
quickest and most usable methods of authentication.
Strengths and Weaknesses of the Selected Statistical Methods
The strengths of the study were that a number statistical methods such as
t-tests, chi-square tests, Mann-Whitney U tests as well as ANOVA were used.
This allowed for various forms of statistically significant comparisons to be made.
The participants had to also complete a Health Literacy Assessment (HLA) in
order to assess health literacy and complete the Montreal Cognitive Assessment
(MoCA) to access cognitive function. This in my opinion helped to reduce bias in
the test set as well as ensure a level of consistency between the participants in
order to get a better statistical comparison with regards to the authentication
methods.
The sample size used was 102 participants and they the selection involved
two groups, an 18-30 and an over 50 group. The participants were chosen from
various locations such as university undergraduate programs, online, public
libraries, senior education sessions, senior computer clubs and community

centers. The sample population used are from educated, English speaking
backgrounds and may not be representative of the population as a whole as they
do not include patients with low levels of health or literacy who may be more
likely to use a health app. The sample size of 102 is also relatively small and may
not represent all the characteristics of a user completely.

References
1. Smith, A. (2017, January 12). Record shares of Americans now own
smartphones, have home broadband, from http://www.pewresearch.org/fact-
tank/2017/01/12/evolution-of-technology/
2. Lee, J. (2016, January). Future of the Smartphone for Patients and Healthcare
Providers. Retrieved February 13, 2018,
fromhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC4756052/
3. Grindrod, K., Khan, H., Hengartner, U., Ong, S., Logan, A. G., Vogel, D., . . .
Yang, J. (n.d.). Evaluating authentication options for mobile health
applications in younger and older adults. Retrieved February 13, 2018,
from http://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.018
9048#
4. Grindrod K, Khan H, Hengartner U, Ong S, Logan AG, Vogel D, et al. (2018)
Evaluating authentication options for mobile health applications in younger
and older adults. PLoS ONE 13(1):
e0189048. https://doi.org/10.1371/journal.pone.0189048
Appendix
S1 Questionnaire: Demographics questionnaire
Please complete the following questionnaire. If you have difficulty, the research coordinator
can assist you. The following questions will help us understand more about you and your
experience with passwords.
1. The following is a list of password options. Select all options that you have used before:
 PIN
o E.g., 1234 or 0984 or 2098 or 2093
 Simple password, or dictionary word, such as

o E.g., sunny, today, person, school, Susan
 Secure password of 8+ digits including a small letter, big letter, number and/or symbol
o E.g., AsoineN1%
 Secure password remembered using a phrase

o E.g., A really good grade is 90% = Arggi90%
 Image-based passcode: selecting the right images or the right spots on provided images to
authenticate yourself;
 Fingerprint
 Pattern Lock
o E.g.
2. Do you own a computer (desktop or laptop)?

 Yes
 No ---skip to question 6
3. How often do you use your computer?

 Daily
 Weekly
 Monthly
 Rarely
4. What kind of password do you need to enter to unlock your computer?

 PIN
 Simple password
 Secure password

 Pattern lock
 Image based passcode
 Fingerprint
 I don’t need to enter a password, pattern, image based passcode or fingerprint
5. Do you need to enter a password to unlock a specific software program on your computer
(specify:)?
6. Do you own a regular cellphone such as a flip phone, not including smartphone?
 Yes
7. How often do you use your regular cellphone?

 Daily
 Weekly
 Monthly
 Rarely
8. What kind of password do you need to enter to unlock your regular cellphone?
 PIN
 Simple password
 Secure password
 Pattern lock
 Fingerprint
 I don’t need to enter a password, image based passcode, pattern or fingerprint
9. Do you own a smartphone such as an Apple iPhone, Samsung Galaxy or a Blackberry Z, etc?
 Yes
10. How often do you use your smartphone?

 Daily
 Weekly
 Monthly
 Rarely
11. What kind of password do you need to enter to unlock your smartphone?
 PIN
 Simple password
 Secure password
 Pattern lock
 Fingerprint
12. Do you need to enter a password to unlock a specific software program on your smartphone
(specify)?
13. Do you own a handheld tablet computer such as an Apple iPad or a Google Nexus or Samsung
Galaxy TAB?
 Yes
14. How often do you use your tablet?

 Daily
 Weekly
 Monthly
 Rarely
15. What kind of password do you need to enter to unlock your tablet?
 PIN
 Simple password
 Secure password
 Pattern lock
 Fingerprint
16. Do you need to enter a password to unlock specific software program on your tablet (specify)?
17. Do you use the same password for multiple software programs on your personal computer,
smartphone and/or tablet computer?
 Yes
 No
 I don’t own a computer, smartphone or tablet computer
18. How often do you write down your passwords and store them somewhere else?
 Always
 Very Often
 Sometimes
 Rarely
 Never
19. How often do you forget your password(s)?

 Always
 Very Often
 Sometimes
 Rarely
 Never—skip to question 21
20. What do you do when you forget a password? Explain:
21. What do you do to help remember passwords? Explain:
22. How often do you have difficulty entering/typing your password into your smartphone or tablet
computer?
 Always
 Very Often
 Sometimes
 Rarely
 Never
 I don’t own a smartphone or tablet computer
Note: The following questions help us to ensure that we include a variety of people in our
study. You can choose to not answer the question if you’d like.
23. What is your education (select all that apply)?

 Below high school  University
 High school  Graduate Degree (MA, PhD)
 Trade school  Professional Degree (MD, MBA)
 College
24. What is your annual household income (the combined income of all individuals living in your home)?
 Less than $20,000
 $20,000-$49,999
 $50,000-$79,999
 More than $80,000
 I don’t know/refuse to answer
25. What is your gender?

 Man
 Woman
 Other (e.g. Transgender)
26. Which of the following best represents your ethnicity (select all that apply):
 Caucasian
 Aboriginal
 Black
 Arab
 Chinese
 Japanese
 Korean
 West Asian
 South Asian
 Southeast Asian
 Filipino
 Hispanic/Latino
 Other (Explain):
Running Head: EVALUATING AUTHENTIATION OPTIONS
15
27. Do you have chronic health conditions in the past three months?
 Yes (Please specify: _________________________________________)
 No
28. Are you on prescription medication?

 No
29. Do you take dietary supplements?

 No
**Thank you for completing this questionnaire. The research coordinator will now perform a
short assessment of your health literacy. Please notify the research coordinator that you are
ready to begin the assessment.
Word
1. Kidney 4. Nutrition 7. Alcoholism 10. Dose 13. Directed 16. Diagnosis
2. Occupation 5. Miscarriage 8. Pregnancy 11. Hormones 14. Nerves 17. Hemorrhoids
3. Medication 6. Infection 9. Seizure 12. Abnormal 15.Constipation 18. Syphilis
Health Literacy Assessment Script
Research Coordinator: The following is a short assessment of your health literacy. (A score between
0 and 14 suggests the examinee has low health literacy). Look at the list of words you have. Read
the first word out loud. Next, I’ll read two words and I’d like you to tell me which of the two words is
more similar to or has a closer association with the word you just read. If you don’t know, please say
‘I don’t know’. Don’t guess.
Next, read the second word… (Continue until the list is complete).
Stem Key or Distracter Don't know

1. Kidney __Urine __Fever __Don’t know
2. Occupation __Work __Education __Don’t know
3. Medication __Instrument __Treatment __Don’t know
4. Nutrition __Healthy __Soda __Don’t know
5. Miscarriage __Loss __Marriage __Don’t know
6. Infection __Plant __Virus __Don’t know
7. Alcoholism __Addiction __Recreation __Don’t know
8. Pregnancy __Birth __Childhood __Don’t know
9. Seizure __Dizzy __Calm __Don’t know
10. Dose __Sleep __Amount __Don’t know
11. Hormones __Growth __Harmony __Don’t know
12. Abnormal __Different __Similar __Don’t know
13. Directed __Instruction __Decision __Don’t know
14. Nerves __Bored __Anxiety __Don’t know
15. Constipation __Blocked __Loose __Don’t know
16. Diagnosis __Evaluation __Recovery __Don’t know
17. Hemorrhoids __Veins __Heart __Don’t know

18. Syphilis __Contraception __Condom __Don’t know
S2 Questionnaire. Usability questionnaire.
Usability and Learnability of Authentication Measures Survey
Session: PIN*
A reminder on the types of Authentication Measures:
1. PIN
 E.g., 1234 or 0984 or 2098 or 2093
2. Simple password or your choosing

 E.g., sunny, today, person, school, Susan
3. Secure password of 8+ digits including a letter, number and/or symbol

 E.g., AsoineN1%
4. Secure password remembered using a phrase

 E.g., A really good grade is 90% = Arggi90%
5. Image Based Passcode: select the right spot on provided images in certain order;
6. Fingerprint
7. Pattern lock
 E.g.
*(Same questionnaire used for Pattern, Fingerprint, and Image)

1. In your daily life, do you agree: “it is important to secure your personal health
information using a PIN or Password”?
 Strongly Agree
 Agree
 Neutral
 Disagree
 Strongly Disagree
2. When you want to protect your personal health information on a computer or mobile
device, what type of password do you typically use?
 PIN
 Simple password
 Secure password
 Pattern lock
 Fingerprint
 I don’t use a password, image based passcode, pattern or fingerprint
3. Given the choice, when you want to protect your personal health information on a
computer or mobile device, what type of password do you most prefer to use on a
mobile device?
 PIN
 Simple password
 Secure password
 Pattern lock
 Fingerprint
The following questions are about your experience using the PIN (the following
questions were also used for GRAPHICAL, PATTERN, and FINGERPRINT).
4. How secure do you think a PIN is compared to no password to unlock your mobile
device?
 Much more secure
 Somewhat more secure
 Not more or less secure
 Somewhat less secure
 Much less secure
 I don’t know
5. How secure do you think a PIN is compared to the type of password you typically
use in your daily life?
 Much more secure
 Somewhat more secure
 Not more or less secure
 Somewhat less secure
 Much less secure
 I don’t know
6. Rate your agreement with the following statements about the PIN method?
Strongly Agree Neutral Disagree Strongly
Agree Disagree
I think entering PIN takes a lot of
time
I think PIN method is annoying
I think the PIN method is tiring
7. Rate your agreement with the following statements about the PIN method?
# Strongly Agree Agree Neutral Disagree Strongly
Disagree
1 I think I would use the PIN
frequently
2 I found the PIN
unnecessary complex
3 I found the PIN easy to use
4 I think I would need the
support of a technical
person to be able to use the
PIN method
5 I thought there was too
much inconsistency in using
the PIN method
6 I would imagine most
people would learn to use
the PIN very quickly
7 I found the PIN very
cumbersome to use
8 I felt confident using the PIN
9 I needed to learn a lot of
things before I could get
going with the PIN
10 I found the PIN was well
integrated with the various
functions of the task.
S3 Questionnaire. Exit questionnaire.

1. On a scale of 0 to 10, rate how much you liked each method that you tried (0=did
not like at all; 10 liked very much).
Did not like at all Neutral Liked very much

0 1 2 3 4 5 6 7 8 9 10
PIN
Picture
Pattern
Fingerprint
2. Considering everything together, including security, reliability, accessibility, etc.,

please rank the 4 password strategies you tried during the study from most
preferred to least preferred.
(Most preferred)
(1)
(2)
(3)
(4)
(Least preferred)
***Please pass this back to the researcher***

The following questions will be asked by the research coordinator:
3. (If there is mismatch ranking between question 1 and 2): I noticed that you
ranked the [e.g., PIN] highest but you liked the [e.g., fingerprint] more. Why
is that?
4. What did you like and dislike about each method you tried:
Like Dislike
PIN
Picture
Pattern
Fingerprint
5. What are some scenarios where a particular scheme would not work in
your life?
6. Is there anything else that you’d like to add?

Journal Review Final

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Journal Review Final

Caricato da

Copyright:

Formati disponibili

Running head: EVALUATING AUTHENTICATION 1

Evaluating authentication options for mobile health applications

in younger and older adults

University of San Diego

Dr. Thidarat Tinnakornsrisuphap, PhD

March 12, 2018

Evaluating authentication options for mobile health applications in younger and

It is estimated that approximately 77% of Americans own a smartphone

in healthcare (Grindrod et al, n.d). Improving the adoption of health apps on

smartphones provides another means of increasing patient access to health

and is the assistant professor at the School of Pharmacy, University of Waterloo,

Ontario, Canada. Her expertise is in the role of eHealth technologies particularly

in the use of mobile phones in community-based primary care, pharmacy practice,

patient-engaged research and health professional education. She has authored a

number of publications and is also a licensed pharmacist and a member of the

Ontario Pharmacy Association.

How the research was conducted?

university undergraduate programs and online through Kijiji.ca, an online buy-

eligible participants were required to have had prior experience using a

smartphone or tablet, be able to speak English, complete a Health Literacy

Cognitive Assessment (MoCA) to access cognitive function. Each participant

was also given a $10 honorarium.

Each participant was tested on four common user authentication

cell from a 3 by 5 grid superimposed on an image (figure 1(b)), and 3) a pattern-

in a 3 by 3 grid (figure 1(c)). The biometric technique involved using the

participants fingerprint (figure 1(d)). Each participant entered each authentication

Data collection involved all participants completing a paper-based

questionnaire which asked about their experience with technology, security

preferences and demographics (Appendix 1: S1 Questionnaire). After testing the

scale (Appendix 2: S2 Questionnaire) and finally after each participant completed

The design of the study follows a methodological approach where

set as well as ensure a level of consistency between the participants.

Collected Data, Analysis, and Results

The study used a number of statistical methods such as t-tests, chi-square

authentication options a mixed model analysis of variance (ANOVA) was used to

A total of 102 participants were recruited of which 43 were in the younger

female and 38% were male.

Table 1 compares the four different authentication techniques with success

attempt (Mean of 0.02).

younger participants were able to complete the fingerprint method. In contrast

Figure 1: Boxplot of systems usability scale score for PIN, PATTERN,

A mixed model ANOVA was used to determine the presence of

significant differences in mean SUS scores between authentication techniques and

common user authentication techniques. More consumers are using their

smartphones for health apps however, methods to secure stored information on

were interested in testing the reliability and usability of common authentication

participants took two to three seconds longer than younger participants to

authenticate using the pin, pattern and graphical methods.

successful in authenticating using a fingerprint. The authors stated that it was

related to dry skin, shaking hands or poor coordination. This is an important

quickest and most usable methods of authentication.

Strengths and Weaknesses of the Selected Statistical Methods

t-tests, chi-square tests, Mann-Whitney U tests as well as ANOVA were used.

This allowed for various forms of statistically significant comparisons to be made.

The participants had to also complete a Health Literacy Assessment (HLA) in

(MoCA) to access cognitive function. This in my opinion helped to reduce bias in

order to get a better statistical comparison with regards to the authentication

various locations such as university undergraduate programs, online, public

libraries, senior education sessions, senior computer clubs and community

backgrounds and may not be representative of the population as a whole as they

not represent all the characteristics of a user completely.

1. Smith, A. (2017, January 12). Record shares of Americans now own

smartphones, have home broadband, from http://www.pewresearch.org/fact-

17. Hemorrhoids Veins Heart __Don’t know

Please pass this back to the researcher