Miranda Lucas

Usability Test Assignment

Usability Test for KansasCity.com

Test Preparation
Before presenting my testers with the tasks I went through and
completed the test myself. By doing this I became aware of what my testers
would be going through and I could better understand their actions and
thought process when completing the tasks. Taking the test for myself also
allowed me to make sure that there was a way to complete each task; this
way I know Im not setting each tester up for a guaranteed failure.
I worked on the tasks in my room on my laptop using Internet Explorer.
Once I arrived at KansasCity.com I looked around the home page of the site.
At first glance I noticed that the page is predominantly text based. There are
pictures to go along with the articles, but the text is what stuck out to me the
most. It seems well organized, but not as visually appealing as I would
personally like a website to be.
The first task asked for information pertaining to the 87th Annual
Academy Awards. I had to look up what films were nominated for best
picture, and to do this I used the search tool located in the upper right-hand
corner of the page. I searched 87th Annual Academy Awards. My search
received thousands of results, but I clicked on the first link Photo Gallery:
Oscar nominations announced Thursday. The link directed me to another

page with Oscar related pictures and a short description giving some insight
about the Academy Awards. The description didnt list the nominations for
best film so I used a different search in the search tool. This time I tried
Oscar best picture nominations. The article I clicked originally was the first
search result, but I chose to scroll farther down and there I found Oscars
best picture nominations are an elite eight, and thats just fine with me. This
article listed all the films nominated this year for best picture and gave short
descriptions about each.
The second piece of information I had to look up regarding the Oscars
was when it would be aired on television. I resorted to my original search,
87th Annual Academy Awards. I chose the first result again and it in the
short description it stated the Oscars would be held February 22nd. I
completed both parts of the first task without too much trouble, but I
definitely felt inconvenienced by having to really dig for the information that
I wanted.
For the second task I needed to find a section within the site that
covered community news called 816. I found the 816 section within the
news header on the navigation bar. My next task was to find a story within
this section and go through the necessary steps needed to print the article.
Along the way I encountered a few pop-up ads that I found frustrating
because they were getting in my way of the content I was looking for. The
article I chose is called The Business of Fighting. Once the article loaded I

used ctrl + P to print. A print preview popped up and from there all I needed
to do was select the print option.
For task number three I needed to figure out what upcoming events
would be happening in Jackson and Cass counties. At first I used the
searched tool to look up Jackson county and Cass County, but the top
results were things unrelated to the information I was looking for. I proceeded
to search the same thing but with the word events in my search bar. This
search refined my results and I found the updated calendar of upcoming
events for Jackson and Cass counties. Even though I found the article I was
unable to actually get the information I needed because the website wouldnt
allow me to continue until I had signed in or created an account for ninetynine cents. I selected no, and was taken back to the websites home page.
The final task was asking for me to look up job related information. I
needed to search for a job in any specific city of my choice using any
keyword. To get to this point I clicked a link at the very top right of the page
that said JOBS. The link directed me to Career Builder within the Kansas
City Star and there in the search tool I looked for restaurant jobs within
Maryville, Missouri. I chose the first listing for a Restaurant Franchise
Opportunity for Mr. Goodcents. At this point the test asks for me to share
the job listing on social media. I searched for a share button but there was no
option to do this, so to share the page I would have to copy the URL and
paste the link into the social media myself. It seems easy enough but it is

really inconvenient to have to go through all the steps when there could be a
share button on the page.
I managed to complete all the tasks without too much frustration. A lot
of the information was not so easily retrievable which could be a bad thing
for a user in a hurry or a user who simply doesnt want to waste their time
looking through pages and pages within a website when they can just as
easily get the same information from a search engine like Google or Yahoo.
For each test I conduct the tester will be recorded using the video
camera on my laptop. This way I can document everything they say and I
can get a better visual of their emotions when they are succeeding or
struggling with certain tasks.
Choosing Participants
Tester 1: Courtney Coble
Courtney is a current student attending Northwest Missouri State
University. Courtney is a nineteen year old female with no occupation while
away at school, but she does work at the local Target near her home in St.
Joseph Missouri. I believe Courtney makes a reasonable tester, first off,
because she is familiar with using the internet. She estimates that she
spends around twenty hours a week on the web. She guesses that of the
twenty hours, seventy percent is spent on social media while thirty percent is
spent doing other miscellaneous things on the internet. Courtney has never
been to KansasCity.com prior to the usability test; therefore she is not

familiar with its content. Although she has never been to this site in
particular, she has been to other sites of the same nature to look up
information for school-related projects. Because of this she feels that she has
a good grasp on how to maneuver around the site being tested. This is
another quality that I feel makes Courtney a reasonable tester. Her web
browsing is usually done from her laptop, so this test should feel comfortable
for her. She is well educated on how to use the internet and for this reason
she is my high-experience tester.
Environment for Tester 1

Location of Test: The test was taken in Courtneys room. This location
was chosen because this is where she usually does her homework and
spends her time browsing the web. I believe keeping her in her own
element will make her feel comfortable and at ease while taking the

usability test.
Physical Environment: While taking the test Courtney was sitting on
her bed in her dorm room in Roberta Hall. The light was on, but
somewhat dim. Her blinds were closed so there was no natural light
coming in which contributed to the dim feeling. She had a friend in the
room, but the friend remained quiet throughout the test and didnt
present herself as a distraction while taking the test. Courtney also had

the TV turned on with a low volume setting.

Technical Environment: The test was taken on a laptop provided by
Northwest. The resolution was set to the recommended 1366 x 786

and the laptop had full connection to the Wi-Fi. Internet Explorer was
the browser used for testing and there were no browser add-ons in use
allowing the website to completely do what is intended.
Tester 2: Ryan Klassen
Ryan is a twenty-three year old male and a recent graduate of
Northwest Missouri State University. Ryan is currently an employee of Simply
Siam, a Thai restaurant in Maryville, Missouri. He is spending one last
semester in Maryville although he is no longer attending school. He claims to
spend around fifteen hours a week on the internet and of those fifteen hours
eighty percent are spent on social media and the other twenty percent are
spent doing miscellaneous things on the web. Ryan does not own a laptop
and doesnt spend much time on websites other than Twitter, Facebook or
Netflix. The miscellaneous twenty percent of his internet usage seems to be
spent gaming online. All of his time spent online is through a mobile device
or through his game system. I consider Ryan my low-experience tester
because he hasnt had a lot of interaction with websites and he says he isnt
always sure how to use them to their full potential. Ryan is a reasonable
tester for this website because after his time in Maryville he plans on moving
to Kansas City, Missouri to start his career. Ryan plans on becoming more
interactive with the internet, but doesnt have a need for it at the moment. A
site like KansasCity.com would be helpful to someone like Ryan moving to the
city. It is a good source of information for what is going on in the area.

Environment for Tester 2

Location of Test: Ryan took the test in the basement of his home in
Maryville. This location was chosen for the same reasons Courtneys
test was in her room. This is where he spends the majority of his time
and where he feels most comfortable being. That way he feels at ease

through all points of the usability test.

Physical Environment: The basement was on the darker side with
only two lamps providing the light in the room. He sat on his couch
while taking the test and there was no one else in the room besides his
pet cat, Minnie. The cat was asleep on a chair and was not an obvious
distraction for Ryan as he took the test. The TV was off as well and it

the room was silent.

Technical Environment: Ryan borrowed my Northwest laptop for the
test seeing that he did not have one of his own. The resolution was set
to the standard 1366 x 786 and computer had a full connection to the
houses Wi-Fi. Just like Courtney, Ryan used Internet Explorer for the
browser and no add-ons were in use for the duration of the test.

Test Results
Initial Site Thoughts

Tester 1 Report: After initially viewing the home page, Courtney

thought the whole site was just a lot of links to different news stories.
She saw things on the page that she assumed were happening in the
news regarding tips, sports and general information. She specifically

said, All I see is a bunch of articles. She didnt find the site to be very
enticing because it was too text heavy with not enough pictures.
Courtney doesnt think this is a website she would visit often because
she like things that are more fun looking and visually appealing. She is
not a person big into news stories, so these are things should would

need to keep interested.

Tester 2 Report: Once arriving to the site Ryan quickly took notice
that site is made for the Kansas City Star newspaper. Knowing this
information he thought the site would give him news about things like
sports and politics. The first articles he noticed were about the Kansas
City Chiefs and Mizzou sports which definitely sparked his interest.
Ryan found the site to be visually appealing unlike tester 1. He figured
this was maybe the case because he knew what kind of information he
was looking at so he thought it was organized nicely. He mentioned the
advertisements on the page didnt bother him because he figured they
wouldnt be obscene or in your face too much because the Kansas City
Star is a well-respected newspaper.

Task 1: Determine the following information about the 87th Annual

Academy Awards:
Tester 1

What films are nominated for best picture: Courtney went directly
to the search tool without hesitation to look for results regarding the
87th Annual Academy Awards. Her search returned her with a lot of

photo galleries, but she chose to go with the first result called Who
Tied for most Oscar Nominations because she assumed that this
would include who was nominated for best picture. The link brought
her to a photo gallery where she spent very little time looking for the
answer before she clicked the back button to take her back to her
original search. I could tell she was beginning to get frustrated because
no other results listed on the page pertained to information she was
trying to find. So at this point she resorted to looking at the
entertainment section on the navigation bar. She spent a lot of her
time here and found nothing related to what she was trying to find. The
sound in her voice and her facial expressions showed that she was not
having an easy time with the task. She returned to the search tool and
searched Academy Awards best picture nominees. The search
brought up more photo galleries which only added to her mood, but
she continued to look through the results until she found an article
without a photo gallery. She chose Birdman, Budapest top Oscar
nominations with nine nods each. Here is where she finally found her
answer.
o Answer: Boyhood, Whiplash, The Theory of Everything, The
Imitation Game, American Sniper, Selma, Birdman and The

Grand Budapest Hotel

o Completed: Yes
When will the Oscars be on television: Courtney stayed on the
article to see if she could find the answer. She read the whole article
before she realized it did not contain what she was looking for. She

returned to the search tool and searched When will the Oscars be on
television? She started to rub her eyes after staring at the screen for
so long looking through the results. She voiced that she was getting
very annoyed with all the photo galleries because she thought this was
a news site not just a bunch of pictures. She also thought this would
be a lot easier than it was turning out to be. She then tried to start a
new search with Oscar premier. She gave up on the task when the
search returned even more photo galleries with headlines that didnt
pertain to what she was asking for.
o Answer: N/A
o Completed: No
Tester 2

What films are nominated for best picture: Ryan started by using
the search tool provided by the website. He searched the phrase 87th
Academy Awards. He mentioned he could maybe be more specific,
but that this search phrase would probably give him what he needed.
Once arriving to the results page a very large advertisement popped
up and blocked his view of the page. He found this to be annoying, but
exited out of it and proceeded with his task. He scrolled down on the
page until he found a link titled Photo Gallery: Oscar nominations
announced Thursday. The description of the gallery didnt answer his
question, but instead of finding a new search he looked through the

gallery images until he found a photo with best picture nominations on

it.
o Answer: Boyhood, Whiplash, The Theory of Everything, The
Imitation Game, American Sniper, Selma, Birdman and The

Grand Budapest Hotel

o Completed: Yes
When will the Oscars be on television: Instead of creating a new
search in the search tool, Ryan decided to stay within his last search
that helped him find the best picture nominations. From the article he
was at listed above, he went back to the results page and started from
the top of the links. He clicked on Photo Gallery: Birdman wins best
picture. He read the description of the gallery which gave him the

date the Oscars were televised.

o Answer: Sunday, February 22nd 2015
o Completed: Yes
Recommendations to improve UX for this task, linked to Jacob
Nielsons 10 Usability Heuristics: Tester 1 had more trouble with
this task than tester 2. Her issue with the task was that her search
results werent obviously giving her the information that she was trying
to find. One of the heuristics I believe will potentially help other users
facing this same issue is Consistency and Standards (Users should not
have to wonder whether different words, situations, or actions mean
the same thing. Follow platform conventions). Tester 1 couldnt figure
out what information was being given to her which caused her to be
confused and frustrated. Maybe if the description of the titles on the
page gave more insight and was clear as to what the link would

provide users would avoid clicking through unnecessary pages. I think

making the search tool on the site more useful will greatly improve
user experience. As of right now the results kind of very and arent
very specific to what the user searched for. I believe giving users the
option of having advanced search will definitely help with problems in
this area, and help users who are trying to find specific information.
Task 2: Find the section of the site titled 816 which covers
community news:
Tester 1

In this section, find a story of interest and go through the

steps to order reprints of the article. Go as far as possible
without actually ordering: Courtney immediately went to the news
category of the navigation bar where she found the 816 section within
seconds. She clicked on the link and looked through the article until
she found one of interest. She chose Should Medical School be Free?
An advertisement popped up on the screen and when she exited out of
it she was taken back to the home page which she found to be very
frustrating. She retraced her steps until she found the article again.
The same thing happened, so she tried for a third time. The same
advertisement popped up on the screen and for the first time in three
attempts she read that a membership was required in order to

continue reading. The one month trial was ninety-nine cents which she
refused to pay, and there was no option to continue for free.
o Completed up to printing point: No
Tester 2

In this section, find a story of interest and go through the

steps to order reprints of the article. Go as far as possible
without actually ordering: Ryan returned to the home page to begin
the second task. He looked to the navigation bar to find if the 816
section would be located there, and he found it under the news
category. He was scrolling through the articles and was given three
pop-up advertisements in the matter of a minute, and he was very
frustrated and distracted by this. Ryan went to the second page and
chose an article about the principal of Lees Summit West High School.
He scanned the article for a second and scrolled to the bottom of the
page to click on an arrow button that gave him a printing option. He
selected print and a print preview popped up on the screen that would

allow you to confirm your print decision.

o Completed up to printing point: Yes
Recommendations to improve UX for this task, linked to Jacob
Nielsons 10 Usability Heuristics: A heuristic linked to making this
task easier for users is Recognition rather than Recall (Minimize the
user's memory load by making objects, actions, and options visible.
The user should not have to remember information from one part of
the dialogue to another). Although tester 2 was able to reach the

printing option at the bottom of the page, he missed the first one given
to him right at the beginning of the article. I think making the options
larger and more visible on the page will help users realize what they
can do sooner without having to think much about it. The site does do
a good job with use of accelerators. When I took the test myself I was
able to print the article using ctrl + P. The site also does a good job of
keeping the navigation consistent throughout the website. Although
tester 2 returned to the home page to start his search, tester 1 stayed
on her current page and found the exact same information without
having to click around as much. The site as a whole could work on the
pop-up advertisements. Users will enable plug-ins to avoid things like
these and it would be less frustrating for a user to just see an
advertisement displayed on the page rather than popping up on the
screen. It distracts them from the important information on the page
which relates back to the aesthetic and minimalist design heuristic
(reduce unnecessary dialogue so the user isnt distracted from the
important content).
Task 3: Determine what events will take place in the following weeks in
Jackson and Cass counties. (This is the seven says following the day of the
test):
Tester 1

For this task Courtney returned to the search tool where she searched
Jackson County. Her results were not what she wanted. They included
articles about Jackson County or criminal charges happening within the
county, but no upcoming events. She tried her search again except this
time she added events to the phrase. The results again, gave her
information other than what she was looking for. She then began to
look through the navigation bar to find if the events section or calendar
would be located there. She looked for a couple minutes, but couldnt
find anywhere within the navigation that related to upcoming events.
She was starting to get very confused and then she tried her luck with
the search tool again, but this time she searched Cass County. She
found an article and photo gallery about a Cass County fair, but the
article was posted 7 months ago and doesnt relate to the information
she was trying to find. She spent a long time on this task trying
different keywords and phrases, but in the end could not give me the
information the task asked for.
o Answer: N/A
o Completed: No

Tester 2

Ryan referred back to the news category on the navigation and

struggled between which section to choose to find the information for
Jackson and Cass counties. After a couple seconds of deliberation he
decided to search Jackson and Cass in the search tool. The results

brought him to a Jackson and Cass counties calendar of events. He

clicked on the link but was not allowed to view the page without
signing up for a ninety-nine cent trial or becoming a member. This was

the same issue that occurred earlier in test one.

o Answer: N/A
o Completed: No
Recommendations to improve UX for this task, linked to Jacob
Nielsons 10 Usability Heuristics: Like I said for task one, what
really will help improve user experience for this task would be to create
a better search tool for users. Although tester two was able to find the
information, tester one could not get to it based on her search. Another
reoccurring theme throughout this site that was brought up a second
time on this test is the fact that users cant access the information they
are looking for without having to pay a fee. In both occurrences, users
immediately chose not to continue through the process. I would
recommend offering users a free one month trial instead of a ninetynine cent one. I know that it doesnt seem like a lot of money, but both
times users automatically rejected the idea of paying.

Task 4: Determine the following information related to finding a job:

Tester 1

Search for a job in any specific city (tester can choose) using
any keyword: At first Courtney looked through the business tab on
the navigation bar and realized she was only seeing articles about
business, not actual jobs ads. She found a job search tool within the

classifieds category on the navigation. She searched with keyword

secretary within St. Joseph, Missouri. The search brought her
nowhere and she was stuck on the same page. She tried this multiple
times and returned no results each time. She then tried a different
approach, within the classified she clicked on the view all top jobs
link. The link took her to a new website called CareerBuilder.com where
it showed the top jobs in Kansas City. She voiced that she was
frustrated because the search tool didnt work and didnt allow her to
specify the city she wanted to work in, but she settled for looking
through jobs within the Kansas City area. She tried sales as a new

keyword and chose a job in pharmaceutical sales.

o Completed: Yes
Find a job listing and share it using social media: She opened the
link to her pharmaceutical sales job listing and searched the page to
share it on social media. Courtney looked all over the page, but found
nothing that could help her share the page to a social media. Her only
option to share was through email and that was not her goal of the
task. She searched again a few more times before deciding that it
wasnt possible to do.
o Completed: No

Tester 2

Search for a job in any specific city (tester can choose) using
any keyword: Ryan went straight to the classified category on the
navigation and found the refined search tool. His keyword was food

and he chose Maryville, Missouri as his city. Nothing happened when he

clicked the search button, so he tried again. After his second try he was
taken to a new search page for Career Builder within KansasCity.com,
but he wasnt sure how he got to it. He specifically said, Whoa, Im not
sure what just happened, but I guess I will try the search again. So he
used the new search tool on the page and used the keyword food and
Maryville, Missouri again which brought up a lot of different results. His
results didnt seem to pertain to the search he was going for, but he

chose an account managing position.

o Completed: Yes
Find a job listing and share it using social media: Ryan took a
long time looking at the page to find an option to share it on social
media, but was unsuccessful. The only sharing option he found was to

send the listing via email.

o Completed: No
Recommendations to improve UX for this task, linked to Jacob
Nielsons 10 Usability Heuristics: The error prevention heuristic
would be very beneficial to improve this task. An error occurred when
bringing up results for both testers one and two. Its important to check
to make sure everything on the site works properly so that users do not
encounter errors like this and become turned off by the site.

Testing Conclusions

Completion rate: There were a total of seven tasks throughout the

test. Tester one completed three out of the seven tasks giving her a

completion rate of 43%. Tester two completed five out of the seven
tasks giving him a completion rate of 71%. Both testers finished below
the typical task completion rate of 78%. I believe both testers would
have finished a little bit higher if they had not been hindered by the

membership requirement.
Similarities and differences: The two testers were similar in a lot of
their approaches. They both heavily relied on the sites search tool to
complete most of their tasks. The difference really was what phrases
they each chose to search. I believe tester two completed more of his
tasks because he wasnt so quick to go over articles without giving
them a closer look. It seemed he trusted the system more. Maybe
tester one was so easily frustrated because she has more experience
with other websites, so she expected quicker results that gave her
exactly what she was asking for. She mentioned throughout her test,
This is too frustrating I could find this information way easier on
something like Google. Another difference was the initial response
each tester had from viewing the home page. Tester one did not like
the look of it, while tester two actually enjoyed it. Maybe this could
have also contributed to the difference in their completion rates.