25April,2014

1400TownsendDrive
Houghton,MI49931

ProfessorKeshabAcharya
MichiganTechnologicalUniversity
DepartmentofHumanities
Houghton,MI49931

DearProfessorAcharya,

Thankyouforgivingustheopportunitytostudytheusabilityofawebsiteorapplication.We
researchedtheusabilityoftheCraigslistapplicationforiPhone.Enclosedisourfinalreport,Craigslist
iOSApplicationUsabilityTesting.Inthisreport,weconcludedthattheapplicationwasserviceable
andcompleteditspurpose,butcouldbeimprovedtobecomemoreeffective.

Ifwecanprovideanyadditionalinformationasyoureviewthisreport,pleasedonothesitatetocontact
us.Welookforwardtohearingyourthoughtsonthesubject.Wehopethatthisreportmeetsyour
expectations.

VeryRespectfully,

KevinErkkila
MatthewMansfield
KellyRouse

CraigslistiOSApplicationUsabilityTesting
FinalReport

KevinErkkila
MatthewMansfield
KellyRouse

HU3120TechnicalandProfessionalCommunication

KeshabAcharya

25April,2014

TableofContents

Abstract
Introduction
ProjectDescription
Methodology
TestResults
Findings
Recommendations
Conclusion
References
Appendices
AppendixA:PretestQuestionnaire
AppendixB:TestScenarios
AppendixC:PosttaskQuestionnaire
AppendixD:PosttestQuestionnaire
AppendixE:ConsentForm

4
5
5
6
7
8
9
9
9

10
11
12
13
15

Abstract
BecauseoftheneedtoadequatelytesttheCraigslistiOSmobileapplication,ourteamperformeda
formalusabilitytestwithseveralusersrepresentingthetargetagegroup.Wetested3peopleinthe18
to20yearoldagegroup,bothmaleandfemale,whohadatleastamoderateamountofexperience
withmobileapplications.

Ourtestsconsistedofwatchingthetestsubjectswhiletheydidaseriesoftasks.Thesetasksincluded
buyingandsellingitems,findingalocalmusician,andcreatinganddeletinganaccount,amongother
things.Thetestsweredesignedtoshowthestrengthsandweaknessesoftheapplication,asshownin
thisreport.

Ourconclusionsincludeddeterminingthatsomenavigationwasdifficult,andthatthereshouldbebetter
optionsforfindinghousing.Thatbeingsaid,wealsocandeterminethattheapplicationwasrelatively
userfriendly,andallthreeusersdeterminedthattheywouldrecommendittotheirfriends.

Introduction
InMarchof2014,itwasdeterminedthattheCraigslistiOSmobileapplicationneededtobehave
formalusabilitytestsperformed.BecausewedeterminedthatCraigslistwasarelativelynewsitethat
targetedayoungaudience,andthattheusersofApplesiOSaremostlyyoungindividuals,wedecided
todotestsonpeople18to20yearsofage.
Thefeaturesthatwedecidedtotestwereasfollows:
RegisteringforCraigslist
Postinganadvertisement
Findinganapartment
Findingabike
Changinglocation
Formingaband
Deletinganaccount

ProjectDescription
Thissectiondescribesthegoals,participantselectionanddemographics,andtestdesignofourusability
studyofCraigslistsmobileapplicationforiOS.

TestGoals
ThegoalofourstudywastodeterminethestrengthsandweaknessesoftheiOSversionofCraigslists
mobileapplication.ThiswasinordertoprovideCraigslistwithalistofsustainsandimprovesfortheir
developmentoftheapplication.

Background
Craigslistwasoriginallyfoundedin1996asanonlinemarketplacewhereindividualscanpostand
respondtoadvertisementsforgoodsandservices.Sincethen,ithasexpandedtoover700sitesin70
countries.Themobileapplicationhasbeenoutforseveralyears,buttheapplicationappearsdated.We
wantedtotesttheusabilityoftheapplication,withparticularfocusonaccountmanagement,postingand
respondingtoadvertisements,andusingthesearchfunction.

Participants
Ourtargetdemographicfortestingwereyoungindividualsfamiliarwithsmartphones,butnotwiththe
mobileapplicationforCraigslist.Toachievethis,werecruitedtwomenandandonewoman,allages
18to20,fromourfriendgroups.TwoofthesepeopleownedAndroiddevices,whileoneusedan
iPhone.Allthreehadbeenusingtheirdevicesforatleast18months.Oneindividualwasfamiliarwith
usingapplicationstobuyandsellgoodsfromtheirphone,butnoneofthemwerefamiliarwiththe
Craigslistapplicationspecifically.However,twoparticipantshadusedthedesktopbasedversionof
Craigslist.
5

Methodology
TestStructure
Weappliedthestructuredobservationsmodeloftestingtoourstudy.Participantswereprovidedwith
acopyofthetestproceduretofollow,aswellasquestionnairestofilloutduringthecourseoftesting.
WeconductedourtestinginastudyroomoftheMichiganTechlibrary.Wehadonepersonfacilitating
thetestprovidingguidancetotheparticipantsandansweringtheirquestions.Anotherpersonwas
recordingthespecificsofthetest,suchasthetimingofeachscenario.Athirdpersonwasaparticipant
observerinordertoseethereactionsoftheparticipants.

TestProcedure
Wedesignedasetof6scenarioseachtakingroughly5minutestocomplete.Wedesignedthescenarios
tobeaslifelikeaspossibletodeterminethetrueusabilityofthetests.Thepreandposttest
questionnairesthatwegavethesubjectswereusedtodeterminetheusersexperiencewithiOSand
mobiledevices,andtohelpdeterminetheresultsofthetest.

Scenario1:RegisteringanaccountforCraigslist.
5minutes

TaskA:Registeranaccount

Scenario2:Postinganadvertisement.
5minutes

TaskA:Postanadvertisementforapencilthatwasprovided

Scenario3:Findinganapartment
5minutes

TaskA:Findanapartmentthatallowsyoutohaveacatcostinglessthan$350/month

Scenario4:Findingabike
5minutes

TaskA:Changelocation
TaskB:Findabikeforsale

Scenario5:Formingaband
5minutes

TaskA:Findadrummer
TaskB:Findabassplayer

Scenario6:Deletinganaccount

5minutes

TaskA:Deletetheaccount

Weallocatedatotalof30minutesforthesubjectstocompletethe6tasks,andweallocated5minutes
toeachtaskindividually.Noneofthetaskswerethatcomplicated,andalloftheindividualsthatwehad
fortestingwaseasilyabletocompletethetestsinthetimegiven.

Asfarasthequestionnairesgo,wedesignedthemwithafewpurposesinmind:thepretest
questionnairewasmadetodeterminetheusersexperiencebeforewedidanythingelse.theposttask
questionnairewasdesignedtodeterminetheusersthoughtsonthetasksafterthefact.Theposttest
questionnairewasusedtodeterminethegeneralthoughtsoftheuseroneverything.

TestResults
TimingResults
Thethreetestparticipantscompletedthescenariosinthetimesshowninthetablebelow.

Scenario1 Scenario2 Scenario3 Scenario4

(5min)
(5min)
(5min)
(5min)

Scenario5
(5min)

Scenario6
(5min)

Participant1
(novice)

4min

5min

5min

5min

7min

30sec

Participant2
(novice)

5min

5min

5min

3min

3min

3min

Participant3
(novice)

2min

4min

4min

3min

2min

1min

Explanationsforexceededtimelimits.
Scenario5:Participant1exceededthetimelimitinscenario5becauseheseemedtohavelocatedthe
wrongpagewhiletryingtofindadrummerandabassplayertojoinhisband.

Typesofdatacollected
Quantitativedata:Ourquantitativedatawasthedatathatwewereabletomeasure,thatwouldallow
ustoanalyzehowtheparticipantsperformedandcomparedtotheotherparticipants.

Qualitativedata:Qualitativedataiswhatmakesupamajorityofourcollecteddata.Someofthe
informationwecollectedincludedthesubjectsopinionsandimpressionsoftheCraigslistapplication,as
wellaswellasitsqualityandfunctionality.

Quantitativedatacollected
Thefollowinginformationwascollectedduringtheusabilitytesting:
Timeittooktocompleteeachtask
Totalparticipantswhofinishedscenarioswithingiventime
Totalparticipantswhotookextratimetofinishscenarios
Totalproblemsencountered
Totalunsuccessfultriesbyparticipant
Numberoftimesparticipantsgotlostduringscenarios

Qualitativedatacollectedduringtesting
Throughoutthetestwecollectedqualitativedataconsistingofobservationsthesubjectsmadeaboutthe
Craigslistapplication.Theobservationsgiveusmoreinsightontheusabilityoftheapplication,andhelps
usunderstandthedifficultiesthatCraigslistusersencounter.
Wecollectedinformationincluding:
Numberoftimesparticipantsrereadthescenarios
Numberoftimestheparticipantslookedconfused
Numberoftimesthewrongpagewasaccessedontheapplication
Theparticipantsbodylanguage(calmvs.irritated)

Usabilitycriteria
Inorderforus,asthetesters,togethelpfulfeedbackontheCraigslistapplication,wehadthe
participantsfilloutposttaskandposttestquestionnaires.Thishelpedusidentifytheusabilityproblems
thatwereassociatedwiththeapplication.Wecreatedquestionsthatwouldtelluswhetherornotthe
applicationmetthefollowingcriteria:
1.Navigation:Wasiteasytogettowhereyouwantedintheapplication?
2.Usefulness:Wastheinformationprovidedofvalidusetotheuser?
3.Appearance:Didtheapplicationappealtotheuser?
4.Feedback:Doestheapplicationprovideappropriatefeedbacktotheuser?

Ourusabilitytestingresultsproducedfindingsinalloftheseareas.

Findings
WefoundavarietyofissueswiththeCraigslistmobileapplication.Firstofall,someusershaddifficulty
findingtheappropriatepagestomanagetheiraccountortochangetheirlocations.Thisisbasic
functionalitythatneedstobeeasytofind.Wealsofoundthatsearchingforadvertisementsbasedon
8

extracriteriawasclumsy.Usersstruggledtofiltertheresultstheyweregettingtofindtheappropriate
ad.Finally,wenoticedthattheentireapplicationwasvisuallyunappealing.

Recommendations
Basedoffoftheresultsofourtests,wehadanumberofrecommendedchangesfortheapplication.
First,werecommendthattheiconsbemademoreobviousforeasiernavigation.Atthemoment,the
iconsaredifficulttofind,andthatgreatlyincreasesthetimespentusingtheapplication.Anotherwayto
solvethisproblemwouldbetocreateahomepagewithlinkstootherplaces.

Anotherrecommendationthatwehadwastoreformatthelayouttobemoreuserfriendly.Thecurrent
layoutisunintuitive,andbecauseofthatitcausesalotofconfusion.Itisalsofairlyclutteredasitisright
now.

Thelastrecommendationthatwehaveistocreatenewiconsforadvancedsearches.asitisnow,there
isnotagoodwaytodosearches,becausethereisnowaytodoanykindofadvancedsearches.

Conclusion
Inconclusion,wefoundthatthereweremanywaysinwhichtheCraigslistiOSmobiledevice
applicationcouldbeimproved.Theapplicationhadafewissuesthatmadeitlessuserfriendly,and
becauseofthatwerecommendedchangestoimproveuserfriendliness.Ourresearchdeterminedthat
thebestwaytodothiswasthroughchangestothelayout,andmakingbuttonsmorevisible.Another
optionthatwaspresentedwastoaddinahomemenuthathadeasywaystogettoalargegroupof
placesintheapplication.

References
Gurak,LauraJ.,andJohnM.Lannon.StrategiesforTechnicalCommunicationinthe
Workplace.2nded.Boston:Pearson,2013.Print.
HotmailUsabilityTestFinalReport.PDFDocumentprovidedonCanvas.Listedas
UsabilityTestReportFinal(Sample.pdf

APPENDIXA:PretestQuestionnaire
Pleasefillinorcirclethefollowingquestions.

Name:_________________________Phone#:___________________________

Age:18202125263031394049505960+
Gender:
Male
Female

Pleaseanswerthefollowingquestionsaboutyoursmartphoneapplicationexperience:
1.DoyouuseaniPhoneorsmartphone?

Yes
No

2.Whatsystemdoyouprefertouse?

Android
Apple
Windows
Other?

3.Whichofthefollowingdoyouprimarilyusetoconnecttotheinternetwithonyourphone?

ConnecttoWiFi
Useserviceprovidersinternet

4.Whoisyourserviceproviderforyoursmartphone?

At&t
Verizon
Sprint
Other?

5.Howlonghaveyoubeenusingasmartphone?

Lessthan6months 612months
1218months
Morethan18months

6.Doyouuseyourphoneforbuying/sellingapplications?

Yes
No

7.HaveyouusedCraigslistonapersonalcomputer?

Yes
No

10

APPENDIXB:TestScenarios
Scenario1(5minutes)
Createanaccountusingthegiveninformationbelow.Thefacilitatorwillthenconfirmtheaccount.
EmailAddress:hu3120testing@gmail.com
Password:2014spring

Scenario2(5minutes)
Yourelookingtosellthepencilthatitisfrontofyou.Postanadforthispencil,andincludethe
followinginformation.
Descriptionoftheitem
Sellingpriceof$100orbestoffer
ListitinHoughton,MI

Scenario3(5minutes)
Yourelookingforaplacetorentforthefallof2014.Youneedsomethinginwalkingdistance
thatwillaccommodateyouandyourcat,andisunder$350/mo.

Scenario4(5minutes)
Yourelookingtobuyabicycletoridethissummer.Youareheadedhomerightafterexams,
andyoudratherbuyitinyourhometownsoyoudonothavetohaulit.

Scenario5(5minutes)
Youplayleadguitarandarelookingtostartabandtoplaygigsinthearea.Locateadrummer
andbassplayerwithintheareathatcouldpotentiallyjoinyourband.

Scenario6(5minutes)
Younolongerneedthisapplicationonyourphone.Pleasedeletethisaccount.

11

APPENDIXC:PosttaskQuestionnaires
(Scenarionumberandsubject)Thisquestionnaireisdesignedtotellushowyoufeelaboutthe
CraigslistAppbasedonthetaskyoujustperformed.Pleasecirclethenumberthatmostclearly
expresseshowyoufeelaboutaparticularstatement.

1.Usingtheapplicationwas:

1
2
3
4
5
Very
Easy
Moderate
Difficult
Very
Easy
Difficult

2.Navigatingthroughtheapplicationwas:

1
2
3
4
5
Very
Easy
Moderate
Difficult
Very
Easy

Difficult

3.Findingtherequiredinformationwithintheapplicationwas:

1
2
3
4
5
Very
Easy
Moderate
Difficult
Very
Easy

Difficult

Comments:

______________________________________________________________
______________________________________________________________
______________________________________________________________

12

APPENDIXD:PosttestQuestionnaire
Thisquestionnaireistotellusmoreaboutyourexperienceusingthephoneapplicationtoday.
Pleaseselectthenumberthatmostclearlyexpresseshowyoufeelabouteachtopic.

Asopposedtootherphoneapplicationsthatyouhaveused,ratethisapplicationintheareasdescribed
below.

Easeofnavigation:

1
2
3
4
5
Very
Easy
Neither
Difficult
Very
easy
easynor
difficult
difficult

Usefulnessoftheinformation:

1
2
3
4
5
Very
Useful
Neither
Notvery
Notuseful
useful
usefulnor
useful
atall
useless

Thelookandfeeloftheapplication:

1
2
3
4
5
Excellent
Good
Neutral
Notvery
Notappealing
Noopinion appealing
atall

Explainthebasisforyourrating.

Canyourecallhowdifferentsectionsoftheapplicationwereidentified?
___________Yes
___________No

Ifyes,pleaseexplainhowdifferentsectionsoftheapplicationareidentified.

Afterusingthisapplication,wouldyoubeinterestedinusingitagain?
__________Yes
__________No

Pleaseexplainthebasisforyouranswer.

13


Ifyouwouldbeinterestedinusingtheapplicationagain,whatinformationwouldyoulookfornexttime
youvisited?

Whatinformationwouldyouwanttohavethatyoudidntseeontheapplication?

ListthreeinterestingthingsthatyoulearnedaboutCraigslist.
1.
2.
3.

Asaresultinusingthisapplication,rateyouroveralllevelofsatisfactionordissatisfactionwiththeapp.

1
2
3
4
5
Very
Satisfied
Neither
Somewhat
Very
Satisfied
satisfiednor dissatisfied dissatisfied
dissatisfied

Pleaseexplainthereasoningofyourrating.

Afterusingthisapplication,wouldyouconsidermakinganaccountwithCraigslist?
__________Yes
__________No

Pleaseexplainthereasonofyouranswer.

Wouldyourecommendthatafriendcheckoutthisapplication?
__________Yes
__________No

Pleaseexplainthereasonforyourresponse.

14

APPENDIXE:CONSENTFORM

___________________________
(namehere)

IagreetoparticipateintheresearchtitledUndergraduateStudentUsabilityTestingofthe
CraigslistMobileApplication,whichisbeingconductedbyMatthewMansfield,KevinErkkila,and
KellyRouse,underthedirectionofKeshabAcharya,HumanitiesandTechnicalCommunication
Department,MichiganTechnologicalUniversity.Iunderstandthatthisparticipationisentirely
voluntaryIcanwithdrawmyconsentatanytimewithoutpenaltyandhavetheresultsofthe
participation,totheextentitcanbeidentifiedasmine,returnedtome,removedfromthe
researchrecords,ordestroyed.

Theresearchhasbeenexplainedtomeasfollows:

ThepurposeofthestudyistoevaluatetheusabilityoftheCraigsListMobileApplication.
Theresearcheswishtodeterminehoweasytheapplicationistouse.
Participantswillworkusingamobiledevicethatisprovided,withanaccountthatwillbe
provided.Theparticipantwillalsobesuppliedatasktocomplete.Whiletheparticipant
attemptsthetask,theywillbeobservedsothattheresearchteamcanreviewthe
participantsinteractions.Thetaskwilltakeapproximately30minutes.
Theparticipantsidentitywillbekeptentirelyanonymousinanytranscriptionsandreports
generatedbythistest.Anycommentsorexpressionsmadeduringthistestmaybeused
forthepurposeofevaluatingthemobileapplication.Allrecordsofthisresearchwillbe
handedovertothehumanitiesdepartment.

Theresearcherwillansweranyfurtherquestionabouttheresearch.

Pleasesignbothcopiesofthisform,andkeeponeofthem.

Signature:________________________________Date:_____________________

15