RESPONSE VARIATION IN E-MAIL SURVEYS: AN EXPLORATION

by Kim Bartel Sheehan A i tant Pr!"e !r# Uni$er ity !" Ore%!n an& Sally '( M)Millan A i tant Pr!"e !r# Uni$er ity !" Tenne ee

*ire)t )!rre +!n&en)e t! the "ir t a,th!r at: Kim Sheehan -./0 Uni$er ity !" Ore%!n E,%ene# OR 1/234 5 heehan6!re%!n(,!re%!n(e&, Ph!ne: 02--427-.388 9a:: 02--427-427.

The a,th!r ;i h t! than5 Mariea <!y an& =harle 9ra>er "!r their %,i&an)e in )!n)e+t,ali>in% an& )!n&,)tin% the re ear)h +r!?e)t re+!rte& in thi t,&y(

Re +!n e Variati!n in E-Mail S,r$ey RESPONSE VARIATION IN E-MAIL SURVEYS: AN EXPLORATION

Ab tra)t A e-mail an& !ther relate& te)hn!l!%ie ha$e &i"", e& ra+i&ly int! a lar%e an& heter!%ene!, +!+,lati!n# re ear)her ha$e be%,n t! e:+l!re the +! ibility !" , in% e-mail a a t!!l "!r ,r$ey re ear)h( <!;e$er# t,&ie !" the te)hni@,e ha$e +rimarily )!m+are& re +!n e rate "!r t,&ie that , e b!th e-mail an& +! tal mail ,r$ey te)hni@,e ( Re ear)h int! e-mail a a ,r$ey meth!& nee& t! &e$el!+ the 5in& !" ri)hne that i "!,n& in the literat,re !n tra&iti!nal +! tal mail ,r$ey meth!& ( Thi arti)le ta5e a "ir t te+ in &e$el!+in% that literat,re ;ith an in&e+th )!m+ari !n !" three t,&ie that , e& e-mail ,r$ey "!r &ata )!lle)ti!n( *etail are +r!$i&e& !n meth!& "!r am+lin% an& ,r$ey te)hni@,e ( <y+!the i>e& relati!n hi+ bet;een i ,e alien)e an& re +!n e rate# an& bet;een +re- ,r$ey n!ti"i)ati!n an& re +!n e time# ;ere %enerally ,++!rte&(

Re +!n e Variati!n in E-Mail S,r$ey

Re ear)her A in$e ti%ati!n !" )!m+,ter-me&iate& )!mm,ni)ati!n a a t!!l "!r )!n&,)tin% re ear)h an& )!lle)tin% )!n ,mer &ata ha been !n the in)rea e a Internet , a%e am!n% +e!+le ar!,n& the ;!rl& )!ntin,e t! %r!;( T!&ay# a many a -33 milli!n +e!+le ;!rl&;i&e ha$e a))e t! e-mail B*O=# -118C# 83 +er)ent !" all , er l!% !n t! the Internet !n a &aily ba i # an&

the &em!%ra+hi) +r!"ile !" Internet , er in the Unite& State i be%innin% t! mirr!r that !" the %eneral +!+,lati!n BKeh!e# Pit5!; an& M!rt!n# -11/C( *ata )!lle)te& $ia h!me +a%e ,r$ey !n the D!rl& Di&e Deb B ,)h a the Ee!r%ia Te)h t,&ie C are the m! t +,bli)i>e& e""!rt "!r )!lle)tin% in"!rmati!n $ia the Internet( <!;e$er# re ear)her Be(%( Ba)hmann# El"rin5 an& Va>>ana# -117F Deible an& Dalla)e# -118F S)hae"er an& *illman# -118C re)ently ha$e be%,n t! analy>e the , e !" ele)tr!ni) mail Be-mailC t! &i eminate ,r$ey an& )!lle)t &ata( Re ear)h int! the $iability !" e-mail a a ,r$ey meth!& ha "!), e& +rimarily !n )!m+arin% re +!n e rate an& re +!n e +ee& !" e-mail ,r$ey t! th! e !" +! tal mail( O$erall# the e t,&ie mail ha %reat +!tential "!r ,r$ey re ear)her ( Re ear)her ha$e re+!rte& a ;i&e $ariati!n in re +!n e rate an& +ee& !" re +!n e "!r email ,r$ey B ee Table -C( Thi i n!t ,r+ri in% be)a, e !" the $ariety !" am+le +!+,lati!n an& re ear)h t!+i) re+!rte& in th! e t,&ie ( A re ear)her +lannin% an e-mail ,r$ey t!&ay ha minimal in"!rmati!n !n ;hi)h t! ba e e timati!n !" re +!n e rate an& there"!re ;ill ha$e &i""i),lty in &eterminin% am+le i>e( The limite& +,bli he& re ear)h !n e-mail meth!&!l!%y al ! +r!$i&e little in"!rmati!n t! a i t re ear)her ;ith !ther ba i) re ear)h &e i%n i ,e ,)h a ,%%e t that e-

@,e ti!nnaire &e$el!+ment an& re +!n&ent )!nta)t ( Re ear)her ha$e a ;ealth !" in"!rmati!n !n re +!n e e""e)t "!r +! tal mail ,r$ey b,t the literat,re a&&re in% ,)h e""e)t "!r e-mail ,r$ey i minimal( The +,r+! e !" thi t,&y i t! +r!$i&e re ear)her ;ith in"!rmati!n that )an a i t in the

&e i%n an& im+lementati!n !" e-mail ,r$ey re ear)h( The literat,re !n re +!n e e""e)t in +! tal

Re +!n e Variati!n in E-Mail S,r$ey mail ,r$ey +r!$i&e a "rame;!r5 "!r &i ), i!n !" 5ey &e i%n i ,e ( Sam+lin% an& ,r$ey te)hni@,e "!r three t,&ie that , e& e-mail ,r$ey are &e )ribe& in &etail( 9inally# ;e e:amine the im+a)t !" t!+i) alien)e an& +re-n!ti"i)ati!n# t;! 5ey +re&i)t!r !" re +!n e in a +! tal mail ,r$ey # !n re +!n e rate !" e-mail ,r$ey ( Re$ie; !" the Literat,re E-Mail S,r$ey E-mail ha been )hara)teri>e& a a G+r!mi in% mean "!r )!n&,)tin% ",t,re ,r$ey H BS)hae"er an& *illman# -118C# an& n,mer!, re ear)her ha$e re)!%ni>e& the bene"it that e-mail +r!$i&e !$er +! tal mail( The e bene"it in)l,&e )! t a$in% "r!m eliminati!n !r re&,)ti!n !" +a+er )! t an& mailin% )! t BPar5er# -11.C an& the ra+i& +ee& !" re +!n e BBa)hmann# El"rin5 an& Va>>ana# -117F Mehta an& Si$a&a # -110C( In "a)t# a )!n i tent "in&in% !" the t,&ie that )!m+are re +!n e +ee& !" ,r$ey &eli$ere& $ia e-mail an& +! tal mail i that e-mail re +!n e are ret,rne& m,)h m!re @,i)5ly than +! tal mail re +!n e BBa)hmann# El"rin5 an& Va>>ana# -117F Kie ler an& S+r!,ll# -187F S)hae"er an& *illman# -118F Deible an& Dalla)e# -118C( In the e t,&ie # e-mail re +!n e +ee& ran%e& "r!m "i$e t! ten &ay # )!m+are& t! the re +!n e +ee& !" +! tal mail ,r$ey # ;hi)h ran%e& "r!m ten t! "i"teen &ay B ee Table -C( Re +!n e rate t! e-mail ,r$ey # h!;e$er# &! n!t )!n i tently h!; bene"it !$er +! tal mail# an& in !me )a e "all bel!; ;hat may be een a a))e+table le$el !" re +!n e( Kie ler an& S+r!,ll B-187C an& Par5er B-11.C re+!rte& e-mail re +!n e rate !" !$er 70 +er)ent# ;ith b!th t,&ie h!;in% e-mail re +!n e rate i%ni"i)antly hi%her than the )!m+arable +! tal mail

meth!&( S)hae""er an& *illman B-118C an& Mehta an& Si$a&a B-117C "!,n& n! i%ni"i)ant &i""eren)e in re +!n e rate bet;een the t;! m!&e ( Se$eral !ther t,&ie Be(%( S)h,l&t an& T!tten# -112F T e et al# -110F Deible an& Dalla)e# -118C "!,n& that e-mail re +!n e rate ;ere

Re +!n e Variati!n in E-Mail S,r$ey l!;er than th! e !" +! tal mail( Re +!n e rate "!r e-mail ,r$ey B ee Table -C $ary "r!m a l!; !" 7 +er)ent BT e et al# -110C t! a hi%h !" /0 +er)ent BKie ler an& S+r!,ll# -187C(

The e &i""eren)e in re +!n e rate are n!t ,r+ri in% %i$en ;hat i 5n!;n ab!,t re +!n e e""e)t in +! tal mail ,r$ey ( The t,&ie h!;n in Table - ha$e h!m!%ene!, am+le # mall

am+le i>e # an& &i$er e ,r$ey t!+i) ( The ty+e !" am+le +!+,lati!n are either em+l!yee !" a in%le )!m+any B, e& in t;! t,&ie C !r Uni$er ity +r!"e !r an& *ean B, e& in "i$e t,&ie C# ;ith !nly !ne t,&y )!n i tin% !" a am+le !" Internet , er BMehta an& Si$a&a # -110C( S,r$ey t!+i) ran%e& "r!m )!r+!rate an& Internet )!mm,ni)ati!n t! b, ine ethi) an& TIM( Ei$en the la)5 !" )!n i ten)y in n,mer!, $ariable in the e t,&ie # the ran%e !" re +!n e rate an& +ee& i ,n&er tan&able( Dhat i mi in% "r!m the ),rrent b!&y !" re ear)h i a )!m+ari !n !" e-mail ,r$ey re +!n e bey!n& the im+le )!m+ari !n t! re +!n e rate !" +! tal mail ,r$ey ( The b!&y !" 5n!;le&%e ab!,t +! tal mail ,r$ey meth!&!l!%y ,%%e t a n,mber !" i ,e that m, t be )!n i&ere& &,rin% the &e i%n an& im+lementati!n !" a +! tal ,r$ey an& that ha$e the +!tential t! e""e)t re +!n e rate an& +ee&( The e e""e)t may al ! be rele$ant "!r e-mail ,r$ey ( P! tal Mail S,r$ey A re$ie; !" the rele$ant literat,re re%ar&in% +! tal mail meth!&!l!%y ,%%e t that n,mer!, &e i%n an& im+lementati!n i ,e may e""e)t b!th re +!n e rate an& +ee& in thi m!&e( The literat,re i ri)h in meta-analy e that +r!$i&e in&i)ati!n !" ,)h e""e)t # an& many !" the e i ,e ;ill al ! be rele$ant "!r e-mail t,&ie ( The literat,re ha re+!rte& the "!ll!;in% e""e)t in +! tal mail ,r$ey : Per !nali>ati!n !" )!$er letter( Per !nali>in% letter a&&re e& t! +e)i"i) in&i$i&,al ha been h!;n t! in)rea e re +!n e rate in !me +! tal mail ,r$ey B*illman# -1/8F -11-C# ;hile !ther B*,n)an# -1/1C "!,n& n! e""e)t !n re +!n e rate &,e t! )!$er letter +er !nali>ati!n( In e-

Re +!n e Variati!n in E-Mail S,r$ey mail ,r$ey # the i ,e !" +er !nali>ati!n i )!m+le:( A )ertain &e%ree !" +er !nali>ati!n ;ill be a,t!mati) in e-mail be)a, e the in&i$i&,alA e-mail a&&re ;ill a++ear !n a hea&er that i !"ten

$i ible thr!,%h!,t the rea&in% !" a me a%e BS)hae"er an& *illman# -118C( Bey!n& thi # h!;e$er# e-mail )an be +er !nali>e& ;ith a %reetin% !r !me !ther ty+e !" rele$ant in"!rmati!n that relate +e)i"i)ally t! the re)i+ient( P! ta%e( The )!n en , am!n% re ear)her a++ear t! be that in)l,&in% a tam+e& en$el!+e B$er , a b, ine re+ly en$el!+eC +r!&,)e hi%her re +!n e rate in +! tal mail ,r$ey BArm tr!n% an& L, 5# -18/F 9!:# =ra 5 an& Kim# -188F Yammarin!# S5inner an& =hil&er # -11-C( Thi e""e)t i n!t yet rele$ant t! e-mail be)a, e +! ta%e i n!t yet nee&e&( <!;e$er# in&i$i&,al ;h! +ay "!r e-mail , a%e either by the me a%e !r by the am!,nt !" time +ent !nline# may "eel that the re ear)her h!,l& +r!$i&e !me mall reimb,r ement "!r that )! t( The e )! t may al ! limit the re +!n e +!tential# a e-mail re)i+ient may a,t!mati)ally &elete the me a%e in !r&er t! a$!i& ,)h )! t ( 9in&in% a ;ay t! a&&re In)enti$e ( Small )a h in)enti$e the e i ,e may )hallen%e re ear)her (

ent ;ith the maile& ,r$ey )an in)rea e re +!n e rate

B9!:# =ra 5 an& Kim# -188F E!y&er# -18.F Y, an& =!!+er# -184C( <!;e$er# &imini hin% ret,rn !n the i>e !" the in)enti$e are e$i&ent# in&i)atin% that in)rea in% the i>e !" the in)enti$e &!e n!t ne)e arily in)rea e the re +!n e rate( It i n!t ),rrently +! ible t! +r!$i&e m!netary in)enti$e thr!,%h e-mail# alth!,%h it i +! ible t! +r!$i&e !ther ty+e !" in)enti$e B ,)h a the !""er !" harin% re ear)h re ,lt C( Re ear)her h!,l& )!n i&er ;ay t! &e$el!+ +! ible in)enti$e that

mi%ht be Gatta)he&H t! e-mail( 9!r e:am+le# &i )!,nt )!,+!n "r!m an !nline $en&!r mi%ht be +r!mi e& t! in&i$i&,al ;h! )!m+lete the ,r$ey( S+!n !r hi+( Meta-analy e B9!:# =ra 5 an& Kim# -188F E!y&er# -18.F <eberlein an& Ba,m%artner# -1/8C ,%%e t that +!n !r hi+ !" a t,&y by a Uni$er ity )an re ,lt in hi%her re +!n e rate "!r +! tal mail ,r$ey than )an +!n !r hi+ "r!m a )!r+!rati!n( <!;e$er#

Re +!n e Variati!n in E-Mail S,r$ey Yammarin!# S5inner an& =hil&er B-11-C &i& n!t "in& ,++!rt "!r the $al,e !" a Uni$er ity +!n !r hi+ t! e""e)t re +!n e rate( S+!n !r hi+ !" e-mail ,r$ey )ann!t be a e:+li)it a ;ith +! tal mail ,r$ey Bi(e( the , e !" a +!n !rin% !r%ani>ati!nA letterhea& i n!t a$ailableC# b,t +!n !r hi+ )an be ma&e im+li)itly thr!,%h tatement in the ,r$ey in tr,ment an& thr!,%h the en&erA e-mail a&&re e Bi(e( an G(e&,H ,""i: !n an a&&re e&,)ati!nal in tit,ti!nC( I,e ti!nnaire &e i%n( *e i%n i ,e # ,)h a the len%th !" the @,e ti!nnaire# )an e""e)t re +!n e( The l!n%er the @,e ti!nnaire# the le li5ely +e!+le are t! re +!n& B<eberlein an& Ba,m%artner# -1/8F Steele# S)h;en&i% an& Kil+atri)5 -11.F Yammarin!# S5inner an& =hil&er # -11-C( Thi e""e)t i hi%hly rele$ant t! e-mail ,r$ey # ;here ,r$ey len%th may be mea ,re& n!t !nly in the n,mber !" +rinte& +a%e b,t al ! in term !" )reen len%th Bthe n,mber !" )reen )!ntainin% the ,r$eyC( Be)a, e an a$era%e +rinte& +a%e )an ta5e ,+ t;! !r three )!m+,ter )reen # re +!n&ent may be "a)e& ;ith +re ,mably len%thy ,r$ey !" a &!>en )reen !r m!re( ;!,l& in&i)ate a !)iati!n ;ith an

An!nymity( Dhile !me re ear)her ha$e "!,n& that an!nymity in)rea e re +!n e rate t! +! tal mail ,r$ey BYammarin!# S5inner an& =hil&er # -11-C# !ther t,&ie ha$e in&i)ate& that thi i n!t ne)e arily tr,e B*,n)an# -1/1F Kan,5 an& Beren !n# -1/0C( Thi i a 5ey i ,e "!r email ,r$ey be)a, e it i &i""i),lt t! a)hie$e tr,e an!nymity in that m!&e( T! &! ! re@,ire re +!n&ent t! a))e an!nym!, remailer t! re +!n&# an& thi may be bey!n& the te)hni)al

)!m+eten)e !" !me Internet , er ( <!;e$er# re ear)her )an a ,re e-mail ,r$ey re +!n&ent !" )!n"i&entiality by in"!rmin% them that their e-mail a&&re e ;ill n!t be re)!r&e& ;ith their ,r$ey re +!n e an& that the ,r$ey &ata ;ill be )!n i&ere& !nly in the a%%re%ate( I ,e Salien)e( In +! tal mail ,r$ey # the alien)e !" an i ,e t! the am+le& +!+,lati!n ha been "!,n& t! ha$e a tr!n% +! iti$e )!rrelati!n ;ith re +!n e rate( Salien)e ;a &e"ine& a t!+i) that &ealt ;ith an im+!rtant i ,e that ;a al ! ),rrent !r timely BMartin# -110C( <eberlein

Re +!n e Variati!n in E-Mail S,r$ey an& Ba,m%artner B-1/8C "!,n& that i ,e alien)e ha& a tr!n%er im+a)t !n re +!n e rate than

&i& any !ther i ,e !r re ear)h &e i%n &e)i i!n in)l,&in% a&$an)e n!ti)e# "!ll!;-,+ )!nta)t # !r m!netary in)enti$e ( R!ber !n an& San& t!rm B-113C an& Martin B-110C al ! "!,n& that alien)e ;a a 5ey +re&i)t!r !" re +!n e rate "!r +! tal mail ,r$ey ( Un&er tan&in% the +!+,lati!n t! be am+le& i an im+!rtant "ir t te+ in &eterminin% i ,e alien)e( Re ear)her ;h! , e e-mail ,r$ey may be able t! be%in t! +re&i)t re +!n e rate !n the ba i !" h!; alient an i ,e i t! the in&i$i&,al ;h! ;ill be !li)ite& t! +arti)i+ate in the e-mail ,r$ey( Re +!n&ent =!nta)t ( 9!:# =ra 5 an& Kim B-188C "!,n& that +re-n!ti"i)ati!n by letter le& t! in)rea e in re +!n e rate "!r +! tal mail ,r$ey ( <!;e$er <eberlein an& Ba,m%artner B-1/8C "!,n& little !r n! e""e)t a !)iate& ;ith +re-n!ti"i)ati!n( Se$eral t,&ie !" +! tal mail ,r$ey BKan,5 an& Beren !n# -1/0F M,r+hy# *aley an& *alenber%# -11-F Tayl!r an& Lynn# -118C "!,n& re +!n e +ee& ;a "a ter "!r +re-n!ti"ie& re +!n&ent than "!r th! e ;h! ;ere n!t +re-n!ti"ie&( Yammarin!# S5inner an& =hil&er B-11-C ,%%e te& that "!ll!;-,+ mailin% an& re+eate& )!nta)te& eeme& t! ha$e a %reater e""e)t !n re +!n e rate am!n% th! e ;h! re)ei$e the ,r$ey be)a, e !" an in tit,ti!nal a""iliati!n than am!n% th! e ;h! re)ei$e a %eneral )!n ,mer ,r$ey( Little )!n en , ;a "!,n& !n the $al,e !" m,lti+le +re- an& +! t- ,r$ey )!nta)t in +! tal mail-ba e& ,r$ey ( Re ear)her , in% +! tal mail "!r &eli$ery !" me a%e m, t ;ei%h the +!tential bene"it !n re +!n e rate a%ain t the )! t !" m,lti+le mailin% ( Be)a, e +ee& !" re +!n e ha been een a a 5ey bene"it t! e-mail ,r$ey # enhan)in% re +!n e +ee& i im+!rtant "!r re ear)her ;h! ;i h t! ma:imi>e the +!tential !" the m!&e( An& be)a, e m! t re ear)her )an en& m,lti+le e-mail me a%e "!r little !r n! )! t# the im+a)t !" m,lti+le )!nta)t !n re +!n e be)!me a hi%hly rele$ant ,b?e)t "!r e-mail ,r$ey (

Re +!n e Variati!n in E-Mail S,r$ey <y+!the e Thi t,&y re+re ent a "ir t te+ in e:aminin% e-mail !n the ba i !" meth!&!l!%i)al

"a)t!r that %r!; "r!m the ri)h literat,re !n +! tal ,r$ey ( T;! hy+!the e re%ar&in% re +!n e e""e)t ha$e been &e$el!+e& ba e& !n the "inal t;! "a)t!r re$ie;e& ab!$e( < -: Rate !" re +!n e t! e-mail ,r$ey ;ill in)rea e a i ,e alien)e in)rea e ( < .: S+ee& !" re +!n e t! e-mail ,r$ey ;ill be "a ter "r!m in&i$i&,al ;h! re)ei$e& a +re-n!ti"i)ati!n !" the ,r$ey than "r!m th! e ;h! &i& n!t re)ei$e +re-n!ti"i)ati!n( Meth!&!l!%y Three e+arate t,&ie that , e& e-mail "!r )!lle)ti!n !" &ata ;ere e:amine& "!r thi arti)le( St,&y - ;a )!n&,)te& in early -11/F in&i$i&,al in$ite& t! re +!n& t! the ,r$ey ;ere all &e$el!+er !" health-relate& Deb ite ( St,&y . ;a )!n&,)te& in the ,mmer !" -117F re +!n&ent ;ere "a),lty an& t,&ent at a ma?!r !,thea t re ear)h ,ni$er ity( *ata "!r St,&y 4 ;ere )!lle)te& "r!m N!$ember -11/ thr!,%h 'an,ary -118 "r!m Internet , er ;ith a +er !nal email a))!,nt in the Unite& State ( *e +ite the &i""erent am+lin% "rame # the t,&ie ;ere imilar in e$eral ;ay B ee Table .C( The ,r$ey in tr,ment ;ere )!m+arable in term !" len%th !" the ,r$ey an& ty+e !" )ale , e& t! an ;er the ,r$ey ( All t,&ie menti!ne& a Uni$er ity a""iliati!n# an& , e& a "!ll!;-,+ remin&er me a%e( Re ear)h re ,lt ;ere !""ere& in all t,&ie a an in)enti$e# an& all t,&ie +r!mi e& )!n"i&entiality !" re +!n e ( <!;e$er# the t,&ie &i""er in t;! 5ey ;ay ( 9ir t# the t,&ie &i""er in i ,e alien)e( The t!+i) !" St,&y - ;a hi%hly alient t! the ,b?e)t +!+,lati!n( St,&y - a 5e& )reat!r !" health-relate& Deb ite t! +r!$i&e in"!rmati!n ab!,t the ite they ha& )reate& Be(%( ;hen it be%an# +,r+! e !" the ite# et)(C a ;ell a m!re %eneral in"!rmati!n ab!,t the in&i$i&,alA +er)e+ti!n !" the Deb( Th, # the in&i$i&,al ha& a &ire)t +er !nal intere t in m! t !" the @,e ti!n ( The t!+i) !" St,&y . ;a m!&erately alient t! the +!+,lati!n( The t!+i) ;a intr!&,)e& t! re +!n&ent a a t,&y !" Internet , a%e habit # an&

Re +!n e Variati!n in E-Mail S,r$ey

-3

the alien)e "!r thi %r!,+ ;a the "a)t that the in&i$i&,al )!lle)tin% the in"!rmati!n ;a a t,&ent at the ,ni$er ity ;here the re +!n&ent ;ere a""iliate&( The t!+i) !" St,&y 4 ;a n!t alient t! the ,b?e)t +!+,lati!n( It ;a +re ente& a a G&!)t!ral t,&ent ,r$eyH !" Internet , a%e habit ( Se)!n&# the t,&ie &i""er in term !" +re-n!ti"i)ati!n( St,&y - &i& n!t +re-n!ti"y re +!n&ent # St,&ie . an& 4 &i& en& a +re-n!ti"i)ati!n me a%e( Thi +re-n!ti"i)ati!n e-mail me a%e e:+laine& the +,r+! e !" the re ear)h an& n!ti"ie& ,b?e)t that they ;!,l& re)ei$e a ,r$ey ;ithin a &e i%nate& time +eri!&( S,b?e)t ;ere t!l& that they )!,l& &e)line t! +arti)i+ate by re+lyin% t! thi "ir t me a%e an& a 5in% that the ,r$ey n!t be ent( Thi te)hni@,e i imilar t! &ire)t mar5etin% +ra)ti)e , e& by !r%ani>ati!n en&in% an item ,nle the )!n ,mer &e)line ( Le ,)h a b!!5 an& re)!r& )l,b that &e"a,lt t! than t;! +er)ent !" the ,ni$er ity "a),lty#

ta"" an& t,&ent !" St,&y . &e)line& t! +arti)i+ate( Sli%htly m!re than -- +er)ent !" the in&i$i&,al ;ith +er !nal e-mail a))!,nt in St,&y 4 &e)line& t! +arti)i+ate in thi ,r$ey( Be)a, e the +r!)e&,re "!r )!n&,)tin% e-mail ,r$ey are relati$ely ne;# a&&iti!nal meth!&!l!%i)al in"!rmati!n i +r!$i&e& t! %,i&e re ear)her ;h! ;i h t! , e thi te)hni@,e( In +arti),lar# ;e +r!$i&e in"!rmati!n !n am+lin% an& ,r$ey te)hni@,e , e& "!r the e three t,&ie ( Sam+lin% St,&y - BDeb-Site *e$el!+er C( The Yah!! &ire)t!ry !" health-relate& Deb ite ;a , e& a the ,ni$er e "!r St,&y -( Yah!! &!e n!t e@,entially n,mber ite ;ithin it )ate%!rie ( There"!re# a trate%y ;a nee&e& "!r i&enti"yin% the n,mber !" ite in the ,ni$er e# ran&!mly ele)tin% ite # an& &eterminin% h!; t! a++ly ran&!m n,mber t! +e)i"i) ite ( The i>e !" the ,ni$er e ;a &etermine& by a&&in% t!tal that Yah!! re+!rte& "!r ea)h health-relate& )ate%!ry !n the G!+enin% +a%eH B!r in&e:C !" the Yah!! )ate%!ry li tin% ( At the time !" the t,&y# the n,mber "!ll!;in% ea)h !" the e )ate%!rie ;ere a&&e& t! !btain the ,ni$er e i>e !" -2#/12(

Re +!n e Variati!n in E-Mail S,r$ey A li t !" -#303 ,ni@,e ran&!m n,mber ran%in% "r!m - t! -2#/12 ;a &ra;n( The e

--

n,mber ;ere a++lie& by tartin% at the t!+ !" the "ir t !n-line +a%e !" Yah!! li tin% an& ;!r5in% thr!,%h t! the b!tt!m !" that +a%e( 9!r e:am+le# the "ir t item !n the "ir t +a%e !" the health )ate%!ry ;a a t!+i) li tin% "!r Alternati$e Me&i)ine( Dhen that li tin% ;a ele)te&# an!ther

)reen a++eare& ;hi)h in)l,&e& a&&iti!nal ,b-)ate%!rie Be(%(# in tit,te # ma%a>ine # et)(C a ;ell a &ire)t lin5 t! ite ( S,b-)ate%!rie ;ere "!ll!;e& ,ntil they re ,lte& in a ite li tin%( 9!,r "a)t!r le& t! re&,)ti!n !" the am+le i>e( 9ir t# !me ite ha& t! be eliminate& be)a, e they &,+li)ate& either the URL B,ni"!rm re !,r)e l!)at!r J the a&&re e-mail a&&re a&&re !" the iteC !r the

!" an!ther ite( In ea)h )a e# the "ir t ite ;ith the &,+li)ate URL an&K!r e-mail

;a 5e+t an& the e)!n& ;a &i )ar&e&( Se)!n&# !me URL li te& by Yah!! ;ere n!t

",n)ti!nin%( Thir&# !me ite &i& n!t ha$e an e-mail a&&re ( 9inally# in !me )a e an e-mail a&&re ;a "!,n& b,t e-mail ;a ,n&eli$erableF -8(7 +er)ent !" the ele)te& ite )!,l& n!t be

)!nta)te& be)a, e !" mi in% !r n!n-",n)ti!nin% e-mail a&&re e ( A"ter a&?, tin% "!r the ab!$e "a)t!r # the t!tal $ali& am+le i>e "!r St,&y - ;a 842( St,&y . BUni$er ity 9a),lty# Sta"" an& St,&ent C( The ,ni$er ity &ire)t!rie !" "a),lty# ta"" an& t,&ent +r!$i&e& the am+le "!r St,&y .( The e &ire)t!rie +r!$i&e name # a&&re ee # +h!ne n,mber # an&# ;here a$ailable# e-mail an& Deb +a%e a&&re e "!r in&i$i&,al li te&( A t!tal !" 083 name ;ere ele)te&F t;! thir& B487C ;ere "r!m the "a),ltyK ta"" &ire)t!ry an& !nethir& B-12C ;ere "r!m the t,&ent &ire)t!ry( A ran&!m n,mber ;a , e& t! ele)t the "ir t +a%e "r!m ;hi)h t! &ra; name ( Be%innin% !n that +a%e# e$ery ei%hth name ;a ele)te& ,ntil the it ;a

&e ire& n,mber !" name ;a rea)he&( I" the ei%hth name &i& n!t ha$e an e-mail a&&re 5i++e& an& re+la)e& ;ith the ne:t entry that in)l,&e& an e-mail a&&re (

St,&y 4 BIn&i$i&,al ;ith Per !nal E-mail A))!,nt C( A t;!- ta%e am+lin% te)hni@,e ;a , e& "!r St,&y 4( 9ir t# the re ear)her ran&!mly ele)te& Internet Ser$i)e Pr!$i&er "r!m an !n-

Re +!n e Variati!n in E-Mail S,r$ey line li t( *,rin% thi ele)ti!n# the re ear)her )he)5e& t! en ,re that the ISP +r!$i&e& er$i)e t! ta%e(

-.

in&i$i&,al rather than b, ine e ( A t!tal !" 00 ISP ;ere ele)te& in thi

In the e)!n& ta%e# the ISP &!main name Be(%( earthlin5()!mC ;a entere& in the 9!,r-ear)h en%ine( Thi ear)h en%ine i &e i%ne& t! "in& in&i$i&,al A e-mail a&&re e ( The ear)h en%ine )!,l& ear)h , in% any )!mbinati!n !" "ir t name# la t name# &!main name# tate# an& )!,ntry( BSin)e the time that thi t,&y ;a ,n&erta5en# the 9!,r-- ear)h en%ine ;a !l& t!

Yah!!( The ear)h +arameter all!;e& by the er$i)e ha$e been )han%e&C( 9!r St,&y 4# the !nly ear)h +arameter ;ere that in&i$i&,al m, t be in the Unite& State an& that they m, t ha$e ha& an a))!,nt ;ith a +e)i"i) ISP( Dith the e +arameter # 9!,r-+r!&,)e& a li t !" +er !n ;h! ha& an e-mail a&&re ;ith the +e)i"ie& ISP( I" .33 !r "e;er

in&i$i&,al , e& that ISP "!r e-mail# all e-mail a&&re e ;ere +r!$i&e&( I" m!re than .33 in&i$i&,al ha& e-mail a))!,nt ;ith the ISP# a ran&!m am+le !" .33 e-mail a&&re e ;a %enerate&( In thi )a e# the t!tal n,mber !" in&i$i&,al ;ith e-mail a&&re e ;a al ! re+!rte&( E-mail li t +r!$i&e& by the 9!,r-- ear)h en%ine ;ere am+le& a nee&e& t! en ,re that "!r ea)h ISP the ,r$ey re)i+ient ;!,l& be &ra;n at ran&!m( Three ty+e !" am+lin% ;ere , e&( 9ir t# i" the 9!,r-- ear)h en%ine re+!rte& that m!re than -#333 e-mail a&&re e ;ere +r!$i&e& by the ISP# then the .33 ran&!m e-mail a&&re e ret,rne& by 9!,r-- ;ere , e&( Se)!n&# i" the 9!,r-- ear)h en%ine ret,rne& .33 ran&!m e-mail a&&re e b,t re+!rte& that -#333 !r "e;er in&i$i&,al ha& an e-mail a))!,nt ;ith the ISP# ;e "elt it ne)e ary t! am+le( In thi )a e# !ne"!,rth !" the .33 e-mail a&&re e ;a ele)te& at ran&!m( 9inally# i" "e;er than .33 e-mail

a&&re e ;ere ret,rne&# ;e am+le& "r!m the li t +r!$i&e& by 9!,r--( One-"!,rth !" the e a&&re e ;ere ele)te& at ran&!m( U in% the e am+lin% te)hni@,e # 0#333 e-mail a&&re e ;ere ele)te& t! re)ei$e the ,r$ey( Pre-te t in&i)ate& that a ,b tantial +er)enta%e !" e-mail a&&re e ;!,l& n! l!n%er be a)ti$e( In thi )a e# -#./7 !" the e-mail a&&re e ;ere n!t &eli$erable(

Re +!n e Variati!n in E-Mail S,r$ey S,r$ey Te)hni@,e

-4

A +re$i!, ly menti!ne&# many tan&ar& ,r$ey te)hni@,e ;ere , e& in all three t,&ie ( All ,b?e)t ;ere %i$en the !+ti!n n!t t! re +!n&( =!n"i&entiality !" re +!n&ent ;a a ,re&( The intr!&,)ti!n t! the ,r$ey in)l,&e& an e timate !" h!; m,)h time ;!,l& be re@,ire& "!r )!m+letin% the ,r$ey( <!;e$er# the e e-mail ,r$ey &i""ere& "r!m +! tal ,r$ey te)hni@,e in !ther ;ay ( One ma?!r &i""eren)e bet;een the e t,&ie an& +! tal ,r$ey re ear)h i )! t( Primary )! t "!r the !n-line ,r$ey ;ere +a+er an& +rinter )artri&%e "!r +rintin% re ,lt an& time "!r ,+l!a&in% an& &!;nl!a&in% me a%e ( De e timate that +! tal mail ,r$ey )! t at lea t -3 time a m,)h a e-mail ,r$ey ( Three a&&iti!nal te)hni@,e are &i""erent in the e-mail ,r$ey "!rmat: en&in% e-mail# remin&er # an& re +!n&in%( Sen&in% E-mail( 9!r St,&y -# in&i$i&,al me a%e ;ere ent t! ea)h ,b?e)t( E-mail a&&re e ;ere )!+ie& t! the e-mail me a%e "r!m a +re$i!, ly )reate& &ataba e( Dhile thi ;a time )!n ,min%# it enable& the re ear)her t! 5ee+ re)!r& !" ;hen me a%e ;ere ent an& re)ei$e& in the ame &ataba e that ;a , e& t! re)!r& re +!n e ( U in% thi te)hni@,e ab!,t .33 ,r$ey ;ere maile& ea)h &ay( Limitin% !,tb!,n& me a%e t! .33 +er &ay re&,)e& the +! ibility !" !$erl!a&in% either the e-mail y tem !r the re ear)herA e-mail in-b!:( The te)hn!l!%y !" en&in% ,r$ey by e-mail tem+t the re ear)her t! Gbat)hH e-mail a&&re e ( St,&y . , e& thi te)hni@,e( The re ear)her )reate& a li t !" ,b?e)t an& ent the me a%e t! all !" them in !ne me a%e( Dhile thi te)hni@,e a$e a %reat &eal !" time !n the "r!nt en& !" the +r!?e)t# it )an )reate +r!blem ;hen ,b?e)t re +!n&( *e +ite in tr,)ti!n # !me re +!n&ent , e& the Gre+ly t! allH ",n)ti!n !" their e-mail rea&er( Thi re ,lte& in +! ibly ann!yin% me a%e bein% re)ei$e& by all ,r$ey ,b?e)t ( S!me ba& ;ill %enerate& "r!m the e me a%e ;a tran "erre& t! the re ear)her(

Re +!n e Variati!n in E-Mail S,r$ey 9!r St,&y 4# a +r!%ram ;a ;ritten t! mer%e a li t !" e-mail a&&re e ;ith the ,r$ey

-2

an& t! en& th! e ,r$ey $ia e-mail( Thi te)hni@,e eliminate& the +r!blem !" m,lti+le re)i+ient "!r the ,r$ey me a%e an& it ;a m!re e""i)ient than the te)hni@,e , e& "!r St,&y -( U in% thi te)hni@,e# ab!,t 233 ,r$ey ;ere maile& ea)h &ay( <!;e$er# there i a +!tential &!;n i&e t! thi ty+e !" ma mailin%( B!th the en&in% an& re)ei$in% )!m+,ter y tem may re%i ter ,)h ma mailin% a G +amH !r ?,n5 mail( In "a)t# thi &i& ha++en( The re ear)herA ,ni$er ity a))!,nt ;a tem+!rarily , +en&e& ,ntil he )!,l& a ,re the y tem mana%er that he ;a )!n&,)tin% le%itimate re ear)h( 9,rtherm!re# !ne !" the ISP et ,+ a "ilter that +r!hibite& her "r!m

)!mm,ni)atin% ;ith any!ne ;h! ha& an e-mail a))!,nt thr!,%h that ISP( The e a&&re e ;ere then re+la)e& in the am+lin% +!!l( Remin&er ( A ;ith +! tal ,r$ey # a remin&er ;a ent i" a )!m+lete& ,r$ey ha& n!t

been re)ei$e& in a +e)i"ie& time( Thi a +e)t !" the meth!&!l!%y re$eal a i%ni"i)ant relati$e a&$anta%e !" e-mail ,r$ey ( Dith +! tal ,r$ey # the re ear)her m, t all!; time "!r the ,r$ey t! be re)ei$e& an& ret,rne& $ia +! tal er$i)e mail be"!re en&in% a remin&er( O"ten# a m!nth !r m!re ela+ e bet;een "ir t an& e)!n& mailin% ( 9,rtherm!re# re +!n e an& remin&er mi%ht G+a H in the mail( *,rin% the time it ta5e "!r the e tran a)ti!n t! !)),r# the ,b?e)t may "!r%et ;hether he !r he ha re +!n&e& t! the ,r$ey( The remin&er mailin% ;a ent "r!m "i$e t! e$en &ay a"ter the !ri%inal e-mail ,r$ey ;a

ent( Be)a, e !" the +ee& ;ith ;hi)h me a%e )!,l& be ent# there ;a little &an%er !" me a%e G+a in%H in e-mail( Dhile !me re +!n&ent in&i)ate& that they ha& n!t re)ei$e& the "ir t me a%e# many im+ly a+!l!%i>e& "!r bein% b, y an& @,i)5ly re +!n&e& t! the remin&er( The remin&er me a%e ent "!r the e t,&ie %enerate& "r!m .4 t! 28 +er)ent !" the t!tal re +!n e (

Re +!n&in%( S!me in&i$i&,al re+!rte& ha$in% &i""i),lty in re +!n&in% t! the ,r$ey( A"ter e-mail e:)han%e ;ith the e in&i$i&,al # it eeme& that the +rimary +r!blem ;a that they &i& n!t

Re +!n e Variati!n in E-Mail S,r$ey ele)t the Gre+lyH ",n)ti!n !" their e-mail +r!%ram( They eeme& t! be ,na;are that they )!,l&

-0

n!t ty+e re +!n e t! the ,r$ey ;hile they ;ere till in the Grea&H m!&e !" their e-mail +r!%ram( Other &i& n!t reali>e that they )!,l& in ert their re +!n e int! the b!&y !" the ,r$ey( Thi may eem !&& "!r +e!+le ;h! +r!bably , e e-mail ;ith !me re%,larity( De , +e)t that !ne rea !n "!r thi +r!blem may ha$e been the n!$elty !" an e-mail ,r$ey( S!meh!;# in&i$i&,al &i& n!t ma5e the )!nne)ti!n that thi ;a ?, t li5e any !ther e-mail J !ne m, t ele)t the re+ly ",n)ti!n be"!re re +!n&in% an& re +!n e )an be inter +er e& ;ith the !ri%inal me a%e( Re +!n&ent t! all three t,&ie ;ere !""ere& the !+ti!n !" +rintin% !,t the e-mail ,r$ey an& )!m+letin% an& ret,rnin% it in thi har& )!+y $er i!n( Thi !+ti!n ;a !""ere& be)a, e !me ,b?e)t mi%ht n!t "eel )!m"!rtable ;ith the +r!)e !" )!m+letin% the ,r$ey $ia e-mail# !r mi%ht

a++re)iate the an!nymity !" en&in% the ,r$ey thr!,%h +! tal mail( 9!r all ,r$ey # le than three +er)ent !" re +!n&ent )h! e thi !+ti!nF m! t !" the e re +!n&ent in)l,&e& their name !n the +a+er )!+y( Thi in&i)ate& t! , that an!nymity i n!t m,)h !" an i ,e "!r the e !n-line re ear)h ,b?e)t ( 9inally# re +!n&ent eeme& t! be m,)h m!re ;illin% t! re+ly t! !+en-en&e& @,e ti!n in

the e-mail "!rmat than in tra&iti!nal +a+er ,r$ey ( The rea !n may be that m! t !" the e re +!n&ent "in& it "a ter an& ea ier t! ty+e re +!n e than t! han& ;rite them( Al !# in !me )a e # re +!n&ent , e& the ),t-an&-+a te ",n)ti!nality !" &i%ital )!mm,ni)ati!n t! +r!$i&e an ;er t! !+en-en&e& @,e ti!n ( 9!r e:am+le# St,&y - a 5e& re +!n&ent the +,r+! e !" their Deb ite ( Many )!+ie& their mi i!n tatement "r!m their Deb ite an& +a te& them int! the email ,r$ey( The n!$elty !" the e-mail ,r$ey may ha$e al ! em+!;ere& in&i$i&,al t! a&& )!mmentary t! re +!n e , in% )ale ( 9!r e:am+le# many re +!n&ent a&&e& )!mmentary t! e:+lain ;hy they ele)te& a )ertain re +!n e !n a @,e ti!n(

Re +!n e Variati!n in E-Mail S,r$ey 9in&in% In )!m+arin% the e three t,&ie # !,r +rimary intere t ;a in h!; i ,e alien)e an& +re-

-7

n!ti"i)ati!n mi%ht e""e)t re +!n e rate an& re +!n e +ee&( De e:+e)te& that %r!,+ re)ei$in% the m!re alient ,r$ey ;!,l& ha$e hi%her re +!n e rate an& that in&i$i&,al ;h! ha& re)ei$e& a +re-n!ti"i)ati!n me a%e mi%ht be G+rime&H "!r the ,r$ey an& th, re +!n& m!re @,i)5ly( <y+!the i <y+!the i - +re&i)te& that rate !" re +!n e t! e-mail ,r$ey ;ill in)rea e a i ,e alien)e in)rea e ( There"!re# it ;a e:+e)te& that the re +!n e rate "!r St,&y - ;!,l& be hi%her than St,&y .# an& the re +!n e rate "!r St,&y . ;!,l& be hi%her than St,&y 4( In "a)t# re +!n e rate "!r the three t,&ie ;ere Bin !r&er "r!m St,&y - t! St,&y 4C: 2/(2# 2/(.# an& .2(3 +er)ent( Table 4 ,mmari>e in"!rmati!n ab!,t am+le i>e an& re +!n e rate "!r the three t,&ie ( Analy i !" Varian)e re$eale& i%ni"i)ant &i""eren)e bet;een St,&y - an& St,&y 4 an& bet;een St,&y . an& St,&y 4 B9L-48(-4# + M (33-C( The e &i""eren)e ;ere in the e:+e)te& &ire)ti!n( <y+!the i - i there"!re ,++!rte&( I ,e alien)e a++ear t! ha$e a +! iti$e e""e)t !n re +!n e rate "!r e-mail ,r$ey ( A email te)hn!l!%y be%in t! &i"", e thr!,%h!,t the +!+,lati!n# re ear)her m, t in ,re that the ,ni$er e "r!m ;hi)h they ele)t e-mail a&&re e i !ne that re+re ent in&i$i&,al "!r ;h!m the re ear)h t!+i) ;ill be alient( A the ba e !" e-mail , er be)!me m!re heter!%ene!, # re ear)her ha$e b!th m!re !++!rt,nitie an& m!re )hallen%e a they attem+t t! mat)h +!tential re +!n&ent ;ith alient i ,e ( Parti),lar attenti!n h!,l& be +ai& t! the ,b?e)t line !" e-mail me a%e a ;ell a t! the &e )ri+ti!n !" the ,r$ey that i +r!$i&e& in +re-n!ti"i)ati!n me a%e an& in the intr!&,)ti!n t! the ,r$ey(

Re +!n e Variati!n in E-Mail S,r$ey <y+!the i . <y+!the i . +re&i)te& that +ee& !" re +!n e t! e-mail ,r$ey ;ill be "a ter "r!m

-/

in&i$i&,al ;h! re)ei$e& a +re-n!ti"i)ati!n !" the ,r$ey than "r!m th! e ;h! &i& n!t re)ei$e +ren!ti"i)ati!n( There"!re# it ;a e:+e)te& that +arti)i+ant in St,&y - ;!,l& re +!n& m!re l!;ly than re +!n&ent in St,&y . an& St,&y 4( Re +!n e +ee& "!r the e t,&ie are Bin !r&er "r!m St,&y - t! St,&y 4C 2(11 &ay # 2(0/ &ay # an& 4(0/ &ay ( Analy i !" Varian)e re$eale& i%ni"i)ant &i""eren)e bet;een St,&y - an& St,&y 4 B9L-3(82# +M(33-C( <!;e$er# there ;a al ! a i%ni"i)ant &i""eren)e bet;een St,&y . an& St,&y 4# an& n!t a i%ni"i)ant &i""eren)e bet;een St,&y - an& St,&y .( <y+!the i . i there"!re +artially ,++!rte&( Pre-n!ti"i)ati!n a++eare& t! in)rea e re +!n e +ee& "!r St,&y . an& St,&y 4# alth!,%h !nly St,&y 4 ;a i%ni"i)antly &i""erent than St,&y -# ;hi)h &i& n!t +re-n!ti"y re +!n&ent (

<!;e$er# the t;! t,&ie ;ith +re-n!ti"i)ati!n ;ere al ! i%ni"i)antly &i""erent "r!m ea)h !ther( One rea !n "!r the !me;hat l!;er re +!n e "!r St,&y . )!,l& be that the ,r$ey ;a ent t!

the Uni$er ity )!mm,nity &,rin% the ,mmer( Alth!,%h a))!,nt ;ere )he)5e& +ri!r t! en&in% the ,r$ey t! ma5e ,re re)i+ient ha& re)ently , e& their e-mail# the ,mmer )he&,le may ha$e re ,lte& in a &e)rea e in the "re@,en)y !" )he)5in% e-mail "!r member !" thi *i ), i!n E-mail ,r$ey ha$e arri$e&( It i time "!r re ear)her t! %! bey!n& im+le )!m+ari !n !" e-mail an& +! tal mail re +!n e rate an& re +!n e +ee&( Re ear)her m, t be%in t! "!), !n why re +!n e rate# re +!n e +ee&# an& !ther meth!&!l!%i)al i ,e $ary am!n% e-mail ,r$ey ( Thi arti)le +r!$i&e an im+!rtant "ir t te+ in thi meth!&!l!%i)al &e$el!+ment by e:aminin% "a)t!r that may im+a)t !n re +!n e rate an& re +!n e +ee&( An!ther meth!&!l!%i)al )!n i&erati!n hi%hli%hte& in thi arti)le i the im+!rtan)e an& )hallen%e !" i&enti"yin% a++r!+riate ,ni$er e "r!m ;hi)h t! am+le( St,&ie re+!rte& in thi am+le(

Re +!n e Variati!n in E-Mail S,r$ey arti)le , e& b!th !n-line an& !""-line &ire)t!rie t! )reate a am+lin% +!!l( On-line !,r)e !" email a&&re e may !""er !me ,ni@,e )hallen%e in term !" &ra;in% a ran&!m am+le( 9!r e:am+le# ele)tin% e-mail a&&re e "r!m the li t !" Deb ite +r!$i&e& by Yah!! re@,ire& m,lti+le te+ an& te&i!, han& )!,ntin%( B,t# in !me )a e # the !n-line !,r)e )an treamline the re ear)herA ?!b( 9!r e:am+le# the ran&!m li t !" e-mail a&&re e +r!$i&e& by the 9!,r-ear)h en%ine a,t!mate& the +r!)e

-8

!" ele)tin% a ran&!m am+le( A e-mail &i"", e # re ear)her

h!,l& e:+erien)e little &i""i),lty in "in&in% !,r)e "r!m ;hi)h t! &ra; am+le ( The 5ey )hallen%e ;ill be t! en ,re that th! e li t are re+re entati$e !" the ,ni$er e "r!m ;hi)h the re ear)her ;i h t! am+le( An!ther )hallen%e ;ill be t! anti)i+ate the +er)enta%e !" ,n, able a&&re e that may be %enerate& "r!m th! e li t ( The t,&ie re+!rte& in thi arti)le ill, trate the "a)t that many !" the tan&ar& re ear)h te)hni@,e , e& "!r +! tal ,r$ey re ear)h )an be ea ily a++lie& t! e-mail ,r$ey ( 9!r e:am+le# remin&er mailin% )an be , e& t! b!! t the !$erall re +!n e rate( Similarly# the ann!,n)ement me a%e ent +ri!r t! the ,r$ey a++ear t! +ee& the re +!n e time "!r the ,r$ey( Alth!,%h alien)e !" t!+i) a++eare& t! e""e)t re +!n e rate # it i im+!rtant t! n!te that all three t,&ie a&&re e& Internet-relate& t!+i) F th, a++r!+riate in !r&er t! a e am+lin% the Internet +!+,lati!n ;a

attit,&e an& !+ini!n !" th! e in&i$i&,al &ire)tly in$!l$e& ;ith

an& im+a)te& by the te)hn!l!%y( Dhat i n!t 5n!;n# h!;e$er# i h!; re +!n&ent ;!,l& rea)t t! n!n-Internet relate& re ear)h( A the Internet +!+,lati!n be%in t! mirr!r the t!tal +!+,lati!n in the Unite& State # it may be tem+tin% t! ,r$ey ),rrent , er !n all ty+e !" ,b?e)t !" intere t t! A&$erti in% re ear)her ( De ;!,l& )a,ti!n th! e t,&yin% br!a&er i ,e that the re ,lt re+!rte& in thi arti)le may n!t ne)e arily a++ly t! their +e)i"i) "iel& !" intere t( S!me te)hn!l!%i)al +r!blem till nee& t! be a&&re e&( 9!r e:am+le# re ear)her nee& t! ma5e ,re they are neither a)), e& !" n!r %,ilty !" the 5in& !" G +ammin%H that )an )ri++le mail

Re +!n e Variati!n in E-Mail S,r$ey er$er ( *e +ite a ,ran)e !" a)a&emi) +r!+riety in )!n&,)tin% St,&y 4# many in&i$i&,al in&i)ate& they &i& n!t belie$e the re ear)her ;a a)t,ally an a)a&emi) re ear)her( In tea& they belie$e& that the me a%e ha& been ent by !me!ne ;h! ;a en%a%e& in the +ra)ti)e !"

-1

G ,%%in%H B ellin% ,n&er the %,i e !" re ear)hC( Until m!re le%itimate re ear)her , e thi meth!& an& it re)ei$e )re&ibility am!n% the Internet +,bli)# imilar it,ati!n )!,l& ari e( A&&iti!nally# ;!r&in% nee& t! be &e$el!+e& that ;ill )lari"y the te)hni)al +r!)e !"

re +!n&in% t! an e-mail ,r$ey( Dhile +! tal mail ,r$ey are "ille& !,t , in% a ;ritin% in tr,ment# e-mail ,r$ey all!; "!r m,lti+le re +!n e "!rmat ,)h a in ertin% the ),r !r int! the &e ire&

+!int an& ty+in% in an an ;er# an ;erin% $ia a G"!rm#H an& +r!$i&in% an ;er in a e+arate me a%e( All "!rmat may n!t ;!r5 !n all y tem B"!r e:am+le# !me AOL member )!,l& n!t re +!n& by in ertin% the ),r !r at a +e)i"i) +!int in the te:tC( A +!tential limitati!n t! thi t,&y i that the &e%ree !" +er !nali>ati!n $arie& !me;hat

am!n% the t,&ie e$en th!,%h all ,r$ey ;ere +er !nali>e& t! !me &e%ree( In St,&y -# re +!n&ent ;ere re@,e te& t! an ;er @,e ti!n re%ar&in% a +e)i"i) Deb ite that they ha& &e$el!+e&# an& the +e)i"i) URL ;a +r!$i&e& t! ea)h +!tential re +!n&ent ( Thi may ha$e in)rea e& b!th +er !nali>ati!n an& i ,e alien)e( St,&ie . an& 4 &i& n!t ha$e thi ty+e !" +er !nali>ati!n( A&&iti!nally# St,&y . ;a maile& , in% a bat)h mail +r!%ram( Dith thi meth!&# the re)i+ientA name i in)l,&e& in the t,&y# yet the mail +r!%ram may in&i)ate that he !r he ;a +art !" a mailin% Gli t(H Dhether thi ty+e !" mailin% te)hni@,e e""e)t re +!n e rate a either a +rimary !r m!&eratin% e""e)t h!,l& be re ear)he& ",rther( 9in&in% relate& t! e-mail ,r$ey h!,l& en)!,ra%e re ear)her ( U in% e-mail# re ear)her )an @,i)5ly a e )!n ,mer !+ini!n( 9,rther re ear)h h!,l& "!), !n ;ay t! in)rea e re +!n e

rate an& )a+itali>e !n !ne !" the m! t e$i&ent bene"it !" e-mail ,r$ey : the "a t re +!n e +ee&( E$en am!n% G l!;erH re +!n&ent %r!,+ # m! t ,r$ey are an ;ere& in le than !ne ;!r5 ;ee5(

Re +!n e Variati!n in E-Mail S,r$ey

.3

In )!n)l, i!n# e-mail ,r$ey !""er an e:)itin% !++!rt,nity "!r re ear)her : )! t are l!;# re +!n e rate are %!!&# an& re +!n e time are @,i)5( A re ear)her a++ly the e te)hni@,e t! many ty+e !" a&$erti in% @,e ti!n # the re ear)h )!mm,nity )an ;!r5 t!%ether t! re"ine e-mail ,r$ey meth!&!l!%y(

Re +!n e Variati!n in E-Mail S,r$ey Table -( S,mmary !" S,r$ey Re ear)h Meth!& U in% E-mail A,th!r Kie ler N S+r!,ll B-187C Par5er B-11.C Sam+le Em+l!yee !" a 9!rt,ne 033 =!m+any Em+l!yee !" ATNT Mar5etin% N MIS Pr!"e !r BUSC U enet U er S,r$ey T!+i) =!r+!rate =!mm,ni)ati!n Internal =!mm,ni)ati!n Share;are =!+yin% Meth!& Mail E-Mail T!tal Mail E-Mail T!tal Mail E-Mail T!tal Mail E-Mail T!tal Mail E-Mail T!tal Mail E-Mail T!tal Mail 9a: E-Mail Deb 9!rm T!tal Mail E-Mail T!tal Sam+le Si>e --0 --0 .43 /3 /3 -23 .33 .-8 2-8 431 -8. 21.33 .33 233 ..2 ..2 228 .33 .33 .33 .33 833 ..7 ..7 20. T!tal Re +!n e // 87 -74 ./ 28 /0 --4 2. -00 -/4 11 ./. 02 -. 77 -2/ --/ .72 /3 03 28 0. ..3 -43 -4.7. Re +!n e Rate 7/O /0O 48O 78O 07(0O -1(4O 07(0OP 02(4OP ./O 7O 70(7O 0.(0O 40(/O .3(1O .1(8O 4.(/O 0/(0O 08(3O Re +!n e Time B&ay C -3(8 1(7 NA NA NA NA NA NA 1(/1 8(31 --(-8 2(78 -.(1 8(8 7(/(2 -2(41 1(-7

.-

S)h,l&t N T!tten B-112C Mehta N Si$a&a B-110C T e# et al B-110C

Internet =!mm,ni)ati!n B, ine Ethi)

Uni$er ity P!+,lati!n B<!n% K!n%C B, ine S)h!!l *ean MIS Pr!"e !r BUSC

Ba)hman# El"rin5 N Va>>ana B-117C Deible an& Dalla)e B-118C

TIM

Internet U e

S)hae"er an& *illman B-118C

Uni$er ity 9a),lty

Un5n!;n

P *i""eren)e n!t i%ni"i)ant

Re +!n e Variati!n in E-Mail S,r$ey

..

Table .( S,mmary !" the Three St,&ie St,&y Sam+lin% "rame =reat!r !" healthrelate& Deb ite Val,e !" ite )reat!r F ite +,r+! e an& ",n&in% 'an,ary-9ebr,ary -11/ 40 -7 St,&y . 9a),lty# ta"" an& t,&ent at a ma?!r re ear)h ,ni$er ity Attit,&e t!;ar& !n-line +ri$a)y St,&y 4 In&i$i&,al ;ith +er !nal e-mail a))!,nt Attit,&e an& beha$i!r a !)iate& ;ith !nline +ri$a)y N!$ember -11/'an,ary -118 47 -8

St,&y t!+i)

Time 9rame N,mber !" @,e ti!n N,mber !" )!m+,ter )reen the ,b?e)t m, t re$ie; B, in% PineC Mailin% meth!& S!li)itati!n ent be"!re ,r$ey Remin&er ent t! n!nre +!n&er a"ter h!; many &ay Per)ent !" t!tal re +!n e that )ame a"ter the remin&er Per)ent re +!n&in% , in% +! tal rather than e-mail

May-',ly# -117 20 .3

In&i$i&,al N! /

Bat)h Ye 7

Mer%e +r!%ram Ye 0

28O .(/O

.0O -(8O

.4O -(/O

Re +!n e Variati!n in E-Mail S,r$ey

.4

Table 4( S,r$ey Re +!n e Rate Vali& Sam+le Si>e St,&y - J Deb *e$el!+er St,&y . J Uni$er ity St,&y 4 JIn&i$i&,al ;ith Per !nal e-mail a))!,nt 842 083 4#/.2 T!tal Re +!n e 410 ./2 810 Re +!n e Rate 2/(2 2/(. .2(3

Re +!n e Variati!n in E-Mail S,r$ey Re"eren)e Arm tr!n% '(S( an& E( '( L, 5( GRet,rn P! ta%e in Mail S,r$ey # A Meta-analy i (H P,bli) O+ini!n I,arterly 0- B18/C: .44-.28( Ba)hmann# *(# '( El"rin5 an& E( Va>>ana( GTra)5in% the Pr!%re Mar5etin% Re ear)h 8 B-117C: 4--40( !" E-mail Ver , Snail Mail(H

.2

*e+artment !" =!mmer)e( GThe Emer%in% *i%ital E)!n!my(H B-118C( A$ailable: htt+:KK;;;(e)!mmer)e(%!$K&anintr!(htm *illman# *( A( GThe *e i%n an& A&mini trati!n !" E-mail S,r$ey (H Ann,al Re$ie; !" S!)i!l!%y -/ B-11-C: ..0-21( *illman# *( A( Mail an& Tele+h!ne S,r$ey : The T!tal *e i%n Meth!&( Ne; Y!r5: '!hn Diley an& S!n # -1/8( *,n)an# D( '( GMail I,e ti!nnaire in S,r$ey Re ear)h: A Re$ie; !" Re +!n e In&,)ement Te)hni@,e (H '!,rnal !" Mana%ement 0# - B-1/1C: 41-00( 9!:# R(# M(R( =ra 5# an& '( Kim( GMail S,r$ey Re +!n e Rate : a Meta-Analy i !" Sele)te& Te)hni@,e "!r In&,)in% Re +!n e(H P,bli) O+ini!n I,arterly 0.B-188C: 27/-21-( ( E!y&er# '( =( G9,rther E$i&en)e !n 9a)t!r A""e)tin% Re +!n e Rate t! Maile& I,e ti!nnaire (H Ameri)an S!)i!l!%i)al Re$ie; 2/ B-18.C: 003-004( <eberlein# T( A an& R( Ba,m%artner( G9a)t!r A""e)tin% Re +!n e Rate t! Maile& I,e ti!nnaire : A I,antitati$e Analy i !" the P,bli he& Literat,re(H Ameri)an S!)i!l!%i)al Re$ie; 24 B-1/8C: 22/-27.( Kan,5# L( an& =( Beren !n( GMail S,r$ey an& Re +!n e Rate : A Literat,re Re$ie;(H '!,rnal !" Mar5etin% Re ear)h -. B-1/4C: 22-04( Keh!e# =(# '( Pit5!;# an& K( M!rt!n( GEi%hth DDD U er S,r$ey(H B-11/C( A$ailable: htt+:KK;;;())(%ate)h(e&,K%$,K, erQ ,r$ey K ,r$ey-32--11/K ( Kie ler# S( an& L(S( S+r!,ll( GRe +!n e E""e)t in the Ele)tr!ni) S,r$ey(H P,bli) O+ini!n I,arterly 03 B-187C: 23.-2-4( Martin# =( L( GThe Im+a)t !" T!+i) Intere t !n Mail S,r$ey Re +!n e Beha$i!r(H '!,rnal !" the Mar5et Re ear)h S!)iety 47# 2B-112C: 4./-44/( Mehta# R( an& E( Si$a&a ( G=!m+arin% Re +!n e Rate an& Re +!n e =!ntent in Mail Ver , Ele)tr!ni) S,r$ey (H '!,rnal !" the Mar5et Re ear)h S!)iety 2# 4/ B-110C: 2.1-223( M,r+hy# P( R(# '( *aley an& *( R( *alenber%( GE:+l!rin% the E""e)t !" P! t)ar& Pren!ti"i)ati!n !n In&, trial 9irm A Re +!n e t! Mail S,r$ey (H '!,rnal !" the Mar5et Re ear)h S!)iety 44# 2 B-11-C: 440-420(

Re +!n e Variati!n in E-Mail S,r$ey

.0

Par5er# L( G=!lle)tin% *ata the E-mail Day(H Trainin% an& *e$el!+ment ',ly B-11.C: 0.-02( R!ber !n# M(T( an& E( S,n& tr!m( GI,e ti!nnaire *e i%n# Ret,rn Rate # an& Re +!n e 9a$!rablene in an Em+l!yee Attit,&e I,e ti!nnaire(H '!,rnal !" A++lie& P y)h!l!%y /0B-113C: 402-40/( S)hae"er# *( R( an& *( A( *illman( G*e$el!+ment !" a Stan&ar& Email Meth!&!l!%y: Re ,lt !" an E:+eriment(R P,bli) O+ini!n I,arterly 4# 7. B-118C: 4/8-413( S)h,l&t# B(A( an& '( T!tten( GEle)tr!ni) Mail $ ( Mail S,r$ey Re +!n e Rate (H Mar5etin% Re ear)h B-112C: --/( Steele# T( '(# D( L( S)h;en&i% an& '( A( Kil+atri)5( G*,+li)ate Re +!n e t! M,lti+le S,r$ey Mailin% : A Pr!blemSH '!,rnal !" A&$erti in% Re ear)h Mar)hKA+ril B-11.C: .7-42( Tayl!r# S# an& P( Lynn( GThe E""e)t !" a Preliminary N!ti"i)ati!n Letter !n Re +!n e t! a P! tal S,r$ey !" Y!,n% Pe!+le(H '!,rnal !" the Mar5et Re ear)h S!)iety .# 23 B-118C:-70--/8( T e# A(# K(=( T e# =(<( Yin# =(B( Tin%# K(D( Yi# K(P Yee# an& D(=( <!n%( G=!m+arin% T;! Meth!& !" Sen&in% O,t I,e ti!nnaire : E-mail $er , Mail(H Journal of the Market Research Society, 4, 37 (1995): 441-445. Deible# R( an& '( Dalla)e( GThe Im+a)t !" the Internet !n *ata =!lle)ti!n(H Mar5etin% Re ear)h 10, 3 (1998): 19-23. Yammarin!# 9('(# S( S5inner an& T( L( =hil&er ( GUn&er tan&in% Mail S,r$ey Re +!n e Beha$i!r(H P,bli) O+ini!n I,arterly 00 B-11-C: 7-4-741( Y,# '( an& <( =!!+er( GA I,antitati$e Re$ie; !" Re ear)h *e i%n E""e)t !n Re +!n e Rate t! I,e ti!nnaire (H '!,rnal !" Mar5etin% Re ear)h .3 B-184C: 47-22(