Chapter 9 Survey Research

SUMMARY

Survey research is the most frequently used mode of gathering data in the social sciences. It consists
ofSelecting a sample of respondents and administering a questionnaire or an interview to them.
Surveys may be used for descriptive, explanatory, and exploratory purposes. The unit of analysis is
usually the individual, but surveys can be used for other units of analysis as well. Survey research
is particularly well suited for studying attitudes and orientations in a l~ge population. Surveys are
increasingly used for less legitimate purposes than social research, including marketing surveys and
political surveys such as "push polls."

Most concepts are measured thro~gh questions and statements. Questions may take one of two-
forms. Closed-ended questions ~re easily process~d, but sometimes overlook alternative ariS"wer
categories. Open-ended questions afford a wider range of responses, but they are more difficult to
process and may yield irrelevant answers.

Questions should also be relevant to the respondent, very clear, and as short as possible. Respondents
should be capable of answ~the questions and should be willing toanswer the questions. It is best
to avoid negative items as well as biased items and terms. Combining two (or more) questions into
one is known as a double-barreled question and produces both confusion for the responden~nd
ambiguity for the researcher. The researcher should also ~void using Questions that p...romotesocial
~esirability-answers that make respondents look gQQQ.. In the final analysis, the overriding guideline
is that questions should reflect the purpose of the study.

Questionnaire format and appearance are critical. Questionnaires that are well organized, uncluttered,
and attractive reduce the likelihood that respondents will overlook or ignore items or dispose of the
questionnaire. The best format for questions is the use of boxes adequately sRaced. C~ntingency
questions can helpjJrevent respondents from being confused and annoyed-questions that are relevant
toonlv some of the respondents are identified. A series of questions with the same set o{~r
categories can be effectively presented through a matrix format, which saves space and time. But
occasionally respondents begin marking the same answers without carefully reading the items, a
problem known as a response set.

The order of question presentation is also critical because ~swers to one question can affect answers
to later questions. Also, it is usually best to begin questionnaires with the most interesting set of
questions to generate interest, although it is generally best to begin interviews with demographic
questions to enhance rapport. Clear instructions at the beginning of a questionnaire, as well as at the
beginning of each section, are mandatory. Special situations will require special instructions. Finally,
it is important to pretest your Questionnaire to help reduce error. Although it is not essential that
pretest participants reflect a representative sample, they should use people for whom the question-
naire is at least relevant. ~ ..

~lf-ad~ered questi()~airesa.re generally executed by mail, although sometimes they are

administered to a group of respondents, and sometimes they are delivered and picked up at a later
time. Self-mai~g questionnaires require no return envelops making it easier to return the instru-

168
 w:;-<.Zr 9 Survey Research

~uestionnaires can be sent via first-class postage or bulk rate and returned via postage stamps
~~ess-reply permits. When each option would be used is determined largely by the budget and
~cted return rate.

=.oring the returns of questionnaires is an important part of the study and can be accomplished
--:::..:.return rate graph plotting the number of questionnaires returned each day. Such a graph can
-:;:e clues about when follow-up mailings should occur, and it may be ~seful in estimating
"="=Sponsebias. Follow-up mailings may contain only a letter of additional encouragement or may
~-"?1n a new copy of the questionnaire as well. F_ollow-up mailings significantly increase return
""'--:
two follow-ups mailed two or three weeks apart are best.

-~ are no strict standards for getermining ap acceptable response rate. Iiigh response rates
=~e the probability that the respondents accurately represent the samE!e, thus reducing the
::z:.:e of response bias. But a demQllStrat~plack of response bias is more crucial than the re~I:>onse
......;;
itself. A response rate of at least 50 percent is adequate, qQ..Rercentis considered good, and 70
~nt is considered very good.

~"~views provide a number of advantages over the self-administered questionnaire. They attain
£;:.er response rates, decrease the number of "don't know" and "no answer" responses, h~ correct
::-...J.i.!sing
ite~, and provide the opportunity to observe the social situation as well as to ask
.':..~llOns.

:s critical that the interviewer serve as a neutral conduit through which questions and answers are
=:1Smitted. Interviewers should have a pleasant demeanor and dress in a fashion similar to the
:eople being interviewed. Familiarity with the instrument will lessen the time required and miniilliZe
:rJolems with individual items. Interviewers should follow the question wording exactly and record
--:"sponsesexactly. One particular skill required is.J2robing, whereby the interviewer redirects inap-
=8priate answers while maintaining the neutral role. When multiJ2leinterviewers are employed, the
-:~archer should thoroughly train and supervise the interviewers. Researchers should also prepare
~ifications, which are explanatory and clarifYing comments about handling difficult or confusing
;:--:uationsthat may arise with a specific item.

elephone surveys save money and time, sometimes enhance willingness to give socially disap-
-=-

7="Ovedanswers, and yiEld greater control when several interviewers are em2!Qyed. They also
.:..:J~e personal safety concerns. But they come with several problems. They have a ~ad re]2!!tatiQ,n,
:or example, and people can hang up easily before the interview is completed. Computer-assisted
~lephone interviewing is frequently used to enhanc~ the reliability and validity of the results.
:omputer-assisted survey methods are also increasingly being used for in-person interviews as well
::.sfor analyzing data as the data are gathered. Recent innovations in computer technology provide
~veral options for computer administered questionnaires. Online polling offers advantages over
Jther forms but comes with some disadvantages, particularly concerning representativeness.

Self-administered questionnaires and interviews each have advantages and disadvantages. Questio~
:laires are generally cheaper and quicker and are more appropriate in dealing with more sensitive
issues. Interviews help reduce incomplete answers, are more effective in dealing with complicated

169
Chapter 9 Survey Research

issues, enable a researcher to conduct a survey based only on a sample of addresses, and provide the
opportunity to make important observations apart from the responses to the items. Interviews
ge~ yield higher response rates. -

The survey design itself has strengths and weaknesses compared to other designs. It is particularly
aImffipriate for describing the characteristics of large 1lQI2ulations,thereby making large samples
feasible. In some respects, surveys are quite flexible, and the use of standardized questions enhances
~eliability. But these advantages come at some cost. Standardization often results in overlooking
o~her appropriate responses and may generate inflexibility in modifying questions. Surveys seldom
analyze the context of social life. They are frequently labeled as artificial because the topic of study
may not be amenable to measurement through questionnaIres, and the act of studying that topic may
affect the results. In short, survey research is weak on validity and strong on reliability.

Researchers with a need for survey data cannot always afford to conduct large-scale surveys them-
selves. Hence many researchers employ secondary analysis of data gathered for some other purpose
by someone else. Such research has experienced increased popularity due to the establishment of
data archives, research centers that collect and distribute many different types of data sets. Such
analysis can be done quite rapidly and inexpensively, particularly with those that are online. The key
~dvantages of secondary analysis include the low cost and the ability to use data gathered by well
known researchers and data collection agencies. The k~y problem with secondary analysis is validity;
when one researcher collects data for a given purpose, other researchers have no assurance that such
da!a will be appmpriate for their research interests. Comparability of measures is a particular
problem. Regarding ethics, surveys often ask for private information (which should remain confiden-
tial), and asking questions may cause psychological discomfort (which should be minimized).

TERMS

1. bias (277) 12. meta-analysis (306)

2. closed-ended question (272) 13. open-ended question (272)
3. Computer Assisted 14. probe (293)
Personal Interviewing (CAPI) (299) 15. questionnaire (272)
4. Computer Assisted 16. respondent (270)
Self Interviewing (CASI) (299) 17. response rate (288)
5. Computer Assisted Self- 18. response set (281)
Administered Questionnaire (CSAQ) (299) 19. return rate graph (287)
6. Computer Assisted 20. secondary analysis (304)
Telephone Interviewing (CATI) (297) 21. self-administered questionnaire (286)
7. contingency question (279) 22. social desirability (277)
8. data archives (304) 23. specifications (294)
9. double-barreled question (273) 24. telephone surveys (295)
10. interview (291) 25. Touchtone Data Entry (TDE) (299)
11. matrix questions (280) 25. Voice Recognition (VR) (299)

170