12/28/2017 Survey Research and Questionnaires

Survey Research and Questionnaires

Survey Research
Questionnaire Design
Survey Administration
Sampling Procedures
Measurement Error
Ethics of Survey Research
Advantages and Disadvantages of Survey Research

Survey Research
Survey research is a commonly used method of collecting information about a population of interest. There are many different types of surveys, several ways to administer t
hem, and many methods of sampling. There are two key features of survey research:

Questionnaires -- a predefined series of questions used to collect information from individuals

Sampling -- a technique in which a subgroup of the population is selected to answer the survey questions; the information collected can be generalized to the entire
population of interest

Questionnaire Design
The two most common types of survey questions are closed-ended questions and open-ended questions.

Closed-Ended Questions

The respondents are given a list of predetermined responses from which to choose their answer
The list of responses should include every possible response and the meaning of the responses should not overlap
An example of a close-ended survey question would be, "Please rate how strongly you agree or disagree with the following statement: 'I feel good about my work on t
he job.' Do you strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, or strongly disagree?"
A Likert scale, which is used in the example above, is a commonly used set of responses for closed-ended questions
Closed-ended questions are usually preferred in survey research because of the ease of counting the frequency of each response

Open-Ended Questions

Survey respondents are asked to answer each question in their own words
Responses are usually categorized into a smaller list of responses that can be counted by the study team for statistical analysis

Considerations for Designing a Questionnaire

It is important to consider the order in which questions are presented. Sensitive questions, such as questions about income, drug use, or sexual activity, should be pu
t at the end of the survey. This allows the researcher to establish trust before asking questions that might embarrass respondents. Researchers also recommend putti
ng routine questions, such as age, gender, and marital status, at the end of the questionnaire
Double-barreled questions, which ask two questions in one, should never be used in a survey. An example of a double barreled question is, "Please rate how strongly
you agree or disagree with the following statement: 'I feel good about my work on the job, and I get along well with others at work.'" This question is problematic bec
ause survey respondents are asked to give one response for two questions
Researchers should avoid using emotionally loaded or biased words and phrases

Visit the following websites for more information about questionnaire design:

A Brief Guide to Questionnaire Development (http://www.indiana.edu/~educy520/sec5982/week_3/questionnaire_development_frary.pdf)

Survey Design (http://www.surveysystem.com/sdesign.htm)

Glossary terms related to questionnaire design:

Double-Barreled Question (/childcare/research-glossary#Double Barreled Question)

Pretesting (/childcare/research-glossary#Pretesting)
Questionnaire (/childcare/research-glossary#Questionnaire)

Survey Administration
Surveys can be admininistered in three ways:

Through the mail

Advantage: Low cost
Disadvantage: Low response rate
By telephone
Advantages: Higher response rates; responses can be gathered more quickly
Disadvantage: More expensive than mail surveys
Face-to-face
Advantages: Highest response rates; better suited to collecting complex information
Disadvantage: Very expensive

Visit the following website for more information about survey administration:

What is a Survey? (http://www.amstat.org/sections/srms/pamphlet.pdf)

Glossary terms related to survey administration:

Attrition (/childcare/research-glossary#Attrition)
Completion Rate (/childcare/research-glossary#Completion Rate)
Cooperation Rate (/childcare/research-glossary#Cooperation Rate)

https://www.researchconnections.org/childcare/datamethods/survey.jsp 1/3
12/28/2017 Survey Research and Questionnaires
Refusal Rate (/childcare/research-glossary#Refusal Rate)
Response Categories (/childcare/research-glossary#Response Categories)
Response Rate (/childcare/research-glossary#Response Rate)

Sampling Procedures
One of the primary strengths of sampling is that accurate estimates of a population's characteristics can be obtained by surveying a small proportion of the population. Four
sampling techniques are described here:

Simple Random Sampling

Simple random sampling is the most basic form of sampling

Every member of the population has an equal chance of being selected
This sampling process is similar to a lottery: the entire population of interest could be selected for the survey, but only a few are chosen at random
Researchers often use random-digit dialing to perform simple random sampling. In this procedure, telephone numbers are generated by a computer at random and c
alled to identify individuals to participate in the survey

Cluster Sampling

Cluster sampling is generally used when it is geographically impossible to undertake a simple random sample
Cluster sampling requires that adjustments be made in statistical analyses

For example, in a face-to-face interview, it is difficult and expensive to survey households across the nation. Instead, researchers will randomly select geographic areas (for
example, counties), then randomly select households within these areas. This creates a cluster sample, in which respondents are clustered together geographically.

Stratified Sampling

Stratified samples are used when a researcher wants to ensure that there are enough respondents with certain characteristics in the sample
The researcher first identifies the people in the population who have the desired characteristics, then randomly selects a sample of them
Stratified sampling requires that adjustments be made in statistical analyses

For example, a researcher may want to compare survey responses of African-Americans and Caucasians. To ensure that there are enough Afrian-Americans in the survey, t
he researcher will first identify the African-Americans in the population and then randomly select a sample of African-Americans.

Nonrandom Sampling

Common nonrandom sampling techniques include convenience sampling and snowball sampling
Nonrandom samples cannot be generalized to the population of interest. Consequently, it is problematic to make inferences about the population
In survey research, random, cluster, or stratified samples are preferable

Visit the following websites for more information about sampling procedures:

Sampling

Sampling (http://www.socialresearchmethods.net/kb/sampling.php) (Cornell University)

Sampling Methods (http://www.statpac.com/surveys/sampling.htm) (StatPac)

Glossary terms related to sampling procedures:

Convenience Sampling (/childcare/research-glossary#Convenience Sampling)

Oversampling (/childcare/research-glossary#Oversampling)
Probability Sampling (/childcare/research-glossary#Probability Sampling)
Purposive Sampling (/childcare/research-glossary#Purposive Sampling)
Quota Sampling (/childcare/research-glossary#Quota Sampling)
Random Sampling (/childcare/research-glossary#Random Sampling)
Random Selection (/childcare/research-glossary#Random Selection)
Representativeness (/childcare/research-glossary#Representativeness)
Sample (/childcare/research-glossary#Sample)
Sample Size (/childcare/research-glossary#Sample Size)
Sampling (/childcare/research-glossary#Sampling)
Sampling Design (/childcare/research-glossary#Sampling Design)
Sampling Frame (/childcare/research-glossary#Sampling Frame)
Snowball Sampling (/childcare/research-glossary#Snowball Sampling)
Stratification (/childcare/research-glossary#Stratification)
Stratified Sampling (/childcare/research-glossary#Stratified Sampling)

Measurement Error
Measurement error is the difference between the target population's characteristics and the measurement of these characteristics in a survey. There are two types of measur
ement error: systematic error and random error.

Systematic Error

Systematic error is more serious than random error

Occurs when the survey responses are systematically different from the target population responses
For example, if a researcher only surveyed individuals who answered their phone between 9 and 5, Monday through Friday, the survey results would be biased towar
d individuals who are unemployed
Sources of bias include
Nonobservational error -- Individuals in the target population are systematically excluded from the sample, such as in the example above
Observational error -- When respondents systematically answer surveys question incorrectly. For example, surveys that ask respondents how much they w
eigh will probably underestimate the population's weight because respondents are likely to underreport their weight

https://www.researchconnections.org/childcare/datamethods/survey.jsp 2/3
12/28/2017 Survey Research and Questionnaires
Random Error

Random error is an expected part of survey research, and statistical techniques are designed to account for this sort of measurement error
Occurs because of natural and uncontrollable variations in the survey process, i.e., the mood of the respondent

For example, a researcher may administer a survey about marital happiness. However, some respondents may have had a fight with their spouse the evening prior to the su
rvey, while other respondents' spouses may have cooked the respondent's favorite meal. The survey responses will be affected by the random day on which the respondents
were chosen to participate in the study. With random error, the positive and negative influences on the survey measure balance out.

Visit the following website for more information about measurement error:

Reducing Measurement Error (http://web.archive.org/web/20160302201323/http://mospi.nic.in/informal_paper_17.htm)

Glossary terms related to measurement error:

Interviewer Error (/childcare/research-glossary#Interviewer Error)

Nonsampling Error (/childcare/research-glossary#Nonsampling Error)
Nonresponse Error (/childcare/research-glossary#Nonresponse Error)
Nonresponse Rate Bias (/childcare/research-glossary#Nonresponse Rate Bias)
Sampling Bias (/childcare/research-glossary#Sampling Bias)

Ethics of Survey Research

Informed Consent

Respondents should give informed consent before participating in a survey. In order for respondents to give informed consent,

The researcher must inform the respondents of the study's purpose, content, duration, and potential risks and benefits
The researcher must inform the respondents that they do not have to answer all the survey questions
The researcher must inform the resondents that they can stop participating in the study at any point

Confidentiality and Anonymity

It is absolutely imperative that researchers keep respondents' identities confidential. To ensure confidentiality, researchers should not link respondents' identifiers to their
survey responses when using data. Common identifiers include names, social security numbers, addresses, and telephone numbers.

Anonymity

Anonymity is an even stronger safeguard of respondent privacy. If a researcher assures anonymity, it means that the researcher is unable to link respondents' names to thei
r surveys.

Visit the following websites for more information about anonymity:

CASRO Code of Standards and Ethics for Survey Research (http://www.casro.org/?page=TheCASROCode)

The Responsible Collection, Retention, Sharing and Interpretation of Data (http://onlineethics.org/CMS/research/modindex/moddata.aspx)
Tips on Informed Consent (http://www.hhs.gov/ohrp/policy/ictips.html)

Glossary terms related to ethics:

Anonymity (/childcare/research-glossary#Anonymity)
Confidentiality (/childcare/research-glossary#Confidentiality)
Informed Consent (/childcare/research-glossary#Informed Consent)

Advantages and Disadvantages of Survey Research

Advantages

Sample surveys are a cost-effective and efficient means of gathering information about a population
Survey sampling makes it possible to accurately estimate the characteristics of a target population without interviewing all members of the population

Survey sampling is particularly useful when the population of interest is very large or dispersed across a large geographic area.

Disadvantages

Surveys do not allow researchers to develop an intimate understanding of individual circumstances or the local culture that may be the root cause of respondent beha
vior
Respondents often will not share sensitive information in the survey format
A growing problem in survey research is the widespread decline in response rates

Research Connections is supported by grant #90YE0104 from the Office of Planning, Research and Evaluation, Administration for Children and Families (OPRE), U.S. Depart
ment of Health and Human Services. The contents are solely the responsibility of the National Center for Children in Poverty and the Inter-university Consortium for Political
and Social Research and do not necessarily represent the official views of OPRE, the Administration for Children and Families, or the U.S. Department of Health and Human S
ervices.

© 2016 The Regents of the University of Michigan

Select Language GO

https://www.researchconnections.org/childcare/datamethods/survey.jsp 3/3