Norinah Mohd Ali

CHA130026

Notes
Petra Lietz. (2010). Research into Questionnaire Design : A Summary of the
Literature. International Journal of Market Research. Vol 52:2. pp 249-272
What is survey?
A complex communication process whereby the product of the interaction
between researchers and respondents leads to sharing and creating of
meaning.
The process begins with the comprehension of the question and proceeds to
the retrieval of relevant information from memory. Then, it involves a
judgement and estimation process that is related to the respondent's
motivation and preparedness to be truthful.
The respondent's internally generated answer is matched to the response
categories provided in the questionnaire.
Questions:
must tune to respondents understanding
its include - question length, question wording, question order
try to avoid negative impact on sample quality due to non-response
Question Length
keep questions as short as possible
words can be 16 - 20 words per sentence.
questions can consist of more than one sentence.
longer questions lead to more accurate reporting - can convey the idea that
the task is important and deserves serious effort. (16 to 64 words).
Grammar
keep minimum complexities
employ active rather than passive voice
repeat noun instead of pronouns
avoid possessive forms.
Specificity and Simplicity
use specific terms instead of general terms
breaking down more complex question to simpler one
provide behavioural illustrations of certain concepts (e.g. Chronic health
condition can be represent by the number of doctor the patient see in last 12
months (in picture)).
avoid vagueness words such as probably, maybe or perhaps.
Social desirability (SD) - want to look good
Don't use difficult vocabulary - this will lead to respondents feel stupid and
uneducated. - then they will answer 'don't know'- increase the probability on
getting this answer. - then it will inaccurately reflect respondents actual
behaviours in numbers of ways:
o respondents might choose answer that is favoured by society (not their
own answer) - this can lead to report does not accurate.
1

Norinah Mohd Ali

CHA130026

o because of the social prestige, respondents will give their opinion to make
them look good
o respondents fear of being identified
in order to reduce respondents to give social desirable answers:
o ask indirect questioning such as "What do you believe OTHER PEOPLE
think about....?"
o phrasing the question so that it can facilitate respondents' ignorance.
o use introductory phrase such as "Do you happen to know...."
o include questions
that are worded as neutrally as possible,
that have propose value on a certain topic not only in one but
different directions
that suggest the normalcy of socially deviant behaviour
Double-barrelled questions
avoid using two different verbs or two different concepts e.g do you have time
to read newspaper... - some have time- some have read, so which one?
Negatively worded question
take longer to process
have a greater likelihood of respondents making mistake
avoid no/not word
e.g "what is your view about the statement that conservationists should not be
so uncooperative with the government?" rephrase to "What is your view about
the statement that conservationists should cooperate with the government?" much faster to understand.
Adverbs of frequency
"usually", "frequently", "regularly" have quite different meanings for different
respondents, and depending on the question content as well as on the
numeric values assigned if these terms are used as labels of a response
scale.
solution: offer participants more specific quantifiers in the response options e.g
"never or almost never", "once or twice a month", "once or twice a week" and
"always or almost always".
Question order
QO effects arise when answering behaviour changes depending on the
position of a question during the interview.
can lead to problem: threaten the validity of the results and the generalisability
of results to the population.
effects of part-whole combinations, part-part combinations and salience
o part-whole combination occur where one question is more general with
respect to a certain concept while the other is more specific.
o part-part combinations arise where questions are asked at the same level
of specificity and respondents adapt their answers as a result of normative
2

Norinah Mohd Ali

CHA130026

consistency. E.g. US into foreign(Q1) vs foreign into US(Q2). Agree for Q2

is higher when Q1 is asked first.
o salience are said to occur when response behaviour changes as a result of
a topic having been raised as part of the questioning process, hence
conveying the importance of that topic to respondents. E.g. Demographic
question like age, edu, income should come at the end to avoid negative
feelings from the respondents.
questions should be constructed to be as clear, simple, specific and relevant.
Responses:
Don't know (DK) option
first we decide whether all respondents should answer all questions or we
filtered out respondents with little or no knowledge by not be asked certain
questions
% respondents choosed DK will increase if DK is explicitly offered.
Opinion Floating
respondents who give a substantive response when the DK is not offered, but
who choose this option when it is offered, seem that this respondents vary
their responses depending on the response options on offer.
concluded that "whether filtered or standard questions should be used in a
questionnaire would seem to depend on whether an investigator is interested
mainly in an 'uninformed opinion' on an issue or mainly in underlying
disposition.
Opinion Filtering
How to filter out respondents?: ask question such as:
o Do you have an opinion on this or not?
o Have you been interested enough to favour one side over the other?
But be careful on who we thought to be ignorant because:
o Respondents self identification as being ignorant might vary
systematically as a consequence of question topic as well as respondents
characteristics such as gender and age.
o A serious consequences of filtering out the respondents is the impact on
the representativeness of the sample
Rule of thumb in survey research is to consider a sample as being not
representative of the intended target population if information is obtained from
less than 80% of originally selected participants
Number of response scale options
5 points scales and 7 points scales are most commonly used in research
7 points scales is more reliable than 5 points scales because it allow greater
differentiation of responses
11 points scales had consistently higher reliability and validity coefficients and
lower invalidity coefficients
However, Foddy (1993) said that 5 points scales are preferable in situations
where respondents are asked for absolute judgements; and 7-9 points scales
3

Norinah Mohd Ali

CHA130026

are more appropriate in situations where more abstract judgement are sought
from respondents.

Odd or even number of response scale options

A decision has to be made as to whether to offer respondents an even or an
odd number of response scale options
Whether should have middle option or not (respondents not to commit
themselves to a direction in their opinion or attitude).
Research by Kalton et al 1980 found that the tendency to choose a middle
option is not generally dependent on age, education or gender
OMuicheartaigh et al. (2000) said, if exclude middle alternative, responses to
the DK option increase only slightly (1-2%), but slightly higher for the weak
agree/disagree responses (8.5%). Extreme agree/disagree only 4.1%.
Krosnick (1991) hypothesizes that because many survey participants are likely
to have low motivation and may find the task of responding difficult and
exhausting then they select the response alternative that involves the least
amount of thinking and justifying.
OMuicheartaighs hypothesis that If exclude middle alternative, people will
reporting meaningful attitudes that they would otherwise not have bother to
describe
This is not true when result showed that the response scales without middle
point had lower validity and higher random error variance - indicating that
people randomly chose other available response options when the middle
option was not available.
OMuicheartaigh also found a phenomenon called acquiescence - which refer
to the tendency of respondents to agree with any statement regardless of its
content
Labelling of response scale options
Decision whether unipolar (0 to 10) or bipolar (-5 to +5) or verbal scale (agree,
slightly agree, neither agree nor disagree) or whether to label all responses
options or only some of the response scale options.
Result LR:
o respondents is likely to choose positive ratings on the bipolar than on
the unipolar
o midpoint of both numerical (-5 to +5) and (1 to 10) was chosen far more
frequently for bipolar verbal anchors (much less power, given much
more power) than the unipolar verbal anchors (not given any more
power, given much more power)
o the lowest scale points (0 or -5) were chosen far more frequently if the
verbal anchor were unipolar
4

Norinah Mohd Ali

CHA130026

o the words disgusting, unsatisfactory, neutral, desirable and

excellent produced normal distributions that overlapped little, whereas
words such as acceptable, important and indifferent polarised
respondents.
o Words fantastic and excellent to be the most positive adjectives, and
horrible and terrible to be the most negative adjectives.
o the term delightful, respondents varied the least, whereas for the term
unacceptable respondents varied the most .
Order and direction of response options
Primacy effect = Respondents will select earlier alternatives more frequently
than later alternatives, especially when alternatives are presented on show
cards.
Recency effect = Respondents select the later alternatives, and is thought to
apply mainly when respondents only hear the alternatives.
The phenomenon of shifting frames of reference refers to the possibility that
the selection of a certain alternative depends on whether the more favourable
alternatives are presented earlier or later.
Result LR:
o Direction of response option did not affect mean scores and standard
deviations significantly as long as the strongly agree option
corresponded to the highest numerical value (=8) and strongly
disagree to the lowest numerical value (=1).

Conclusion

Norinah Mohd Ali

CHA130026