Quality & Quantity 38: 367379, 2004.

2004 Kluwer Academic Publishers. Printed in the Netherlands.

367

Investigating Question Meaning and Context

Through In-Depth Interviews
PAUL BEATTY

National Center for Health Statistics

WENDY DAVIS HICKS

U.S. Census Bureau

EMILIE SCHMEIDLER and CORINNE KIRCHNER

American Foundation for the Blind

Key words: in-depth interviewing, cognitive interviewing, survey methods, questionnaire design,
disability statistics.

1. Introduction
The objective of this article is to demonstrate the usefulness of in-depth interviewing to illuminate the meaning of survey findings. In particular, the study described
here was designed to explore the meaning of two survey statistics addressing the
prevalence of visual impairment in the United States. Two surveys have been used
in recent years to provide such estimates: the National Health Interview Survey
(NHIS) and the Survey of Income and Program Participation (SIPP). Both surveys
have included a question about the inability to read newspaper print at a functional
level as a key measure. The NHIS measure asks Wearing your (glasses/contact
lenses) if that is how you see best . . . can you see well enough to read newspaper
print? The SIPP measure asks Do you have difficulty seeing the words and letters
in ordinary newspaper print even when wearing glasses or contact lenses, if you
usually wear them?
These questions are not identical, but both refer to the same conceptual domain (difficulty reading print), both refer to problems not correctable by ordinary
glasses or contact lenses, and both calibrate the problem by referring to a print
size that virtually everyone is familiar with. Agencies that base their program
planning on such survey data, such as the National Library Service for the Blind
and Physically Handicapped (NLS), believed that either measure should effectively
identify the populations they intended to serve. Yet statistics from the two surveys

Author for correspondence. Dr. Paul Beatty, National Center for Health Statistics, 3311 Toledo

Road, Room 3218, Hyattsville, MD 20782, U.S.A.

368

PAUL BEATTY ET AL.

are quite different: SIPP data from 199192 data yield an estimate of 9.7 million
people 15 years old and older (McNeil, 1993). NHIS data (from 1977 and 1984,
adjusted to 1990 population parameters) yield an estimate of 4.3 million people
(approximately 4.2 million people 15 years old and older).1
From a policy standpoint, this difference is quite problematic. NLS and other
agencies needed to decide which statistic to use, hopefully based on an understanding of their actual meanings. As a result, NLS supported a research initiative
designed to go beyond the statistics and to explore why respondents answer the
questions as they do. In particular, the present study explored what specific aspects
of each question invoked different decision processes. Can the differences between
the two survey estimates be explained entirely in terms of question wording, or
are they also due to other aspects of question format and the larger context the
questions appear in? Is there a better question- or at a minimum, one that makes
more sense for use in policy planning? In an effort to address these questions, we
conducted in-depth interviews with respondents who had various degrees of visual
impairment. Respondents in the study not only answered the survey questions
of interest, but also provided their interpretations of the questions and detailed
explanations of their visual abilities.
In-depth interviewing is not new to survey research. In recent years, survey
researchers have employed a particular variant known as cognitive interviewing
as a tool for questionnaire development (Willis et al., 1999; Wilson and Beatty,
1999). Yet cognitive interviewing techniques are generally applied to draft survey questions with an eye toward devising improvements (Willis, 1994; DeMaio
and Rothgeb, 1996; Forsyth and Lessler, 1991). The usefulness of this type of
interviewing for understanding the frame of reference of previously administered
questions has not been discussed nearly as often.
The few studies that have used in-depth interviews to study existing questions
have done so in a preliminary manner. For example, Sudman et al., (1997) used
focus groups and in-depth interviews to study the cognitive aspects of self-report
data on cancer examinations however, the findings were primarily used to develop an additional record-check study. While this is an excellent use of in-depth
interviews, it is also important to demonstrate how such interviews can themselves
contribute to our understanding of social science problems. Narrative responses
offer a window into the actual experiences of respondents that statistics alone do
not. Furthermore, these narratives can be coded as well as analyzed qualitatively,
yielding insights from both angles. Researchers from qualitative backgrounds have
long argued that such research can illuminate findings from prior quantitative
studies rather than the more common view that in-depth interviews are merely
preliminary studies that need to be validated by other types of research (Weiss,
1994). It is our conviction that in-depth interviewing can be usefully viewed as a
component of ongoing investigations, rather than a preliminary step before real
research begins.

INVESTIGATING QUESTION MEANING AND CONTEXT THROUGH IN-DEPTH INTERVIEWS

369

2. Evaluating Differences Between the Two Surveys

Since our study was designed to explore the difference between two survey statistics, we began with a careful consideration of the differences between the surveys
that created them. Both the SIPP and NHIS are large scale U.S. Federal statistical
surveys, based on national probability samples of the non-institutionalized population. Both use standardized interviewing techniques (Fowler and Mangione, 1990)
and data for both are collected by the U.S. Census Field Representatives who have
completed the same general training. Nevertheless, there are important differences
between the overall survey designs and the visual disability questions themselves.
2.1. DIFFERENCES BETWEEN SIPP AND NHIS DATA COLLECTIONS
First, respondent selection rules for the two surveys are somewhat different. The
1984 NHIS Supplement on Aging was written specifically for self-report. However,
respondents in the 1977 NHIS and the 199192 SIPP sometimes answered print
disability questions by proxy for other members of the household. According to
1992 SIPP data, 5.8% of self-respondents reported difficulty reading newspaper
print, but only 3.8% of proxy respondents indicated that the sampled individual
had difficulty reading newspaper print. Perhaps a greater proportion of self-reports
could have led to higher reports of print disability. However, while the prevalence
of proxy responses in the 1977 NHIS was not recorded, we do know that the 1984
component of the NHIS estimate was based entirely on self-responses yet the
reports of print disability are lower on the NHIS. Therefore, this explanation seems
less promising than some alternatives.
Second, the content of the two questionnaires is quite different. The SIPP
primarily consists of questions about participation in government funded financial
assistance programs. The print disability question appears in a periodic supplementary module on health and disability, which covers a range of topics. In
contrast, the NHIS focuses on health and health related behavior. Furthermore, the
NHIS print disability question follows a series of vision-related questions about
glaucoma, cataracts, use of magnification devices to do close work, and so on. It
is therefore important to consider the possibility that context effects from earlier
questions influence responses to print disability questions.
2.2. DIFFERENCES BETWEEN QUESTIONS ON THE SIPP AND NHIS
The SIPP and NHIS print disability questions themselves, which appear in Appendix A, differ in several ways. The SIPP asks whether respondents have
difficulty seeing the words and letters in ordinary newspaper print, while the
NHIS asks whether respondents can see well enough to read newspaper print.
These seem to assess different levels of severity one could be able to read while
still having difficulty doing so. However, it is not completely clear whether the
difference is sufficient to account for a statistical gap of this magnitude.

370

PAUL BEATTY ET AL.

There are other minor differences: the SIPP mentions ordinary newspaper
print while the NHIS simply says newspaper print. The lack of the word ordinary could allow greater reports of ability to read print, if large headlines were
counted. The lower reports of print disability on the NHIS are consistent with this
explanation, although this seems like a very minor difference in wording. Another
issue is that the NHIS asks about ability to read, which could be interpreted as
referring to literacy. However, if respondents answered on the basis of literacy, we
would expect that the NHIS would generate higher indications of print disability
(capturing both those who could not see and those who could not read). The actual
statistical difference works in the opposite direction of this explanation.
Another design difference involves the direction in which the questions are
asked. A yes response to the NHIS means that the respondent is able to read
newspaper print. In contrast, a yes response to the first SIPP item indicates
that the respondent has difficulty reading newspaper print. A bias toward yes
responses in general would decrease estimates of print disability from the NHIS,
and increase estimates of print disability from the SIPP. This is consistent with
the statistical difference we observed. There is some evidence of acquiescence for
attitudinal questions i.e., respondents may lean toward yes responses out of
a general tendency to agree with the central premise of opinion questions (Schuman and Presser, 1981: 203217). However, most research on acquiescence has
focused on agree/disagree questions and argues that the effect is based on either
a desire to seem agreeable or a tendency to be uncritical of general statements.
To our knowledge, no one has proposed a reason to expect similar effects regarding health-related judgments. Therefore acquiescence does not seem to be a
particularly promising explanation.
Another difference in the wording of the two measures involves the placement
of the instruction informing respondents to answer in reference to wearing glasses
or contact lenses, if that is how they see best. In SIPP, this instruction is explicit.
The NHIS, in contrast, includes this instruction only as part of a lead-in to a series
of questions. The print disability question is the third in this series; therefore the
lead-in could be forgotten and the frame of reference not included while thinking
of a response. Yet, if respondents considered the use of glasses in the SIPP more
often than in the NHIS, we would expect higher rates of print disability from the
NHIS. This is in the opposite direction of the difference we observed.

3. Methods
Our interviewing protocol was designed to help tease out the specific sources of
the difference between the NHIS and SIPP estimates i.e., whether it could be
attributable to major concepts, individual words within the questions, or the context
from surrounding questions. We developed three different interviewing protocols,
outlined in Table 1.

INVESTIGATING QUESTION MEANING AND CONTEXT THROUGH IN-DEPTH INTERVIEWS

371

Table I. Three interview conditions

Form 1

Form 2

Form 3

abbreviated version of
NHIS core interview
general vision-related
questions from NHIS
NHIS print reading
disability measure
cognitive/narrative
probing
abbreviated version of
core SIPP interview

abbreviated version of
NHIS core interview

abbreviated version of
of core SIPP interview

NHIS print reading

disability measure
cognitive/narrative
probing
abbreviated version of
core SIPP interview

SIPP print reading

disability measure
cognitive/narrative
probing

SIPP print reading

disability measure
cognitive/narrative
probing

SIPP print reading

disability measure
cognitive/narrative
probing
abbreviated version of
core NHIS interview
general vision-related
questions from NHIS
NHIS print reading
disability measure
cognitive/narrative
probing

Each version began with an abbreviated questionnaire from either the NHIS
or the SIPP, ending with the print disability measures. Probing began at that
point, using a semi-scripted interviewing protocol. Some of the suggested probes
included:
What does this question sound like it is asking to you?
?
How did you decide that your answer was
How easy was it for you to decide that?
When answering, were you thinking about being able to see newspaper print,
or were you thinking about being able to read newspaper print? Is there a
difference?
In your own words, can you tell me how well you can see newspaper print?
Interviewers were also free to add new probes if responses seemed to warrant additional inquiry. Respondents were then told that we wanted to ask some questions
from another survey, some of which may sound similar to items they had already
answered but we wanted them to try to answer them regardless of what their
previous responses had been. After answering the second set of survey questions,
additional probes were administered. Finally, we re-read both print disability items
and asked respondents how (or if) they differed.
The primary difference between versions was whether respondents first
answered the NHIS (Versions 1 and 2) or the SIPP (Version 3). Version 2 omitted
a series of NHIS vision-related questions which appeared immediately prior to the

372

PAUL BEATTY ET AL.

print disability question. This design allowed us to make several comparisons. First,
we compared responses and interpretations to the NHIS with preceding questions,
the NHIS in isolation, and the SIPP measures. Also, since all respondents answered
both the SIPP and NHIS print disability questions, we asked them for direct comparisons. We were also able to combine responses from various versions for certain
analyses. For example, we hypothesized that the vision-related questions from
NHIS might influence responses to subsequent print disability questions. These
questions appeared before the NHIS print disability measure on Forms 1 and 3, but
not on Form 2.
Resources were available to conduct thirty interviews for the project. Respondents were recruited from two clinical settings at Johns Hopkins University, as well
as the Virginia Department for the Visually Handicapped and a Washington, DCbased ophthalmology practice of retina specialists. These settings were used in
order to find respondents within a specific range of impaired visual acuity, and
within a limited geographical area. All respondents had been previously diagnosed
as having a visual acuity between 20/60 and 20/100 in the better eye, with best
correction. This range was chosen with the expectation that such respondents
would not be completely blind but would not be able to read newspaper print
with complete ease. To protect confidentiality, staff at these organizations contacted potential respondents to request permission to give their names, addresses
and phone numbers to Census Bureau staff for the purpose of participating in our
research. If permission was granted, the respondents were contacted to schedule an
interview. All respondents were interviewed in their homes and were paid $25 for
their participation in a 30 minute interview.
In addition to meeting the visual criteria, we selected the 30 respondents to vary
in terms of age, race and education. Most visual impairment is among older people,
and almost all of our respondents were above the age of 55. However, six respondents between 18 and 35 years of age were included under the assumption that their
experiences and interpretations might be different. We included both blacks and
whites in our sample because they have different rates of clinically measured visual
impairment; approximately 1/3 of our respondents were black. We varied education
level because comprehension of questions could differ along those lines. We were
also open to the possibility that illiterate (or less literate) respondents might answer
the NHIS on the basis of their ability to read; however, we were unable to screen
for this in advance, and determined during the interviews that all respondents were
literate.
Respondents were randomly assigned to one of the three interviewing conditions. Eleven respondents were interviewed with Form 1, nine were interviewed
with Form 2, and ten were interviewed with Form 3. Interviews were conducted by
one of two interviewers, both of whom had several years of experience conducting
in-depth interviews. Each interview was transcribed so that verbatim text could
be analyzed, and frames of reference could be coded e.g., whether respondents
included glasses or other aids while answering the question; whether respondents

INVESTIGATING QUESTION MEANING AND CONTEXT THROUGH IN-DEPTH INTERVIEWS

373

were thinking of headlines, regular print text, or some other size of print; and so
on.2 We also coded objective information such as whether respondents regularly
wore glasses or used magnification devices.
4. Results
4.1. INTERPRETATION OF CENTRAL CONCEPTS IN PRINT DISABILITY
QUESTIONS

Respondents explanations indicated that they most commonly answered both

questions based on a functional ability to read print. The ability to make out
some letters, or the ability to slowly piece through the headline of a story, was
usually insufficient for them to respond that they had the ability to read print, for
both NHIS and SIPP questions. For example, a respondent who answered that he
could not read print (based on the NHIS question) explained that I can read headlines, but . . . [reading] headlines is not reading the newspaper. Although the NHIS
question does not specify ordinary print, respondents tended to interpret the
question as referring to an ability to read something substantial. As one respondent
paraphrased, the NHIS question asked if he can pick up the paper and read it.
A few respondents indicated that their answers might vary depending on whether
they were talking about regular articles or smaller-print sections, but their answers
were ultimately based upon regular articles. For example, one respondent indicated
without hesitation that she could read the newspaper unless we meant obituaries.
Another indicated that she could read newspapers except the little teeny [letters].
When asked to compare the meanings of questions, virtually all respondents (in
all three interviewing conditions) thought that the central concept of being able to
read print was expressed equivalently by the SIPP and NHIS questions. In other
words, the ability to see words and letters is the same as being able to see well
enough to read. One respondent pointed out the literal difference and noted that
it did not change the overall meaning: the [SIPP] question asks me, do you have
difficulty seeing the words and letters. I dont have difficulty seeing that there are
words and letters there . . . actually, I can see the letters, I just cant read them.
Yet he added that individual words dont do you any good if you cant read [the
content of articles]. His actual response was that he did have difficulty, and that
both questions referred to a pragmatic ability to read.
In spite of conceptual similarities, the questions do address different severity
levels of print disability. While 23 of 30 respondents indicated that they had difficulty reading print based on the SIPP question, only 18 of 30 reported difficulty
based on the NHIS. This difference operates in the same direction as national data,
with higher reports of print disability on the SIPP. One respondent explained his
NHIS response (he could read print) by noting that he could read with special
reading glasses, for perhaps 1530 minutes. Although he was capable of reading a
regular newspaper article, his abilities were limited enough to admit on the SIPP
that difficulty was involved. He believed that most people can clearly say that

374

PAUL BEATTY ET AL.

they either can or cannot read newspaper print, but that isnt true in some cases
. . . I cant see that all the time . . . My vision changes as I am looking. Other apparent disagreements between SIPP and NHIS responses reflected similar thinking.
For example, another respondent explained his NHIS answer by noting that I read
the paper every day, but when answering the SIPP admitted that he sometimes
had difficulty depending on the time of day and sometimes I just cant focus.
Thus, even though the central concepts of the question are the same, the severity
level implied by each question is likely to be a factor in some responses.

4.2. FRAME OF REFERENCE WHILE ANSWERING THE QUESTIONS : GLASSES

AND OTHER AIDS

Virtually all respondents assumed that the questions referred to their ability to read
with glasses or contact lenses, as opposed to with the naked eye. The SIPP question
explicitly instructs respondents on this point. The NHIS is somewhat less clear,
because the instruction to include glasses appears within an introduction several
questions earlier, and some respondents expressed initial uncertainty about how to
answer (well . . . [I cant read] without my glasses). In such cases, the interviewers
asked respondents to interpret the question as they thought most appropriate, and
all respondents who wore glasses concluded that the NHIS question should be
answered based on their reading ability with glasses.
In contrast, most respondents assumed that it was inappropriate to indicate an
ability to read print if they could do so only with stronger magnification devices. Of
the respondents who explained that they could read print only with high-powered
magnifiers, only one decided to answer the NHIS item positively. Eight other respondents who could read only with high-powered magnifiers answered negatively,
explaining that they did not believe the question intended for them to consider
such circumstances. One respondent initially answered yes, but then reconsidered:
well, not without magnification. When pressed for an answer, he concluded I
think [the question] means without this (indicating his magnifier). The answer to
that is no. Another explained simply that he answered no because I have to use a
magnifier.
Altogether, respondents tended to answer the print disability question based
on their everyday abilities using only unremarkable corrective devices such as
glasses. Yet on the NHIS, respondents were less immediately clear about what to
include or exclude. Although they ultimately seemed to reach the same conclusions
about corrective devices as the SIPP responses, they did indicate some uncertainty,
which may have made respondents more susceptible to some potential response
effects that we discuss below.

INVESTIGATING QUESTION MEANING AND CONTEXT THROUGH IN-DEPTH INTERVIEWS

375

Table II. Response to SIPP print disability question according to context

Form 1
SIPP preceded by
NHIS context Qs
SIPP response:
Can read print
Difficulty reading print
Total

Forms 2 & 3
SIPP not preceded by
NHIS context Qs

5
6

2
17

11

19

4.3. CONTEXT OF THE PRINT DISABILITY QUESTIONS

A notable difference between the two questionnaires is that the print disability
questions appear in different contexts. The NHIS question follows a lengthy series
of vision-related questions, while the SIPP print question is the only vision-related
question in its instrument. Is there any evidence that respondents answer differently
depending on whether or not they answer other vision-related questions first?
If there is a context effect, it would be difficult to pick this up through narrative
responses alone. As noted previously, most respondents thought the questions were
very similar in meaning, and furthermore we would not expect them to be aware of
all factors that influence their responses. Yet we can look at how actual responses
varied according to interviewing condition. There were modest differences between
NHIS responses that were preceded by other visual ability questions (Forms 1 and
3) and those not preceded by such questions (Form 2). In the former case, 10 of
21 respondents (47.6%) indicated that they could read print, while only 2 of 9
respondents (22.2%) indicated this in the latter. While not statistically significant,
it does show a modest tendency for respondents to rate their vision more highly
when they first answered the preceding vision questions.
The questions about magnification and current visual conditions do not appear
on the actual SIPP instrument. However these additional questions did appear prior
to the SIPP print disability question on Form 1 of our study. On Forms 2 and 3, participants answered the SIPP question before any of the additional NHIS questions
about vision or magnification had been administered. Table 2 shows how responses
to the SIPP print disability question varied based on whether or not participants had
previously answered these additional vision questions.
We would expect the response distributions in the two columns to be equivalent
unless some particular characteristic of the interview forms affected participants
responses. The actual difference observed approaches significance (Fishers Exact
Test, two-sided, p < 0.068), suggesting that questionnaire content may play a
role in how people respond. Note also that while reports of print-reading disability
did vary across the forms, a simple measure of actual visual abilities did not

376

PAUL BEATTY ET AL.

approximately six respondents in each condition expressed inability to read entire

words in fast enough succession to comprehend a complete sentence, while the rest
indicated that they were able to do so.

5. Discussion
Results of our interviews suggest that the central concepts addressed in the SIPP
and NHIS print disability questions are very similar. Subtle wording differences
between newspaper print and ordinary newspaper print, or see well enough
to read and seeing words and letters are unlikely to account for the large statistical differences generated by the two measures. Respondents tended to base their
answers to either question on a functional ability to read print they indicated
print reading disability if they could not read the text of a standard newspaper
article through the use of ordinary devices such as glasses or contact lenses.
However, the measures do specify different criteria for indicating print disability
respondents met the SIPP criterion of having difficulty more easily than the
NHIS criterion of not being able to see well enough to read print. This difference
between these criteria is non-trivial. Even with a very small sample, we observed
different prevalence based on the two questions. Respondents who indicated print
disability with the SIPP question but not the NHIS question also provided narrative information that highlighted the differences in meaning between these two
thresholds of responses.
Context effects are also likely to play a role in creating the discrepancy. There
is some evidence that the vision questions preceding the print disability measure
on the NHIS influence respondents to evaluate their vision more positively. The
key issue seems to be whether or not to include magnification devices while answering. Numerous respondents mentioned magnifiers in explaining their decision
processes. While both print disability questions are ambiguous regarding whether
or not respondents should include magnifiers in their responses, the NHIS includes
an earlier question about using magnification devices to do close work. It is likely
that this question forces respondents to explicitly consider whether or not magnification devices are appropriate for inclusion. We conclude that respondents who
received the context questions ultimately tended to exclude the use of magnifiers
from their responses (for a more detailed discussion of exclusion effects due to
context, see Sudman et al., (1996), and Schwarz and Bless (1992)).
Our findings echo those of Todorov (2000), who also concluded that context
effects from vision-related questions decreased the frequency of reports about
serious difficulty seeing on the NHIS. His study differs from ours in a number of ways: Todorov did not specifically examine print disability questions, and
furthermore, the questionnaires he studied did not include some of the contextcreating questions that ours did (e.g., the preceding question about using magnifiers
was absent from Todorovs study). He also compared different versions of the
NHIS, rather than comparing the NHIS and SIPP. Also, while Todorov had a very

INVESTIGATING QUESTION MEANING AND CONTEXT THROUGH IN-DEPTH INTERVIEWS

377

large sample that allowed for statistical analysis, his use of previously collected
survey data meant that he was unable to directly manipulate the parameters of
his experiment. Our study and Todorovs complement each other nicely, both suggesting that the potential for context effects should be carefully considered when
reporting statistics of this type, while using very different methods to reach these
conclusions.
In fact, both our study and Todorovs suggest that it may be more appropriate
to consider estimates of phenomena such as visual impairment to be products
of modules rather than individual questions. This has important implications for
survey researchers, who often transplant individual questions from one survey to
another. Such questions may not be as portable as researchers think, and it may
be more appropriate to replicate larger batteries of questions rather than individual
items or, at a minimum, to evaluate the potential of context-creating questions
from the original questionnaire before pulling out an individual question. When
considering statistical discrepancies in general, it may be that apparently similar
questions are functionally more different than they appear due to their contexts.
We think that this study reveals how in-depth interviewing can contribute to
the investigation of the meaning of survey findings. Although in-depth interviews
have been successfully employed for years in questionnaire design, they also offer unique insights into various survey problems. Such interviews provide details
about respondent frames of reference while answering questions, yielding narrative
insights that are impossible to glean from statistics alone. The narrative data can
also be coded to explore particular ideas quantitatively. These interviews can be
conducted at low cost as well. In-depth interview studies deserve consideration not
just as rough exploratory projects, but as part of a larger research picture with some
unique angles to contribute to the study of measurement error in surveys.
Appendix A. Print Reading Disability Measures
SIPP
Question content preceding the print disability item all respondents:

Recent work experience

Income sources from the Federal Government
Retirement, disability, or pension income
Medicare and Medicaid coverage
Current health status
Use of aids to get around

(Do you/Does Name) have difficulty seeing the words and letters in ordinary
newspaper print even when wearing glasses or contact lenses if (you/Name)
usually wear(s) them?

378

PAUL BEATTY ET AL.

[ ] Yes, has difficulty

[ ] No difficulty skip to 3
NHIS
Question content preceding the print disability item all respondents:
Work/retirement status
Disability income
Difficulty performing activities (walking, standing, sitting, reaching, grasping,
lifting)
Questions on the following topics were omitted for respondents using Form 2:
Current visual conditions (cataracts, glaucoma, color-blindness, detached
retina, blindness, other difficulty seeing)
Use of eyeglasses or contact lenses
Ever had cataract surgery
Currently have lens transplant
Use magnifying glass to do up-close work
The next few questions are about how well you can see (wearing your
(glasses/contact lenses) if that is how you see best.)
1) Can you see well enough to recognize the features of people if they are within
2 or 3 feet?
2) Can you see well enough to watch t.v. 8 to 12 feet away?
3) Can you see well enough to read newspaper print?
[ ] Yes
[ ] No

Notes
1. See Nelson and Dimitrova (1993) for further explanation of the NHIS estimates, which were
created by combining data from different years and target populations and temporally adjusted.
One may question whether such complex estimation procedures may be responsible for the
discrepancy. While Nelson and Dimitrova did not provide a confidence interval for the NHISbased estimate, the SIPP estimate has a 90% confidence interval of (9.3 million, 10.0 million).
Although the NHIS confidence interval may be wider, it is highly unlikely that it could be wide
enough to account for a discrepancy of this size.
2. We recorded the survey responses for all 30 respondents, but due to tape recorder malfunctions
were only able to transcribe and code the content of 26 interviews. This coding was done independently by both of the project interviewers. Initial agreement of codes was 90%, and all
disagreements were reconciled prior to analysis.

INVESTIGATING QUESTION MEANING AND CONTEXT THROUGH IN-DEPTH INTERVIEWS

379

References
DeMaio, T. J. & Rothgeb, J. M. (1996). Cognitive interviewing techniques: in the lab and in the
field. In: N. Schwarz & S. Sudman (eds), Answering Questions: Methodology for Determining
Cognitive and Communicative Processes in Survey Research. San Francisco, CA: Jossey-Bass.
Forsyth, B. H. & Lessler, J. T. (1991). Cognitive laboratory methods: A taxonomy. In: P. P. Biemer, R.
M. Groves, L. E. Lyberg, N. A. Mathiowetz & S. Sudman (eds), Measurement Error in Surveys.
New York: Wiley.
Fowler, F. J. & Mangione, T. W. (1990). Standardized Survey Interviewing: Minimizing InterviewerRelated Error. Newbury Park, CA: Sage.
McNeil, John M. (1993). Americans with Disabilities, 199192. U.S. Bureau of the Census Current
Population Reports, P70-33. U.S. Government Printing Office, Washington, DC.
Nelson, K. A. & Dimitrova, G. (1993). Statistical brief #36: Severe visual impairment in the United
States and in each state, 1990, Journal of Visual Impairment and Blindness, 8085.
Schwarz, N. & Bless, H. (1992). Constructing reality and its alternatives: An inclusion/exclusion
model of assimilation and contrast effects in social judgment. In: L. L. Martin & A. Tesser (eds),
The Construction of Social Judgements. Hillsdale, NJ: Erlbaum.
Schuman, H. & Presser, S. (1981). Questions and Answers in Attitude Surveys. New York: Academic
Press.
Sudman, S., Bradburn, N. & Schwarz, N. (1996). Thinking About Answers: The Application of
Cognitive Processes to Survey Methodology. San Francisco: Jossey-Bass.
Sudman, S., Warnecke, R., Johnson, T., ORourke, D., Davis, A. M. & Jobe, J. B. (1997). Understanding cognitive processes used by women reporting cancer prevention examinations and tests.
Journal of Official Statistics 13: 305315.
Todorov, A. (2000). Context effects in national health surveys: Effects of preceding questions on
reporting serious difficulty seeing and legal blindness. Public Opinion Quarterly 64: 6576.
Weiss, Robert S. (1994). Learning from Strangers: The Art and Method of Qualitative Interview
Studies. New York: Press Press.
Willis, G. B. (1994). Cognitive interviewing and questionnaire design: A training manual. Cognitive Methods Staff Working Paper Series, No. 7. Hyattsville, MD: National Center for Health
Statistics.
Willis, G. B., DeMaio, T. & Harris-Kojetin, B. (1999). Is the bandwagon headed to the methodological promised land? Evaluating the validity of cognitive interviewing techniques. In: M. Sirken,
D. Herrmann, S. Schechter, N. Schwarz, J. Tanur & R. Tourangeau (eds), Cognition and Survey
Research. New York: Wiley.
Wilson, B. F. & Beatty, P. (1999). Cognitive aspects of survey methodology: An oral history. In:
M. Sirken, T. Jabine, G. Willis, E. Martin & C. Tucker (eds), A New Agenda for Interdisciplinary Survey Research Methods: Proceedings of the CASM II Seminar. Hyattsville, MD: National
Center for Health Statistics.