THEORY TO DATA COLLECTION INSTRUMENTS 1

Developing Data Collection Instruments

In this document is a brief review of the relationship between the guiding theory of your research study and the data collection

instrument. Additionally I will review some guiding principles for each type of question that might be in the data collection

instrument.

Theory and the Data Collection Instrument

The Topic of Inquiry leads to the organization (taxonomy and categorization) of the Literature Review, which leads to the

development of the quantitative theoretical framework, or the qualitative theoretical construct, or conceptual framework, literature

map. These are the source of the questions developed for interviews, questionnaires, and surveys. For the purposes of this experience

I define Interviews use open-ended questions that encourage lengthy and detailed responses. Questionnaires contain semi-structured

questions with a short answer response. Surveys contain directed questions with a quantifiable measured response.

The process of developing the data collection instrument provides a direct relationship between the theory and the data

collection instrument. Data collected should always be related to the theory. This is the essence of reliability in scientific research. It

is also necessary to be mindful of your unit of analysis. If you are analyzing at the individual level your questions should be related to

this level or the individual perspective of other units of analysis, such as the team, organization, or social institution.

The diagrams below might lead you to believe there is a one question to one concept relationship. There are two strategies

regarding the concept to question ratio. In one strategy with quantitative directed questions and qualitative semi-structured questions

you might have multiple questions concerning each concept; the point is that you do not leave assumptions unquestioned or gaps in
THEORY TO DATA COLLECTION INSTRUMENTS 2

the questions, which leave data uncollected. Although you might have multiple questions for each concept, you will not have multiple

concepts for one question. In a second strategy with qualitative open-ended questions you are attempting to encourage the participant

to respond regarding the concepts with one or more questions that are over arching in nature, or of a broader scope.

Qualitative Data Collection

Open-Ended Questions are artfully crafted to be as inclusive of the factors making up the qualitative theoretical construct, or

conceptual framework, literature map in as few questions as possible. The artful open-ended question does not lead or bias the

participant’s response. This leaves room for the participant to respond from their perspective as it naturally occurs to them. If you

have too many questions, you are really constructing a questionnaire of semi-structured questions. When using open-ended questions

the interviewer should prepare a few questions to use in probing for greater detail and clarifying questions to reduce the assumptions

left unquestioned. It is difficult to anticipate these probing or clarifying questions, engaging in the practice of prewriting will prepare

you to improvise good questions during the interview. A good open-ended question might encourage the participant to reflect on and

then respond from her or his experience without prompting for the specific concepts or variables. Begin with a discussion of the

purpose of the research study. Then discuss briefly the theoretical perspective. This grounds the participant into the research study

and makes it more likely their responses will be appropriate. Although this can constitute a form of leading the participant to give the

responses you are looking for. Be cautious with how much information you give prior to the interview and in revealing your bias.

Refer to Figure 1 and Table 1 for a progression from theory to open-ended questions.
THEORY TO DATA COLLECTION INSTRUMENTS 3

Theoretical Construct Data Collection Instrument

Concept Question
Factor Concept Question Factor

Concept Question
Question
Factor Concept Question Factor
Concept Question

Concept Question
Theory

Theory
Factor Concept Question Factor

Concept Question
Factor Concept Question Factor

Concept Question
Factor Concept Question Question
Factor
Concept Question
Concept Question
Factor Factor
Concept Question

Figure 2. Theory to open-ended data collection instrument. In this model the researcher would ask one question for each factor of the

theory of Emotional Intelligence.

THEORY TO DATA COLLECTION INSTRUMENTS 4

Exercise 1. Theory to open-ended question progression

Theory Factor or Concept Qualitative Open-ended Questions Probing and Clarifying Questions
Variable
Emotional Empathy Understanding Others – sensing Discussion of the purpose of the P. What do you think that ____ was
Intelligence others’ feelings and perspectives, research study – Understanding the role attempting to gain through this
taking an active interest in their of empathy in determining leadership behavior? How might you label the
concerns succession in work teams. ethics of _____’s motivation in
Developing Others – Sensing others’ Brief discussion of the theoretical communicating this?
development needs and fostering perspective – Empathy in Emotional
their abilities Intelligence. C. What does the acronym ____
Service Orientation – Anticipating, mean in this context?
recognizing, and meeting others’ I would like you to relate to me some How do the others respond in
needs instances that exemplify how you make similar situations as you have just
Leveraging Diversity – Cultivating use of empathy in your relationships described?
opportunities through different kinds with the others on your team, how you
of people interact with them based on that
Political Awareness – Reading a empathetic understanding, how it might
group’s emotional currents and effect your relationships with them, and
power relationships how it might impact your team’s
performance.

Exercise:

In small groups develop two open-ended questions with their related two probing and two clarifying questions.
THEORY TO DATA COLLECTION INSTRUMENTS 5

Semi-Structured Questions should only respond to one concept in the qualitative theoretical construct, or conceptual

framework, literature map. In other words the response to a question of these types should only satisfy one concept. Questions of

these types that satisfy or answer more than one question are not clear and concise enough, and the resulting data will not yield

specific and reliable data. The well constructed semi-structured question limits the scope of the participant’s response and yet does

not lead or bias that response. Note that semi-directed questions should not require probing or clarifying and are appropriate for a

self-administered instrument. See Figure 2 and Table 2 below for an example of the progression from a theory to its factors, then to

questions.
THEORY TO DATA COLLECTION INSTRUMENTS 6

Concept Question
Factor Concept Question Factor

Concept Question
Factor Concept Question Factor
Concept Question

Concept Question
Theory

Theory
Factor Factor
Concept Question

Concept Question
Factor Concept Question Factor

Concept Question
Factor Concept Question Factor
Concept Question
Concept Question
Factor Factor
Concept Question

Figure 2. Theory to semi-structured data collection instrument.

THEORY TO DATA COLLECTION INSTRUMENTS 7

Exercise 2. Theory to semi-structured question progression

Theory Factor or Variable Concept Qualitative Semi-Directed Questions

Emotional Intelligence Empathy Understanding Others – sensing others’
feelings and perspectives, taking an
active interest in their concerns
Please feel free to offer examples or details in your responses to
the following questions.
1. How do you describe your ability to listen to others and to
perceive their body language and vocal tone?
2. How would your coworkers describe your skill in listening
well when they are discussing a difficult issue with you?
3. Share an example of a colleague indicated they are
experiencing difficulty?

Developing Others – Sensing others’ 1.

development needs and fostering their
abilities
Service Orientation – Anticipating, 1.
recognizing, and meeting others’ needs
Leveraging Diversity – Cultivating 1.
opportunities through different kinds of
people
Political Awareness – Reading a 1.
group’s emotional currents and power
relationships

Exercise:

In small groups develop at least two semi-structured questions for each concept that is part of the factor of Empathy.
THEORY TO DATA COLLECTION INSTRUMENTS 8

Quantitative Data Collection

Directed Questions should only respond to one concept in the quantitative theoretical framework. In other words the

response to a question of these types should only satisfy one concept. Questions of these types that satisfy or answer more than one

question are not clear and concise enough, and the resulting data will not yield specific and reliable data. The directed (closed-ended)

question provides the participant with a specific range of responses yet does not lead or bias the response. An important consideration

for directed questions is the level of measurement you will use. The level of measure is either: nominal, ordinal, or interval/ratio. For

direct comparisons of responses you should use the same level of measurement. In SPSS these questions are referred to as variables

and in testing your model and hypothesis you will be comparing the variables (questions) against each other. See Figure 3 and Table 3

below for an example of the progression from a theory to its factors, then to questions.
THEORY TO DATA COLLECTION INSTRUMENTS 9

Dependent Concept Question Dependent

Variable Concept Question Variable

Independent Concept Question Independent

Concept Question
Variable I Variable I
Concept Question
Independent Concept Question Independent
Theory

Theory
Variable II Concept Question Variable II
Independent Concept Question Independent

Variable III Concept Question Variable III

Independent Concept Question Independent

Concept Question
Variable IV Variable IV
Concept Question
Independent Independent
Concept Question
Variable V Concept Question Variable V

Figure 3. Theory (theoretical framework) to directed question data collection instrument.

THEORY TO DATA COLLECTION INSTRUMENTS 10

Exercise 3. Theory to directed question progression.

Theory Factor or Concept Quantitative Questions Level of

Variable Measurement
Emotional Empathy Understanding Others – sensing 1. Rate your ability to listen to others and perceive their Nominal
Intelligence others’ feelings and perspectives, body language and vocal tone or inflection?
taking an active interest in their
concerns 2. On a scale from very good to very poor indicate how Nominal
your coworkers would rate your skill in listening well.

3. How willing are you to broach a sensitive topic with a Ordinal

colleague who indicates they are experiencing
difficulty?

4. How well would your coworkers rate your skill in Ratio/interval

understanding them? Use a scale of 1 to 5 where 1 = not
skilled at all and 5 = very skilled.

Developing Others – Sensing 1.

others’ development needs and
fostering their abilities
Service Orientation – Anticipating, 1.
recognizing, and meeting others’
needs
Leveraging Diversity – Cultivating 1.
opportunities through different
kinds of people
Political Awareness – Reading a 1.
group’s emotional currents and
power relationships

Exercises:

In small groups develop at least two directed questions and choose an appropriate level of measurement format for each

concept that is part of the variable Empathy. The questions should represent the full diversity of measurement formats.
THEORY TO DATA COLLECTION INSTRUMENTS 11

Recommended Reading

Creswell, J. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. (4th ed.). Thousand Oaks, CA: Sage

Publishing

Maxwell, J. A. (2013). Qualitative research design: An interactive approach. (3rd ed.). Thousand Oaks, CA: Sage Publishing

Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Jossey-Bass

Saldaña, J. (2011). Fundamentals of qualitative research. In P. Leavy (series Ed.) Series in Understanding Qualitative Research.

Oxford., UK: Oxford University Press

Saldaña, J. (2013). The coding manual for qualitative researchers. London, UK: Sage Publishing.

Schensul, S. L., Schensul, J. J., & LeCompte, M. D. (1999). Essential ethnographic methods: Observations, interviews, and

questionnaires. Walnut Creek, CA: Altamira Press.

LeCompte, M. D. & Schensul, J. J. (2013). Essential ethnographic methods: Observations, interviews, and questionnaires. Walnut

Creek, CA: Altamira Press.