Quality of Thought

Descriptors

Excellent: Exceptional, complete, clear, exceeds performance indicators

Good: Excellent, complete, meets performance indicators
Average: Approaching performance indicators, missing some detail, not fully developed
Fair: Developing competence, vague, weak, needs more detail
Poor: Not present

Category

Category Description

Q1 - Q 2 (5 points each), Q3-Q6 (3 points

Each Question each) Q 7 (6 points) Q 8 (3 points)Q8 (3
(95 points all) points) Q9 (8 points) Q10-Q25 (3 points
each) and Q26 (8 points) points.
Students answers are free of grammatical
Grammar &
errors and is written in a manner where a
Writing Style
college level reader can grasp the essays
(5 points)
meaning/argument in a single rapid reading.

Descriptor

Points
Awarded/Max.
Points

1) A good research plan, according to your textbook, includes:

A) Talking about a research design or plan. The research design or
plan provides the overall structure for the procedures the
researcher follows, the data the researcher collects, and the data
analyses the researcher conducts. Simply put, research design is
planning.
Nothing helps a research efort be successful so much as
carefully planning the overall design. More research efort is
wasted by going of half-preparedwith only a vague set of ideas
and proceduresthan in any other way. You will be much more
efficient and efective as a researcher if you identify your
resources, your procedures, and the forms that your data will take
always with the central goal of solving your research problem in
mindat the very beginning of your project.
2) The basic format of the quantitative research process includes (in
part), in chronological order:
A) The basic format of the quantitative research process includes :
1) A question is posed: In the mind of the researcher, a
question arises that has no known resolution.
2) Its a matter of words: The researcher converts the question
to a clearly stated research problem.
3) Its worth a guess! : The researcher poses a temporary
hypothesis or series of hypotheses.
4) The search is on! The researcher searches the literature for
ideas that shed light on the problem and for strategies that
may help to address it.
5) Data! Hard data! And nothing but the data: The researcher
collects data that potentially relate to the problem.
6) How do the data fit together: The researcher arranges the
data into a logical organizational structure?
7) The data speak: The researcher analyzes and interprets the
data to determine their meaning.
8) Its either . . . or . . .: Either the data seemingly resolve the
research problem or they do not. Either they support the
hypotheses or they do not.

3) When considering general criteria for high-quality research

projects, universality means:
A) Universality: The research project should be one that might be
carried out by any competent person. As a researcher, you are a
director and produceran agent whose function is to collect,
organize, and report what the collected data seem to indicate
but another, equally knowledge-able individual might take your
place and complete the project with essentially the same results.
4) When considering general criteria for high-quality research
projects, reliability means:
A) Replication: The research should be repeatable. Any other
competent researcher should be able to take your problem and,
collecting data under the same circumstances and within the
same parameters you have used, achieve results comparable to
those you have obtained.
5) Jack, an anthropologist, has been living in an Incan village for
three years. During that time Jack has become expert in Incan
weaving by observing and imitating the master weavers in the
community. For Jack, observing weaving constitutes a primary
data or secondary data
A) Primary Data:
6) Jackie wrote a book about her experience as an anthropologist
living in an Incan village for three years and becoming an expert
weaver. Her book was widely read by other anthropologists,
including Olivia, who is an expert in Navaho weaving. For Olivia,
Jackies book constitutes a primary data or secondary data
A) Secondary data:

7) In planning for data collection, the researcher has to answer

questions about the data,
a. Identify them, and

b. Why it is important to answer them

A) a) Basic to any research project are four fundamental questions
about the data.
1. What data are needed? This question may seem like an
overly simple one, but in fact a specific, definitive answer is
fundamental to any research efort. On a sheet of paper,
write the answers to the following questions: To resolve the
problem, what data are mandatory? What is their nature?
Are they historical documents? Interview excerpts?
Questionnaire responses? Observations? Measurements
made before and after an experimental intervention?
Specifically, what data do you need, and what are their
characteristics?
2. Where are the data located? Those of us who have
taught courses in research methodology are constantly
awed by the fascinating problems that students identify for
research projects. But then we ask a basic question: Where
will you get the data to resolve the problem? Some
students look bewildered and remain speechless or else
mutter something such as, Well, they must be available
somewhere. Not somewhere, but precisely where? If you
are doing a study of documents, where are the documents
you need? At exactly which library and in what collection
will you find them? What society or what organization has
the files you must examine? Where are these organizations
located? Specify geographicallyby town, street address,
and postal code! Suppose a nurse or a nutritionist is doing a
research study about Walter Olin Atwater, whose work has
been instrumental in establishing the science of human
nutrition in the United States. Where are the data on
Atwater located? The researcher can go no further until that
basic question is answered.
3. How will the data be obtained? To know where the data
are located is not enough; you need to know how you might

acquire
them.
With
privacy
laws,
confidentiality
agreements, and so on, obtaining the information you need
might not be as easy as you think. You may indeed know
what data you need and where you can find them, but an
equally important question is, how will you get them?
Careful attention to this question marks the diference
between a viable research project and a pipe dream.
4. How will the data be interpreted? This is perhaps the
most important question of all. The three former hurdles
have been overcome. You have the data in hand. But you
must also spell out precisely what you intend to do with the
data to solve the research problem or one of its sub
problems.

b) To avoid serious trouble later on, the researcher must answer these
questions specifically and concretely. Clear answers can help bring any
research planning and design into focus. Now go back and look
carefully at how you have worded your research problem. Will you be
able to get data that might adequately provide a solution to the
problem? And if so, might they reasonably lend themselves to
interpretations that shed light on the problem? If the answer to either
of these questions is no, you must obviously rethink the nature of your
problem. If instead, both answers are yes, a next important step is to
consider how you might best measure what you observe.

8) Define Measurement exactly as given in your textbook.

A) We think of measurement in terms of such objects as rulers,
scales, gauges, and thermometers. In research, measurement
takes on a somewhat diferent meaning
Measurement is limiting the data of any phenomenon
substantial or insubstantial so that those data may be

interpreted and, ultimately, compared to a particular qualitative

or quantitative standard.

9) a. Define Validity of the measurement instruments as given in

your textbook.
b. Identify the diferent forms of this validity.
A) a) The validity of a measurement instrument is the extent to
which the instrument measures what it is intended to measure.
b) The validity of a measurement instrument can take several
diferent forms, each of which is important in diferent situations:
1) Face validity is the extent to which, on the surface, an
instrument looks like it is measuring a particular
characteristic. Face validity is often useful for ensuring the
cooperation of people who are participating in a research
study. But because it relies entirely on subjective judgment,
it is not, in and of itself, a terribly dependable indicator that
an instrument is truly measuring what the researcher wants
to measure.
2) Content validity is the extent to which a measurement
instrument is a representative sample of the content area
(domain) being measured. Content validity is often a
consideration when a researcher wants to assess peoples
achievement in some areafor instance, the knowledge
students have learned during classroom instruction or the
job skills workers have acquired in a training program. A
measurement instrument has high content validity if its
items or questions reflect the various parts of the content
domain in appropriate proportions and if it requires the
particular behaviors and skills that are central to that
domain.

3) Criterion validity is the extent to which the results of an

assessment instrument correlate with another, presumably
related measure (the latter measure is, in this case, the
criterion). For example, a personality test designed to
assess a persons shyness or outgoingness has criterion
validity if its scores correlate with other measures of a
persons general sociability. An instrument designed to
measure a salespersons efectiveness on the job should
correlate with the number of sales the individual actually
makes during the course of a business week.
4) Construct validity is the extent to which an instrument
measures a characteristic that cannot be directly observed
but is assumed to exist based on patterns in peoples
behavior (such a characteristic is a construct). Motivation,
creativity, racial prejudice, loveall of these are constructs,
in that none of them can be directly observed and
measured. When researchers ask questions, present tasks,
or observe behaviors as a way of assessing an underlying
construct, they should obtain some kind of evidence that
their approach does, in fact, measure the construct in
question.

c. Why there are diferent forms of validity?

A) It is critical to note that the validity of any measurement
instrument can be vary considerably depending on the
purpose for which it is being used. In other words, the
validity of instrument in specific to the situation.
1. Table of specification
2. Multitrait - multimethod approach
3. Judgment by a panel experts

10)
Sean, a high school wrestler, has agreed to participate in a
study of cardiovascular conditioning. He is left somewhat
confused when, at the first research session, he is asked to
complete a questionnaire about commonly purchased grocery
items. Seans confusion indicates a lack of ______ regarding the
task.
A) Face Validity

11)
A researcher decides to use a high school sample to test the
relationship between her new measure of empathy and a wellestablished measure of interpersonal sensitivity. She finds that
the two instruments are highly related, which supports the ______
of the new instrument.
A) Criterion validity

12)
A researcher designed a new questionnaire to measure
political conservatism. To test out his new instrument, he asks
people leaving their polling place on election day to report their
degree of political conservatism on a scale of 1 (low) to 10 (high)
and then complete his questionnaire. Results show that people
who identified themselves as political conservatives also had the
highest scores on his questionnaire. This is an indication of the
______ of the new instrument.
A) Criterion validity

13)
Define Reliability of the Measurement Instruments as given
in your textbook. Identify the diferent forms of this reliability
A) Reliability is the consistency with which a measuring instrument
yields a certain, consistent result when the entity being measured
hasnt changed. As we have just seen in our waist-measuring
situation, instruments that measure physical phenomena arent
necessarily completely reliable.

Like validity, reliability takes diferent forms:

1) Interrater reliability is the extent to which two or more
individuals evaluating the same product or performance
give identical judgments.
2) Test-retest reliability is the extent to which a single
instrument yields the same results for the same people on
two diferent occasions.
3) Equivalent forms reliability is the extent to which two
diferent versions of the same instrument (e.g., Form A
and Form B of a scholastic aptitude test) yield similar
results.
4) Internal consistency reliability is the extent to which all of
the items within a single instrument yield similar results.

14)
Dick and Jane are studying aggression among preschool
children. Separately, they each watch a videotape of four

children interacting in a playroom and then rate each child on the

level of aggression displayed during the play session. They
compare their completed ratings and are pleased to note they are
highly similar. This is an example of ______ reliability.
A) Interpreter reliability

15)
Dr. Drive is testing a 10-question measure of achievement
motivation. He notes that if his respondents agree with the first
question, they tend to also agree with the other nine. Likewise, if
his respondents disagree with the first question, they tend of
disagree with the other nine. This is an indication of good ______
reliability
A) Internal consistency reliability

16)
Professor Pickle is studying civic-mindedness among senior
citizens. She administers a survey of civic mindedness to a
senior citizens group in early April, then again in early May. By
comparing the two sets of scores, the professor can assess the
______ reliability of her measure of civic-mindedness
A) Test-retest reliability

17)
Professor Pickle is studying civic-mindedness among senior
citizens. She administers a survey of civic mindedness to a
senior citizens group and then she asks Professor Jones to
develop and administer another survey to measure the same. By

comparing the two sets of scores, the professor can assess the
______ reliability of her measure of civic-mindedness.

18)
When Dr Moumen divides his class into two groups and
used diferent form of his quizzes to each group. The results of
the quizzes of the two groups are similar. Dr Moumen is assessing
the -------------- reliability of measuring the concepts of research
methods.

a) Test- retest reliability

19)
Identify three of the techniques for strengthening the
internal validity of a study.
A) The three of the techniques for strengthening the internal validity
of a study
1) A controlled laboratory study. An experiment is conducted in a
laboratory setting so that environmental conditions can be
carefully regulated.
2) A double-blind experiment. In a situation where two or more
diferent interventions are being compared, neither the
participants in the study nor the people administering the
interventions (e.g., teachers, research assistants) know which
intervention each participant is receiving. Such lack of knowledge
(blindness) decreases the likelihood that peoples expectations
for outcomes might influence the actual outcomes.
3) Unobtrusive measures. People are observed in such a way that
they do not know
4) their actions are being recorded. We ofer two real-life examples
to illustrate. In one case, a university library measured student
and faculty use of diferent parts of the library by looking at wear-

and-tear patterns on the carpet. In another situation, researchers

for the U.S. National Park Service looked at hikers frequency of
using diferent hiking trails by installing electronic counters in
hard-to-notice locations beside the trails (R. K. Ormrod & Trahan,
1982). (Note that ethical issues sometimes arise when we
observe people without their permission.
5) Triangulation. Multiple sources of data are collected with the hope
that they will all
6) converge to support a particular hypothesis or theory. This
approach is especially common in qualitative research; for
instance, a researcher might engage in many informal
observations in the field and conduct in-depth interviews, then
look for common themes that appear in the data gleaned from
both methods. Triangulation is also common in mixed-methods
designs, in which both quantitative and qualitative data are
collected to answer a single research question.
20)
Identify three of the techniques for strengthening the
external validity of a study.
A) The external validity of a research study is the extent to which
its results apply to situations beyond the study itselfin other words,
the extent to which the conclusions drawn can be generalized.
Following are three commonly used strategies that enhance the
external validity of a research project:
1) A real-life setting. Earlier we mentioned that researchers
sometimes use laboratory experiments to help them control the
environmental conditions in which a study takes place.
Laboratory studies have a downside, however: They provide an
artificial setting that may be quite diferent from real-life
circumstances. Research that is conducted in the outside world,
although it may not have the tight controls of a laboratory
project, may be more valid in the sense that it yields results with
broader applicability to other real-world.
2) A representative sample. Whenever researchers seek to learn
more about a particular category of objects or creatures
whether they are studying rocks, salamanders, or human beings
they often study a sample from that category and then draw
conclusions about the category as a whole. (Here is a classic
example of inductive reasoning.) For example, to study the
properties of granite, researchers might take pieces of granite
from anywhere in the world and assume that their findings based

on those pieces might be generalizable to granite found in other

locations. The same might hold true for salamanders if
researchers limit their conclusions to the particular species of
salamander they have studied. Human beings are another
matter. The human race is incredibly diverse in terms of culture,
childrearing practices, educational opportunities, personality
characteristics, and so on. To the extent that researchers restrict
their research to people with a particular set of characteristics,
they may not be able to generalize their findings to people with a
very diferent set of characteristics. Ideally, then, researchers
want participants in a research study to be a representative
sample of the population about which they wish to draw
conclusions.
3) Replication in a diferent context. Imagine that one researcher
draws a conclusion from a particular study in a specific context,
and another researcher who conducts a similar study in a very
diferent context reaches the same conclusion, and perhaps
additional researchers also conduct similar studies in dissimilar
contexts and, again, draw the same conclusion. Under such
circumstances, these studies, taken together, provide evidence
that the conclusion has validity and applicability across diverse
contexts and situations.
21)
The requirement to keep information provided by research
participants confidential applies _____________________________
A) Yes, the information to keep confidential to all persons under
circumstances.
22)
When is altering of data is allowable?
A) The altering of data is allowable under no circumstances.
23)
Dr. Davenport wants to conduct a study of whether high
school students learn more efficiently while seated at a desk
compared to in an easy chair. Participants will be seniors in local
high school history classes. She hopes to publish her findings in
the History Teacher Quarterly. Does Dr. Davenport need internal
review board (IRB) approval for this study?
A) Yes

24)
Dr. Davenport wants to conduct a study of whether high
school students learn more efficiently while seated at a desk
compared to in an easy chair. She knows there is a large
research literature regarding similar influences on learning, and
she wants to see if the findings hold true with a group of high
school history students. Moreover, as a researcher she is uneasy
with a lack of structure. You recommend that Dr. Davenport
conduct a ______ study.
a. quantitative
or
b. qualitative
A) Quantitative

25)
Dr. Greenhill wants to know how it is that some early
adolescents come to make a connection between their personal
lifestyle and environmental problems, while others dont.
Furthermore, among those who see the connection, why do some
become committed to environmentalism while others do not? Dr.
Greenwald looks forward to probing the thoughts of young teens
on these issues and trying to see the questions and issues
through 14-year-old eyes. You recommend that Dr. Greenhill
conduct a ______ study.
a.
qualitative

quantitative

or

b.

A) Qualitative

26) Professor Harris is constructing a demographic questionnaire for

use in a research project.
a. One question asks students to report whether they are currently
living in an urban, suburban, or rural setting.

b. One question asks students to report their age in years.

c. One question asks students to report their highest level of
education by choosing from these options: some high school,
completed high school, some college, or completed 4-year
college degree.
d. One question asks students to report how politically conservative
they are. It includes a 7-point scale where 1 is not at all
conservative and 7 is extremely conservative.
Identify the type of scale of each question:
A) a. Nominal Scale
b. Ratio Scale
c. Nominal Scale
d. Interval Scale