Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Experimental
RESEARCH
• …to provide
answers to
research
questions
• ...to control
variance
(differences)
Subject
Characteristics
(Selection Bias/Differential Selection) -- The
groups may have been different from the start. If
you were testing instructional strategies to
improve reading and one group enjoyed reading
more than the other group, they may improve
Loss of Subjects
(Mortality) -- All of the high or low scoring
subject may have dropped out or were
missing from one of the groups. If we
collected posttest data on a day when the
honor society was on field trip at the
treatment school, the mean for the
treatment group would probably be much
lower than it really should have been.
Location
Perhaps one group was at a
disadvantage because of their
location. The city may have
been demolishing a building
next to one of the schools in
our study and there are
constant distractions which
interfere with our treatment.
Instrumentation
Instrument Decay
Data Collector
Characteristics
Data
Collector
Bias
Testing
The act of taking a pretest or posttest may influence the
results of the experiment. Suppose we were conducting
a unit to increase student sensitivity to prejudice. As a
pretest we have the control and treatment groups watch
Shindler's List and write a reaction essay. The pretest
may have actually increased both groups' sensitivity
and we find that our treatment groups didn't score any
higher on a posttest given later than the control group
did. If we hadn't given the pretest, we might have seen
differences in the groups at the end of the study.
History
Something may happen at one site during our study that
influences the results. Perhaps a classmate dies in a car
accident at the control site for a study teaching children bike
safety. The control group may actually demonstrate more
Maturation
Hawthorne Effect
The subjects may respond differently just because they are
being studied. The name comes from a classic study in
which researchers were studying the effect of lighting on
worker productivity. As the intensity of the factory lights
increased, so did the worker productivity. One researcher
suggested that they reverse the treatment and lower the
lights. The productivity of the workers continued to increase.
It appears that being observed by the researchers was
increasing productivity, not the intensity of the lights.
John
Henry
Effect
Resentful
Demoralization of
the Control Group
Regression
(Statistical Regression) -- A class that scores
particularly low can be expected to score slightly
higher just by chance. Likewise, a class that
scores particularly high, will have a tendency to
score slightly lower by chance. The change in
these scores may have nothing to do with the
treatment.
Implementation
Experimental Treatment
Diffusion
Sometimes the control group
actually implements the
treatment. If two different
techniques are being tested in
two different third grades in the
same building, the teachers
may share what they are
doing. Unconsciously, the
control may use of the
Multiple-treatment
interference
(catalyst effect)
If a researcher were to apply
several treatments, it is difficult to
determine how well each of the
treatments would work individually.
It might be that only the combination
of the treatments is effective.
Hawthorne effect
(attention causes differences)
Subjects perform differently because they
know they are being studied. "...External
validity of the experiment is jeopardized
because the findings might not generalize
to a situation in which researchers or
others who were involved in the research
are not present" (Gall, Borg, & Gall, 1996,
p. 475)
Novelty and
disruption effect
(anything different makes a difference)
A treatment may work because it is novel and the subjects
respond to the uniqueness, rather than the actual
treatment. The opposite may also occur, the treatment
may not work because it is unique, but given time for the
subjects to adjust to it, it might have worked.
Experimenter effect
(it only works with this experimenter)
The treatment might have worked
because of the person implementing it.
Given a different person, the treatment
might not work at all.
Pretest sensitization
(pretest sets the stage)
A treatment might only work if a
pretest is given. Because they
have taken a pretest, the subjects
may be more sensitive to the
treatment. Had they not taken a
pretest, the treatment would not
have worked.
Posttest sensitization
(posttest helps treatment "fall into place")
The posttest can become a learning
experience. "For example, the posttest might
cause certain ideas presented during the
treatment to 'fall into place' " (p. 477). If the
subjects had not taken a posttest, the
treatment would not have worked.
Interaction of
history and
treatment effect
(...to everything there is a time...)
Not only should researchers be cautious about
generalizing to other population, caution should be
taken to generalize to a different time period. As
time passes, the conditions under which treatments
Measurement of
the dependent
variable
(maybe only works with M/C tests)
A treatment may only be evident with
certain types of measurements. A teaching
method may produce superior results
when its effectiveness is tested with an
essay test, but show no differences when
Interaction of time
of measurement
and treatment
effect
(it takes a while for the treatment to kick in)
It may be that the treatment effect does not occur until several
weeks after the end of the treatment. In this situation, a
posttest at the end of the treatment would show no impact,
XO
This simple design is known as a one-shot case
study design.
OXO
This is known as a one-group pretest-posttest design.
OXO
O O
OXO
O O
ROXO
RO O
ROXO
RO O
ROXO
RO O
R XO
R O
RO O
treatment and one of
the non-pretest groups
R XO
is assigned to
treatment.
R O