CHAPTER 4

ISSUES IN SINGLE
SUBJECT RESEARCH
Pelaez, EDP 7058

THE CONCEPTS OF
PREDICTION, VERIFICATION,
AND REPLICATION

The concepts of prediction, verification, and

replication relate to the issues of reliability and
validity as specifically applied to single subject
research.
When these concepts can be demonstrated
within a single subject research design, the
functional relationship between the independent
and dependent variables is evident.
The reliability and validity of the study are
verified assuming the extraneous variables and
systematic bias cannot be reasonably perceived
to account for the changes that have occurred.
2

Prediction
Prediction refers to the idea that if there is no effect
attributable to the independent variable, the
dependent variables data path will remain
unchanged; and one could predict that it will remain
unchanged.
EX: A researcher has collected baseline data on an
individuals target behavior. After stability has been
achieved during the baseline phase, the intervention
phase would be introduced.
If the intervention had no effect on the dependent
variable, one could logically assume the data path from
baseline to intervention phases would depict no
appreciable change. Therefore, one could predict that
the data path will remain unchanged despite a phase
change.
Should the data path change and that change be
maintained one could reject the prior notion and possibly
verify that the intervention has an effect on the
dependent variable or target behavior.
3

Verification
Verification is the confirmation that the dependent
variable is changing in a predictable fashion as the
independent variable is systematically applied.
EX: A baseline phase is completed and an
intervention is introduced. If the independent
variable will have no effect, we could predict there
would be no change in the data path with that
phase change.
We may predict conversely that the data are likely
to reveal a change in the desired direction as a
result of the implementation of the independent
variable.
When this happens, we have verified our hypothesis
that the dependent variable will change predictably
with the introduction of the independent variable.
Replication is needed also to complete the
demonstration of a functional relationship.
4

Replication
Replication refers to the repeating of the observed
predictions and verifications within the same study.
EX: A practitioner gathers baseline data on math
accuracy and implements a reinforcement program,
and the desired changes are observed. One could
state that both a prediction and a verification are
present, but replication is not present.
For replication to occur, the researcher might return
to a baseline condition and measure the outcome.
Should the target behavior change during this phase
prediction has been verified.
The researcher may then reintroduce the
intervention, which allows for replication of the
original prediction and verification. This makes a
stronger case for the demonstration of a functional
relationship (ABAB design).
5

REPLICATION CONT.
The more replications included within a study, the
less changes in the dependent variable are
attributable to extraneous or confounding
variables.
Also, the case for the power (robustness) of an
independent variable to influence a dependent
variable is strengthened when other researchers
seek to replicate effects with other individuals,
with similar behaviors, in different settings, and
so on.

RELIABILITY AND VALIDITY

There are three major areas of concern relating to
reliability and validity in single subject research.
Interobserver reliability (sometimes referred
to as interrater reliability or interobserver
agreement) is important to establish confidence in
the measurement of the dependent variable.
Internal validity is important to establish the
believability of the functional relationship and,
therefore, establish confidence in the results and
conclusions drawn.
External validity is related to the confidence
others may have that the same independent
variable will yield similar results in similar studies.

Interobserver Reliability
The researcher must identify and define the target
behavior in such a way that there is confidence that
at least two people can observe the individual and
agree whether or not the behavior has occurred, or
to what extent, or for how long, and so on.
The researcher identifies at least two people to serve
as observers (one can be the researcher). Then the
observers are trained and practice observing the
individual and scoring the dependent variable.
Following practice sessions, the observers should
compare results for each and every occurrence or
measurement of the target behavior (dependent
variable).

Interobserver Reliability cont.

They should reach an understanding of both why and
why not the target behavior has occurred. When
disagreements occur, it is particularly important to
understand and reach agreement as to what the correct
measurement was and why.
Interobserver reliability should be calculated among and
between observers. The coefficients should be
calculated in a rigorous manner to ensure adequate
reliability.
Some experts (e.g., Barlow & Hensen , 1984) argue that
observers should be nave to the intervention and
purpose of the study, should not be involved in
calculating interobserver agreement, and should remain
unaware of when reliability checks are scheduled.
However, since the researcher is often involved in data
collection such precautions are not always possible.
9

Reactivity
Reactivity refers to the individual being observed
altering his or her behavior (i.e. target behavior) as
a response to being observed.
Reactivity generally diminishes as the number of
observations increases.
Practice observations in the experimental setting
can reduce the possibility of reactivity when the
official data collection begins, although the
presence of observers regularly before the
introduction of the independent variable may have
an unknown and confounding result.
Reactivity may also be overcome by extending
baseline measurements until there is stability, but
also a reasonable assumption that the performance
on the dependent variable truly represents previous
levels unaffected by observer presence.
10

Observer Drift
Observer drift refers to a change in interpretation
of when the target behavior occurs or not (or at
what level, or how intensely, or for how long,
etc.) from the original operational definitions.
Observer drift occurs generally when a number of
observations have been made, particularly if the
communication and comparison of scores among
raters has not been regularly reviewed and
discussed.
EX: Two observers are recording sitting in the seat
behavior, and one observer begins to score
occurrences even if the individual has begun to
slump over and props herself with her hand. The
other observer, simultaneously scoring the
individuals performance, may disagree on what the
actual performance was and therefore threaten
interobserver reliability.
11

VALIDITY
There are several types of validity associated with
measurement in particular (e.g., face validity,
content validity, and predictive validity). These
are used less frequently in single subject research
then in group studies, but understanding them is
important.
Internal and external validity are concepts that
apply to both group and single subject research.

12

Internal Validity
Internal validity refers to the degree to which the
researcher has adequately controlled the independent,
dependent, and extraneous variables so that changes in
the dependent variable are directly attributable to the
presence or absence or intensity of the independent
variable.
EX 1: A researcher is attempting to help an individual
improve his articulation. The researcher implements
a treatment that is intended to accomplish that
objective. After several weeks, the researchers
measurements verify that the individuals articulation
improves, and there is no reason to suspect that that
any other factor or variable accounted for the
improved articulation . The researcher has
convincingly demonstrated the functional relationship
between the independent variable and the dependent
variable.
13

Internal Validity cont.

EX 2: As an example of diminished internal
validity, the same change in the target
behavior occurs. However, the individual
reports that he has been reading materials that
tell how other people overcame their
articulation difficulties. Also, he and his
mother have been praying for divine
assistance. The researcher must now
acknowledge that the treatment alone may not
account for the changes in the target behavior.

14

Extraneous Variables
Extraneous variables refer to virtually anything
that may affect the demonstration of the
functional relationship between the independent
and dependent variables.
History is used to refer to the passage of time
and both foreseen and unforeseen events that
arise.
Maturation refers to the natural development
of an individual that occurs over time.
Attrition is the loss of subjects during the
course of the study.
15

Extraneous Variables cont.

Multiple treatment interference is a very real

threat in many single subject studies.
Interference may occur when more than one
independent variable (e.g., positive reinforcement
followed by negative reinforcement) is used.
EX: A package treatment is used (e.g., positive
reinforcement plus verbal prompting plus
response interruption). Because more than
one intervention is being used, it may be
difficult if not impossible for the researcher to
accurately determine which of the components
of the independent variable or variables
actually accounted for changes in the
dependent variable.
16

Extraneous Variables cont.

Treatment drift refers to individuals involved in
administering the independent variable producing
personal modifications (consciously or
unconsciously) that may influence the impact of the
independent variable on the dependent variable.
Systematic bias refers to the researcher
introducing some element to the experimental
conditions that is likely to influence changes in the
dependent variable regardless of the manipulation
of the independent variable.

17

External Validity
External validity refers to the degree to which the
researcher may have confidence that she or he or
other researchers will obtain the same or similar
results if they use the same or very similar
experimental procedures with other individuals,
with other target behaviors, or in other settings.
The more an experimental effect is replicated by
the same or other researchers, the greater the
external validity.

18

Types of Replication
Direct replication: The researcher attempts to
duplicate the procedures as precisely as possible.
Intrasubject replication: The same subject is
used in a subsequent study.
Intersubject replication: The researcher
maintains every aspect of an earlier study, but
with different although similar subjects.
Systematic replication: The conditions from a
previous study are varied, but still obtaining
similar results.

19

Educational Significance
Educational significance refers to the concern
that, although statistically significant results may
be achieved the results should merit conclusions
that the interventions used also translated into
real world significance.
The question is asked, Did the interventions
result in outcomes that are meaningful to
practitioners and to the lives of the participants in
the study?

20

ETHICS
There are many considerations in the use of applied
behavior analysis and single subject research;
empirical and social validity are equally important.
Social validity refers to the degree to which other
people think that the targeted changes in behavior
are important and that the methods used to
encourage behavior change are acceptable.
Empirical validity refers to the measurements that
actually demonstrate that the proposed behavioral
changes will indeed positively affect the individuals
life.

21