Planned Comparisons

What Does the F-Ratio Tell Us?

• The F-ratio (called an omnibus or overall

F) provides a test of whether or not there a
treatment effects in an experiment
• A significant F-ratio suggests that there
are differences between of means
• However, the F-ratio does not tell us which
means differ
 Assessing Mean Differences
• With only two treatment groups we know
that the two groups differ if the F is
statistically significant
• With three or more means we need to do
further tests to see which pairs of means
contribute to the significant F
• This can be done informally using
confidence intervals, or more precisely
using significance testing procedures
 Statistical Comparison Procedures

• Comparison (or contrast) procedures are

used to test more specific hypotheses
about differences between means
• These comparison procedures fall into two
categories:
– Planned comparisons
– Post hoc (unplanned) comparisons (e.g.,
Tukey’s honestly significant difference,
Scheffe…)
Planned Comparisons (Contrasts)
• An experiment is often designed to provide
information regarding several research
questions
• If particular comparisons are of interest
prior to the data analysis, they may be
analyzed as planned comparisons
• Planned comparisons can be performed
without reference to the omnibus F-ratio,
i.e., You can just directly test the contrast
and it is not necessary to test the overall F.
 Reading Comprehension Study
• Remember the study investigating the
effects of two types of drugs on reading
comprehension
– Group 1: Placebo (control group)
– Group 2: Drug A
– Group 3: Drug B
 Comparison Examples
• One research question may involve a
comparison of the two drugs, Drug A
versus Drug B (a pairwise comparison)
• Another question may involve the
comparisons of the two drug groups to the
control group (a complex comparison)
• There other questions like, Drug A versus
control and Drug B versus control
(pairwise comparisons)
 Simple vs. Complex Comparisons
• Comparison 1 is a • Comparison 2 is a
simple comparison complex comparison

H0: 2 = 3 H0: (2 + 3)/2 = 1

H1: 2  3 H1: (2 + 3)/2  1
 Contrast Coefficients
• The contrasts are formed by applying a set
of weights, called contrast coefficients,
to the means

 = c11 + c22 + c33 + … + caa = cii

• c1, c2, c3, …, ca are the weights, or

contrast coefficients.
•  is the comparison difference among
means.
 Developing Contrasts
• Contrasts are usually developed such that
the weights sum to zero
• For previous examples, the coefficients
are as follows

Simple = (0)1 + (1)2 + (-1)3

Complex = (2)1 + (-1)2 + (-1)3
More on Contrast Coefficients
• A mean is excluded from a comparison
whenever it is assigned a weight of
zero
• There are many coefficient sets that could
be used for a given comparison
• The only restriction is that the sum of the
weights must be zero
• Integer coefficients are generally preferred
 Any contrast/comparison
includes:

1. Multiplication of means by a set of

coefficients.
2. Algebraic summation of the weighted means.
 Steps in Testing Contrasts
1. Calculate comparison difference among
means, Ψ.
2. Calculate SScomparison for the comparison.
3. Calculate MScomparison for the comparison.
4. Calculate F=MScomparison/MSS/A for the
comparison.
5. Compare Fcomparison to Fcritical to test the
significance of the comparison.
 Information needed to test Contrasts
1. Means in each group.
2. Sample Size in each group.
3. Coefficients to capture the comparison.
4. MSS/A from the overall analysis. This will be
the error term for testing all contrasts in
between subjects ANOVA designs.
 Research Scenario
• A researcher is interested in comparing two
drugs and a placebo condition on reading
comprehension in a sample of hyperactive
boys (K&W p. 22). Fourth grade boys are
randomized to one of three groups; (1)
Placebo Control, (2) Drug A, (3) Drug B. One
hour after receiving either drug or placebo,
each boy studies an essay for 10 minutes
and then is given an objective reading
comprehension test. IV is Drug Condition,
and DV is reading comprehension score.
15
 Reading Comprehension Example
Data (K&W, p. 22)
• The group means are 15, 6, and 9
• The grand (overall) mean is 10
Report
Reading
txgroup Mean N Std. Deviation
Control Group 15.00 5 3.87
Drug A
6.00 5 3.53
Drug B 9.00 5 4.18
Total 10.00 15 5.28

16
 SPSS ANOVA Output
• The SPSS ANOVA output is as follows

ANOVA
reading
Sum of
Squares df Mean Square F Sig.
Between Groups 210.000 2 105.000 7.000 .010
Within Groups 180.000 12 15.000
Total 390.000 14

17
 Information needed to test Contrasts
for Reading Example

1. Means in each group; 15, 6, 9.

2. Sample Size in each group; n=5.
3. Coefficients to capture the comparison; to be
shown below.
4. MSS/A from the overall analysis. This will be
the error term for testing all contrasts in
between subjects ANOVA designs; MSS/A=15,
from the overall ANOVA analysis,dfS/A=12.
 Comparison of Control and Drug A

• The group means were 15, 6, and 9

• Calculations for the simple contrast
comparing control and Drug A means are:

ˆ  (c1 )YA1  (c2 )YA2  (c3 )YA3  (1)(15)  (1)(6)  (0)(9)  9

(n)(ˆ ) 2 5(9) 2 405

SS comparison     202.5
c (1)  (1)  (0)
2 2 2 2
2
 Comparison of Control and Drug A

MScomparison=SScomparison/dfcomparison=202.5/1 =
202.5
• F = MScomparison/MSS/A = 202.5/15 = 13.5
• Fcritical = F(1,12) = 4.75
• The Fcomparison > Fcritical so reject HO that
there is no difference between the Control
and Drug A conditions.
• The difference between control (Mean 15)
and Drug A (Mean 6) was statistically
significant.
Complex Comparison Example
 Comparison of Drug A and Drug B
to Control: Complex Comparison
• The group means were 15, 6, and 9
• Calculations for the complex contrast are

ˆ  (c1 )YA1  (c2 )YA2  (c3 )YA3  (2)(15)  (1)(6)  (1)(9)  15

(n)(ˆ ) 2 5(15) 2 1125

SScomparison     187.5
c (2)  (1)  (1)
2 2 2 2
6
Comparison of Drug A and Drug B to
control: Complex Comparison
MScomparison=SScomparison/dfcomparison=187.5/1 =
187.5
• F = MScomparison/MSS/A = 187.5/15 = 12.5
• Fcritical = F(1,12) = 4.75
• The Fcomparison > Fcritical so reject HO that
there is no difference between the Drug A
and Drug B compared to the control
condition.
 Interpreting the Complex
Comparison
• The complex comparison essentially
averaged the means for the two Drug
conditions
• The aggregated Drug groups mean was (9
+ 6) / 2 = 7.5
• The control group mean was 15
• The significant F test tells us that Drug
of any kind produced lower reading
comprehension scores than the control
Advice on Planned Comparisons
• Planned contrasts should be used if they
address the substantive hypotheses of
interest
• As Keppel notes, “the meaningfulness of a
comparison is of critical importance in the
analysis of an experiment and not its
inclusion in a set of comparisons.”
 Planned Comparisons Using SPSS

• When doing planned comparisons using

ONEWAY procedure, SPSS uses a t-
statistic
• The GLM procedure uses an F
• However, for any two-group comparison, it
is always the case that t2 = F
• Note also that the two-tailed p-value on
the SPSS printout is identical to the p-
value associated with the F-ratio