CochranMantelHaenszeltest

forrepeatedtestsof
independence
Summary
UsetheCochranMantelHaenszeltestwhenyouhavedatafrom
22tablesthatyou'verepeatedatdifferenttimesorlocations.Itwilltell
youwhetheryouhaveaconsistentdifferenceinproportionsacrossthe
repeats.

Whentouseit
UsetheCochranMantelHaenszeltest(whichissometimescalled
theMantelHaenszeltest)forrepeatedtestsofindependence.Themost
commonsituationisthatyouhavemultiple22tablesofindependence
you'reanalyzingthekindofexperimentthatyou'danalyzewithatestof
independence,andyou'vedonetheexperimentmultipletimesorat
multiplelocations.Therearethreenominalvariables:thetwovariables
ofthe22testofindependence,andthethirdnominalvariablethat
identifiestherepeats(suchasdifferenttimes,differentlocations,or
differentstudies).ThereareversionsoftheCochranMantelHaenszel
testforanynumberofrowsandcolumnsintheindividualtestsof
independence,butthey'rerarelyusedandIwon'tcoverthem.
Forexample,let'ssayyou'vefoundseveralhundredpinkknit
polyesterlegwarmersthathavebeenhiddeninawarehousesincethey
wentoutofstylein1984.Youdecidetoseewhethertheyreducethe
painofankleosteoarthritisbykeepingtheankleswarm.Inthewinter,
yourecruit36volunteerswithanklearthritis,randomlyassign20towear
thelegwarmersundertheirclothesatalltimeswhiletheother16don't
wearthelegwarmers,thenafteramonthyouaskthemwhethertheir
anklesarepainfreeornot.Withjusttheonesetofpeople,you'dhave
twonominalvariables(legwarmersvs.control,painfreevs.pain),each
withtwovalues,soyou'danalyzethedatawithFisher'sexacttest.
However,let'ssayyourepeattheexperimentinthespring,with50
newvolunteers.Theninthesummeryourepeattheexperimentagain,
with28newvolunteers.Youcouldjustaddallthedatatogetheranddo
Fisher'sexacttestonthe114totalpeople,butitwouldbebettertokeep
eachofthethreeexperimentsseparate.Maybelegwarmersworkinthe
winterbutnotinthesummer,ormaybeyourfirstsetofvolunteershad
worsearthritisthanyoursecondandthirdsets.Inaddition,pooling
differentstudiestogethercanshowa"significant"differencein
proportionswhenthereisn'tone,orevenshowtheoppositeofatrue
difference.ThisisknownasSimpson'sparadox.Forthesereasons,it's
bettertoanalyzerepeatedtestsofindependenceusingtheCochran
MantelHaenszeltest.

Nullhypothesis
Thenullhypothesisisthattherelativeproportionsofonevariableare
independentoftheothervariablewithintherepeatsinotherwords,
thereisnoconsistentdifferenceinproportionsinthe22tables.Forour

imaginarylegwarmersexperiment,thenullhypothesiswouldbethatthe
proportionofpeoplefeelingpainwasthesameforlegwarmerwearers
andnonlegwarmerwearers,aftercontrollingforthetimeofyear.The
alternativehypothesisisthattheproportionofpeoplefeelingpainwas
differentforlegwarmerandnonlegwarmerwearers.
Technically,thenullhypothesisoftheCochranMantelHaenszeltestis
thattheoddsratioswithineachrepetitionareequalto1.Theoddsratioisequal
to1whentheproportionsarethesame,andtheoddsratioisdifferentfrom1
whentheproportionsaredifferentfromeachother.Ithinkproportionsareeasier
tounderstandthanoddsratios,soI'llputeverythingintermsofproportions.But
ifyou'reinafieldsuchasepidemiologywherethiskindofanalysisiscommon,
you'reprobablygoingtohavetothinkintermsofoddsratios.

Howthetestworks
Ifyoulabelthefournumbersina22testofindependencelikethis:
ab
cd

and(a+b+c+d)=n,youcanwritetheequationfortheCochranMantel
Haenszelteststatisticlikethis:

2MH={|[a(a+b)(a+c)/n]|0.5}2

(a+b)(a+c)(b+d)(c+d)/(n3n2)

Thenumeratorcontainstheabsolutevalueofthedifferencebetween
theobservedvalueinonecell(a)andtheexpectedvalueunderthenull
hypothesis,(a+b)(a+c)/n,sothenumeratoristhesquaredsumof
deviationsbetweentheobservedandexpectedvalues.Itdoesn'tmatter
howyouarrangethe22tables,anyofthefourvaluescanbeusedasa.
Yousubtractthe0.5asacontinuitycorrection.Thedenominatorcontains
anestimateofthevarianceofthesquareddifferences.
Theteststatistic, 2MH,getsbiggerasthedifferencesbetweenthe
observedandexpectedvaluesgetlarger,orasthevariancegetssmaller
(primarilyduetothesamplesizegettingbigger).Itischisquare
distributedwithonedegreeoffreedom.
DifferentsourcespresenttheformulafortheCochranMantel
Haenszeltestindifferentforms,buttheyareallalgebraically
equivalent.TheformulaI'veshownhereincludesthecontinuity
correction(subtracting0.5inthenumerator),whichshouldmaketheP
valuemoreaccurate.SomeprogramsdotheCochranMantelHaenszel
testwithoutthecontinuitycorrection,sobesuretospecifywhetheryou
useditwhenreportingyourresults.

Assumptions
Inadditiontotestingthenullhypothesis,theCochranMantel
Haenszeltestalsoproducesanestimateofthecommonoddsratio,a
wayofsummarizinghowbigtheeffectiswhenpooledacrossthe
differentrepeatsoftheexperiment.Thisrequireassumingthattheodds
ratioisthesameinthedifferentrepeats.Youcantestthisassumption
usingtheBreslowDaytest,whichI'mnotgoingtoexplainindetailits
nullhypothesisisthattheoddsratiosareequalacrossthedifferent
repeats.

Ifsomerepeatshaveabigdifferenceinproportioninonedirection,
andotherrepeatshaveabigdifferenceinproportionsbutintheopposite
direction,theCochranMantelHaenszeltestmaygiveanonsignificant
result.SowhenyougetanonsignificantCochranMantelHaenszeltest,
youshouldperformatestofindependenceoneach22tableseparately
andinspecttheindividualPvaluesandthedirectionofdifferencetosee
whethersomethinglikethisisgoingon.Inourlegwarmerexample,if
theproportionofpeoplewithanklepainwasmuchsmallerfor
legwarmerwearersinthewinter,butmuchhigherinthesummer,and
theCochranMantelHaenszeltestgaveanonsignificantresult,itwould
beerroneoustoconcludethatlegwarmershadnoeffect.Instead,you
couldconcludethatlegwarmershadaneffect,itjustwasdifferentinthe
differentseasons.

Examples
Whenyoulookatthebackofsomeone'shead,thehaireitherwhorls
clockwiseorcounterclockwise.LauterbachandKnight(1927)compared
theproportionofclockwisewhorlsinrighthandedandlefthanded
children.Withjustthisonesetofpeople,you'dhavetwonominal
variables(righthandedvs.lefthanded,clockwisevs.
counterclockwise),eachwithtwovalues,soyou'danalyzethedatawith
Fisher'sexacttest.
However,severalothergroupshavedonesimilarstudiesofhair
whorlandhandedness(McDonald2011):

Studygroup
whitechildren

Britishadults

Pennsylvaniawhites

Welshmen

Germansoldiers

Germanchildren

NewYork

Americanmen

Handedness
Clockwise
Counterclockwise
percentCCW
Clockwise
Counterclockwise
percentCCW
Clockwise
Counterclockwise
percentCCW
Clockwise
Counterclockwise
percentCCW
Clockwise
Counterclockwise
percentCCW
Clockwise
Counterclockwise
percentCCW
Clockwise
Counterclockwise
percentCCW
Clockwise
Counterclockwise
percentCCW

Right
708
169
19.3%
136
73
34.9%
106
17
13.8%
109
16
12.8%
801
180
18.3%
159
18
10.2%
151
28
15.6%
950
218
18.7%

Left
50
13
20.6%
24
14
38.0%
32
4
11.1%
22
26
54.2%
102
25
19.7%
27
13
32.5%
51
15
22.7%
173
33
16.0%

Youcouldjustaddallthedatatogetheranddoatestofindependenceon
the4463totalpeople,butitwouldbebettertokeepeachofthe8
experimentsseparate.Someofthestudiesweredoneonchildren,while
otherswereonadultssomewerejustmen,whileothersweremaleand
femaleandthestudiesweredoneonpeopleofdifferentethnic
backgrounds.Poolingallthesestudiestogethermightobscureimportant
differencesbetweenthem.
AnalyzingthedatausingtheCochranMantelHaenszeltest,the
resultis 2MH=6.07,1d.f.,P=0.014.Overall,lefthandedpeoplehavea
significantlyhigherproportionofcounterclockwisewhorlsthanright
handedpeople.
McDonaldandSiebenaller(1989)surveyedallelefrequenciesatthe
LaplocusinthemusselMytilustrossulusontheOregoncoast.Atfour
estuaries,wecollectedmusselsfrominsidetheestuaryandfroma
marinehabitatoutsidetheestuary.Therewerethreecommonallelesand
acoupleofrareallelesbasedonpreviousresults,thebiologically
interestingquestionwaswhethertheLap94allelewaslesscommon
insideestuaries,sowepooledalltheotherallelesintoa"non94"class.
Therearethreenominalvariables:allele(94ornon94),habitat
(marineorestuarine),andarea(Tillamook,Yaquina,Alsea,or
Umpqua).Thenullhypothesisisthatateacharea,thereisnodifference
intheproportionofLap94allelesbetweenthemarineandestuarine
habitats.
Thistableshowsthenumberof94andnon94allelesateach
location.Thereisasmallerproportionof94allelesintheestuarine
locationofeachestuarywhencomparedwiththemarinelocationwe
wantedtoknowwhetherthisdifferenceissignificant.

Location
Tillamook

Yaquina

Alsea

Umpqua

Allele
94
non94
percent94
94
non94
percent94
94
non94
percent94
94
non94
percent94

Marine
56
40
58.3%
61
57
51.7%
73
71
50.7%
71
55
56.3%

Estuarine
69
77
47.3%
257
301
46.1%
65
79
45.1%
48
48
50.0%

Theresultis 2MH=5.05,1d.f.,P=0.025.Wecanrejectthenull
hypothesisthattheproportionofLap94allelesisthesameinthemarine
andestuarinelocations.
Duggaletal.(2010)didametaanalysisofplacebocontrolled
studiesofniacinandheartdisease.Theyfound5studiesthatmettheir
criteriaandlookedforcoronaryarteryrevascularizationinpatientsgiven
eitherniacinorplacebo:

Study

FATS

AFREGS

ARBITER2

HATS

CLAS1

Niacin
Placebo
Niacin
Placebo
Niacin
Placebo
Niacin
Placebo
Niacin
Placebo

Revascularization
2
11
4
12
1
4
1
6
2
1

No

revasc.
46
41
67
60
86
76
37
32
92
93

Percent
revasc.
4.2%
21.2%
5.6%
16.7%
1.1%
5.0%
2.6%
15.8%
2.1%
1.1%

Therearethreenominalvariables:niacinvs.placebo,
revascularizationvs.norevascularization,andthenameofthestudy.
Thenullhypothesisisthattherateofrevascularizationisthesamein
patientsgivenniacinorplacebo.Thedifferentstudieshavedifferent
overallratesofrevascularization,probablybecausetheyuseddifferent
patientpopulationsandlookedforrevascularizationafterdifferent
lengthsoftime,soitwouldbeunwisetojustaddupthenumbersanddo
asingle22test.TheresultoftheCochranMantelHaenszeltestis
2MH=12.75,1d.f.,P=0.00036.Significantlyfewerpatientsonniacin
developedcoronaryarteryrevascularization.

Graphingtheresults
TographtheresultsofaCochranMantelHaenszeltest,pickoneof
thetwovaluesofthenominalvariablethatyou'reobservingandplotits
proportionsonabargraph,usingbarsoftwodifferentpatterns.

Lap94alleleproportions(with95%
confidenceintervals)inthemusselMytilus
trossulusatfourbaysinOregon.Graybars
aremarinesamplesandemptybarsare
estuarinesamples.

Similartests
SometimestheCochranMantelHaenszeltestisjustcalledthe
MantelHaenszeltest.Thisisconfusing,asthereisalsoatestfor
homogeneityofoddsratioscalledtheMantelHaenszeltest,anda
MantelHaenszeltestofindependenceforone22table.Manteland
Haenszel(1959)cameupwithafairlyminormodificationofthebasic

ideaofCochran(1954),soitseemsappropriate(andsomewhatless
confusing)togiveCochrancreditinthenameofthistest.
Ifyouhaveatleastsix22tables,andyou'reonlyinterestedinthe
directionofthedifferencesinproportions,notthesizeofthe
differences,youcoulddoasigntest.
TheCochranMantelHaenszeltestfornominalvariablesis
analogoustoatwowayanovaorpairedttestforameasurement
variable,oraWilcoxonsignedranktestforrankdata.Inthearthritis
legwarmersexample,ifyoumeasuredanklepainona10pointscale(a
measurementvariable)insteadofcategorizingitaspain/nopain,you'd
analyzethedatawithatwowayanova.

Howtodothetest
Spreadsheet
I'vewrittenaspreadsheettoperformtheCochranMantelHaenszel
test.Ithandlesupto5022tables.Itgivesyouthechoiceofusingornot
usingthecontinuitycorrectiontheresultsareprobablyalittlemore
accuratewiththecontinuitycorrection.ItdoesnotdotheBreslowDay
test.

Webpages
I'mnotawareofanywebpagesthatwillperformtheCochran
MantelHaenszeltest.

R
SalvatoreMangiafico'sRCompanionhasasampleRprogramforthe
CochranMantelHaenszeltest,andalsoshowshowtodotheBreslow
Daytest.

SAS
HereisaSASprogramthatusesPROCFREQforaCochran
MantelHaenszeltest.Itusesthemusseldatafromabove.Inthe
TABLESstatement,thevariablethatlabelstherepeatsmustbelisted
firstinthiscaseitis"location".
DATAlap;
INPUTlocation$habitat$allele$count;
DATALINES;
Tillamookmarine9456
Tillamookestuarine9469
Tillamookmarinenon9440
Tillamookestuarinenon9477
Yaquinamarine9461
Yaquinaestuarine94257
Yaquinamarinenon9457
Yaquinaestuarinenon94301
Alseamarine9473
Alseaestuarine9465
Alseamarinenon9471
Alseaestuarinenon9479
Umpquamarine9471
Umpquaestuarine9448
Umpquamarinenon9455
Umpquaestuarinenon9448
;
PROCFREQDATA=lap;
WEIGHTcount/ZEROS;
TABLESlocation*habitat*allele/CMH;

RUN;

Thereisalotofoutput,buttheimportantpartlookslikethis:
CochranMantelHaenszelStatistics(BasedonTableScores)

StatisticAlternativeHypothesisDFValueProb

1NonzeroCorrelation15.32090.0211
2RowMeanScoresDiffer15.32090.0211
3GeneralAssociation15.32090.0211

Forrepeated2x2tables,thethreestatisticsareidenticaltheyarethe
CochranMantelHaenszelchisquarestatistic,withoutthecontinuity
correction.Forrepeatedtableswithmorethantworowsorcolumns,the
"generalassociation"statisticisusedwhenthevaluesofthedifferent
nominalvariablesdonothaveanorder(youcannotarrangethemfrom
smallesttolargest)youshoulduseitunlessyouhaveagoodreasonto
useoneoftheotherstatistics.
TheresultsalsoincludetheBreslowDaytestofhomogeneityof
oddsratios:
BreslowDayTestfor
HomogeneityoftheOddsRatios

ChiSquare0.5295
DF3
Pr>ChiSq0.9124

TheBreslowDaytestfortheexampledatashowsnosignificant
evidenceforheterogeneityofoddsratios( 2=0.53,3d.f.,P=0.91).

References
Cochran,W.G.1954.Somemethodsforstrengtheningthecommon 2
tests.Biometrics10:417451.
Duggal,J.K.,M.Singh,N.Attri,P.P.Singh,N.Ahmed,S.Pahwa,J.
Molnar,S.Singh,S.KhoslaandR.Arora.2010.Effectofniacin
therapyoncardiovascularoutcomesinpatientswithcoronaryartery
disease.JournalofCardiovascularPharmacologyandTherapeutics
15:158166.
Lauterbach,C.E.,andJ.B.Knight.1927.Variationinwhorlofthehead
hair.JournalofHeredity18:107115.
Mantel,N.,andW.Haenszel.1959.Statisticalaspectsoftheanalysisof
datafromretrospectivestudiesofdisease.JournaloftheNational
CancerInstitute22:719748.
McDonald,J.H.2011.Mythsofhumangenetics.SparkyHousePress,
Baltimore.
McDonald,J.H.andJ.F.Siebenaller.1989.Similargeographicvariation
attheLaplocusinthemusselsMytilustrossulusandM.edulis.
Evolution43:228231.
ThispagewaslastrevisedJuly20,2015.Itsaddressis
http://www.biostathandbook.com/cmh.html.Itmaybecitedas:
McDonald,J.H.2014.HandbookofBiologicalStatistics(3rded.).Sparky
HousePublishing,Baltimore,Maryland.Thiswebpagecontainsthecontentof

pages94100intheprintedversion.
2014byJohnH.McDonald.Youcanprobablydowhatyouwantwiththis
contentseethepermissionspage
(http://www.biostathandbook.com/permissions.html)fordetails.