Psychometrics:

Anintroduction

Psychometrics:Anintro

Overview

Abriefhistoryofpsychometrics
Themaintypesoftests
The10mostcommontests
Whypsychometrics?:Clinicalversus
actuarialjudgment

Psychometrics:Anintro

Abriefhistory
Testingforproficiencydatesbackto2200B.C.,whenthe
Chineseemperorusedgruelingteststoassessfitnessfor
office

Psychometrics:Anintro

FrancisGalton
ModernpsychometricsdatestoSirFrancisGalton(1822
1911),CharlesDarwinscousin
Interestedin(infact,obsessedwith)individual
differencesandtheirdistribution
18841890:Tested17,000individualsonheight,
weight,sizesofaccessiblebodyparts,+behavior:
handstrength,visualacuity,RTetc
Demonstratedthatobjectivetestscouldprovide
meaningfulscores

Psychometrics:Anintro

JamesCattell
JamesCattell(studiedwithWundt&Galton)firstused
thetermmentaltestin1890
Histestswereinthebrassinstruments
traditionofGalton
mostlymotorandacuitytests
FoundedPsychologicalReview(1897)

Psychometrics:Anintro

ClarkWissler
ClarkWissler(Cattellsstudent)didthefirstbasicvalidationalresearch,
examiningtherelationbetweentheoldmentaltestscoresandacademic
achievement

Hisresultswerelargelydiscouraging
Hehadonlybrightcollegestudentsinhis
sample
Whyisthisaproblem?
Wisslerbecameananthropologistwitha
strongenvironmentalistbias.

Psychometrics:Anintro

AlfredBinet
Goodenough(1949):TheGaltonianapproachwaslikeinferringthenatureof
geniusfromthethenatureofstupidityorthequalitiesofwaterfromthose
of.hydrogenandoxygen.

AlfredBinet(1905)introducedthefirst
modernintelligencetest,whichdirectly
testedhigherpsychologicalprocesses(real
abilities&practicaljudgments)
i.e.picturenaming,rhymeproduction,weight
ordering,questionanswering,worddefinition.

AlsomotivatedIQ(Stern,1914):mental

agedividedbychronologicalage
Psychometrics:Anintro

Theriseofpsychometrics
LewisTerman(1916)producedamajorrevisionofBinetsscale
RobertYerkes(1919)convincedtheUSgovernmenttotest1.75millionarmyrecruits
PostWWI:Factoranalysisemerged,makingotheraptitudeandpersonalitytestspossible

Psychometrics:Anintro

Whatisapsychometrictest?
Atestisastandardizedprocedureforsamplingbehavioranddescribingitusingscoresorcategories
Mosttestsarepredictiveofofsomenontestbehaviorofinterest
Mosttestsarenormreferenced=theydescribethebehaviorintermsofnorms,testresultsgatheredfromalargegroupofsubjects(the
standardizationsample)
Sometestsarecriterionreferenced=theobjectiveistoseeifthesubjectcanattainsomeprespecifiedcriterion.

Psychometrics:Anintro

Themaintypesoftests

Intelligencetests:Assessintelligence
Aptitudetests:Assesscapability
Achievementtests:Assessdegreeofaccomplishment
Creativitytests:Assesscapacityfornovelty
Personalitytests:Assesstraits
Interestinventories:Assesspreferencesforactivities
Behavioraltests:Measurebehaviorsandtheir
antecedents/consequences
Neuropsychologicaltests:Measurecognitive,sensory,
perceptual,ormotorfunctions

Psychometrics:Anintro

The10mostcommonlyusedtests
1.)WechslerIntelligenceScaleforChildren(WISC)
2.)BenderVisualMotorGestaltTest
3.)WechslerAdultIntelligenceScale(WAIS)
4.)MinnesotaMultiphasicPersonalityInventory(MMPI)
5.)RorschachInkBlotTest
6.)ThematicApperceptionTest(TAT)
7.)SentenceCompletion
8.)GoodenoughDrawAPersonTest
9.)HouseTreePersonTest
10.)StanfordBinetIntelligenceScale
FromBrown&McGuire,1976

Psychometrics:Anintro

Clinicalversusactuarialjudgment
Clinicaljudgment=reachingadecisionbyprocessing
informationinoneshead
Actuarialjudgment=reachingadecisionwithout
employinghumanjudgment,usingempiricallyestablished
relationsbetweendataandtheeventofinterest
Actuarial:ad.L.actu[{amac}]rius,akeeperof
accounts
Notethatsomeofthedatainanactuarialjudgment
maybequalitativeclinicalobservations,allowinga
mixtureofmethods
Psychometrics:Anintro

Clinicalversusactuarialjudgment
PaulMeehl(1954)firstaddressedthequestion:
Whichisbetter?
Hisgroundrulesforcomparison:
Bothmethodsshoulddrawfromthesame
dataset(thiswasrelaxedbyothers,withno
changesinresults)
Crossvalidationshouldberequired,to
avoidusingvariationspecifictothedataset
Thereshouldbeexplicitpredictionof
success,recidivism,orrecovery

Psychometrics:Anintro

Meehl(1954):Results
Helookedatbetween16and20studies(dependingoninclusioncriteria)
itisclearthatthedogmatic,complacentassertionsometimesheardfromcliniciansthatnaturallyclinicalprediction,beingbasedonreal
understandingissuperior,issimplynotjustifiedbythefactstodate.

Inallbutonecase,predictionsmadebyactuarialmeanswereequaltoorbetterthanclinicalmethods
Inalaterpaper,hechangedhismindabouttheone.

Psychometrics:Anintro

Thirtyyearslater...
Thereisnocontroversyinsocialsciencethat
showssuchalargebodyofqualitatively
diversestudiescomingoutsouniformlyas
thisone.
PaulMeehl,1986

Psychometrics:Anintro

Wherearecliniciansstrengths?I
i.)Theorymediatedjudgments
Ifthepredictorknowstherelevantcausal
influences,canmeasurethem,andhasamodel
specificenoughtotakehim/herfromtheoryto
fact
However,arethereanyreasonstodoubtthis
potentialadvantage?

Psychometrics:Anintro

Wherearecliniciansstrengths?II
ii.)Abilitytouserareevents
Ifthepredictorknowsthatthecurrentcaseisan
exceptiontothestatisticaltrend,s/hecanuse
thatinformationtooverridethetrend
itisintheorypossibletobuildtheseinto
actuarialmethods
Whyisitverydifficultinpractice?
Whymightweworryaboutcliniciansabilityto
incorporaterareeventsintoprediction?

Psychometrics:Anintro

Wherearecliniciansstrengths?III
iii.)Abletodetectcomplexpredictivecures
Humansbeingsstill(fornow)aremastersat
recognizingsomecomplexconfigurations,suchas
facialexpressionsetc.

Psychometrics:Anintro

Wherearecliniciansstrengths?IV
iv.)Abletoreweightutilitiesinrealtime
Forethical,legal,orhumanitarianreasons,
wemightdecidetodothingsdifferently
thanusualinparticularcases.

Psychometrics:Anintro

Whereareactuarialstrengths?I
i.)Immunityfromfatigue,forgetfulness,
hangovers,hostility,prejudice,ignorance,
falseassociation,overconfidence,bias,and
randomfluctuationsinjudgment.

Psychometrics:Anintro

Whereareactuarialstrengths?II
ii.)Consistency&properweighting
variablesareweightedthesamewayevery
time,accordingtotheiractualdemonstrable
contributionstothecriterionofinterest
irrelevantvariablesareproperlyweighted
tozero

Psychometrics:Anintro

Whereareactuarialstrengths?III
iii.)Feedback&baseratesbuiltintothe
system
Cliniciansrarelyknowhowtheyaredoingbecause
theydontgetimmediatefeedbackandbecausethey
haveimperfectmemory
actuarialrecordsconstituteperfectmemoriesof
howthingscameoutinsimilarcasesandcaninclude
alargerandwidersamplethanahumancanever
hopetosee
Psychometrics:Anintro

Whereareactuarialstrengths?IV
iv.)Notoverlysensitivetooptimal
weightings
Evensimplisticactuarialjudgmentsoften
beathumanjudgments

Psychometrics:Anintro