SusanElrod,ProjectKaleidoscopeatAAC&U

DavidBrakke,JamesMadisonUniversity
JudyDilts,JamesMadisonUniversity

CCASAnnualMeeAng
November3,2011
Montreal,Canada
DriversofChangein
UndergraduateBiologyandSTEM
Educa;on
!"#$%"&'()(*+",(-"./$"01
2.
"3$4.5-+6"
74.$-8'29':)'4;-'.+"
h<p://www.nap.edu/catalog.php?record_id=12764
VisionandChangeinBiologyEducaAon
RecommendaAonsfor
change
Learningoutcomes
Research-basedteaching/
learning
AuthenAcassessment
Note:RevisionstoAP
Biology(andother
sciences)
FromtheReport:RecommendaAons
1. IntegrateCoreConceptsandCompetencies
throughouttheCurriculum
E.g.,Denelearninggoalssothattheyfocusonteachingstudents
thecoreconcepts,andalignassessmentssothattheyassessthe
studentsunderstandingoftheseconcepts.
2. FocusonStudent-CenteredLearning
E.g.,ViewtheassessmentofcoursesuccessassimilartoscienAc
research,centeredonthestudentsinvolved,andapplythe
assessmentdatatoimproveandenhancethelearningenvironment.
3. PromoteaCampus-wideCommitmenttoChange
E.g.,Advocateforincreasedstatus,recogniAon,andrewardsfor
innovaAoninteaching,studentsuccess,andothereducaAonal
outcomes.
HealthProfessionsPreparaAon
ScienAccompetenciesin
chemistry,physics,biology,
psychology,mathforpre-
medstudents
ImplicaAonsfor
forthcomingMCATrevision
ImplicaAonsforlower
divisionundergraduate
pre-medcurriculum
FromtheReport:Competencies
E1:Applyquan%ta%vereasoningandappropriate
mathema%cstodescribeorexplainphenomenain
thenaturalworld.
E2:Demonstrateunderstandingoftheprocessof
scien%cinquiry,andexplainhowscien?c
knowledgeisdiscoveredandvalidated.
E3:Demonstrateknowledgeofbasicphysical
principlesandtheirapplica%onstothe
understandingoflivingsystems.
E4:Demonstrateknowledgeofbasicprinciplesof
chemistryandsomeoftheirapplica%onstothe
understandingoflivingsystems.
FromtheReport:Competencies
E5:Demonstrateknowledgeofhowbiomolecules
contributetothestructureandfunc%onofcells.
E6:Applyunderstandingofprinciplesofhow
molecularandcellassemblies,organs,and
organismsdevelopstructureandcarryout
func?on.
E7:Explainhoworganismssenseandcontroltheir
internalenvironmentandhowtheyrespondto
externalchange.
E8:Demonstrateanunderstandingofhowthe
organizingprincipleofevolu%onbynatural
selec%onexplainsthediversityoflifeonearth.
MCATRevision
Fourtestsec;onsandreportfourscores:

1. Molecular,Cellular,andOrganismalProperAesof
LivingSystems
2. Physical,Chemical,andBiochemicalProperAesof
LivingSystems
3. BehavioralandSocialSciencesPrinciples
4. CriAcalAnalysisandReasoningSkills
h_ps://www.aamc.org/iniAaAves/mr5/
Testexaminees
knowledgeanduseoftheconceptsinbiology,
chemistry,physics,biochemistry,cellular/molecular
biology,researchmethods,andsta%s%cs.
knowledgeanduseoftheconceptsinbehavioraland
socialsciences,researchmethods,andsta%s%cs.
abilitytoanalyzeandreasonthroughpassagesin
ethicsandphilosophy,cross-culturalstudies,
popula%onhealth,andawiderangeofsocial
sciencesandhumani%esdisciplines.
MCATRevision
h_ps://www.aamc.org/iniAaAves/mr5/
hup://www.herl.ucla.edu/nlh/PL8l_8esearch8rlef_CL_2010_S1LM.pdf
!"#$%&'()*+,-./'0%1%,&+$'2/1-3*3%'4!(025''67(8'63*)9
TheSTEMPipelineIssue:OneView
TheSTEMIssue:AnotherView
STEMjobswillconAnuetobe
indemand,secondonlyto
healtcare;but,STEMmajors
arebeingdivertedtonon-
STEMjobs
DemandforSTEMtalentwill
growinnon-STEMprofessions;
thosewithSTEMcompetencies
haveincreasedearningpower
WomenandminoriAesaresAll
underrepresentedinSTEMjobs
h_p://cew.georgetown.edu/STEM/
TheBiggerPicture:Pre_yBleak
Mathprociencyofcollegestudentsmarginal
5%offreshmenprocient;10%ofseniors(ETSdata)
Mathprociencydoesntchange(CAAPtestdata)
ANtudesregardingquan;ta;veliteracyincreasemoderately
36%increaseinposiAveaftude;53%nogrowthordecline(Wabash
NaAonalStudy)
Desiretocontributetosciencesdoesntchangemuch
21%increaseoverfouryears(WabashNaAonalStudy)
16.9%offreshmen;20%ofseniors(CollegeSeniorSurvey)
Measuresofreading,wri;ngandcri;calthinkingarenotany
be<er[seealso,AcademicallyAdriJ(Arum&Roksa,2011)]

[Source:Finley,A.(2011)MakingProgress?:WhatAssessmentDataTellsUsAbout
LiberalEduca?onOutcomes.Washington,DC:AAC&U]
ThelargestgaininlearningproducAvityin
STEMwillcomefromconvincingthelarge
majorityofSTEMfacultythatcurrentlyteaches
bylecturingtouseanyformofacAveor
collaboraAveinstrucAon.

--JamesFairweather(2009)ReporttotheNa?onalAcademies
BoardonScienceEduca?on
HoellwarthandMoelter(2011)AmericanJournalofPhysics79(5):540.
30%
NSSESTEM/non-STEMAnalysis
GoodNews:Paritywithnon-STEMstudentsin
acAve/collaboraAvelearningandstudent-faculty
interacAons
BadNews:STEMstudentsreportfewer
experienceswithhigherorder,integraAveand
reecAvelearningthannon-STEMcounterparts

[Source:NelsonLaird,etal.(2011)PeerReview13(3):23-26]
N
S
S
E

2
0
1
0

A
n
n
u
a
l

R
e
p
o
r
t
:

h
_
p
:
/
/
n
s
s
e
.
i
u
b
.
e
d
u
/
h
t
m
l
/
a
n
n
u
a
l
_
r
e
s
u
l
t
s
.
c
f
m

DiscussionI:SoWhat?Drivers
Whichoftheseissuesarerelevantdriversfor
advancinglifescienceandSTEMeducaAonon
yourcampus?
Whatareotherimportantdrivers?
WherearethegapsormissedopportuniAes
forworkingtoimproveundergraduate
educaAon?
CHALLENGEOFEXISTINGEFFORTS
MosteortstoreformundergraduateSTEM
educaAon(MISTAKINGLY)startfroma
presumpAvereformmodel,onebasedprimarily
onin-classroominnovaAon,thatindividual
facultymemberimprovementswillleadto
aggregatechange.
--JamesFairweather(2009)ReporttotheNa?onalAcademies
BoardonScienceEduca?on
CHALLENGEOFEXISTINGEFFORTS
STEMreformrequiresacAveintervenAonby
academicleadersatthedepartmental,collegeand
insAtuAonallevel.Itrequireseortstoencourage
aculturewithinacademicprogramsthatvalues
teaching.
(Fairweather,2009,ReporttotheNa?onal
AcademiesBoardonScienceEduca?on)
OnLeadership,Managementand
Change
Strongleadershipcanhelpcreatethevision,
setthetoneoftheclimate,
emphasizethevaluesthataremostcriAcal,
andbuildtrustamongpeople.
Strongmanagementensuresthattheappropriate
execuAonoffuncAonsandfollow-through
areenabledthroughassessment.
However,changingthecultureistheengine
thatdrivestransforma;on.
Source:FREEMANA.HRABOWSKIIII,JACKSUESS,ANDJOHNFRITZ
AssessmentandAnaly?csinIns?tu?onalTransforma?on
EDUCAUSEReviewMagazine,Volume46,Number5,September/October2011

HowdoyouplanforSuccessinSTEM?
AligninsAtuAonalgoalsandprioriAeswith
respecttoSTEMlearningandstudentsuccess/
integraAonwithcampusstrategicplanorother
campus-wideiniAaAves
Characterizecurrentprogresstowardgoals,
includinganalysisofinsAtuAonaldata;current
strategies
IdenAfygapsormissedopportuniAesandareas
offuturework
IdenAfybestpracAcesthathelpyoumake
progress
38%
54%
48%
63%
65%
68%
73%
69%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
LaAna/oRespondents OtherRespondents
PercentageGradua;ng"On
Time"(i.e.,in2006-07)
None 1HIP 2HIPs 3ormoreHIPs
[
V=.109(.094)]
[
V=.255(.007)]
Source:DoesPar?cipa?oninMul?pleHighImpactPrac?cesAectStudentSuccessatCalStateNorthridge?byBefnaHuber
(unpublishedpaper,2010).

ImpactofParticipationinHigh-ImpactPracticesonPercentageof
SeniorNSSERespondentsGraduatingonTime,byRacialand
EthnicBackground
Whatarethecharacteris;csof
STEM-engaged?
InsAtuAonalgoalsforSTEMlearning(specied
outcomes)andstudentsuccess(retenAon,
progressionandcompleAon)
FocusonanddocumentaAonofstudentlearning
andsuccess
EecAvelearningenvironments(teaching,spaces);
assessment
ConAnuousprogramimprovementthatislinkedto
insAtuAonalgoalsandprioriAes
Note:SeeAAASV&CReport,amongothers
Howcansuccessbemeasured?
Studentlearningdata(pre/posttests,useof
rubricstoassesslearning/projects)
RubricforassessingeecAveandengaged
STEMteaching;guidelinesforfacultyreview,
tenureandpromoAon
Studentsuccess/retenAondatarstyear
courses,sophomoregatewaycourses
ProgramcompleAondatapercentagesof
studentswhograduate,disaggregated
NaAonalSurveyofStudentEngagementdata
InsAtuAonalRetenAonData
90.5%
75.4%
72.6% 73.2%
64.7%
64.4%
0.9%
7.0%
6.7% 5.3%
11.8%
11.5%
60.0%
65.0%
70.0%
75.0%
80.0%
85.0%
90.0%
95.0%
100.0%
AtMatricula;on OneYear
Reten;on
TwoYear
Reten;on
ThreeYear
*Persistence
FourYear
*Persistence
FiveYear
*Persistence
SixYear
Gradua;onRate
S
u
c
c
e
s
s

R
a
t
e

Fall2004First-TimeFreshmanCohort
AchievementGapofUnderRepresentedMinori;es

91.4%
75.9%
76.5%
78.5%
79.3%
82.4%
UnderRepresented
Minori;es
n=317
(NaAveAmerican,Black,Hispanic)
MajorityStudents
n=2,582
*Persistenceisacombina;onofstudentswhoeithergraduatedORares;llenrolled.
DiscussionII:NowWhat?Strategies
WhatcharacterisAcswoulddescribeaSTEM
engagedcampus?
Whatbenchmarksmightbeusedtomeasure
STEMengagementorsuccess?
Whatdatashouldbeusedtomeasuresuccess
inSTEMeducaAon?
ProjectKaleidoscope(PKAL)
NaAonalorganizaAon
dedicatedtoadvancing
whatworksin
undergraduateSTEM
educaAon
Crossdisciplinary
communityofnearly7,000
peopleatover1,000
colleges,universiAes,and
organizaAons
AssociaAonofAmerican
Colleges&UniversiAes
(AAC&U)aliaAon
h_p://www.aacu.org/pkal