22/2/2015

SuperMemo:Analysis

SuperMemoHome

ABC

Demos

Hints&Tips

Reference

Overviews

Glossary

Buy

Support

Search

Analysis
Editable(wiki)versionofthispage

Contents
1Distributions
1.1Intervals
1.2AFactors
1.3Repetitions
1.4Lapses
2ForgettingCurves
33DCurves
4Graphs
4.1ForgettingIndexvs.Priority
4.2Firstinterval
4.3DFactorvs.AFactor
4.4FirstGradevs.AFactor
4.5Gradevs.ForgettingIndex
53DGraphs
6Approximations
7Matrices
8Use
8.1Usestatistics
8.2Usecontextmenu
8.3Examples
8.3.1Use:Elementcount:Topicsandtasks
8.3.2Use:Newlymemorized
8.3.3Use:Efficiency:Forgettingindex
8.3.4Use:Priorityprotection:Items
8.3.5Use:Priorityprotection:Topics
9FAQ
10Furtherreading

ThetabbeddialogavailablewithTools:Statistics:Analysis(Shift+Alt+A)inSuperMemoprovidesgraphsandtablesthatillustratethecurrentstateofthelearningprocessin
thecurrentlyopenedcollection.
InadditiontoAnalysis,SuperMemoprovidesmultipleotheranalyticaltools.

Distributions
Intervals
Intervalsdistributionofinterrepetitionintervalsinagivencollection.Bychangingthesettingofthecomboboxatthebottomofthedialog,youcanseeintervaldistributionsfor
allelementsinthecollection,foronlyitems,andforonlytopics.

AFactors
AFactorsdistributionofAFactorsinagivencollection.ThedistributionitselfisnotusedinAlgorithmSM15,andmerelyresultsfromit.Notethattopicsanditemsusedifferent
AFactorcategories.ChooseItemsonlyorTopicsonlyatthebottomtoseedistributionsforitemsortopics.IfyouchooseAllelementsyouwillseealessmeaningful
conglomeratedistributionindexedbyAFactorcategorynumber.

http://www.supermemo.com/help/analysis.htm

1/10

22/2/2015

SuperMemo:Analysis

Repetitions
Repetitionsdistributionofthenumberofrepetitionsofitemsortopicsinagivencollectiononlymemorizedelementsareconsideredinthedistribution,i.e.thereisnozero
repetitionscategory.

Lapses
Lapsesdistributionofthenumberoftimesparticularitemsinthecollectionhavebeenforgottenonlymemorizedelementsareconsidered.Youshouldneverhavemorethan
10%ofitemswithmorethan34lapses.Ifthisisnotthecase,youshouldreexaminethewayinwhichyouformulateitems(see20RulesofFormulatingKnowledge).Astopics
arenever"forgotten"atrepetitiontime,therearenolapseslistedinLapsesifyouselectTopicsonlyintheelementtypecombobox.

ForgettingCurves
400forgettingcurvesareindependentlyplottedinordertocomputetheRFmatrixinAlgorithmSM15.Thesecorrespondto20repetitionnumbercategoriesand20AFactor
categories.Fordatarepresentationconvenience,thecolumnsoftheRFmatrixforthefirstrepetitionareindexedbythenumberofmemorylapsesratherthanbytheAFactor.By

http://www.supermemo.com/help/analysis.htm

2/10

22/2/2015

SuperMemo:Analysis

choosingapropercombinationoftabsatthebottomofthegraph,youcanselectaforgettingcurveofinterest.Horizontalaxisrepresentstimeexpressedas:(1)UFactor,i.e.the
ratioofsubsequentinterrepetitionintervals,or(2)days(onlyforthefirstrepetition).Verticalaxisrepresentsknowledgeretentioninpercent.

Figure: Bluecircles inthepicturerepresentdatacollectedduringrepetitions.Thelargerthecircle,thegreaterthenumberofrepetitionsrecorded.The redcurve

correspondswiththebestfitforgettingcurveobtainedbyexponentialregression.Forillstructuredmaterialtheforgettingcurveiscrooked,i.e.notexactly
exponential.Thehorizontal greenline correspondswiththerequestedforgettingindex,whilethevertical greenline showsthemomentintimeinwhichthe
approximatedforgettingcurveintersectswiththerequestedforgettingindexline.ThismomentintimedeterminesthevalueoftherelevantRFactor,andindirectly,
thevalueoftheoptimuminterval.Forthefirstrepetition,RFactorcorrespondswiththefirstoptimuminterval.ThevaluesofOFactorandRFactoraredisplayedat
thetopofthegraph.Theyarefollowedbythenumberofrepetitioncasesusedtoplotthegraph.Atthebeginningofthelearningprocess,thereisnorepetition
historyandnorepetitiondatatocomputeRFactors.Itwilltakesometimebeforeyourfirstforgettingcurvesareplotted.Forthatreason,theinitialvalueoftheRF
matrixistakenfromthemodelofalessthanaveragestudent.Themodelofaveragestudentisnotusedbecausetheconvergencefrompoorerstudentparameters
upwardsisfasterthantheconvergenceintheoppositedirection.TheDeviationparameterdisplayedatthetoptellsyouhowwellthenegativelyexponentialcurve
fitsthedata.Thelesserthedeviation,thebetterthefit.Thedeviationiscomputedasasquarerootoftheaverageofsquareddifferences(asusedinthemethodof
leastsquares).

http://www.supermemo.com/help/analysis.htm

3/10

22/2/2015

SuperMemo:Analysis

Figure:InSuperMemo16,forgettingcurvescanbenormalizedoverAFactors(AllAFs)atdifferentrepetitioncategories(AllReps)(asinthepictureabove).Asa
result,youcandisplay:
1. yourcumulativeforgettingcurve( bluedots )
2. itsnegativeexponentialapproximationusedbySuperMemo( redline )
3. individualdatapointsfromallforgettingcurvescollectedbySuperMemo( yellowcircles ).
Time(normalized)standsat100%formaximumUfactorforeachentryoftheRFmatrix.Decaystandsforthedecayconstantoftheforgettingcurve.Cases
showsthenumberofrepetitioncasesusedinplottingthegraph.RepetitionsstandsforthenumberofrepetitionsexecutedwhendeterminingRFactors.Deviation
isasquarerootoftheaverageofsquareddifferencesbetweenthedataandtheapproximatedforgettingcurve.

3DCurves
Thesegraphsshowtheforgettingcurvesin3dimensions.Thereare20graphsavailablewithAFactortabsatthebottom.EachgraphcorrespondstoadifferentAFactor(from1.2
to6.9).TheXaxisrepresentstime(asinForgettingcurves).TheYaxisrepresentsrepetitioncategory(roughlycorrespondingtotherepetitionnumber).TheverticalZaxis
representsretentioninpercent(i.e.howmuchisrememberedatagivenmomentintime).Forbettervisibility,youcanrotate3DforgettingcurveswithRotate,orsmooththem
withSmooth.RotateswingsthegraphonitsZaxisby90degreesclockwise(asviewedfromabove).ThismeansthatXandYaxesgetswapped.Notethatsmoothingproceeds
alongtheYaxis.Thismakesitpossibletoalignincompletelyplottedforgettingcurvesfromdifferentrepetitioncategories.However,onceyouclickRotate,Yaxissmoothingwill
produceadifferentresult(e.g.afterthefirstRotate,smoothingwillreducekinksalongtherepetitioncategories).Forthebestillustrationoftheforgettingprocess,smooththe
graphbeforeyourotateit.

Figure:Exemplary3DgraphofforgettingcurvesforAFactor=3.6.ThegraphhasbeensmoothedalongtheYaxis(byaveragingresultscorrespondingtodifferentrepetition
categories),androtatedonceforbettervisibility(theslopeillustratesforgettingintimethatisroughlysimilarforvariousrepetitioncategoriesexceptthosewithhighrepetition
numberthatmayreflectscarcityofdataforlongintervals).

Graphs
ForgettingIndexvs.Priority
ForgettingIndexvs.Prioritythisgraphshowsyouhowforgettingdependsonthepriorityofthelearningmaterial.Ifyoualwaysgothroughyouroutstandingmaterialintime
andifyousticktothe20rulesofknowledgeformulation,yourmeasuredforgettingindexshouldbethesameastherequestedforgettingindex.Thisgraphwillthenshowaflat
straightlinepassinghorizontallyattheleveloftheforgettingindex.However,onceyoustartoverloadingthelearningprocessandusingPostponeorMercy,yourmeasured
forgettingindexwillincrease.Ifyoualwayssortyourrepetitionsbypriority(e.g.byleavingLearn:Sorting:Autosortrepetitionschecked),youshouldstillensurethatyour
measuredforgettingindexequalstherequestedforgettingindexforhighpriorityelements(i.e.ontheleftsideofthegraph).Yourmeasuredforgettingindexforlowpriority
materialwillbehigher,andthegreatertheoverloadthegreaterthedifference.Ifyouchangeyourrepetitionsortingcriteriabyincreasingrandomizationofrepetitions,thisgraph
willbeflatterandthemeasuredforgettingindexforhighprioritymaterialwillbehigher.

Firstinterval
Firstintervalthelengthofthefirstintervalafterthefirstrepetitiondependsonthenumberoftimesagivenitemhasbeenforgotten.Notethatthefirstrepetitionheremeans
thefirstrepetitionafterforgetting,notthefirstrepetitionever.Inotherwords,atwicerepeateditemwillhavetherepetitionnumberequaltooneafterithasbeenforgottenthe
repetitionnumberwillnotequalthree.Thefirstintervalgraphshowsexponentialregressioncurvethatapproximatesthelengthofthefirstintervalfordifferentnumbersofmemory
lapses(includingthezerolapsescategorythatcorrespondswithnewlymemorizeditems).Inthegraphbelow,bluecirclescorrespondtodatacollectedinthelearningprocess(the
greaterthecircle,themorerepetitionshavebeenrecorded).

http://www.supermemo.com/help/analysis.htm

4/10

22/2/2015

SuperMemo:Analysis

Figure:Inthegraphabove,whichincludesdatafromover130,000repetitions,newlymemorizeditemsareoptimallyrepeatedaftersevendays.However,itemsthathavebeen
forgotten10times(whichisrareinSuperMemo)willrequireanintervaloftwodays.(Duetologarithmicscaling,thesizeofthecircleisnotlinearlyproportionaltothedatasample
thenumberofrepetitioncasesforlapses=0isbyfarlargerthanforlapses=10,ascanbeseeninDistributions:Lapses.)

DFactorvs.AFactor
DFactorvs.AFactorDFAFgraphshowsdecayconstantsofthepowerapproximationofRFactorsalongcolumnsoftheRFmatrix.YouneedtounderstandSuperMemo
Algorithmtounderstandthisgraph.ThehorizontalaxisrepresentsAFactor,whiletheverticalaxisrepresentsDFactor(i.e.DecayFactor).DFactorisadecayconstantofthe
powerapproximationofcurvesthatcanbeinspectedintheApproximationstaboftheAnalysisdialogbox.

FirstGradevs.AFactor
FirstGradevs.AFactorGAFgraphcorrelatesthefirstgradeobtainedbyanitemwiththeultimateestimationofitsAFactorvalue.Ateachrepetition,thecurrentelement's
oldAFactorestimationisremovedfromthegraphandthenewestimationisadded.ThisgraphisusedbySuperMemoAlgorithmtoquicklyestimatethefirstvalueofAFactorat
themomentwhenallweknowaboutanelementisthefirstgradeithasscoredinitsfirstrepetition.

Gradevs.ForgettingIndex
Gradevs.ForgettingIndexFIGgraphcorrelatestheexpectedforgettingindexwiththegradescoredatrepetitions.YouneedtounderstandSuperMemoAlgorithmto
understandthisgraph.Youcanimaginethattheforgettingcurvegraphmightusetheaveragegradeinsteadoftheretentiononitsverticalaxis.Ifyoucorrelatedthisgradewith
theforgettingindex,youwouldarriveattheFIGgraph.Thisgraphisusedtocomputeanestimatedforgettingindexthatisinturnusedtonormalizegrades(fordelayedor
advancedrepetitions)andestimatethenewvalueofitem'sAFactor.Thegradeiscomputedusingtheformula:Grade=Exp(A*FI+B),whereAandBareparametersofan
exponentialregressionrunoverrawdatacollectedduringrepetitions.

http://www.supermemo.com/help/analysis.htm

5/10

22/2/2015

SuperMemo:Analysis

3DGraphs
3DgraphsthatvisuallyrepresentRF,OF,andCasesmatrices(see:Matricesbelow)

Approximations
20powerapproximationcurvesthatshowthedeclineofRFactorsalongcolumnsoftheRFmatrix.YouneedtounderstandSuperMemoAlgorithmtounderstandthesegraphs.
ForeachAFactor,withincreasingvaluesoftherepetitionnumber,thevalueofRFactordecreases(atleasttheoreticallyitshoulddecrease).Powerregressionisusedtoillustrate
thedegreeofthisdeclinethatisbestreflectedbythedecayconstantcalledhereDFactor.BychoosingtheAFactortabatthebottomofthegraph,youcanviewacorresponding
RFactorapproximationcurve.Thehorizontalaxisrepresentstherepetitionnumbercategory,whiletheverticalaxisrepresentsRFactor.ThevalueofDFactorisshownatthetop
ofthegraph.ThebluepolylineshowsRFactorsasderivedfromrepetitiondata.TheredcurveshowsthefixedpointpowerapproximationofRFactor.Thegreencurveshowsthe
fixedpointpowerapproximationofRFactorderivedfromtheOFmatrixthisisequivalenttosubstitutingtheDFactorobtainedbyfixedpointpowerapproximationofRFactorsfor
theDFactorobtainedfromDFAFlinearregression.Afixedpointapproachisusedforbothapproximationssincefortherepetitionnumberequaltotwo,RFactorequalsAFactor.

Matrices
YouneedtounderstandSuperMemoAlgorithmtounderstandthesetables.
OFmatrixmatrixofoptimalfactorsindexedbytherepetitionnumberandAFactor(exceptforthefirstrepetition,AFactorisreplacedwithmemorylapses)
RFmatrixmatrixofretentionfactors
CasesmatrixofrepetitioncasesusedtocomputethecorrespondingentriesoftheRFmatrix(doubleclickanentrytoviewtherelevantforgettingcurve).Thismatrixcanbe
editedmanually(e.g.ifyouwanttochangetheweightofsomemeasurementsduringrepetitions)
IntervalsmatrixofoptimumintervalsderivedfromtheOFmatrix
DFactorsvectorofDFactorvaluesfordifferentAFactorvalues(alsorepetitioncasesusedincomputingparticularDFactors)

Use
Graphsofchangesinthestatisticsofthelearningprocessovertime.Thesegraphsrecorddailychangestostatisticssuchasthenumberofmemorizeditems,measuredforgetting
indexorthenumberofoutstandingitemsforagivenday.Thefirstnonzerovalueofthedayisrecorded.Ifthatvalueincreasesafterwards,thatincreaseisignored(exceptafew
casessuchasthestatisticofnewlymemorizeditems,etc.).Ifyouhaveabreakinlearning,thelastrecordednonzerovaluewillpropagatethroughtheperiodinwhichyoudidnot
useSuperMemo(exceptafewcasessuchasthestatisticsofrepetitionsmade,etc.).
Tozoominonaportionofthegraph(asinpicturesbelow),sweeptheportionsofthegraphsthataretoberemovedwiththemouse,i.e.pointatthedayoffocus,movethe
mouseleftandreleasethemousebutton.Thiscutsawaypartofthegraphinthedirectionthatthemousewasmoved.YoucanalsouseCtrl+LeftandCtrl+Rightbuttonstozoom
in(seebelow)

Usestatistics
HerearethestatisticsrecordeddailybySuperMemoanddisplayedontheUsetab:
1. Workdone
Usetimeyourdailytotaltimeinwhichyouactivelyusedagivencollection(asrecordedinTools:SleepChart)
Recalltimeyourdailytotaltimewhichyouspentontryingtorecallanswerstoquestions
Repetitionsyourdailyrecordednumberofitemandtopicreviews
Itemrepetitionsyourdailyrecordofitemrepetitions
Reviewsyourdailyrecordoftopicreviews
2. Outstandingnumberofoutstandingelementsscheduledforagivenday
Outstandingnumberofalloutstandingelementsscheduledforagivenday
Outstandingitemsnumberofoutstandingitemsscheduledforagivenday

http://www.supermemo.com/help/analysis.htm

6/10

22/2/2015

SuperMemo:Analysis
Outstandingtopicsnumberofoutstandingtopicsscheduledforagivenday

3. Burden
BurdencurrentBurdenparameterrecord.Burdenestimatesthenumberofelementsscheduledforrepetitiondailyontheassumptionofnodelaysinthelearning
process.Burdenisthesumofintervalreciprocals.Burden=Itemburden+Topicburden
ItemburdencurrentBurdenforitems
TopicburdencurrentBurdenfortopics
4. Elementcount
Elementstotalnumberofelementsinthecollectionasrecordedonagivenday(i.e.topics+items+tasks)
Itemsnumberofitemsinthecollectionasrecordedonagivenday
Topicsandtasksnumberoftopicsandtasksinthecollectionasrecordedonagivenday(seeanexemplarygraphbelow)
Tasklist(nameofthecurrentlyselectedtasklist)numberoftasksonthecurrentlyusedtasklist(itsnameisdisplayedinparenthesis).UsetheTasklistcombobox
intheToolstoolbartochangethecurrentlyselectedtasklist.
5. Memorized
Memorizednumberofelementstakingpartinthelearningprocess(i.e.notdismissedandnotpending)
Memorizeditemsnumberofitemstakingpartinthelearningprocess
Memorizedtopicsnumberoftopicstakingpartinthelearningprocess
6. Newlymemorized
Newlymemorizednewelementsmemorizedonagivenday(seeanexemplarygraphbelow)
Newitemsnewitemsmemorizedonagivenday
Newtopicsnewtopicsmemorizedonagivenday
7. Efficiency
Forgettingindexmeasuredforgettingindex
Retentionretentionasmeasuredoneachday
Topicloadproportionoftopicsscheduledforreviewonagivendayamongallelementsscheduled.Topicload=Outstandingtopics/Outstandingelements.Topic
loadisexpressedinpercentforanygivenday.Topicloadshouldnotbeconfusedwithproportionoftopicsinsortingcriteria.Proportionoftopicsisdetermined
independentofthetopicloadifpossible(e.g.forzerotopicload,youcannotachieveproportiongreaterthanzero,etc.).Ifyoudonotuseautosorting,topicloadtells
youwhatproportionofyourrepetitionsismadeoftopics.
8. Overloadthenumberofexecutedrepetitionsdividedbythenumberofoutstandingrepetitions(i.e.thepercentoftheactuallyexecutedrepetitions)
Elementsthepercentofalloutstandingrepetitionsexecuted
Itemsthepercentofoutstandingitemrepetitionsexecuted
Topicsthepercentofoutstandingtopicreviewsdone
9. Priorityprotectionyourprocessingcapacityforhighprioritymaterialonagivenday.IfyourgraphoscillatesaroundPriorityof3%,youwillknowthatonlytop3%ofyour
learningmaterialisguaranteedatimelyrepetition.Youcanincreasethatnumberbydoingmorework,reducinginflowofnewmaterial,deprioritizinglessimportant
elements,orreducingtherandomizationdegreefortheoutstandingqueuesortingcriteria.ReadaboutthisimportantparameterinPrioritizationRulebookinthepriority
queuearticle
Itemsthehighestpriorityitem(withthelowest%)thatwasmissedinrepetitions.(Theactual%intheoutstandingqueuecanbemuchhigher(becauseyoumostly
reviewhighpriorities))
Topicsthehighestprioritytopic(withthelowest%)thatwasmissedinrepetitions.(seeanexemplarygraphbelow)

Usecontextmenu
Next(Ctrl+Tab)switchtothenextmajorUsestatistictab
Previous(Shift+Ctrl+Tab)switchbacktothepreviousmajorUsestatistictab
First(Home)switchtothefirstmajorUsestatistictab(WorkdoneinSuperMemo15)
Cutoffleft(Ctrl+Rightarrow)cutofftheleftpartofthegraphtozoominontherestofit
Cutoffright(Ctrl+Leftarrow)cutofftherightpartofthegraphtozoominontherestofit
Cutofftop(Ctrl+Downarrow)cutoffthetoppartofthegraphtozoominontherestofit
Close(Esc)closetheAnalysiswindow

Examples
Use:Elementcount:Topicsandtasks

http://www.supermemo.com/help/analysis.htm

7/10

22/2/2015

SuperMemo:Analysis

Figure:Exemplarygraphshowingthetotalnumberoftopicsstoredinthecollection.Thedatadisplayedatthebottomdependsonthepositionofthemouse.Inthisexample,it
indicatesthatonMar6,2009,therewere251,228topicsintheincrementalreadingprocess.

Use:Newlymemorized

Figure:Exemplarygraphshowingthenumberofelementsmemorizedonindividualdays.Bysweepingawaytheleftandrightpartofthegraph,itispossibletozoominona
smallsectionofthelearningprocess.TheperiodshownonthegraphspansfromJul1,2008toMar19,2009.

Use:Efficiency:Forgettingindex

http://www.supermemo.com/help/analysis.htm

8/10

22/2/2015

SuperMemo:Analysis

Figure:Exemplarygraphenablingamoremeaningfulanalysisoftheforgettingindex.ChangestoforgettingindexinAnalysisusethedailymeasuredforgettingindex(previously:
lessinformativecumulativemeasuredforgettingindexvaluewastakenfortheentireperiodsincethelastuseofTools:Statistics:Resetparameters:Forgettingindex
record).Notethatthepriorityqueuemaydistorttheactualretentioninyourcollectionasmeasuredvaluesareprimarilytakenfromtopprioritymaterial.Thusmeasuredforgetting
indexshouldbeunderstoodas"forgettingindexmeasuredatrepetitions",notas"overallmeasuredforgettingindex".

Use:Priorityprotection:Items

Figure:Exemplarygraphshowingthehighestpriorityitem(withthelowestpriority%)thatwasmissedinrepetitionsonagivenday.Inthepicture,youcanseetheincreaseinthe
itempriorityprotectionfrom0.5%to11%withinamonth.Thisispossiblebyfocusingonhonestprioritization,doinglotsofrepetitions,avoidingmassupprioritizations,etc.

Use:Priorityprotection:Topics

http://www.supermemo.com/help/analysis.htm

9/10

22/2/2015

SuperMemo:Analysis

Figure:Exemplarygraphshowingthehighestprioritytopic(withthelowestpriority%)thatwasmissedinrepetitions.Thedatadisplayedatthebottomdependsonthepositionof
themouse.Inthisexample,itindicatesthatonJun26,2013,theworstmissamongsttopicswasatopicwithPriorityequalto7.613%ofthequeueofthemostimportanttopicsto
review.

FAQ
IworrymygraphslookweirdandIdosomethingwrong

Furtherreading
FAQ:Memoryandlearning
SuperMemoTheory
SuperMemoAlgorithm

http://www.supermemo.com/help/analysis.htm

10/10