Software Reviews, Walkthroughs, Inspections, and &ive references to 234eil Audits

Better to find an error twice than never to find it at all. [Freedman/Weinberg !" Freedman, #$%$, &$'$ Weinberg, ()andbook of

0pdate terminolog1 according

Wh1 #o We )ave Formal *echnical Reviews5

,$ *o err is human$ 6$ 7ots of errors escape the originator more easil1 than an1one else$ 8$ Reviews are educational$

%urpose9 %rovide
: visibilit1 into state of pro;ect : opportunities for personnel <pro;ect and non.pro;ect= to discuss topics related to pro;ect : intermediate milestones and sense of progress : assessment of technical ade>uac1 of pro;ect

0se the power of a team to ?

: %oint out needed improvements of a product$ : +onfirm the parts in which improvement is not needed or desired$ : 'ake technical work more uniform and predictable in >ualit1, which makes it more manageable$

*echnical review is different from other reviews

: For e@ample, budget and management reviews : *echnical review answers >uestion9 will this product do the ;ob it3s supposed to do : If answer is AnoB, then
C no schedule is on time, and C no cost is cheap enough$

When5
: Single review that occurs on a particular date : Single review that occurs in response to a particular condition : 'ultiple reviews that occur periodicall19 e$g$, monthl1 : 'ultiple reviews that occur in response to a particular condition, e$g$, code review for subs1stem

Denefits
: ,!E reduction in errors in products : F!.-!G cost reduction

Wh1 not ;ust pass it around and have reviewers sign off ?

Wh1 not ;ust pass it around and have reviewers sign off ?
: +omments are too large : +omments are unread : Reviewers don3t have to take responsibilit1 for the content$

Re>uirements
: *eam participation : #ocumented procedures : #ocumented results

*1pes of Reviews
: : : : : 'anagement Review *echnical Review Walkthrough Inspection Audit

*1pes of Reviews
: 'anagement Review
C Hvaluation of pro;ect.level plan or pro;ect progress C 'onitors status of schedules and compliance to re>uirements

: : : :

*echnical Review Walkthrough Inspection Audit

'anagement Review
: 2b;ectives9
C Inform management of pro;ect status C Resolve higher.level issues <management decisions= C Agree on risk mitigation strategies

: #ecisions
C C C C Identif1 corrective actions +hange resource allocation +hange pro;ect scope +hange schedule

*1pes of Reviews
: 'anagement Review : *echnical Review
C Focus on specification and design C #etermine whether products meet specs

: Walkthrough : Inspection : Audit

Scenario for a t1pical review

: 7eader9 Welcome$ We are here to review E for the purpose of I$ : Hach person is introduced$ : *he recorder is introduced$ : An agenda is presented and the review process e@plained$ : *he list of material in the review packet is stated$ : *he review begins?

Review
: Agenda set b1 leader, agreed to b1 committee : Review proceeds line.b1.line$ : Reviewers bring up issues as the1 encounter them$ : Issues are added to an Issues 7ist$ : Agenda changes as issues are encountered$

Round Robin Review

: Force ever1 member to participate$ : Hach person takes the lead for a section9
C C C C paragraph, line of code, function, test case

: &ood wa1 to get involvement from team$

*1pes of Reviews
: 'anagement Review : *echnical Review : Walkthrough
C 0sed to find anomalies and evaluate conformance C *1picall1 used for source code C Hffective form of education

: Inspection : Audit

Walkthroughs
: Producer guides review : Step b1 step presentation of product9 C +ode, design, report, test cases ? : +an cover lots of material : +an have more people attend, less prepared : 'ore work for presenter : 'a1 be difficult to control interactions C *oo man1 interruptions C Alwa1s hard to have a producer present

Walkthroughs
: %articipant should spend , hour preparing for each hour in meeting : Should be scheduled to be brief : Should onl1 review completed code or document : Should re>uire participants to sign report

*1pes of Reviews
: : : : 'anagement Review *echnical Review Walkthrough Inspection
C 'ore formal t1pe of walkthrough C 0sed to identif1 specific errors or problems

: Audit

Inspections
: Rapid evaluation of material with specific aspect in mind : +onfine attention to one aspect onl1
C Hg assume details are correct, but look for missing items at a global level C Hg look for occurrences of a particular kind of bug such as buffer overflow$

: Selection of aspects is ke1 point$

*1pes of Reviews
: : : : : 'anagement Review *echnical Review Walkthrough Inspection Audit
C Independent evaluation of process or product C Hnsures that the process is being followed

Review *eams
: : : : %ermanent or semi.permanent teams IJKJ or LA teams &et to be ver1 good at reviews 4eed to have some wa1 of reviewing the reviewers$

*he team
: 7eader C Mob9 obtain a good review or report wh1 it was not possible : Reflects the >ualit1 of the review process, not the >ualit1 of the product : 'ust be accurate : Recorder C %rovide info for accurate report of review C Short, public notes C +apture essence of each issue C 'ust ensure group has reached conclusion C #on3t video tape

*asks of leader
: : : : : : 'onitor preparedness of team members Set pace Neep meeting on track %oll members to reach consensus Hnsure participation )as right to terminate review if it is unproductive C #isagreements C Fatal flaws

+onsensus
: #ecision of the group is e>ual to the most severe opinion of the group members$ : De conservative$ : Accept the doubts of the most doubting member$ : 4o AI told 1ou so, but I got outvotedB$

Reviewers
: 'ust be >ualified to contribute : 'ust be unbiased : #on3t invite management if it causes conflict
C *he point is to review the pro;ect, not the producers$

Rules for Reviewers

I$ II$ III$ IJ$ J$ JI$ JII$ De prepared Hvaluate product, not people$ Watch 1our language$ 2ne positive, one negative$ Raise issues, don3t resolve them$ Record all issues in public$ Stick to technical issues$

Rules for Reviewers

: :
C C C

If 1ou find something, assume it3s a mistake$ *hese are 1our peers, not 1our enem1$ Remember people are involved Avoid Awh1 did 1ou ?B wh1 didn3t 1ou ?B sa1 instead AI don3t understand ?B
AWh1 did 1ou set the upper bound to ,! here5B AI don3t understand wh1 the upper bound is ,! here$B seems trivial, but it3s not$

: : :

#o not get bogged down in st1le issues$ For e@ample, if efficienc1 isn3t an issue, then don3t make it one$ If it makes things less maintainable, that3s an issue$ If there are standards, then either stick to the standard, or dispose of the standard$

4umber of Reviewers
: : : : Hnsure material is covered )ave enough #on3t have too man1 8.O is good
C +ount participants onl1 C 89 ensures material is understood

: 2utsiders can be good$ 'ust be unbiased

*ime
: At most 6 hours at a time : +ome back again if necessar1 : #epends on
C +omple@it1 C SiPe of pro;ect C +loseness to completion

Hver1one must prepareQ

: Review packet : Hver1thing relevant to making correct ;udgment9 code, specs, test data, results, standards, designs, ? : -!G of failures stem from unprepared teams

*actics for participation

: #evil3s Advocate C 2ne person makes strongest case against a product C Mob is to find fault C 4eeds to be an actor : #ebugging C %roducers leave known bugs in C Lualit1 of review depends on how man1 found : Alarms C *ime each person3s contribution, cut off : Stand.up Reviews C *alk as long as 1ou stand on one leg

Report
: &oal9 #oes the product do the ;ob its supposed to do5 C +omplete$ C +orrect$ C #ependable as basis for other work C 'easurable for purposes of tracking$ : What was reviewed5 : Signatures of reviewers$ : 7eader and recorder$

4otes
: Re>uires learning9 to review and to write : 4eed to allocate time : 4ot a form of pro;ect management
C Dut can provide information for tracking

: Review earl1 and often

C Dut not too often

4eed for Reviews

: Reviews help organiPation deliver >ualit1 product
C H@amine individual parts earl1 in process C Identif1 problems spanning multiple items or phases

: Reviews help identif1 problems earl1 : Reviews help build image of the product and the vision of the product in the minds of the team members

+ost of Software Reviews

: 0p.front cost for
C *raining C Staff preparation C Review +onduct

: +ode review ma1 be -.,FG of total cost of pro;ect : 2ffset b1 savings9 reduced rework later in pro;ect <6F.8FG of total pro;ect cost saved=
C Ra1theon9 pre., --9 R8G of software cost in defect correction C , R <after software reviews instituted=9 FG of pro;ect cost in defect correction

Writing Issues
: : : : &roups of 8 I3ll give 1ou an issue from an issue list, Iou tell me what3s wrong with it$ Issues <and responses= from Handbook of Walkthroughs, Inspections, and Technical Reviews, #aniel Freedman and &erald Weinberg, 8rd Hd, #orset )ouse$

Writing Issues S,
Some of the e@planations of user commands were misinterpreted b1 members of the review committee$

+omments on Issues S,
4ot specific enough$ Which e@planations were misinterpreted5 #on3t make the producers guess, or the1 ma1 change the wrong thing$

Writing Issues S6
A ma@imum of ,! values ma1 be specified even though none of the standard s1stem ADH4# codes has been made not eligible$

+omments on Issues S6
In writing, clarit1 isn3t the most important thing9 it3s the only thing$ If 1ou don3t want to be not understood, don3t never use no double negatives$ Another thingC what is the issue an1wa15 Is the ma@imum too low5 *oo high5 2r is it that some of the codes should have been made not eligible5 2r eligible5 De direct$

Writing Issues S8
*he referenced table of constant values was not part of the review packet$

+omments on Issues S8
Alwa1s give the most direct reference available$ *here could be more than one table, now or in the future$ Wh1 make the producers search5

Writing Issues SR
*he method used for maintaining the message >ueue seems to solve a severe performance problem we3ve been having with the production version of the RNI s1stem$

+omments on Issues SR
*his is an interesting observation, clearl1 and directl1 stated$ Dut what does it have to do with the product under review5 It might be worth millions, but it doesn3t belong on the Issues 7ist for this product$ It should go on the Related issues list$

'ore issues SF
*he price/performance table should be sorted using either Luicksort or Shell sort$

+omments SF
Raise issues, but avoid all temptation to give advice in the Issues 7ist$ *he1 won3t be welcome if the1 come in that form, so if 1ou reall1 must give advice, find some unofficial vehicle$ *ake the producers to lunch, or out for a beer, before 1ou share 1our vast e@perience on matters of sorting$ If 1our idea isn3t worth the price of a beer, wh1 not forget it5

'ore issues S/
*he three diagrams drawn b1 )arold 'itter are not in the standard format <#S.,! = re>uired for such diagrams in our installation$

+omments S/
*his point is nicel1 specific, but wh1 do we have to mention poor )arold5 We are reviewing the product, not the people$ Find another wa1 to identif1 the diagrams and leave people3s names out of the Issues 7ist$

'ore issues SO
*he committee was unable to understand the significance of paragraph 8$,$

+omments SO
4othing3s wrong with this one$ It3s specific about which paragraph is under discussion, and it sa1s the committee doesn3t understand it$ Who can argue with that5 Iet, surprisingl1, people seem to be afraid to e@press issues this wa1 C AWe don3t understand ?B #on3t worr1 about being thought stupid$ If 1ou don3t understand it, it3s at least a potential issue in documentation$ And besides, lack of understanding ma1 mean there is something dreadfull1 wrong$

Iet more issues S*he bubble sort used for sorting the table of price/performance figures is a stupid approach if the table should grow an1 bigger than the present 6! entries$

+omments S*he word AstupidB doesn3t add an1thing at all and might antagoniPe the producers$ *ake it out$ *hen tr1 to e@press factuall1 and >uantitativel1 wh1 the method is inappropriate for larger tables$ And if the approach is stupid, what of it5 As Arthur +$ +larke e@pressed it, AIt has 1et to be proved that intelligence has an1 survival value$B

Iet more issues S

If the DI 4A'H option is not used, the structuring of the structure operands must be e>uivalent to the structuring of the structures in the arra1s of structures$

+omments S
*his wasn3t reall1 taken from an Issues 7ist, but from an old %7/I manual$ Still, I3ve read real issues that were almost as obscure$ I3d advise 1ou to follow the NISS a@iom <ANeep It Simple, Stupid=, e@cept that I would be contradicting m1 own advice from the previous >uestion$

Iet more issues S,!

Frieda Sonntag has no computer science background, and her e@perience with %7/I is nil$ She should never have been assigned the coding of this module$

+omments S,!
It3s non of the business of the review committee who management has assigned to particular ;obs$ *he committee3s business is to review the product and tell what state it3s in$ )ow it got to that state is another issueTand not for the Issues 7ist$ Who is responsible is even less of an issue, and raising it is sure to be ineffective$