Lessons Learned in Software Testing 1

Learnings from Lessons Learned in Software Testing

1. Regression testing is of three kinds: Bug Fix regression after reporting a bug and
hearing later on that it’s fixed. Old Bugs regression is to prove that a change to the
software has caused an old bug fix to become unfixed. Side-effect regression or
Stability regression to prove that the change has caused something that used to
work to now be broken.

2. Cheap Fixes worth fixing. If an issue costs fewer than four hours to fix it (code
change, replacement of a driver or data file on a disk, a change to the user manual
or to the text on the product’s box). Fixing cheap fixes can prevent over half of the
technical support calls.

3. Solution-oriented bug reporting – Deciding on the solution that fits the bug is the
product designer’s role, not a tester’s role. Many testers get so caught up in their
proposed solution that they don’t provide clean, accurate information about the
failure itself. If the tester misunderstands the underlying cause of the failure, his
solution is worthless at best.

4. Verify bug fixes promptly. If you find a problem with the fix quickly and report it
quickly to the programmer, he’ll probably still remember what he did to the code and
be able to take care of the problem right away. The longer you wait, the less the
programmer ill remember.

5. Manual Vs Automation tests. Don’t compare manual tests to automated tests.

Instead, look at automated tests as making it possible for you to extend your reach,
to do things that you wouldn’t do with manual testing.

6. Test Automation is programming. Tool strategies that promise to allow testers to create test
suites without programming fail. Don’t solely depend on junior programmers or programmer
rejects either. Managing, installing, configuring and maintaining the tools require programming
skill.

7. Many programmers mistreat testers. We think the best way to prevent or counteract
bad relations is to build personal relationships based on mutual respect. Assume that the
individuals you work with are worthy of respect and act on that assumption. Do your work in a
way that should earn their respect. Refuse to accept mistreatment or abuse.

8. Don’t try to create a control culture. What appears to be poor and incomplete
decisions are often thoughtful business decisions with which you disagree. It is easy
to believe that you know better and would do it better.

9. Pair testing is different from many other kinds of pair work because testing is an
idea generation activity, rather than a plan implementation activity. Pairing has the

Copyrights - Parimala Shankaraiah – http://curioustester.blogspot.com

Lessons Learned in Software Testing 2

effect of encouraging each tester to explain and react to ideas. When one tester
must phrase his thoughts to another tester, that simple process of phrasing seems to
bring the ideas into better focus and naturally triggers more ideas.

10. Measurement Dimensions. You can measure how much of the product has been
tested in many dimensions: bugs, requirements, code, configurations, change
history, developer, tester, data and so forth. No single measurement dimension is
sufficient.

11. Formal Evaluations are rare – every six months or every year. We encourage you
to actively follow the work of your staff on an ongoing basis. Note that we’re not
suggesting micromanagement: learning what your people do, and coaching them on
the basis of that knowledge is very different from telling them when and how to do
their work.

12. Testing Team. Build a staff whose individual strengths make up for each other’s
weaknesses. In some companies, a non-programming tester will be treated as a
second-class employee no matter how good he is. Our impression is that testers who
have specialized technical skills are often more prized and harder to find than testers
with managerial skills, who are in turn more prized than black box testers and less
experienced test automators.

13. Testing Tools. Whether or not your current employer appreciates you staying up to
date with testing tools, this knowledge will help you when you interview with
potential employers.

14. Writing. Much of your job involves writing reports and memos to persuade other
people to do things. The better you write, the more effective your reports. Take
courses in technical and persuasive writing and find other ways to practice and
improve your writing

15. Categories in a Test Plan – Strategy, Logistics and Work Products

16. Good Test Strategy is Product-specific, Risk-focused, Diversified and Practical

17. Diverse Half-measures. It is better to do more different kinds of testing to a

pretty good level, than to do one or two kinds of testing perfectly. We call this the
principle of diverse half-measures. This strategic principle derives from the
structured complexity of software products. When you test, you are sampling a
complex space. No single test technique will sample this space in a way that finds all
important problems quickly.

18. Test to the maturity of the product – Early in the project, test sympathetically, in
the middle of the project, test aggressively, near the end of the project, test
diversely, in the final days, test meticulously.

Copyrights - Parimala Shankaraiah – http://curioustester.blogspot.com

Lessons Learned in Software Testing 3

There are a lot more nuggets of wisdom in the book. In addition, I wish I could jot
down all the 293 lessons that I learnt from the book. In the interest of time, I won’t
do that. Do read the book. I hope I have whetted your interest.

Copyrights - Parimala Shankaraiah – http://curioustester.blogspot.com