Building quality into projects

November 2010 Orignally published at www.bcs.org.

Gary Sutherland, an operations manager at SQS, calls for project managers to build
quality and not big test budgets into their projects.
One of the main causes of overruns on technology projects is, not surprisingly, late discovery of
issues with the technology and software itself. Adopting a more solution delivery rather than
software delivery mindset helps project teams to engage in ongoing assurance and identify
issues early - avoiding costly rework before it hits the project timescales.

Projects involving software are notorious for overruns and as project managers, were familiar with the signs: business
stakeholders get pressured to accept a reduced scope, the project team gets pressured to deliver more rapidly and external
stakeholders apply pressure to senior management and the project board. Meanwhile, the number of defects begins to rise and,
in many cases, critical defects relating to the basic design or operation of the system are found mid-way or late in the
development.

If we look at the technology related problems, quality issues are generally identified as defects and integration problems during
the various testing phases of the project. There are ways to mitigate these issues and some agile approaches can improve
quality. However, for the vast majority of software development projects there are still basic problems in how quality is built into
software products.

Adding quality by discovering defects

In the traditional model, developers build code and pass it on to testers, who execute a series of pre-defined tests against it.
This approach can only ever discover defects after the fact; hence, the traditional process of test and rework can only add
quality to the solution.
As project managers we build in rework budgets and allow time for testing to allow for the defects that will be discovered,
however, the later a technical issue is discovered, the higher the cost to resolve it - this isnt a linear relationship. The
groundwork for the most difficult to resolve defects is laid early on in requirements and design, those defects tend to be
discovered late, during integration, performance or security testing - when many components are finished or close to
completion.

Building quality in - test everything

An alternative way to tackle the problem is to build quality in from the outset, seeking to prevent those expensive defects from
getting into the design or requirements in the first place.
Planning for three cycles of testing is quite common; sadly discovering that re-tests and rework cause testing to spill over into
later test stages is quite common too. In such a project, as rework is discovered quality assurance (QA) costs rise because
much of the test teams life is spent performing testing or re-testing code that has been reworked; the proportion of time testers
spend focussing on new functionality or system behaviour can drop dramatically.

A better approach based on the principles that QA and test teams already apply is to treat all products delivered through the
project lifecycle as testable (i.e. assurable) items. Just like the code, documents and other artefacts are part of the overall
solution. When done properly, improving quality by addressing technical and business risk factors in early lifecycle products is a
high value-add activity.

The results from taking a more solution-based approach and assuring products earlier in the project can be dramatic. Figure
1[1] shows that the earlier a defect is introduced, the higher the ultimate cost of removing that defect e.g. removing defects
introduced at the requirements stage can cost 100 times more than defects introduced during integration.

Starting assurance earlier in the lifecycle may mean that QA resource is engaged on projects for longer, but the reduction in
workload towards the end of the project more than compensates for this (because fewer defects are discovered and re-tested).

Catching just one major issue early on could be enough to make the difference between on time and overrun.

There is some irony in the fact that one of the best ways to reduce your spend on testing effort is to do more testing, albeit of a
different sort to that which most test and project managers are accustomed to. We think about assurance as an ongoing activity,
not something we do after a piece of software has been built. The fact remains that by assuring and testing products earlier in
our projects, we can significantly increase the likelihood of avoiding technical surprises and thus bringing our projects in on time.

Making it happen - four steps for building quality into your projects
Step one: Mindset is critical. We often hear of the software development lifecycle (SDLC), which tends to focus the attention on
the software. Instead, we should think in terms of a SOLUTION DELIVERY lifecycle, in which all project/programme artefacts
form part of the solution and are subject to scrutiny by testers.

Step two: Identify the key artefacts produced during the solution delivery lifecycle, and define the business and technical risk
based acceptance criteria for them.

Step three: Coach the testers in the review process that you follow - these can range from full blown Fagan type inspections
that appeared in the late 1970s to informal reviews. Exercise caution here: it is all too easy for this kind of assurance activity to
turn into a box-ticking exercise and assurers need to have people skills and objectivity.

Step Four: Record the outcomes of inspections or reviews as you would for a software defect, assigning priorities to the
defects; if using a defect tracking tool, record the fact that they were raised following an inspection and identifying the artefact
that they were found in. Independence from the project is important to ensure objectivity and assurers should not normally be
part of the project delivery team.