Published in PM World Today - October 2008 (Vol.

X, Issue X)

PM WORLD TODAY – PM TIPS & TECHNIQUES – OCTOBER 2008

Testing Phase of Software Project – Much More than Testing

By Michael Szwarc

What is the Contribution to the Company and How Does it Serve me as Manager?

In this article I will describe some benefits from the testing phase of a software development
project, and how to ensure that the company and the project manager can increase them.

Definition of the Testing Phase

It is very difficult to define "the testing phase". In order to illustrate the idea of this article, "the
testing phase" will refer to the phase in which comprehensive testing begins to be carried out on
the development products (usually by designated testers). For discussion purposes,
comprehensive testing begins from the integration stages and up to the system test. Many refer
to this stage as the "QA stage". Because of the complexity of the concept "QA" and vast
description, the article will adhere to the concept "testing phase".

Objectives of the Testing Phase

Before we examine the contribution of the testing phase to the project, it is important to
understand the declared objectives of this phase. The accepted answers to this question are as
follows:

• "To allow interested parties in the product to receive a measure of its quality and
compliance to its requirements"
• "To prove that the software does what it is supposed to do" (partial and problematic
definition)
• "To find the gaps between what the software is supposed to do/not do to what it actually
does/does not do" (a slightly more complete definition)

This is indeed the main purpose of the testing phase - an important and critical phase for the
success of the project.

However, is it the exclusive purpose? Is the job of the testers summed up in this way?
In most cases the answer is positive without much hesitation. His focused task to mark a "V" on
the quality of the product as it is perceived by many, results in lack of administrative attention.
Most of the focus in the project, as well as administrative enthusiasm, is in the development
stage (in its traditional sense: the world of software developers).

In view of this, managers tend to put less emphasis to planning of the testing phase in the
planning stages of the project and many encourage more attention to be paid to this stage (and

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 1
 Published in PM World Today - October 2008 (Vol. X, Issue X)

they are indeed correct). But in fact this encouragement proves that it is not trivial. Otherwise it
would not be necessary …

Many of the project managers recall the testing phase only at the last minute. Few involve and
integrate them still in the planning stages and only the "meticulous" refer to the planning of the
testing as an integral part of the project planning itself.

Additional Duties of "the Testing Phase"

Field experience shows that the simplistic attitude to the testing phase reduces its contribution to
the project. The testing phase conceals much value for the project manager in particular and the
company in general:

a. First and foremost, and in accordance to the declared objective, the testing phase gives a
quality view of the project product. This is the classic and obvious contribution of the
managers involved. Already after rotation of the first testing, the gut feelings of those
involved begin to disappear and become based on more scientific evaluations. There is a
status report that starts to become clear for the first time allowing the project manager a
sigh of relief (even a little) for the first time or alternately to start to worry (but this time
out of knowledge…).

b. Already at the beginning of the testing plan writing stage (before the start of the testing):
many problems appear in the design documents: ambiguity, handling of extreme
cases, vagueness and such. This results from great caution, punctuality and thoroughness
characterizing the testing staff. Their training and personalities are what push them to
ask questions (as opposed to the development people who have a greater tendency to
interpret in a subjective way). Usually the cost of finding these problems at a later stage,
if at all, is much greater.

Example: In a project which was planned to take about 3 months (two weeks design, a
month for development, 2 weeks for testing and a week installation), writing of the testing
plan was delayed by a week before beginning of the testing. During the writing (by
examining many extreme cases) the tester was flooded with a number of questions
relating to matters that were unclear in the design. These questions caused a basic
change in the design and consequently in the development. The result was a total of two
weeks delay in the project. Besides the tester, no one paid attention to these incidents
early.

c. The transfer from the development stage to the testing phase is mostly characterized by
tying different ends of the project products. In many cases a few developers work on
a number of different parts of the software. Sometimes a stage in itself is devoted to the
integration testing. Sometimes the testing phase constitutes the first opportunity to see all
the different components working together. This is the moment in which the 'tunnel

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 2
 Published in PM World Today - October 2008 (Vol. X, Issue X)

diggers' from both sides meet. Sometimes only in this stage are the last details finalized
of the interfaces and the specific configuration.

Example: In a project for a certain client, a specific system configuration was required
which defined some different parameters. During development the software was checked
with a variety of parameters without any connection to the client's specific definitions.
During testing, the testers also insisted to check the specific demands of the client and the
testing revealed functional contradictions precisely in these cases.

d. In continuation to the previous paragraph, the testing phase begins from the installation
(maybe not necessarily the first but definitely significant) of the software and/or the
product. This is the first time that the product itself passes hands (and not only
documents). There are two transfers as follows:

1) Technological: transfer from one environment (development) to another environment

(testing). It is possible to refer to this operation as first rehearsal for operational
installation and to examine the quality of the installation process itself. Example: In
one project, installation of the product in the transfer from the development phase to
the testing lasted about two days. The operational installation plan allowed for a time
frame of only 6 hours. As a result of the extended installation, the company was
forced to allocate another week to reducing the installation time. The project lasted a
week but an unfortunate surprise was avoided on the night of the operational
installation…

2) Transfer of knowledge: The product is more than its technology. The product
combines processes, user interface, terminology and more. In a utopian world the
testers do not need the product in order to be familiar with it (they are already
involved in the design phase). In practice it is not like that. The testing phase has to
begin with proper training of the testing staff. This training is the initial foundation to
the training material of the product. Therefore, the transfer of knowledge constitutes
the first indication of the training process and the required assimilation.

Example: A project that consisted of many user interfaces, the training program was
greatly based on data and conclusions that were reported by the testers (based on
their personal experience).

e. The testing phase is not a consecutive homogeneous phase. By nature this is a cyclic
stage conducted in rotations (without relationship to the methodology of the whole
project management). Important players participate in these rotations: product managers,
development team and project manager. The organization tension created between the
"examiner" and the "examined", the need to make many small decisions (sometimes large
as well) sometimes causes the formation of some diverse inter-organizational
communications: "What is a bug? What is the change? What is critical? When to repair?
Whether to repair?" This communication improves the functioning of the project team in

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 3
 Published in PM World Today - October 2008 (Vol. X, Issue X)

other projects as well and thus contributes to improvement of processes in the company.
For a good reason the tools to manage bugs/changes were the first that were assimilated
in the project management process, in order to improve the communication and
organization abilities.

Example: There was a lack of communication between two companies who were partners
in a development project, which included a number of development and test cycles. The
design and development stages suffered from the differences in understanding and
methodology of the work which almost led to a complete halt of the project in the joint
testing phase. In light of this, a lot of administrative efforts were invested in order to
define the work process and communication which is based on agreed work methods and
supporting technology. These processes that were defined in the testing phase assisted
the rest of the cycles in the project.

f. The testing process includes a structured format of measurement: statistics on bugs,

coverage percentage, functional measurements of the product and a lot of its non-
functional measurements. To the "control panel" of the project manager are added at
once many indicators that reduce the uncertainty and improve his ability to plan the
continuation of the project.

Example: In an irregular project of its kind in the company, careful evaluations for the
time of testing were given (due to the innovativeness in the project: from new interface
and use of a new shelf product). At the beginning of the testing phase, the testing
manager started to advertise on a daily basis statistics on the results of the testing: bugs
and coverage percentages. Analysis of the results showed the pace of the testing and
supplied an objective tool for planning the continuation of the project.

Utilization of Hidden Potential of "Testing Phase"

From the aforementioned it can be seen that the testing process constitutes an important
framework for managers. For the project manager this is a central tool to control the whole
project (and not only the quality of the product). For the rest of the managers this is a good tool
for control of the company, and even more so, an opportunity to learn, to improve and to
assimilate processes.

The testing phase indicates the maturity of the company, quality of design processes,
development processes and more. A good design process will not indicate an inferior test
process but the opposite. From this point of view those who get confused between the concepts
"product testing" and "quality assurance" (QA) are right (out of kindness and not in favor of). In
the wide sense of the meaning, the concept of quality assurance refers to all the company
processes and not only to the quality of the product. And the argument is that the manner of
examining the quality of the product indicates the quality of the company processes.

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 4
 Published in PM World Today - October 2008 (Vol. X, Issue X)

The obvious conclusion is to dedicate much more attention to the testing phase and to provide
the necessary environment in order to ensure its execution in an appropriate and proper manner –
to be precise with correct gating: clarification of the contents being tested, freezing the code,
concern for appropriate and suitable testing environment and passage of all the partners in the
project on the testing plan.

In addition, the project manager should actively involve the testers in all the phases, and even to
strictly manage the milestone of the start of the testing. If possible, it is recommended to begin
testing cycles as early as possible (although there are a lot of disadvantages to an early start and
it is necessary to know how to manage them). Cooperation between the project manager and the
testing manager is critical.

At the company level, importance must be applied to the manning of the testing staff (and
especially its managers) in order to maximize the potential of the comprehensive contribution to
the company beyond the quality of the product itself.

Finally there are cases in which it is correct to use methodologies from the world of testing in
other processes of the company such as use of measurements. An extreme example is including
the development stage in the testing phase in order to force an 'agile' structure to the project if
required (especially if the company is not skilled in this).

Example: In a 4-month project and due to certain conditions (pressure of time and client's
character) it was known in advance that there would be changes in requirements during the
development. The company decided to begin testing cycles at a much earlier stage. All the
mechanisms of bugs review at a high frequency and management of development versions
accordingly served as changing points in the design. In these frequent meetings (once every two
days, in a short format) all the interested parties were present: the designers, the developers, the
testers and the project manager. The project received a natural and familiar agile type of
management without any need in specific training. This decision ensured the success of the
project.

Summary

The testing phase of a project has a main, important and familiar objective but beyond this, it
constitutes a wide range of opportunities to improve processes in the company and conceals
within high value for the project manager. The awareness of this potential at the initial stage and
its use can serve as a strategic asset that ensures success of the projects and as a driving force for
long-term study and development.

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 5
 Published in PM World Today - October 2008 (Vol. X, Issue X)

About the Author:

Michael Szwarc

Author

Michael Szwarc is currently a project manager

at Personeta Inc. which develops converged
Telecom Application Server and personalized
Telecom applications. His former positions include leading several
development groups and project management (with more than 10 years of
experience). Michael has been involved in development and assimilation of
different organizational processes in the areas of project management, HR
management and training programs. Michael can be reached at:
michael.szwarc@gmail.com

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 6