Software Testing 4

Damian Gordon

Test Approaches

Static Testing is the testing of a component or

system at a specification or implementation
level without execution of the software.

Dynamic Testing is the testing of software by

executing the software of either a component or
system.

Static Testing and Dynamic Testing are

complementary methods and tend to find
different types of defects effectively and
efficiently.

Test Approaches

Static Testing detects defects such as

deviations from standards, missing
requirements, design defects, nonmaintainable code and inconsistent
interface specifications.

Dynamic Testing detects defects such as

checking if outputs from the expected
values.

Test Approaches

One key approach in Static Testing is the

Review Process.

Reviews can find defects, are informational,

communicational, and educational.
Participants in the review learn the content of
the software systems, the role of their own
work, help planning for future stages of the
work.
Reviews often represent milestones, and
support the establishments of a baseline for the
software product.

The Review Process

Test Approaches
The type and quantity of defects found
during the review stage can help focus the
testing process.
In some cases customers or users attend
the review process and provide feedback
to the developers and document authors.
Studies have shown the reviews
significantly increase productivity and
product quality.

The Review Process

They can be either informal or formal.

The formality of the review process is
related to factors such as the maturity of
the development process, any legal or
regulatory requirements or the need for an
audit trail.
In practice most reviews are informal.
A two-person team can conduct an informal
review, as the developer/author can get a
colleague to review the code and
documentation.

The Review Process

Phases of a Formal Review

1.
2.
3.
4.
5.
6.

Planning
Kick-Off
Preparation
Review Meeting
Rework
Follow-up

The Review Process

1. Planning
The authors/developers request a review.
A moderator is assigned this is the leader
of the review process.
The project planning needs to incorporate
time to undertake the review.
The planning must start with deciding an
entry criteria to ensure that document is
ready for review.

The Review Process

1. Planning
Entry Criteria
The set of generic and specific conditions
for permitting a process to go forward with
a defined task, e.g. Test phase. The
purpose of entry criteria is to prevent a
task from starting which would entail more
(wasted) effort compared to the effort
needed to remove the failed entry criteria.

The Review Process

1. Planning
Minimum set of criteria for performing entry check:

A short check of a product sample by the moderator does

not reveal a large number of defects, e.g. After 30 minutes
of checking, no more than 3 major defects are found in a
single page or fewer than 10 major defects in total in a set of
5 pages.
The document to be reviewed is available with line numbers.
The document has been cleaned up by running any
automated checks that apply.
References needed for the inspection are stable and
available.
The document author is prepared to join the review team
and feels confident with the quality of the document.

The Review Process

1. Planning
The review will focus on a few different
things:

Focus on higher-level documents, e.g. Does the

design comply to the requirements
Focus on standards, e.g. Internal consistency,
clarity, naming conventions, templates
Focus on related documents at the same level,
e.g. Interfaces between software functions
Focus on usage, e.g. For testability and
maintainablity

The Review Process

2. Kick-Of
The review starts with a kick-off meeting, to
make sure everyone is on the same
wavelength regarding the document under
review.
The meeting consists of a short introduction to
the objectives of the review and the
documents.
Role assignments, checking rate, the pages to
be checked, process changes and possible
other questions are discussed at this meeting.

The Review Process

3. Preparation
Participants identify defects, questions,
and comments, according to their
understanding of the document and their
role.
A checking rate is decided, which is the
number of pages checked per hour,
usually about 5 to 10 pages per hour,
depending on complexity.

The Review Process

4. Review Meeting
Usually made up of the following phases:

Logging phase
Discussion phase
Decision phase

The Review Process

4. Review Meeting
Logging Phase:
The issues that have been identified in the
Preparation stage are logged.
To ensure progress and efficiency, no real
discussion is allowed during the logging phase.
Each defect is logged with a severity:
Critical: defects will cause downstream damage.
Major: defects could cause downstream damage.
Minor: defects are not likely to cause downstream
damage.

The Review Process

4. Review Meeting
Discussion Phase:
Each of the defects that require discussion
are discussed, with a chairman preventing
discussions from getting too personal.

The Review Process

4. Review Meeting
Decision Phase:
At the end of the discussion phase, a
decision is taken about the document
under review. If the number of defects
found per page exceeds a certain level,
the document may need to be reworked,
and reviewed again.

The Review Process

5. Rework
Based on the defects detects, the author
will improve the document under review,
step-by-step.

The Review Process

Specification Terminology
Checklist

Always, every, all, none, never, (absolutely sure?)

Certainly, therefore, clearly, obviously, customarily,
most, (persuasion lingo)
Some, sometimes, often, usually, ordinarily,
customarily, most, (vague)
etc., and so forth, and so on, such as, (not
testable)
Good, fast, cheap, efficient, small, stable,
(unquantifiable)
Handled, processed, rejected, skipped, eliminated
If then (missing else)

The Review Process

Roles and Responsibilities
The moderator
The author
The scribe
The reviewers
The manager

The Review Process

Roles and Responsibilities
The moderator serves as the review
leader, they determine the type of review,
approach and the composition of the
review team.
The moderator performs the entry check,
and the follow-up on the rework.
The moderator also schedules meetings,
disseminates documents, leads
discussions and stores relevant data.

The Review Process

Roles and Responsibilities
The author writes the original document
and seeks to improve the quality of the
document by working with others.

The Review Process

Roles and Responsibilities
The scribe records all of the defects
during the logging meetings.

The Review Process

Roles and Responsibilities
The reviewers (also called checkers and
inspectors) check the documents for
defects.
Reviewers are chosen to represent
different perspectives in the review.

The Review Process

Roles and Responsibilities
The manager decides on the execution of
reviews and determines whether review
process objectives have been met.

The Review Process

Types of Reviews
Walkthrough
Technical Review
Inspection

The Review Process

Types of Reviews
Walkthrough
The author guides the review team through
the document, to achieve a common
understanding and gather feedback.
This means the author does a range of
preparation, and the review team dont need
to do a detailed study before the meeting.
A walkthrough is especially useful for higherlevel documents, like requirements
specifications and architectural documents.

The Review Process

Types of Reviews
Technical Review
This approach focuses on developing a
consensus about the technical content of
the document.
Defects are found by technical experts,
who focus on the content of the document
(as opposed to considering any relevant
legislation and standards, referenced
documents, and intended readership).

The Review Process

Types of Reviews
Inspection
This approach is the most formal review type.
The document is inspected thoroughly be the
reviews before the meeting, comparing the work
product with its sources and other referenced
documents, and using rules and checklists.
The defects found are logged and any discussion
is postponed until the discussion phase.
This makes the inspection meeting a very
efficient meeting.

The Review Process

Lets do one.