Software Testing.

Test Design & Analysis Raluca Gagea

19.07.2012
Agenda

Introduction
Test Analysis & Design in the Fundamental
Test Process
Test Analysis & Design Activities -> Benefits
Test Analysis: identifying test conditions
Test Design: specifying test cases
Requirements
Requirements Based Testing (RBT)
Metrics

2
Introduction

Test Analysis and Design is the activity where general testing objectives are transformed into tangible test
conditions and test designs.

During Test Analysis and Design, we take general testing objectives identified during planning and build test
designs and test procedures (test scripts).

Test Analysis is the process of looking at something (e.g system requirement, a technical specification, the code
itself, a business process) that can be used to derive test information

Test Design is the process of identifying the associated high level test cases for a test item

At its core, testing is the process of comparing "what is" with "what ought to be" '.

[Copeland, 2003]

IN YOUR ZONE 3
Test Analysis & Design in the
Fundamental Test Process

We can divide the activities within the fundamental test process into the following basic steps:

Test Planning and Control

Test Analysis and Design

Test Implementation and Execution

Evaluating Exit Criteria and Reporting

Test Closure Activities

IN YOUR ZONE 4
Test Analysis & Design
Activities -> Benefits

Review the test basis (such as the product risk analysis, requirements, architecture, design specifications and
interfaces), examining the specifications for the software we are testing identify gaps and ambiguities in
the specifications because we are trying to identify precisely what happens at each point in the system and this
also prevents defects appearing in the code.
Identify the test conditions based on analysis of test items, their specifications and what we know about their
behavior and structure gives us a high-level list of what we are interested in testing, we can start to
identify the type of generic test data we might need.
Define the test cases and test procedures (design the tests using techniques to help select representative tests
that relate to particular aspects of the software which carry risks or which are of particular interest, based on
the test conditions and going into more detail) the high risks areas will be covered by tests before the
actual execution phase starts
Evaluate testability of the requirements and system at execution moment, all the requirements are
translated in terms of testable items
Design the test environment set-up and identify any required infrastructure and tools at execution
moment, everything we need to carry out our work is in place

IN YOUR ZONE 5
Test Analysis: identifying test conditions

A test condition is simply something that we could test

When identifying test conditions, we want to throw the net wide to identify as many as we can, and then we
will start being selective about which ones to take forward to develop in more detail and combine into test
cases

A test technique is a process that helps us select a good set of tests from the total number of all possible tests
for a given system

Test conditions should be able to be linked back to their sources in the test basis this is called traceability

Event: The requirements for a given

function or feature were changed.
Consequence: Some of the fields could
have different ranges that can be
entered.
Problem: Which tests were looking at
those boundaries? They now need to be
changed. How many tests will actually be
affected by this change in the
requirements?
Solution: Have requirements that can
easily be traced to the tests.
Event: A set of tests that has run ok
in the past has started to have
serious problems.
Problem: What functionality do
these tests actually exercise?
Solution: Have a traceability
between the tests and the
requirement being tested
Event: Delivering a new release
Consequence: We want to know whether
or not we have tested all of the specified
requirements specification.
Problem: We have the list of the tests
that have passed was every
requirements tested?
Solution: Have requirements that can
easily be traced to the tests.

IN YOUR ZONE 6
Test Analysis: identifying test conditions

After identifying the test conditions, they should be prioritized

Test conditions can be identified for test data as well as for inputs and test outcomes

Test conditions are documented in the IEEE 829 document called a Test Design Specification

IN YOUR ZONE 7
Test Design: specifying test cases
A test case is a set of input values, execution preconditions, expected results and execution post conditions,
developed for a particular objective or test condition, such as to exercise a particular program path or to verify
compliance with a specific requirement.

Test cases are designed by the stepwise elaboration and refinement of the identified test conditions using test
techniques identified in the test strategy. They should be repeatable, verifiable and traceable back to
requirements.
Test case design includes the identification of:
the preconditions such as either project or localized test environment requirements and the plans for
their delivery
the test data requirements
the expected results and post conditions
The expected results include information displayed on a screen in response to an input, but they also include
changes to data and/or states, and any other consequences of the test.

Ideally, the expected results should be predicted before the test is run then your assessment of whether or
not the software did the right thing will be more objective.

IN YOUR ZONE 8
Test Design: specifying test cases
Test cases are documented in the IEEE 829 document called a Test Case Specification

IN YOUR ZONE 9
 Requirements Based Testing (RBT)
Deliver what you promised

IN YOUR ZONE 10
Requirements

A requirement is a singular documented physical and functional need that a particular product or service must be or
perform.
Requirements are also an important input into the verification process, since tests should trace back to specific
requirements. Requirements show what elements and functions are necessary for the particular project
Coverage of requirements is a fundamental need throughout the software life cycle. Be it during design, coding or
testing, the ability to ensure that the software meets the expected requirements is something that every developer and
project manager aspires for
Techniques and tools have been invented to target requirement coverage. Some tools offer the capability of mapping
requirements to the other project artifacts, such as, use cases, test cases and design documents. Some projects prefer
to follow simple spreadsheet based traceability matrix of requirements to various other artifacts
Types of requirements:
Architectural requirements
Structural requirements
Design requirements
Functional requirements
Non functional requirements
Constraint requirements

IN YOUR ZONE 11
Requirements Based Testing (RBT)
The Requirements-Based Testing (RBT) process addresses two major issues:

1) validating that the requirements are correct, complete, unambiguous, and logically consistent

2) designing a necessary and sufficient (from a black box perspective) set of test cases from those
requirements to ensure that the design and code fully meet those requirements.

In designing tests two issues need to be overcome:

1) reducing the immensely large number of potential tests down a reasonable size set and ensuring
that the tests got the right answer for the right reason

2) the RBT process does not assume, going in, that we will see Good Requirements

The overall RBT strategy is to integrate testing throughout the development life cycle and focus on the
quality of the Requirements Specification. This leads to early defect detection which has been shown to be
much less expensive than finding defects during integration testing or later

IN YOUR ZONE 12
Requirements Based Testing (RBT)
To put the RBT process into perspective, testing can be divided into the following eight activities:
1) Define Test Completion Criteria.
The test effort has specific, quantitative and qualitative goals. Testing is completed only when the goals have been
reached
2) Design Test Cases.
Logical test cases are defined by five characteristics:
- the initial state of the system prior to executing the test
- the data in the data base
- the inputs
- the expected outputs
- the final system state
3) Build Test Cases.
There are two parts needed to build test cases from logical test cases: creating the necessary data and building the
components to support testing (e.g., build the navigation to get to the portion of the program being tested).
4) Execute Tests.
Execute the test case steps against the system being tested and document the results.
5) Verify Test Results.
Verify that the test results are as expected.
6) Verify Test Coverage.
Track the amount of functional coverage and code coverage achieved by the successful execution of the set of tests.
7) Manage and Track Defects.
Any defects detected during the testing process are tracked to resolution. Statistics are maintained concerning the
overall defect trends and status.
8) Manage the Test Library.
The test manager maintains the relationships between the test cases and the programs being tested. The test
manager keeps track of what tests have or have not been executed, and whether the executed tests have passed or failed

IN YOUR ZONE 13
Requirements Based Testing (RBT)
The top three reasons why software projects fail are:

- Requirements and specifications are incomplete

- Requirements and specifications change too often

- There is a lack of user input (to requirements)

The RBT process addresses each of these issues:

- It begins at the first phase of software development where the correction of errors is the least
costly

- It begins at the requirements phase where the largest portion of defects have their root case

- It addresses improving the quality of requirements: Inadequate requirements often are the reason
for failing projects

IN YOUR ZONE 14
Metrics

Metrics to monitor test analysis and design may include:

Percentage of requirements or quality (product) risks covered by test conditions

Percentage of test conditions covered by test cases
Number of defects found during test analysis and design
We can track test analysis and design tasks against a work breakdown structure, which is useful in determining
whether we are proceeding according to the estimate and schedule.

IN YOUR ZONE 15
 Raluca Gagea| Acting Test Lead
Raluca.gagea@endava.com
Tel +40 (0)746 938 736|Skype en_rgagea

thank you

IN YOUR ZONE 16