By

Why Test Designing

The Quality of Testing is as good as its Test Design
Usage of formal and customized Test Specification techniques for
deriving Test Cases from the Input Documents (Test Basis) will
help in achieving the following:
Proper Coverage/Depth in testing each of the functions (/system)
Test Specifications prepared by various members in the test team
will be uniform

Test Cases will be more Manageable

1 July 2009

Understanding test design techniques

Classic distinction black-box and white box techniques
Black box - Black-box techniques (also called specification-based
techniques) are a way to derive and select test conditions or test cases
based on an analysis of the test basis documentation, whether
functional or non-functional, for a component or system without
reference to its internal structure.
White box - White-box techniques (also called structural or structurebased techniques) are based on an analysis of the internal structure of
the component or system.
Black box testing may be further classified into specification based or
experience based techniques.

1 July 2009

Categories of test design techniques

Requirement based techniques:
Models, either formal or informal, are used for the specification of
the problem to be solved, the software or its components.
From these models test cases can be derived systematically.
Structure based techniques:
Information about how the software is constructed is used to derive
the test cases, for example, code and design.
The extent of coverage of the software can be measured for
existing test cases, and further test cases can be derived
systematically to increase coverage.
Experience based techniques:
The knowledge and experience of people are used to derive the
test cases.
knowledge of testers, developers, users and other
stakeholders about the software, its usage and its
environment;
knowledge about likely defects and their distribution.

1 July 2009

Requirement based or black box techniques

Equivalence partitioning:
Inputs to the software or system are divided into groups that are
expected to exhibit similar behavior, so they are likely to be
processed in the same way. Equivalence partitions (or classes) can
be found for both valid data and invalid data, i.e. values that should
be rejected. Partitions can also be identified for outputs, internal
values, time-related values (e.g. before or after an event) and for
interface parameters (e.g. during integration testing). Tests can be
designed to cover partitions. Equivalence partitioning (EP) is
applicable at all levels of testing.
Equivalence partitioning as a technique can be used to achieve
input and output coverage. It can be applied to human input, input
via interfaces to a system, or interface parameters in integration
testing.

1 July 2009

Requirement based or black box techniques

Boundary value analysis:
Behavior at the edge of each equivalence partition is more likely to
be incorrect, so boundaries are an area where testing is likely to
yield defects. The maximum and minimum values of a partition are
its boundary values. A boundary value for a valid partition is a valid
boundary value; the boundary of an invalid partition is an invalid
boundary value. Tests can be designed to cover both valid and
invalid boundary values. When designing test cases, a value on
each boundary is chosen.
Boundary value analysis can be applied at all test levels. It is
relatively easy to apply and its defect finding capability is high;
detailed specifications are helpful. This technique is often
considered an extension of equivalence partitioning and can be
used on input by humans as well as, for example, on timing or
table boundaries. Boundary values may also be used for test data
selection.

1 July 2009

Requirement based or black box techniques

Decision table testing:
Decision tables are a good way to capture system requirements
that contain logical conditions, and to document internal system
design. They may be used to record complex business rules that a
system is to implement. The specification is analyzed, and
conditions and actions of the system are identified.
The input conditions and actions are most often stated in such a
way that they can either be true or false (Boolean). The decision
table contains the triggering conditions, often combinations of true
and false for all input conditions, and the resulting actions for each
combination of conditions. Each column of the table corresponds to
a business rule that defines a unique combination of conditions that
result in the execution of the actions associated with that rule. The
coverage standard commonly used with decision table testing is to
have at least one test per column, which typically involves covering
all combinations of triggering conditions.
The strength of decision table testing is that it creates
combinations of conditions that might not otherwise have been
exercised during testing. It may be applied to all situations when
the action of the software depends on several logical decisions.
1 July 2009

Requirement based or black box techniques

State- transition testing:
A system may exhibit a different response depending on current
conditions or previous history (its state). In this case, that aspect of
the system can be shown as a state transition diagram. It allows
the tester to view the software in terms of its states, transitions
between states, the inputs or events that trigger state changes
(transitions) and the actions which may result from those
transitions. The states of the system or object under test are
separate, identifiable and finite in number. A state table shows the
relationship between the states and inputs, and can highlight
possible transitions that are invalid. Tests can be designed to cover
a typical sequence of states, to cover every state, to exercise every
transition, to exercise specific sequences of transitions or to test
invalid transitions.
State transition testing is much used within the embedded software
industry and technical automation in general. However, the
technique is also suitable for modeling a business object having
specific states or testing screen-dialogue flows (e.g. for internet
applications or business scenarios).

1 July 2009

Requirement based or black box techniques

Use-case testing:
Tests can be specified from use cases or business scenarios. A
use case describes interactions between actors, including users
and the system, which produce a result of value to a system user.
Each use case has preconditions, which need to be met for a use
case to work successfully. Each use case terminates with postconditions, which are the observable results and final state of the
system after the use case has been completed. A use case usually
has a mainstream (i.e. most likely) scenario, and sometimes
alternative branches.
Use cases describe the process flows through a system based on
its actual likely use, so the test cases derived from use cases are
most useful in uncovering defects in the process flows during realworld use of the system. Use cases, often referred to as scenarios,
are very useful for designing acceptance tests with customer/user
participation. They also help uncover integration defects caused by
the interaction and interference of different components, which
individual component testing would not see.

1 July 2009

Structure based or white box techniques

Structure-based testing/white-box testing is based on an identified
structure of the software or system, as seen in the following examples:
Component level: the structure is that of the code itself, i.e.
statements, decisions or branches.
Integration level: the structure may be a call tree (a diagram in
which modules call other modules).
System level: the structure may be a menu structure, business
process or web page structure.

1 July 2009

Structure based or white box techniques

Statement testing and coverage
In component testing, statement coverage is the assessment of the
percentage of executable statements that have been exercised by a test
case suite. Statement testing derives test cases to execute specific
statements, normally to increase statement coverage.
Decision testing and coverage
Decision coverage, related to branch testing, is the assessment of the
percentage of decision outcomes (e.g. the True and False options of an IF
statement) that have been exercised by a test case suite. Decision testing
derives test cases to execute specific decision outcomes, normally to
increase decision coverage.
Decision testing is a form of control flow testing as it generates a specific
flow of control through the decision points. Decision coverage is stronger
than statement coverage: 100% decision coverage guarantees 100%
statement coverage, but not vice versa.
Other structure based techniques
There are stronger levels of structural coverage beyond decision coverage,
for example, condition coverage and multiple condition coverage. The
concept of coverage can also be applied at other test levels (e.g. at
integration level) where the percentage of modules, components or classes
that have been exercised by a test case suite could be expressed as module,
component or class coverage. Tool support is useful for the structural testing
of code.
1 July 2009

Illustration of Black-box techniques

Equivalent Partitioning
Boundary Value Analysis

Cause and Effect graphing

Decision Table based Testing

1 July 2009

Guidelines for equivalence classes

1.
2.

3.
4.

If an input condition specifies range,

one valid and two invalid equivalence classes are needed

If a condition requires a specific value,

then one valid and two invalid equivalence classes are

needed
If an input condition specifies a member of a set,

one valid and one invalid equivalence class are needed

If an input condition is Boolean,

one valid and one invalid class are needed

1 July 2009

Illustration for EP
EP may be best explained with an example of a function which has
the pass parameter "month" of a date. The valid range for the
month is 1 to 12, standing for January to December. This valid
range is called a partition. In this example there are two further
partitions of invalid ranges.
The first invalid partition would be <= 0 and the second invalid
partition would be >= 13.
.... -2 -1 0 1 ...................12 13 14 15 .....
--------------------|-------------------|--------------------Invalid partition I Valid partition Invalid partition II
Equivalence partitioning is no stand alone method to determine test
cases. It has to be supplemented by another method.

1 July 2009

Guidelines for BVA

Focus on the boundaries of the input
If input condition specifies a range bounded by a certain values, say, a and
b, then test cases should include
The values for a and b
The values just above and just below a and b

If an input condition specifies any number of values, test cases should be

the minimum and maximum numbers,
the values just above and just below the minimum and maximum
values

1 July 2009

Illustration for BVA

If the same example of a function which has the pass parameter
"month" of a date
Valid Class is 1<= month <=12
Invalid Class 1 is month <1
Invalid Class 2 is month >12
When compared to EP, which says select any test case within a
range and any on either side of it , in BVA the emphasis is on the
edges
1,2 and 11,12 for the edges of the Valid class
7( a middle value ) for the Valid Class
0 and 13 for the Invalid class
Experience has shown that when a test case on or just to one side
of a boundary of an equivalence class is selected, the probability of
detecting a fault increases.
1 July 2009

Cause-Effect Graphing
This method
Captures the relationships between specific combinations of
inputs (causes) and outputs (effects)
Deals with specific cases,
Avoids combinatorial explosion
Explore combinations of possible inputs
Causes/effects are represented as nodes of a cause effect graph

The graph also includes a number of intermediate nodes linking

causes and effects

1 July 2009

Guidelines to create CEG

1.
2.

3.
4.
5.
6.
7.

Study the functional requirements and divide the requirements into

workable pieces
Identify causes and effects in the specifications
i.
Causes: distinct input conditions
ii.
Effects: an output condition or a system transformations.
Assign unique number to each cause and effect
Use the semantic content of the spec and transform it into a Boolean
graph
Annotate the graph with constrains describing combinations of
causes and/or effects
Convert the graph into a limited-entry decision table
Decision table rules are converted to test cases

1 July 2009

Illustration for CEG

Consider the following set of requirements as an
example: Requirements for Calculating Car Insurance Premiums:
R00101 For females less than 65 years of age, the premium is $500
R00102 For males less than 25 years of age, the premium is $3000
R00103 For males between 25 and 64 years of age, the premium is
$1000
R00104 For anyone 65 years of age or more, the premium is $1500
Causes (input conditions)
1. Sex is Male
2. Sex is Female
3. Age is <25
4. Age is >=25 and < 65
5. Age is >= 65

Effects (output conditions)

1. Premium is $1000
2. Premium is $3000
3. Premium is $1500
4. Premium is $500

1 July 2009

Drawing the CE Graph

1 July 2009

Constraint symbols in CEG

1 July 2009

Example of O constraint

CEG #3 for the

Calculating Car Insurance
Premiums example
would be annotated with
a one and only one
constraint between
causes 1 and 2 because
the Sex cause has to be
either Male or Female
but it can never be both.

1 July 2009

Drawing the Decision Table

A decision table is consists of a number of columns (rules) that comprise all
test situations.

1 July 2009

Deriving the Test Cases

Some CEGs may result in more than one test case being created. For
example, CEG #3 resulted in columns 3 and 4 and CEG#4 resulted in
columns 5 and 6 in the decision table. The columns in the decision table are
converted into test cases.

1 July 2009

Guidelines for Decision Table Technique

The below rules can be followed for Test Case Generation
Conditions: inputs
Actions: outputs
Rules: test cases
The above conditions have the following
implications
Rules are complete (i.e., every combination of decision table values
including default combinations are inherent in the decision table)
The rules are consistent (i.e, there is not two actions for the same
combinations of conditions)

1 July 2009

Orthogonal Array Testing Technique

Orthogonal Array Testing is a statistical technique implemented by
Taguchi. It is a statistical way of testing pair-wise interactions. It
provides a uniformly distributed representative coverage of all
variable pair combinations.
It is useful in Integration testing of software components in complex
applications like E-Com.
Sufficient knowledge about the system under test combined with
the power of statistical technique, helps in providing maximum
coverage with minimum number of test cases.

1 July 2009

What is an Orthogonal Array?

Orthogonal Array is a matrix of n rows and k columns with every element being
one of the q symbols 0 to q-1.
The array has strength t if in every n*t sub matrix, the qt possible distinct rows all
appear the same number of times.
OAT is an array of values in which each column represents a variable-factor that
can take a certain set of values called levels.
Each row in an orthogonal array represents a test case.
In OAT the factors are combined pair-wise rather than representing all possible
combinations of factors and levels.
Refer to- http://support.sas.com for pre defined Orthogonal arrays for various values for
n,k,q,t.

1 July 2009

Illustration of OAT
1&2

2&3

1&3

In every 8*2 sub matrix, the 22 possible distinct rows all appear the same number of
times. Hence by OAT, we can arrive at a minimum of 4 possible test cases which is
said to provide maximum coverage.

1 July 2009

Orthogonal Array- Example

Let us consider a Pizza shop offering the following varieties of Pizza

Size

Base

Topping

Small

Thin

Capsicum

Medium

Thick

Onion

Large

Pan

Mushroom

As a part of introductory offer, Pizza shop is planning to offer free Pizzas.

But the question here is What Pizza combinations need to be offered free?

What is the optimal number of combinations that needs to be considered to

cover all varieties?
1 July 2009

Orthogonal Array- Example

The optimal number of combinations possible are:

Size

Base

Topping

Small (0)

Thin (0)

Capsicum (0)

Small (0)

Thick (1) Onion (1)

Small (0)

Pan (2)

Mushroom (2)

Medium (1)

Thin (0)

Onion (1)

Medium (1)

Thick (1) Mushroom (2)

Medium (1)

Pan (2)

Capsicum (0)

Large (2)

Thin (0)

Mushroom (2)

Large (2)

Thick (1) Capsicum (0)

Large (2)

Pan (2)

Onion (1)

Orthogonal Array (n,k,q,t) Orthogonal Array (9,3,3,2)

1 July 2009

Orthogonal Array- Practical Considerations

If in case an orthogonal array has 9 rows, 3 variables. 2 variables have 3 distinct values but the
other variable has only 2 values. In such case, the orthogonal array can be represented as :
L 9 2 31 2

In such cases, the combinations can be formed by selecting any value.

Size

Base

Topping

Small (0)

Thin (0)

Capsicum (0)

Small (0)

Thick (1)

Onion (1)

Small (0)

Any

Mushroom (2)

Medium (1)

Thin (0)

Onion (1)

Medium (1)

Thick (1)

Mushroom (2)

Medium (1)

Any

Capsicum (0)

Large (2)

Thin (0)

Mushroom (2)

Large (2)

Thick (1)

Capsicum (0)

Large (2)

Any

Onion (1)

Care needs to be taken to ensure that the combinations are practically feasible.
1 July 2009

OATS-A Statistical Optimization Technique

How OATS help overcome the challenges of Software Testing
optimization?
Helps optimize testing process to face the fast paced industry
with aggressive time-to-market
Helps to reduce Test Cycle Time
Finds maximum defects with minimal test cases
Provides maximum coverage with minimum test cases

1 July 2009

- 32
-

POC on A Leading Insurance Company in UK

Application: Administrator (Admin System for Avivas Staff Pension scheme)
Project: Goya
A quick case study has been done for a small part of Avivas Staff Pension scheme
Administrator Application that contains three independent variables ( i.e. A -Configure the
scheme , B - Issuing the policy , C - Investment calculation ) and each variable can take on
three values
(A0- Scheme Selection, A1- Start & End date , A2- Contribution to Fund).
(B0- Policy holders details, B1- Contribution, B2- Acceptance of proposal).
(C0- Investment Amount, C1- Number of units, C2-Term of retirement ).
MAPPING THE FACTORS IN OA

TC-1
TC- 2
TC- 3
TC- 4
TC- 5
TC- 6
TC- 7
TC- 8
TC- 9

Configure the scheme (A)

Scheme selection
Scheme selection
Scheme selection
Start & End date
Start & End date
Start & End date
Contribution to fund
Contribution to fund
Contribution to fund

Issuing the policy (B)

Policy holders details
Contribution
Acceptance of proposal
Policy holders details
Contribution
Acceptance of proposal
Policy holders details
Contribution
Acceptance of proposal

Investment calculation (C)

Investment Amount
Number of units
Term of retirement
Number of units
Term of retirement
Investment Amount
Term of retirement
Investment Amount
Number of units

Factor 4

To test all of the possible combinations, it would take a test set

containing 27 test cases (3 x 3 x 3 = 27) in Exhaustive Testing. In real
scenario, the Test analyst has written 26 test cases to test the above
piece of requirement. For designing each test case, the Test Analyst
had to spend 20-30 minutes.
1 July 2009

- 33
-

POC on AVIVA Project GOYA CONTD...

-RESULTS
-Using the OATS technique we can reduce the number of test cases to 9 in order
to test all the pair-wise combinations.
Three suspicious combinations for three variables that are not generated by the
OATS method are included additionally. Total no of scripts are (9+3) = 12
- TEST CASE REDUCTION
The number of test cases were reduced to 12 as against 26; Which is 46% of the
total test cases.
- EFFORT REDUCTION
The Test Case generation time was also reduced by at least 4.6 hrs. Which is also
46% of the total time,
- CYCLE TIME REDUCTION
Due to decreased number of test cases, cycle time also reduced by 46%
- INCREASE PRODUCTIVITY
The average time which require for test case preparation and execution are also
reduced which in turn increases the Productivity.

1 July 2009

- 34
-

Testing Challenges and OATS

Testing Challenges

How OATS helps!!

Too much Effort

Reduces Effort significantly

Optimal number of Test

Cases

Too many Test Cases

Optimal Coverage

Insufficient
Coverage
Too much Time and Cost

Overall 40% to 50% Effort,

Time and Cost Reduction
with Optimal Coverage

Reduces TC Time and Cost

1 July 2009

Summary-Test Case Design Techniques

Test Case designing is an important aspect for Test Coverage.
Depending on the type of testing done ,
Requirement-based Test design techniques
Code-based Test design techniques
For Functionality / Black Box Testing, Requirements are the only source
for Test Specifications.
Equivalence Class Partitioning
Boundary Value Analysis
Cause-Effect Graphing
Decision Table Technique
Orthogonal Array Technique

1 July 2009

Exercise-1
Prepare blackbox test cases on how you would test the Yahoo
webpage
Prepare whitebox test cases for this simple C program
#include <stdio.h>
#define LAST 10

int main()
{ int i, sum = 0;
for ( i = 1; i <= LAST; i++ ) {
sum += i;
} /*-for-*/
printf("sum = %d\n", sum);
return 0;
}

1 July 2009

Exercise-2
Annual Exam Mark Processing for Orlinda Pierce Memorial Hr.Sec. School

Final Grade Calculation

Enter Student Roll No

Enter Final Exam Marks

Enter Class Work Marks

Total Marks

Final Grade

The Candidate has got "Final Grade" grade in the Annual Examination

1 July 2009

Exercise 2-Contd
Functional Specification for the Final Grade Calculation screen
The Final Exam mark is out of 75 and the Class Work mark is out of 25. The Total
Mark is calculated as the sum of the Final Exam Mark and the Class Work Mark.
The Final Grade is calculated in the range A to D. The classification of Final
Grade is done as below
Total Marks greater than or equal to 70 - 'A'
Total Marks greater than or equal to 50, but less than 70 - 'B'
Total Marks greater than or equal to 30, but less than 50 - 'C'
Total Marks less than 30 - 'D'
Where a Total Mark outside its expected range is derived then a fault message
('FM') is generated.
The calculation has to be done upon entering a valid Roll No which is a 5 digit
character data with the starting range of 10000. All other inputs are passed as
integers.
Using EP, BVA and CEG derive at the Test Cases for the above Specification.

1 July 2009

Experience-Based Testing
Use of experience to design test cases
Experience can include
Domain knowledge
Knowledge of developers involved
Knowledge of typical problems
Two main types
Ad Hoc Testing
Exploratory Testing

1 July 2009

TC
S

Experience-Based Testing (contd)

Ad-hoc Testing
Informal testing
No preparation
Not repeatable
Cannot be tracked
Exploratory Testing
Informal
Actually involves test design and control
Useful when no specification is available
Notes are taken and progress tracked

1 July 2009

TC
S

The Art of Test Case Writing

Objective and Importance of a Test Case

The basic objective of writing test cases is to ensure complete test
coverage of the application.
The most extensive effort in preparing to test a software, is writing
test cases.

Gives better reliability in estimating the test effort

Improves productivity during test execution by reducing the
understanding time during execution
Writing effective test cases is a skill and that can be achieved by
experience and in-depth study of the application on which test cases
are being written.

1 July 2009

43

Importance of Test Case-Contd.

Sotware Testing has been recognised as a separate technical
disciple in the software development life cycle
A structured and process oriented approach is recommended in
every phase of the test process

Documenting the test cases prior to test execution ensures that the
tester does the homework and is prepared for the attack on the
Application Under Test
Breaking down the Test Requirements into Test Scenarios and Test
Cases would help the testers avoid missing out certain test conditions

1 July 2009

44

What is a Test Case?

It is the smallest unit of Testing

A test case is a detailed procedure that fully tests a feature or an

aspect of a feature. Whereas the test plan describes what to test, a
test case describes how to perform a particular test.
A test case has components that describes an input, action or event
and an expected response, to determine if a feature of an application
is working correctly.
Test cases must be written by a team member who thoroughly
understands the function being tested.

1 July 2009

45

What is a Good Test Case

A test case which has the following characteristics is considered to be
a good test case
Accurate -addresses the objective
Economical-Limited precise steps
Maintainable , Reusable

Maps to the logical flow of the application

Both Forward & Backward traceable-to a requirement as well as
defect

Self Explanatory and requires no help in understanding during

execution
Self Cleaning & Brief comments in case of Automation test

scripts
1 July 2009

46

Elements of a Test Case

Every test case must have the following details
Anatomy of a Test Case
Test Case ID
Requirement # / Section:
Objective: What is to be verified?
Assumptions & Prerequisites
Steps to be executed:
Test data(if any): Variables and their values
Expected result:
Status: Pass or Fail with details on Defect ID and proofs
[o/p files, printouts, screenshots(optional)]
Comments:

Any CMMi company would have defined templates and standards

to be adhered to while writing test cases.
1 July 2009

47

Language usage in Test Case writing

Use Simple and Easy-to-Understand language

Use Active voice while writing test cases For eg.

Click on OK button
Enter the data in screen1
Choose the option1
Navigate to the account Summary page.

Use words like Verify for starting any sentence in Test Case
description For eg.
Verify whether the account information gets displayed on clicking the Account
Summary menu

Use words like Must and use Future Tense for Expected Results
The application must display the account information screen
An error message must be displayed on entering special characters

1 July 2009

48

Some Tips on Test Case Writing

Don't write long test cases(limit to maximum of 10 steps)
Avoid incoherence. Follow a logical flow from one test case to
another.
Don't skip steps assuming that it is understood

Don't confuse between Action & Results For eg.

The button is pressed

Don't write test cases as paragraphs in a word document. Use

Table/Matrix format or any Spreadsheet format.
Follow a logical flow , but start writing test cases for those
requirements that are frozen and make a mark like TBD for those
unclear sections.

1 July 2009

49

Sample Condition to write a test case

User Registration
Enter the First and Last Name and Click the Submit Button
First Name

None

Second Name

None

Submit

I.

II.

First and last name fields are mandatory and are limited to 50 characters
each. The fields will not accept numerical values and the following
characters: `~!@#$%^&*()_:";'{}[]+<>?,./
Respective Error messages are thrown on NOT meeting the above conditions
III. User will be registered successfully on entering the correct data

1 July 2009

50

Thank You