Textbook SQA+ (Liggesmeyer) P2

POSTGRADUALE FERNSTUDIENGÄNGE
SCIENCE & ENGINEERING
TEXTBOOK V-M.1.1 SOFTWARE ENGINEERING FOR EMBEDDED SYSTEMS
SOFTWARE QUALITY
ASSURANCE
AUTHOR
PROF. DR.-ING. PETER LIGGESMEYER
This work is protected by copyright. All rights thus conferred, particularly the rights to copy and
disseminate as well as the rights to translate and reprint the entire work or parts thereof, are re-
served. Beyond the permissions regulated by copyright, no part of this work may be reproduced
in any way (print, photocopy, microfilm, or any other process) nor processed, copied, or dissem-
inated using electronic systems without the written permission of the University of Kaiserslau-
tern.
4th edition 2017

Table of Contents i
Table of Contents
Glossary v
About the Author xi
References xiii
Objectives of the Textbook xvii
1 Introduction 1
1.1 Learning objectives of this chapter 1
1.2 Introduction 1
1.3 Motivation 2
1.4 Definition of terms 4
1.5 Software quality assurance 8
1.6 Classification of testing, analysis, and verification techniques 10
1.6.1 Dynamic testing 12
1.6.2 Static analysis 13
1.6.3 Formal techniques: symbolic testing and formal verification 14
1.7 Problems for Chapter 1 14
2 Function–Oriented Testing 17
2.2 Introduction 17
2.3 Characteristics and aims of function–oriented tests 17
2.4 Functional equivalence class construction 19
2.4.1 Characteristics and objectives of functional equivalence class
construction 19
2.4.2 Building equivalence classes 19
2.4.3 Evaluating functional equivalence class construction 24
2.5 State–based testing 25
2.5.1 Characteristics and aims of state–based testing 25
2.5.2 Description of state–based testing 25
2.5.3 Evaluation of state–based testing 27
2.6 Other function–oriented testing techniques 28
2.6.1 Transaction flow testing 28
2.6.2 Testing on the basis of decision tables or decision trees 29
2.7 Evaluation of function–oriented testing 30
3 Control Flow Testing 33
3.2 Introduction 33
3.3 Characteristics and objectives of control–flow–oriented testing 34
3.4 Statement coverage test 35
3.4.1 Characteristics and objectives of the statement coverage test 35
3.4.2 Description of the statement coverage test 35
ii Table of Contents
3.4.3 Evaluation of the statement coverage test 36

3.5 Branch coverage test 36
3.5.1 Characteristics and objectives of the branch coverage test 36
3.5.2 Description of the branch coverage test 36
3.5.3 Problems of the branch coverage test 37
3.5.4 Evaluation of the branch coverage test 38
3.6 Condition coverage test 39
3.6.1 Characteristics and objectives of the condition coverage test 39
3.6.2 Simple condition coverage 40
3.6.3 Condition/decision coverage test 43
3.6.4 Minimal multiple condition coverage test 44
3.6.5 Modified condition/decision coverage test 45
3.6.6 Multiple condition coverage test 46
3.6.7 Evaluation of condition coverage 49
3.7 Techniques for testing loops 49
3.7.1 Characteristics and objectives 49
3.7.2 Structured path test and boundary interior path test 50
3.7.2.1 Description 50
3.8 Path coverage test 52
3.8.1 Characteristics and objectives of the path coverage test 52
3.8.2 Evaluation of the path coverage test 52
3.9 Evaluation of control flow testing techniques 52
4 Data Flow Testing 55
4.2 Introduction 55
4.3 Characteristics and objectives of data–flow–oriented tests 55
4.4 Defs/uses test 57
4.5 Evaluation of data flow testing techniques 61
5 Tool–Supported Static Code Analysis 63
5.2 Introduction 63
5.3 Characteristics and objectives of tool–supported static code analysis 63
5.4 Style analysis 65
5.4.1 Characteristics and objectives of style analysis 65
5.4.2 Programming conventions 65
5.4.3 Evaluation of style analysis 67
5.5 Slicing 67
5.5.1 Characteristics and objectives of slicing 67
5.5.2 Evaluation of slicing 71
5.6 Data flow anomaly analysis 71
5.6.1 Characteristics and objectives of data flow anomaly analysis 71
5.6.2 Performing data flow anomaly analysis 71
Table of Contents iii
5.6.3 Possible problems in data flow anomaly analysis 75

5.6.4 Evaluation of data flow anomaly analysis 78
5.6.5 Evaluation of tool–supported static code analysis 79
6 Software Inspections and Reviews 81
6.2 Introduction 81
6.3 Characteristics and objectives of software inspections and reviews 82
6.4 Formal inspections 84
6.4.1 Characteristics and objectives of formal inspections 84
6.4.2 Description of the formal inspection 84
6.5 Informal review techniques with review meetings: structured
walkthrough 91
6.6 Informal review techniques without review meetings 92
6.7 Evaluation of software inspections and reviews 92
Solutions to the Problems 93
Index 103
iv Table of Contents
Glossary v
Glossary
All c–uses criterion: This criterion requires the execution of at least one
definition–clear path with respect to x from ni to every element of dcu(x, ni) for
every node ni and every variable x that is an element of def(ni).
All c–uses/some p–uses criterion: This criterion is fulfilled if for every node ni
and every variable x that is an element of def(ni), a definition–clear path with
respect to x from ni to every element in dcu (x, ni) is tested, or if at least one
definition–clear path is tested with respect to x from ni to one element of dpu(x,
ni) if dcu (x, ni) is empty.
All–def criterion: This criterion requires a number of test paths so that for every
node ni and for every variable x that is an element of def(ni), there is at least one
definition–clear path with respect to x from ni to an element of dcu(x, ni) or dpu(x,
ni). The criterion requires that the test cases should be such that for every
definition (all–defs) of all the variables, at least one c–use or one p–use is tested.
All du–paths criterion: For this criterion to be fulfilled, the requirement of the
all–uses criterion has to be extended so that all the du–paths are tested with
respect to the definitions of all variables.
All p–uses criterion: This criterion is fulfilled if for every node ni and every
variable x that is an element of def(ni), a definition–clear path is contained in the
tested paths with respect to x from ni to all elements of dpu(x, ni). Every
combination of every variable definition with its p–use should be tested.
All p–uses/some c–uses criterion: This criterion requires a definition–clear path
from every definition to every predicate use, and if it is not available, demands
that a corresponding path to a computational use be tested.
All–uses criterion: This criterion subsumes the all p–uses/some c–uses criteria
and all c–uses/some p–uses criteria. It requires the testing of all definitions in
combination with all reachable p–uses and c–uses.
Author: The creator of the product to be inspected who is responsible for
correcting the errors found during inspection.
Availability: The measure of the ability of an entity to be functional at a specified
point in time. It is expressed through the probability that the entity performs the
required function at a defined point in time and under defined operating
conditions.
Backward slicing: A method for generating a slice containing instructions that
influence an observed variable.
Boundary interior testing: An approach to testing where it is assumed that a
complete set of tests must test alternative paths through the top level of a program,
alternative paths through loops, and alternative boundary tests of loops.
vi Glossary
Branch coverage test: A stricter testing technique than the statement coverage
test, whose objective is to execute all branches of the program to be tested, i.e.,
the run–through of all edges of the control flow graph.
Condition coverage test: A testing technique that handles the logical structure of
the conditions in the software.
Condition/decision coverage test: A testing technique that explicitly demands
branch coverage to be established in addition to simple condition coverage.
Control flow graph: A graph in which each node in the graph represents a basic
block, i.e. a straight–line piece of code without any jumps or jump targets; jump
targets start a block, jumps end a block, and directed edges are used to represent
jumps in the control flow.
Control flow testing techniques: Testing techniques are based on the control
structure or the control flow of the software to be tested.
Correctness: The degree of consistency between specification and program, or
the degree by which the program satisfies the customer’s expectations.
Data flow anomalies: Incorrect data flows through code in which the input data
is processed to determine interim results that are written into the memory to then
be reread and to finally be converted into output values.
Data flow attributed control flow graph: The control flow graph extended to
the data flow attributes (def, c–use, and p–use) is called data flow attributed
control flow graph.
Data flow tests: Test techniques in which test completeness is evaluated with
respect to coverage of data accesses.
dcu(x, ni): For a node ni and a variable x where x is an element of def(ni), it is the
set of all nodes nj that contain c–use(x) and for which a definition–clear path
exists with respect to x from node ni to nj.
Decision tables: Tables that represent the logic model of a problem to be solved
by software used to derive test cases.
Decision trees: The logic model of a problem to be solved by software represen-
ted in the form of a tree that is used to derive test cases.
def(ni): The set of the globally defined variables for every node ni of the control
flow graph.
dpu(x, ni): The set of all edges (nj, nk) for which x contains p–use(x) and for
which there exists a definition–clear path with respect to x from ni to (nj, nk).
du–path: A path p = (ni, …nj, nk) with a global definition of x in ni for which the
following holds true:
• p is definition–clear with respect to x, and ni contains a c–use of x and all
nodes ni, …nk are distinct or ni, …nj are distinct and ni = nk
Glossary vii
or
• p′ = (ni, … nj) is definition–clear with respect to x, and (ni, nk) contains a
p–use of x, and all nodes ni,… nj are distinct.
Dynamic testing: Testing techniques that can be used to generate test cases that
are representative, error–sensitive, have low redundancy, are economical, and can
be used to execute software, by providing it with concrete input values.
Equivalence class testing: A method of testing where the complexity of a
problem is reduced through continuous division in such a manner that the result
are equivalence classes from which elementary test cases can be constructed. All
the values of an equivalence class are processed similarly by the software.
Error: The wrong understanding of a specific programming language construct
which may cause a faulty command in the program and which, during the
program’s execution, may exhibit a deviation from the expected behavior or cause
a failure.
Failure: A malfunction that occurs dynamically during product usage.
Fault: A cause for a malfunction or failure that is statically contained in the
program code.
Formal inspection methods: Inspections that are carried out during exactly
defined phases with defined input and output criteria, a defined inspection rate,
and predefined goals to be achieved by different inspectors.
Forward slicing: A method for generating a slice that indicates which
instructions are influenced by a specific variable and how this influence takes
place.
Functional equivalence class testing: Technique that derives test cases from
equivalence classes of input and output values based on the specification of the
system being tested.
Functional testing: Testing techniques that test against specifications describing
the desired software functionality.
Informal review techniques without review meetings: Techniques that require
no review meetings and form the lower end of the scale in manual analysis
methods with respect to efficiency and effectiveness, but require very little effort.
Minimal multiple condition coverage test: A testing technique demanding that
along with the atomic partial decisions and overall decisions, all composite
decisions should be tested against the values true and false.
Moderator: A specialist with training as a moderator of inspections whose main
job is to create synergy within the inspection team.
Modified condition/decision coverage test: A testing technique that requires test
cases to demonstrate that every atomic partial decision can influence the logical
value of the overall decision, independent of the other partial decisions.
viii Glossary
Multiple condition coverage test: A testing technique that demands the test of all
logical combinations of the atomic partial decisions.
Path coverage test: A comprehensive control flow test technique that requires the
execution of all the different paths of a software module.
Principle of Integrated Quality Assurance: A method for quality assurance in
which, for a software development process structured into phases, constructive
methods are applied in each phase, and at the end of each phase, the quality of the
intermediate product is evaluated using analytical methods.
Quality characteristic: Properties of a functional unit, on the basis of which the
quality is described and evaluated without indicating values.
Quality measure: Measures that permit inference on specific values that help to
check whether specification–related quality characteristics have been met.
Quality requirement: A requirement or a set of requirements, which specifies
one or several quality charasteristics of a component or a system.
Quality target specification: A document in which the quality requirements are
defined.
Reader: A person who guides the inspection team through the meeting by reading
out the technical content, explaining it at the same time.
Recording clerk: A person who writes down and classifies all errors found
during inspection and also assists the moderator in preparing the inspection
reports.
Reliability: This is a collective term used to describe performance with respect to
availability, maintainability, and maintenance support, which can be defined as
the quality of an entity with respect to its ability to satisfy its reliability
requirements during or after pre–defined time spans under pre–defined application
conditions.
Robustness: The capability of an entity to function in a defined manner and
produce acceptable reactions under unusual operating conditions.
Safety: The state in which the risk of personal or material damage is within
tolerable limits or a measure for the ability of an item to not endanger people,
property, or the environment.
Simple condition coverage: A testing technique that requires the test of all
atomic component decisions against true and false.
Slicing: A method of testing that generates software slices, which are pieces of
software that can be examined for detecting errors.
Software Quality Assurance: Techniques for achieving a desired level of quality
for software applications.
State–based testing: Test techniques that use state charts to generate test cases.
Glossary ix
Statement coverage test: The simplest control flow test method, whose purpose
is a minimal one–time execution of all statements of the program, i.e., coverage of
all nodes of the control flow graph.
Static analysis: A testing technique where the software to be tested is not
executed and which can, in principle, be performed manually, but which is, in
practice, done using a tool, except in review techniques.
Static code analysis: An approach for testing software without executing it.
Structured testing: A testing approach in which all paths through a functional
module that require less than or equal to k (usually k >=2) iterations of loops are
tested at least once.
Structured walkthrough: This is an informal review technique with review
meetings where no formal procedure and no defined roles for team participants
exist. These reviews have lower efficiency and effectiveness than formal
inspections but the time consumed, the cost, and the use of resources are also less
than those for formal inspection methods.
Structure–oriented testing techniques: Dynamic testing techniques that assess
the completeness of testing using coverage of the source code.
Style analysis: A set of rules that programmers have to use when coding, also
called “programming conventions”.
Transaction flow testing: Test techniques that use a transaction flow represented
by flow diagrams, sequence diagrams, or message sequence charts to generate test
cases.
x Glossary
About the Author xi
About the Author

Prof. Dr.–Ing. habil. Peter Liggesmeyer
Since 2015, Prof. Dr.–Ing. habil. Peter Liggesmeyer has been the Executive
Director of the Fraunhofer Institute for Experimental Software Engineering IESE
in Kaiserslautern. Since 2004, he has been holding the chair of Software
Engineering: Dependability in the Department of Computer Science at the
University of Kaiserslautern, and since 2014 he has been managing the affairs of
the Gesellschaft für Informatik (GI e.V., German Informatics Society) as its
President; since the beginning of 2016 in his second term. He is the founder of the
Fraunhofer Alliance Embedded Systems, whose speaker he was from 2010 to
2013. From 2004 until 2014, Prof. Liggesmeyer was the Scientific Director of
Fraunhofer IESE.
Prof. Liggesmeyer studied Electrical Engineering with an emphasis on Data
Technology at the University of Paderborn and obtained his doctorate with honors
from Ruhr University Bochum in 1992, where he also completed his habilitation
thesis on “Quality Assurance of Software–Intensive Technical Systems” in 2000.
From 2000 to 2004, he was a C4 Professor for Software Engineering and Quality
Management at the Institute of Computer Science at the University of Potsdam
(HPI).
Further stages of his career included setting up a competence center on “Safety
Analysis and Risk Management” in the Central Department of Research and
Development of Siemens AG, Munich (1993 to 2000). From 1988 to 1993, he was
a researcher at the Chair of Software Technology in the Department of Electrical
Engineering at Ruhr University Bochum, where he held regular teaching
assignments from 1993 to 2000. Other teaching assignments took him to the
Technical Universities of Munich and Ilmenau, and to Friedrich Schiller
University Jena. He turned down the offer of a professorship at the Technical
University of Ilmenau in 1999.
Prior to being elected President of the Gesellschaft für Informatik, Prof.
Liggesmeyer had already been its vice–president and long–time spokesman of
both the expert group “Software Technology” and the Division of Software
Technology and Information Systems. In the latter capacity, he was a member of
the Advisory Board. Prof. Liggesmeyer is a co–founder of the Working Group
“Test, Analysis, and Verification of Software” of the GI. He is also a member of
the IEEE Computer Society as well as of the Münchener Kreis. Prof. Liggesmeyer
has received several scientific awards, including the Software Engineering Award
of the Ernst Denert Foundation in 1993. In the context of the Science Year 2014,
he was chairman of the jury that selected “Germany’s Digital Minds” on behalf of
the German Ministry of Education and Research (BMBF).
xii About the Author
Prof. Liggesmeyer is the author of numerous scientific articles and popular books,
particularly the standard reference work “Software Quality” (2002, 2nd edition
2009). He is one of the six authors of the National Roadmap Embedded Systems
(NRMES). He also participates regularly in many national and international
program committees. He has been co–editor of various professional journals,
including “Informatik–Spektrum”, “Informatik Forschung & Entwicklung” (both
Springer–Verlag), “information technology” (Oldenbourg–Verlag), and “Lecture
Notes in Informatics” (GI).
Prof. Liggesmeyer is a member of the scientific steering board of SafeTRANS,
supervisory board member of Schloss Dagstuhl, a member of the scientific
advisory board for the platform “Industry 4.0”, and advisor of the cluster project
“fast”. He is also a member of the executive board of the German Chapter of the
ACM as well as a member of the expert advisory board of the quality seal
“Software made in Germany” and a board member of the ZukunftsRegion
Westpfalz e.V. (Western Palatinate Regional Association). Prof. Liggesmeyer
provides consulting to several institutions on the state and federal level. He is a
member of the platform “Security, Protection and Trust for the Society and the
Economy” of the Federal Ministry of the Interior as well as a member of the
platform “Digitization in Education and Science” of the Federal Ministry of
Education and Research in the German National IT Summit. Furthermore, Prof.
Liggesmeyer is active on the advisory board of the “Alliance for Cyber Security”
of the Federal Office for Information Security. Since 2016, he has been a member
of the State Council for Digital Development and Culture of the State of
Rhineland–Palatinate. In 2015, he was also the chairman of the Commission of
Experts for the Bavarian Center for Digitization (ZD.B). From 2011 to 2014, he
was a member of the University Council at the Darmstadt University of Applied
Sciences by order of the Hessian Ministry for Science and Arts. Furthermore, he
was a liaison officer of the German National Academic Foundation (2009 –
2014).
For more than 25 years, Prof. Liggesmeyer has been in frequent demand as a
lecturer for industry seminars and has been advising many leading companies in
matters of technology issues. He leads several strategic projects at the University
of Kaiserslautern that are funded by the Federal Ministry of Education and
Research (BMBF) and by the EU. He is a member of the Center for Digital
Commercial Vehicle Technology (ZNT) and of the research initiative “amsys”.
His research interests are safety and reliability analysis techniques for cyber–
physical systems and comprehensive security and safety analysis processes for
smart ecosystems, primarily in the application fields of digital commercial vehicle
technology, Industry 4.0, and “Smart Rural Areas”. Prof. Liggesmeyer is co–
author of several patents.
References xiii
References
References for Chapter 1
DeMarco, T.: Structured Analysis and System Specification. Englewood Cliffs:
Prentice Hall, 1985
Booch, G., Rumbaugh, J., Jacobson, I.: The Unified Modeling Language User
Guide. Reading: Addison–Wesley, 1998
Informationstechnik; Bewertung von Software–Produkten; Qualitätsmerkmale
und Richtlinien für deren Anwendung. Berlin: Beuth Verlag, December 1991
IEEE Standard Glossary of Software Test Documentation. IEEE, New York 1983
Qualitätsmanagement – Begriffe. (ISO 8402: 1994); Dreisprachige Fassung EN
ISO 8402: 1995, Berlin: Beuth Verlag, August 1995
Birolini, A.: Zuverlässigkeit von Geräten und Systemen. Berlin, Heidelberg, New
York: Springer, 1997
Normen zu Qualitätsmanagement und zur Darlegung von Qualitätsmanagement-
systemen; Leitfaden zum Management von Zuverlässigkeitsprogrammen. Berlin:
Beuth Verlag, June 1994
Liggesmeyer, P., Rothfelder, M.: System Safety Improvement by Automated Soft-
ware Robustness Evaluation. Proceedings 15th International Conference on
Testing Computer Software, Washington, D.C., 71–77, June 1998
Leveson, N. G.: Safeware: System safety and computers. New York: Addison–
Wesley, 1995
Communications of the ACM, Vol. 33, No. 12, December 1990
Leveson, N. G., Turner, C. L.: An Investigation of the Therac–25 Accidents. IEEE
Computer, 26(7), 18–41, July 1993
Hohlfeld, B.: Zur Verifikation von modular zerlegten Programmen. Dissertation,
Fachbereich Informatik, Universität Kaiserslautern, 1988
Lions, J.L.: ARIANE 5 Flight 501 Failure, Report by the Inquiry Board. Paris,
July 1996
Musa, J. D., Iannino, A., Okumoto, K.: Software Reliability: Measurement,
Prediction, Application. New York: McGraw–Hill, 118, 1987
Jones, C.: Applied software measurement. New York: McGraw–Hill, 142–143,
1991
Boehm, B.: Software engineering economics. Englewood Cliffs: Prentice Hall,
534, 1981
xiv References
Möller, K.–H.: Ausgangsdaten für Qualitätsmetriken – Eine Fundgrube für

Analysen in: Ebert C., Dumke R. (Hrsg.), Software–Metriken in der Praxis,
Berlin, Heidelberg: Springer, 105–116, 1996
Liggesmeyer, P.: Modultest und Modulverifikation – State of the Art. Mannheim,
Wien, Zürich: BI Wissenschaftsverlag, 1990
Fagan, M. E.: Design and code inspections to reduce errors in program
development. IBM Systems Journal, (3), 182–211, 1976
Fagan, M. E.: Advances in Software Inspections. IEEE Transactions of Software
Engineering, Vol. SE–12(7), 744–751, July 1986
Gilb, T., Graham, D.: Software Inspection. Reading: Addison–Wesley, 1993

Myers, G. J.: The Art of Software–Testing. New York: John Wiley & Sons, 1979
Beizer, B.: Software Testing Techniques. New York: Van Nostrand Reinhold,
1990

Girgis, M. R., Woodward, M. R.: An Experimental Comparison of the Error
Exposing Ability of Program Testing Criteria. Proceedings Workshop on
Software Testing, Banff, 64–73, July 1986
Software Considerations in Airborne Systems and Equipment Certification.
RTCA, Inc., 1992
Howden, W. E.: An Evaluation of the Effectiveness of Symbolic Testing.
Software – Practice and Experience, 8, 381–397, 1978
Howden, W. E.: Theoretical and Empirical Studies of Program Testing. IEEE
Transactions on Software Engineering, Vol. SE–4(4), 293–298, July 1978
Howden, W. E.: Theoretical and Empirical Studies of Program Testing.
Proceedings of the 3rd International Conference on Software Engineering,
Atlanta, 235–243, May 1978
Gannon, C.: Error Detection Using Path Testing and Static Analysis. Computer,
12(8), 26–31, August 1979
Gannon, C.: Results for Static Analysis and Path Testing of Small Programs.
General Research Corporation RM–2225, September 1978
Riedemann, E. H.: Testmethoden für sequentielle und nebenläufige Software–
Systeme. Stuttgart: Teubner, 1997
Howden, W. E.: Methodology for the Generation of Program Test Data. IEEE
Transactions on Computers, Vol. C–24, 554–560, May 1975
References xv
Tai, K. C.: Program Testing Complexity and Test Criteria. IEEE Transactions on
Software Engineering, Vol. SE–6(6), 531–538, November 1980
Howden, W. E.: A Survey of dynamic Analysis Methods. Tutorial: Software
Testing and Validation Techniques, New York: IEEE Computer Society Press,
209–231, 1981

Bieman, J. M., Schultz, J. L.: Estimating the Number of Test Cases Required to
Satisfy the All–du–paths Testing Criterion. Proceedings of the ACM SIGSOFT
’89 Third Symposium on Software Testing, Analysis, and Verification (TAV3),
Key West, 179 – 186, December 1989

Korel, B., Laski, J.: Dynamic Program Slicing. Information Processing Letters,
29(3), 155–163, October 1988
Korel, B.: The Program Dependence Graph in Static Program Testing.
Information Processing Letters, 24(2), 103–108, January 1987
Taylor, R. N., Osterweil, L. J.: Anomaly Detection in Concurrent Software by
Static Data Flow Analysis. IEEE Transactions on Software Engineering, Vol. SE–
6(3), 265–278, May 1980
Wilson, C., Osterweil, L.: Omega – A Data Flow Analysis Tool for the C
Programming Language. IEEE Transactions on Software Engineering, Vol. SE–
11(9), 832–838, September 1985

Möller, K.–H.: Ausgangsdaten für Qualitätsmetriken – Eine Fundgrube für
Analysen, in: Ebert C., Dumke R. (Hrsg.), Software–Metriken in der Praxis,
Berlin, Heidelberg: Springer, 105–116, 1996.
Fagan, M. E.: Design and code inspections to reduce errors in program
development. IBM Systems Journal, (3), 182–211, 1976
Fagan, M. E.: Advances in Software Inspections. IEEE Transactions of Software
Engineering, Vol. SE–12(7), 744–751, July 1986
Gilb, T., Graham, D.: Software Inspection. Reading: Addison–Wesley, 1993
Thaler, M., Utesch, M.: Effektivität und Effizienz von Softwareinspektionen, in:
Müllerburg et al. (Hrsg.), Test, Analyse und Verifikation von Software, GMD–
Bericht Nr. 260, München, Wien: Oldenbourg, 183–196, 1996
xvi References
Ebert, C.: Improving the Validation Process for a Better Field Quality in a Product
Line Architecture. Proceedings Informatik 2000, Berlin, Heidelberg, New York:
Springer, 372–388, 2000
Russel, G. W.: Experience with Inspection in Ultralarge–Scale Developments.
IEEE Software, 8(1), 25–31, January/February 1991
Schnurrer, K. E.: Programminspektionen: Erfahrungen und Probleme. Informatik–
Spektrum, Band 11, 312–322, 1988
Kosman, R. J., Restivo, T. J.: Incorporating the Inspection Process into a Software
Maintenance Organization. Proceedings Conference Software Maintenance,
October 1992
Yourdon, E., Structured Walkthroughs (4th ed.). Englewood Cliffs: Prentice Hall,
335–347, 1989
Objectives of the Textbook xvii
Objectives of the Textbook

After reading this textbook, you will
• have an overview of the existing testing methodologies and how they have
been used to test software;
• be able to understand and evaluate the various dynamic and static testing
techniques;
• be able to apply the knowledge gained to decide which testing technique
best suits the software at hand to be tested.
xviii Objectives of the Textbook
Chapter 1 – Introduction 1
1 Introduction
1.1 Learning objectives of this chapter

After reading this chapter, you will be able to
• understand why software quality assurance is important;
• understand the “Principle of Integrated Quality Assurance”;
• define the following terms with respect to software systems:
a) quality
b) quality requirement
c) quality measure
d) failure
e) fault
f) error
g) correctness
h) safety
i) reliability
j) availability
k) robustness
• classify various software testing techniques;
• differentiate between dynamic testing and static analysis.
1.2 Introduction
Every organization that develops software endeavors to deliver the best quality.
One can only attain a goal when it is precisely defined, which is not the case for
the term ‘the best quality’. There are many ways to define software quality. Many
attributes of software contribute towards its overall quality. Not all of these
attributes are equally important to the end user and to the manufacturer. Certain
attributes that are absolutely essential for a particular software product may be
totally irrelevant for another product. Certain attributes in the system have
negative interdependencies. The argument that one wants to implement the best
quality demonstrates that this fundamental fact is not understood. The aim is not
software development with the best, but with the right quality. This demands that
quality requirements be defined in a so–called quality target specification.
Subsequently, measures aimed at attaining the specified quality can be
determined. As a rule, a combination of techniques and organizational means is
2 Chapter 1 – Introduction
required, comprising both construction techniques and analytical examination. It

is important to be economical, i.e., to keep in mind the time frame as well as cost
effectiveness. All of this, as well as the diversity of different software products
that generate the corresponding demand for quality requirements, inevitably
means that there is no universally applicable solution.
1.3 Motivation
With the growing penetration of computers in many application areas, assuring
reliability and correctness of software functionality has gained more importance.
Cost development exhibits a clear rise in software costs in comparison to
hardware costs, even though the life expectancy of software is significantly
longer. Cost–cutting in the area of software development would therefore be
especially economical. Analyzing the costs of software development depending
on the phase of the software development lifecycle, leads to the following
conclusion: By far the greatest proportion of the expenditure arises during the
maintenance phase, after introduction to the market. This is a result of inadequate
software quality caused by errors originating from the software development
phases that are first discovered during field use. If, in addition, a software product
is badly structured, inadequately documented, and difficult to understand, the
correction of errors becomes a time–consuming venture. Likewise, the growing
complexity of software products contributes towards this tendency. The
localization of errors and their elimination is difficult. In particular, in
inadequately structured software, error correction in one part of the system may
result in further errors due to interdependencies with another part of the system. If
the error is caused during the early phases of software development, for example
in the specification of requirements or during the conceptual phase, extensive
modifications often become necessary. This causes high maintenance costs; at the
same time, employees are tied up who are required elsewhere.
There are also methods for building software that is reliable, understandable, and
easy to modify. These methods are either constructive or analytical. In practice,
functional decomposition and object–oriented software development methods are
widely used. Examples are Structured Analysis (SA) /DeMarco 85/, Structured
Design (SD), and the Unified Modeling Language (UML) /Booch et al. 98/.
However, no constructive procedure can guarantee error–free products. For this
reason, the quality needs to be tested with analytical methods. If the software
Principle of development process is structured into phases, constructive methods can be
Integrated applied in each phase, and at the end of each phase, the quality of the intermediate
Quality Assurance
products can be evaluated with analytical methods (Fig. 1.1). This is the so–called
“Principle of Integrated Quality Assurance”. An intermediate product whose
quality is inadequate would not be forwarded to the next phase until a defined,
satisfactory quality level has been attained. This procedure facilitates early
detection of errors and enables their correction at a time when this is possible with
minimal expense. The aim is to detect as many errors as possible during the tests
that immediately follow the development step where the error originated. The aim
is to minimize the number of errors persisting over several phases. The quality
assurance process starts with the first intermediate product in order to detect
quality deviations early and facilitate actions to eliminate them. Merely testing a
‘complete’ program is not enough. There is an interdependency between
constructive and analytical quality assurance. Ideally, each constructive step
should be followed by an analytical one.
The early definition of quality requirements is of crucial importance. This quality
target specification requires defining the desired target values of so–called quality
characteristics at an early stage of the development process. Software testing
occupies a central position in software development. Every programmer would
run newly created software and make a few inputs in order to examine the
generated outputs and compare them with the expected results. He would also
read the printout of the program code to spot bugs. Basically, the first procedure is
the application of an unsystematic dynamic testing procedure and the second
approach resembles manual execution of an unsystematic statistical analysis. On
the one hand, testing methods – in their simpler form – are very common. On the
other hand, the various techniques as well as their systematic and their productive
efficiency are widely unknown. The objective of testing must be the generation of
reproducible results applying a clearly defined procedure. Unfortunately, in
normal practice, this is often not done. In this course, techniques and methods for
analytical software quality assurance will be discussed, with a focus on testing
techniques. Alongside dynamic tests, automated and manual statistical analyses
will be introduced.
Analysis Errors Analysis Analysis Errors

- UML
- Testing of UML tools

- Inspection of UML-Analysis-
documents not
successful
successful
Design
Design Errors - UML Design Errors
- Testing of UML tools

- Inspection of UML-Design-
documents not
successful
successful
Coding
Programming Errors - C++
- Compiler syntax testing

not - Module test
Figure 1.1:
successful
The principle of
integrated quality successful
assurance
1.4 Definition of terms

A few basic, important terms are defined below. The subsequent chapters contain
numerous further definitions:
• Quality, quality requirement, quality parameter, quality measure
• Failure, fault, error
• Correctness
• Safety
• Reliability
• Availability
• Robustness
Quality:
The term ‘quality’ is defined in /DIN EN ISO 9000:2005/ as the degree to which a Early
set of inherent characteristics fulfills requirements. commitment to
quality is
Quality Requirement: important
The term ‘quality requirement’ is the need or expectation that is stated, generally
implied or obligatory. It specifies one or several quality charasteristics of a
component or a system.
Quality Characteristic:
The concrete specification and evaluation of quality is done by using so–called
quality characteristics. They represent properties of a functional unit on the basis
of which the quality is described and evaluated without indicating values. A
quality characteristic can be detailed over several steps by sub–characteristics.
There are various proposals for quality characteristics that are based on
publications by various authors. Examples of these characteristics are security,
reliability, availability, robustness, memory and runtime efficiency, flexibility,
portability, testability, and usability. While quality characteristics such as
flexibility, portability, and testability are important to the manufacturer, factors
such as security, reliability, availability, memory, and runtime efficiency as well
as usability are important to the end user. For this reason, they are relevant for the
manufacturer in the interest of customer satisfaction. The ISO/IEC standards 9126
/ISO IEC 9126 91/ and 25010 /ISO IEC 25010 11/ describe several quality Quality
characteristics. Normally, techniques used to assess the quality characteristic characteristics
can have
‘correctness’ are distinguished from those used to verify other quality reciprocal effects
characteristics such as safety or reliability. In some publications, one can find a
differentiation between functional and non–functional characteristics.
Interdependencies and correlations can exist between different quality
characteristics. Safety and availability are frequently antipodal optimization
targets. Safe systems are created in such a manner that component failures can be
identified, e.g. through redundancy, and the system resolves itself into a safe state.
The normal functionality of the system is not provided in this state, which reduces
its availability. Safe systems, e.g., dual channel or 2–by–2 systems, generally have
lower availability in comparison to a single–channel structure. Software can be
optimized either in terms of memory efficiency, i.e., for minimum memory
demand, or in terms of runtime efficiency, i.e., for minimum runtime demand.
Often, an improvement of one of the quality characteristics results in a
deterioration of the other. Therefore, it does not make sense to demand optimum
quality in every respect. This is principally impossible because of the
interdependencies among the various quality characteristics. The specification of
target values for individual quality characteristics needs to be performed at an
early stage of the development process. This is done by carrying out a so–called
quality target definition.
Quality Measure:
The concrete commitment to specifications of a quality characteristic occurs
through quality measures. These are measures that permit inference on specific
values. For example, MTTF (Mean Time to Failure) can be used as a measure of
the quality characteristic reliability.
Failure:
IMPORTANT: A malfunction or failure occurs dynamically during product usage. During
The terms dynamic testing of software, one does not detect faults but a malfunction or
failures, faults,
and errors may be failure. These are a result of faults in the program. In hardware, failures may result
defined differently from wear and tear.
in some
publications Fault:
In software, a defect or fault constitutes a cause for malfunctions or failures:
Faults are statically contained in the program code. In hardware, too, design flaws
are possible (e.g., under–dimensioning of the cooling of a power semiconductor).
Error:
An error can be the result of a wrong understanding of a particular programming
language construct. Errors, faults, and failures are often related to each another.
Likewise, an error – the wrong understanding of a specific programming language
construct – may cause a faulty command in the program, which during its
execution then exhibits a different result than desired, namely a failure. The term
“error” is often used as a synonym for fault.
Correctness:
An IEEE Proposal /ANSI 729 83/ defines correctness as the degree of consistency
between specification and program, or as the degree by which the program
satisfies the customer’s expectations. Such definitions are geared towards
practice. They are not optimally suited for defining criteria for testing correctness
because their definition is already vague. As an example, the degree of customer
satisfaction depends to a large extent on the concrete user group surveyed. An
occasional user of a system will typically have requirements that are different
from those of a professional/frequent user. Among experts, the following
definition of the term correctness has been established:
Correctness is a • Correctness possesses no gradual character, i.e., software is either correct
binary property or not correct.
• Error–free software is correct.
• Software is correct if it is consistent with its specifications.
• In the absence of a specification, no verification of correctness is possible.
Since with this definition, a program is regarded as incorrect if there is only a
minimal deviation from the specification, one may assume that there is no correct
program above a certain level of complexity. On the other hand, it is normal in
practice to classify errors as tolerable and non–tolerable. Different error priorities

for applications can be defined, for example, the program is considered correct if
it no longer contains any errors at the highest level of priority.
Safety and security:
The standard DIN EN ISO 8402 /DIN EN ISO 8402 95/ defines safety as the state Safety ≠ Absence
in which the risk of personal or material damage is within tolerable limits. of risks
/Birolini 97/ defines safety as a measure for the ability of an item to endanger
neither persons, nor property, nor the environment. One can distinguish the safety
of a failure–free system (accident prevention) from the technical safety of a
failure–prone system. Besides, there is the quality characteristic ‘security’. It
includes the attribute of a system to prevent loss of information and unauthorized
data access.
Reliability:
DIN EN ISO 8402 /DIN EN ISO 8402 95/ defines reliability as a collective term Dependability ≠
for describing performance with respect to availability, maintainability, and Reliability
maintenance support. In DIN ISO 9000 Part 4 /DIN ISO 9000–4 94/, reliability is
defined as the quality of an entity with respect to its ability to satisfy its reliability
requirements during or after pre–defined time spans under pre–defined application
conditions. Reliability in this comprehensive sense is referred to as dependability.
/Birolini 97/ uses a restricted definition of reliability. Here, it is the measure of the
ability of an entity to remain functional, expressed through the probability that a
required function can be performed failure–free under defined operating
conditions for a specified period of time. This definition allows defining reliability
with the help of stochastic techniques. Thus, the expected value of the time span
until system failure – the so–called Mean Time to Failure (MTTF) – is specified
as the measure of reliability. In repairable systems, one uses the expected value
between two consecutive failures – the so–called Mean Time between Failures
(MTBF). Alternatively, the probability of survival R(t) can be used. This is the
probability that the system will still be running at time t after it was started at time
zero. In the following, reliability will be used in this narrower sense. Reliability
thus describes the probability of a failure–free functionality of an entity over a
specified time span and under specified operating conditions. It can be described
with stochastic methods. Reliability in general is not constant. In software, it
grows over the course of error corrections and increases in stability, and decreases
through new errors introduced when functionality is extended. For hardware
components, wear and tear require attention. The observed reliability deviates
further through the way a system is used. It is often possible to generate reliability
predictions using statistical procedures.
Availability:
Availability is a measure of the ability of an entity to be functional at a specified
point in time. It is expressed through the probability that the entity will perform
the required function at a defined point in time and under defined operating
conditions.
Robustness:
Robustness is Robustness means that an entity is capable of functioning in a defined manner and
gradual producing acceptable reactions under unusual operating conditions /Liggesmeyer,
Rothfelder 98/. An entity that, per specification, is correct may therefore still have
low robustness. According to the above definition, a system is correct if it
correctly performs all the functions described in its specification. If there are
situations that are not included in the specification, the system may fail, for
example. Even then it is considered correct. Such cases are typically unusual
operating situations, e.g., faulty or contradictory input data. Robustness is actually
a property of the specification rather than one of the implementation. A robust
system is the result of the correct implementation of a specification that specifies
a defined response even in abnormal situations. Since one cannot include every
conceivable case while setting up the specification, robustness also has a gradual
character. It is important to note that increasing robustness requires additional
development effort, which arises from the creation of specifications, realization,
and testing. As a result of this, it makes sense to set the required degree of
robustness early in the development by creating appropriate specifications.
1.5 Software quality assurance

Software quality assurance provides techniques for achieving the desired level of
application of quality characteristics of software systems. Software quality
assurance and hardware quality assurance are similar to some extent, but have
evolved relatively separately from each other for a long time. Software quality and
quality in general have been refined into various quality characteristics (for
example, reliability, memory and runtime efficiency, maintainability, etc.).
Software increasingly penetrates critical application areas, such as medical
systems and transportation technology. At the same time, there is a considerable
increase in size and complexity, which complicates the quality assurance.
Failures Already, there exist reports on countless system failures pertaining to software
concerning (see /Leveson 95/). In the USA and Canada, a software–related failure by the
software
radiation device THERAC 25 resulted in patients being killed /CACM 90/,
/Leveson, Turner 93/. In an early version of the passenger aircraft B737 it was
observed that the propulsion control unit would sometimes fail while starting at
exactly 60 knots /Hohlfeld 88/. The cause of this failure was a programming error
because no control instructions existed for the exact speed of 60 knots. Even
today, errors cannot be ruled out with certainty. The crash of the European launch
rocket Ariane 5 on its test flight in the mid–1990s was caused by software
components that had been implemented in Ariane 4, and their integration with the
new environment had not been tested satisfactorily /Lions 96/. The first flight of
the new European launch rocket Ariane 5 (ESA) took place on 4 June, 1996 in
Kourou. The rocket was equipped with cluster satellites. About 37 seconds after
ignition, Ariane 5 attained a horizontal speed of more than 32768 internal units.
This value, which was converted into a signed integer variable in Ada, caused an
overflow that was not handled. There was hardware redundancy, but the second
channel ran the same software and thus experienced the same problem. As a
consequence of a complete breakdown of the onboard navigation, diagnosis data
was sent to the host computer, which interpreted it as flight data. Absurd control
commands were given. The rocket threatened to break apart and exploded. The
software had been reused from Ariane 4 and had worked there without any
problems.
Empirical data show that software errors are not exceptions. /Musa et al. 87, p. Empirical data on
118/ found that the average software at the beginning of module testing contains software errors
and their
approx. 19.7 errors per thousand lines of source code. At the beginning of system dependency on
testing, there exist about six errors per thousand lines of source code in the software measures
product. According to this research, released software contains about 1.5 errors are contained in
/Fenton, Ohlsson
per thousand lines of source code. At first, this appears to be a low value. It is 00/
clear, however, that today, large software products consist of a few million lines
of source code. Such a product can thus contain thousands of errors.
Figure 1.2:
Relative share of
development
expenditure
Jones /Jones 91, pp. 142–143/ published data that shows a relatively clear increase
in the expenditure for analytical quality assurance, depending on the total
development expenditure of a project (Fig. 1.2). According to these data, in very
small projects, software coding represents the largest share of the expenditure. In
the largest project presented by Jones /Jones 91/, analytical quality assurance
accounted for 37% of the total development expenditure, while coding only
required 12%.
A renowned research work done by Boehm /Boehm 81, p.534/ arrives at the result
that the ratio between maintenance and development expenditure of software
projects is between 2:1 and 1:1. Because maintenance basically aims at
eliminating every error that was not detected by the time of the product’s release,
it makes sense that maintenance expenditure should be included in analytical
quality assurance. It is clear that in many cases, the largest share of development
expenses is spent on quality assurance.
Relative
count of 10% 40% 50%
emerging
errors
Relative 25%
count of 3% 5% 7% 50% 10%
detected
errors
Cost per
error 250 250 250 1000 3000 12500
Figure 1.3: correction
Error count and (€)
error correction Analysis Design Coding Module System Field
costs test test use
From the cost perspective, early quality assurance is especially important. In a

study carried out by Möller /Möller 96/, which published error cost data, error
correction in analysis, design, and coding amounts to €250 per error (Fig. 1.3). In
module testing, the correction costs are as high as €1000. The correction of an
error detected during system testing costs around €3000. Errors that are
discovered during product usage generate correction costs of €12,500. This nearly
exponential increase in the correction costs calls for early quality assurance due to
economic reasons.
1.6 Classification of testing, analysis, and verification

techniques
Gross In /Liggesmeyer 90/, a classification scheme is proposed for software testing
classification of techniques. In Fig. 1.4, a revised version of this proposal is shown. The revised
testing techniques
classification scheme proposes two classes – static and dynamic techniques.
Symbolic testing techniques and formal validation procedures, which are used to
form their own classes in /Liggesmeyer 90/, have now been grouped as
verification techniques, as a sub–class of static techniques. Actually, all
verification techniques, except for so–called exhaustive tests, are static
approaches. Exhaustive tests require the testing of software with all possible
inputs under all possible operating conditions. Such tests have no practical
relevance and are therefore not shown in Fig. 1.4. Dynamic testing techniques
have high practical significance. However, this does not apply for each testing
technique shown in Fig. 1.4. Particularly important are the functional testing
technique and a few of the control flow testing techniques. In the class of
diversified techniques, the practical relevance of regression tests is especially
high.
static verifiable formal Assertion techniques

symbolic Algebraic techniques
Automaton-based techniques
analytical Measurements
Style analysis
Graphics and Tables
Slicing
Data flow anomaly analysis
Audit and Formal and informal
Review techniques session techniques
Comment techniques
dynamic structure-oriented
control flow-oriented Statement coverage test

Branch coverage test
Condition coverage test
Simple condition coverage test
Condition/decision coverage test
Minimum multiple condition
coverage test
Modified condition/decision
coverage test
Multiple condition coverage test
Structured path test and boundary
Interior path test
LCSAJ based test
Path coverage test
data flow-oriented Defs-/Uses-criterion

all defs-criterion
all p-uses-criterion
all c-uses-criterion
all p-uses/some c-uses criterion
all c-uses/some p-uses criterion
all uses-criterion
all du-paths-criterion
Required k-tuples test
Data context coverage
function-oriented Functional equivalence class construction

State-based test
Cause effect analysis
Syntax test
Transaction flow based test
Test on the basis of decision tables
diversifying Back to back test

Mutation test
Regression test
Range test Path domain test

Test of fault revealing sub-domains
Partition analysis
Figure 1.4:
Statistical test
Error guessing Classification of
Boundary value analysis software testing
Assertion techniques techniques
1.6.1 Dynamic testing

The simplest form of a dynamic test is the execution of the software with inputs
that are generated by the tester in an intuitive and usually not reproducible
manner. Unlike the techniques to be discussed in the following, this is an
unsystematic, ad–hoc approach. Since not all possible data can be selected as test
cases, dynamic testing techniques unfortunately provide an incomplete view of
the tested software. Dijkstra summarized this fact in his statement that testing can
Characteristics of demonstrate the presence of errors, but certainly not their absence. Dynamic
dynamic testing
testing holds an enormous practical significance because of its universal
techniques
applicability. All dynamic tests comprise the following common characteristics:
• The compiled, executable software is provided with concrete input values
and is executed.
• The software can be tested in its actual operating environment.
• Dynamic testing techniques are sampling procedures.
• Dynamic testing techniques cannot prove the correctness of the tested
software.
Dynamic testing The objective of selecting appropriate test cases has given rise to a multitude of
can unfortunately dynamic testing techniques. A statement about the correct or incorrect functioning
not rule out
remaining errors of software can only be made for the selected test data. The correct functioning of
the software for all inputs can only be guaranteed by testing all possible inputs.
This complete testing of all inputs is called exhaustive testing. As a rule,
performing exhaustive tests is not possible for real programs. The objective of
dynamic testing techniques is the generation of test cases that are representative,
error–sensitive, have low redundancy, and are thus economical.
Dynamic testing techniques define different rules for the design and evaluation of
test cases. Testing techniques can be classified, evaluated, and compared to one
White–box tests another using various criteria. Widely used is the differentiation between white–
versus black–box box tests and black–box tests. Contrary to black–box techniques, white–box
tests
techniques utilize the structure of the program code. The division into white–box
and black–box techniques must be considered too coarse against the background
of today’s state of the art. According to these criteria, testing techniques that, after
all, are widely different can be assigned to the same class. Another method of
classification is ranking according to the utilized test reference. Testing
techniques are assigned to a common class if they use identical test references. A
testing technique tests against a test reference. The test reference is used to
Classification evaluate the completeness of the test. Structure–oriented testing techniques
according to the evaluate the completeness of the test by considering the coverage of the structure
test reference
utilized elements in the program code. Statements, branches, or data accesses can be used.
All structural testing techniques are therefore white–box tests. The correctness of
the results is evaluated against the specification. The principle is shown in Fig.
1.5. Functional testing techniques evaluate the completeness of the test as well as
the correctness of the test results against the specification. Functional testing
techniques are black–box techniques, even though not all black–box techniques
are functional. Functional testing and black–box testing are not synonyms. The
back–to–back test, for example, is a black–box technique that is not function–
oriented. By diversifying the testing techniques used, the correctness of the test
results can be evaluated by comparing the results from several versions of the
Function–oriented
software. test ≠ black–box
test
Test cases
Specification
nstart
n1
Interpretation
Completeness n2
n3
Image of the
specification n4
n5
n6
Figure 1.5:
Tester Evaluate nfinal Principle of
outputs structure–oriented
tests
The classification of dynamic tests according to Fig. 1.4 is essentially based on

the test reference criterion. Among the structural testing techniques, the sub–
classes ‘control flow test’ and ‘data flow test’ are formed. The control flow test
techniques aim at covering control structure elements of the software under test.
This class includes well–known testing techniques such as the statement coverage
test and the branch coverage test. The data flow testing techniques define rules for
the complete coverage of data access operations in the software to be tested. Write
and read data access has to be tested in a particular manner according to the
technique used. Functional testing techniques specify rules for implementing test
cases based on the specification.
1.6.2 Static analysis

All static analyses share the following common properties:
• There is no execution of the software to be tested.
• A static analysis can, in principle, be performed without computer support. Features of static
analysis
• No test cases need to be selected.
• Correctness or reliability cannot be verified completely.
Even though a static analysis can be performed manually, tool support is

advisable and should be used in all cases except review techniques.
The use of measurements facilitates quantitative control of the software
development and allows making precise statements about the product attributes.
Especially in the case of programming languages with a liberal syntax, the usage
of constructs that may impair the quality can be governed by means of a style
analysis. The application of programming conventions is required by several
standards for the development of safety–critical software using certain
programming languages such as C. Graphs and tables can be statically generated
from the program code. Slicing permits the identification of cause–effect
relationships between the instructions of software. It investigates the impact of
statements on variables and vice versa and can particularly support debugging.
Data flow anomaly analysis allows automatic and guaranteed identification of
certain errors – the so–called data flow anomalies. These cannot be discovered
with other methods. Therefore, data flow anomaly analysis has highly practical
significance. Review techniques are different from the other static analyses, for
they need to be implemented without any tool support. Empirical examinations
show that formal inspection techniques, for example the so–called Fagan
inspection /Fagan 76, 86/, /Gilb, Graham 93/, are very efficient, but expensive.
1.6.3 Formal techniques: symbolic testing and formal

verification
With the help of formal methods, formal verification procedures attempt to prove
consistency between software and its specification. Therefore, a formal
specification is essential. In practice, the acceptance of formal techniques is low.
These techniques will not be discussed in this course.
1.7 Problems for Chapter 1

Problem 1.1: Problem 1.1:
Reliability and
availability Explain the difference between “reliability” and “availability”.

Failure, fault, error
What is meant by the terms “failure”, “fault”, and “error”?

Your opinion
Give your opinion about the following statements:
true false
Correctness has a binary character.  
Even if there are no defects,  

the program might not be correct.
An artifact is not consistent with  

its specification if it is not correct.
Robustness has a binary character.  
A correct system can have low robustness.  
Robustness is a property only of the  

implementation.
Chapter 2 – Function–Oriented Testing 17
2 Function–Oriented Testing

• develop test cases using important function–oriented techniques such as
a) equivalence class partioning,
b) state based testing,
c) transaction flow based testing;
• understand what decision tables/decision trees are and use them in testing;
• evaluate each of the functional testing techniques described in this chapter.
2.2 Introduction
In the following, dynamic tests will be described, which are used to assess the
completeness of a test by determining the degree to which the specification is
covered by the test cases. As specifications describe the desired software
functionality, testing techniques that test against specifications are called
function–oriented testing techniques. Function–oriented testing techniques are
absolutely essential. Standards always demand meticulous function–oriented test
planning. This is a direct requirement for the use of systematic function–oriented
testing techniques. Function–oriented tests take place in all test phases. In unit
testing, module specifications are considered to be the basis for the test. In
integration tests, interface specifications form the test reference. In system tests,
the test is carried out against the requirements definition. A test without systematic
function–oriented test planning is to be regarded as being inadequate. This chapter
introduces important function–oriented testing techniques that are used to develop
test cases from specifications.
2.3 Characteristics and aims of function–oriented tests

Function–oriented testing techniques place software specifications in the center of
test events. Therefore, one must ensure that a specification is available and used
with respect to every test. Fig. 2.1 shows the principle of function–oriented tests.
The tester reads the specifications and gets an idea of its contents. This is then
translated into test cases with the help of specific guidelines. The
implementation of a test case results in software responses. The correctness of this
response is evaluated according to the image of its specifications. The test has
reached its end if the image of the specification is completely covered by the test
case. The specification is the central point. Function–oriented tests are aimed at
evaluating the completeness of the test, in other words, at deriving test cases and
18 Chapter 2 – Function–Oriented Testing
evaluating the software responses. Because of their derivation from the

specification, the test cases of a function–oriented test are organized
systematically for an audit of the desired functionality. An audit is carried out to
examine the test cases that are organized systematically to test for a particular
functionality. Therefore, it can be tested whether the specification has been
translated completely into the software. A successfully implemented function–
oriented test imparts increased trust in the quality of the software. However, there
are also disadvantages along with the advantages. A function–oriented test cannot
guarantee that the program code is completely covered. If a simple function–
oriented test is used, the software may still contain untested code.
Test cases
Specification
Completeness
Interpretation
Software
Image of the
specification
Figure 2.1:
Principle of Tester Evaluate
function–oriented outputs
tests
The concrete implementation of the software is irrelevant in function–oriented

tests. Therefore, function–oriented testing techniques belong to the so–called
black–box tests. Black–box tests and function–oriented tests are not synonyms.
All function–oriented testing techniques are black–box techniques, but not all
black–box techniques are function–oriented. Function–oriented testing enforces
the generation of test cases from specifications. The term black–box testing means
that the software code is not considered. As function–oriented testing techniques
generate test cases based on the specification and use the specification to evaluate
completeness, they do not need to consider the code.
2.4 Functional equivalence class construction
2.4.1 Characteristics and objectives of functional equivalence

class construction
One of the main problems with dynamic software testing is choosing a few test
cases from an often large number of possible operating situations. It is difficult to
select a few test cases for the purpose of execution from a large number of
potential test cases. Choosing only a few test cases means that the test assumes
that a large number of operating situations need not be tested. Therefore, the
selection of these test cases must be done very thoroughly. One must pay attention
that the selected test cases represent an appropriate substitution for basic
generality. This should also be achieved in such a manner that potential errors
should be detected as reliably as possible. Furthermore, the test cases should be
free of redundancy. Functional equivalence class construction uses a principle that
has proven to be useful in many situations to conquer complexity: the principle of
divide and conquer. Functional equivalence class construction attempts to reduce
the complexity of a problem through continuous division in such a manner that it
results in a very elementary choice of test cases. At the end of this division, there
are a number of these so–called equivalence classes. All the values of an
equivalence class should be processed similarly by the software. Therefore, it is
assumed that every value of an equivalence class is a representative substitution
for all values in the equivalence class. As the equivalence classes are identified
considering the specification, this is called functional equivalence class
construction.
2.4.2 Building equivalence classes

The fundamental approach for identifying functional equivalence classes is based
on continued case differentiation. This takes place for both input and output
conditions of the software under test. Values from one equivalence class shall
functionally perform in an identical manner. They particularly test the same
specified program function. On the one hand, the formation of equivalence classes
ensures that all specified program functions are tested with the values of their
equivalence class. On the other hand, this leads to a limitation of the number of
test cases. Input equivalence classes are directly created for input conditions. If
the specification, for example, only permits positive values for a certain input, two
cases can be differentiated: positive input values and non–positive input values.
• Specified input range of values: 1 ≤ Value ≤ 99 Example 2.1:
Input values
• One valid equivalence class: 1 ≤ Value ≤ 99
• Two invalid equivalence classes: Value < 1 and Value > 99
Positive input values represent the normal case. One speaks of so–called valid
equivalence classes. Non–positive inputs are not permitted. If they occur, an error
exists requiring an error handling routine. These are called invalid equivalence
classes. Equivalence classes for output conditions are formed likewise by means
of case differentiation based on the specification. Once the output equivalence
classes are available, the input values leading to outputs in the respective output
equivalence classes must be determined. The selection of concrete test data from
an equivalence class can be made according to different criteria. A frequently
applied approach is the equivalence class boundary test. This heuristic process is
known as boundary value analysis. It is based on the experience that errors occur
more frequently on the boundary of equivalence classes. Another possible
approach is the test of particular values, for example, the value 0, or a stochastic
procedure. The creation of valid and invalid equivalence classes can already be
found in /Myers 79/. If an input provides for a range of values, this range
constitutes a valid equivalence class that may have to be segmented or limited at
its upper and lower end by introducing invalid equivalence classes.
In detail, the following rules are used to form equivalence classes /Myers 79/:
• If an input condition specifies a value range, one valid and two invalid
equivalence classes are to be formed (see the above example).
Example 2.2: For example, between one and six owner(s) can be registered for one car.
Car, car owners
One valid equivalent class:
• 1–6 owner(s).
Two invalid equivalence classes:
• No owner.
• More than 6 owners.
If an input condition specifies a quantity of values that are to be processed
differently, a separate valid equivalence class has to be formed for each value.
For all other values except the valid ones, one invalid equivalence class is to be
formed.
Example 2.3: Keyboard instruments: piano, harpsichord, spinet, organ
Musical
instruments Four valid equivalence classes:
• Piano
• Harpsichord
• Spinet
• Organ
One invalid equivalence class:
• All others, e.g., violin
If elements of an equivalence class can be further classified into sub–categories
that need to be processed differently, then the equivalence class needs to be
partitioned accordingly. For example, the elements of the equivalence class piano
can be further differentiated as electronic or acoustic piano, which need to be

processed differently, so we must create two equivalence classes:
• Acoustic piano
• Electronic piano
If an input condition stipulates a situation that must be necessarily fulfilled, one
valid and one invalid equivalence class are to be formed.
The first character must be a letter. Example 2.4:
First character
One valid equivalence class:
• The first character is a letter.
One invalid equivalence class:
• The first character is not a letter (e.g., a numeral or special character)
These rules for the construction of input equivalence classes can be translated into
the construction of output equivalence classes. If an output condition specifies a
value domain, then all input values that generate output values inside the
specified output domain can be grouped into a valid equivalence class. All input
values that result in output values below the specified value domain are classified
into an invalid equivalence class. Similarly, all input values that result in output
values above the specified value domain are classified into another invalid
equivalence class.
Output domain: 1 ≤ Value ≤ 99 Example 2.5:
Output domain
Valid equivalence class:
• All inputs that generate output values between 1 and 99.
Two invalid equivalence classes:
• All inputs that generate output values less than 1.
• All inputs that generate output values greater than 99.
The construction of valid and invalid equivalence classes is advisable for
programs that receive their inputs from an input/output channel because error
handling routines must exist for invalid equivalence classes. In order to support
operations that receive their input from parameterized interfaces, limitations with
respect to their inputs should be observed. It is possible that the calling program
already implements certain error handling routines. In this case, additional error
handling in the called operation would cause non–executable code.
The equivalence classes must be numbered clearly. For the generation of test
cases from the equivalence classes, the following two rules apply:
• Test cases for valid equivalence classes are constructed by selecting test
data from as many valid equivalence classes as possible. This reduces the
number of test cases for valid equivalence classes to a minimum.
• Test cases for invalid equivalence classes are constructed by selecting test
data from an invalid equivalence class. These are combined with values
that are selected from valid equivalence classes. Because an error handling
routine must exist for all invalid input values, the input of one erroneous
value per test case tests only the corresponding error handling routine. If
many erroneous data per test case are input, it cannot be determined which
erroneous data has invoked the error handling routine.
Example 2.6: An inventory control program in a shop contains an input possibility for the
Inventory control registration of deliveries. For example, when wooden boards are delivered, the
program
wood type is entered. The program recognizes the wood types oak, beech, and
pine. Furthermore, the length – with a value between 100 and 500 – is entered in
centimeters. An input signifies the number of delivered items, which must be
between 1 and 9999. In addition to this, each delivery maintains an order number.
Every order number for the delivery of wood begins with the letter W.
With the above description and the application of the given conditions, the
equivalence class is constructed as shown in Tab. 2.1.
Table 2.1: Input Valid equivalence classes Invalid equivalence classes

Equivalence
classes Type of wood 1) Oak 4) Everything else, i.e., steel
2) Beech
3) Pine
Length 5) 100 ≤ length ≤ 500 6) Length < 100
7) Length > 500
Number of 8) 1 ≤ number ≤ 9999 9) number < 1

delivered items
10) number > 9999
Order number 11) First character is W 12) First character is not W
Tab. 2.2 represents a complete set of test cases. The test data is chosen from an
equivalence class without the application of a concrete strategy. The resulting test
suite consists of nine test cases. Three test cases are required to cover all valid
equivalence classes. Six test cases are needed to cover the invalid classes. Each
invalid equivalence class is covered separately by a test case. The invalid values
of the parameters are marked gray in the table.
Test cases with valid Test cases with invalid values Table 2.2:
values Test cases
according to
equivalence class
Test case 1 2 3 4 5 6 7 8 9
analysis
(additionally) 1) 2) 3) 4) 6) 7) 9) 10) 12)

tested
5)
equivalence
classes 8)
11)
Type of Oak Beech Pine Steel Beech Beech Beech Beech Beech
wood
Length 200 300 300 300 50 1000 200 200 200
Number of 100 200 100 100 100 100 0 50000 100

delivered
items
Order W1 W1 W1 W1 W1 W1 W1 W1 J2
number
The test cases represented in Tab. 2.3 emerge from the joint application of
equivalence class analysis and boundary value analysis. The boundary values of
the equivalence classes are chosen as test data. The abbreviation L or U after the
description of the tested equivalence class denotes a test of the lower or upper
boundaries of the specified equivalence classes, respectively. The resulting test
suite consists of three test cases with valid boundary values and six test cases for
covering the invalid boundary values. Each boundary value of an invalid
equivalence class is covered separately by a test case. The invalid values of the
parameters are marked gray in the table.
Table 2.3: Test cases with Test cases with invalid values
Test cases valid values
according to
equivalence class
Test case 1 2 3 4 5 6 7 8 9
analysis and
boundary value
analysis (additionally) 1) 2) 3) 4) 6) U 7) L 9) U 10) L 12)
tested
5) 5) U
equivalence
L
classes
8) 8) U
L
11)
Type of Oak Beech Pine Steel Beech Beech Beech Beech Beech
wood
Length 100 500 300 300 99 501 200 200 200
Number of 1 9999 100 100 100 100 0 10000 100

delivered
items
Order W1 W1 W1 W1 W1 W1 W1 W1 J2
number
2.4.3 Evaluating functional equivalence class construction

Functional equivalence class construction is a universally applicable technique.
The application of this procedure is straightforward. It is expected that numerous
software testers who have not worked with concrete methods until now should
already have used similar procedures implicitly. In many cases, implementation of
a functional equivalence class construction in the above–mentioned procedure
only systemizes a technique that is already in place in a simpler form.
The above–mentioned procedure provides tabulated planning and filing of test
cases. Hence, it is expected that in the event of a change in test responsibility,
training should be easier. It is possible to optimize the testing process as
construction of equivalence classes enables faults to be discovered that otherwise
might go undetected and might show up at a later point in time. A few
disadvantages exist along with the advantages. Functional equivalence class
construction does not allow describing the interaction between equivalence
classes. If software responses are bound by a particular connection of equivalence
classes, this cannot be described in a suitable manner by functional class
construction. Functional equivalence class construction is also not appropriate for
testing state–based software.
In summary: Functional equivalence class construction is suitable for software

testing that is not state–based and whose response is not dependent on
complicated input patterns.
2.5 State–based testing
2.5.1 Characteristics and aims of state–based testing

State charts are a widely used description method in technical disciplines. Modern
analysis and design techniques for software, e.g. UML (Unified Modeling
Language), offer state charts as description methods. In UML, state charts can be
used to describe the behavior of classes. State–based testing uses state charts to
define test cases.
2.5.2 Description of state–based testing

State testing aims at the complete test coverage of state charts. The following
diagram specifies a connection setup and tear–down of a telephone call.
Transitions are depicted as straight edges between two states. The event that
causes the state is written before a slash. The action assigned to the transition is
written after the slash. The arrow emerging from the right of the state
“Disconnected” marks the state as the initial state.
Timeout
occurred
Timeout/reset
dialed number Hang up
Hang up/reset
Digit 0, digit 1,
dialed number
…, digit 9/Add
digit to dialed Discon-
Dialing
number, validate nected
dialed number Unhook/reset
dialed number
Dialed number Hang up/reset

invalid dialed number,
reset connection
Dialed number
Timeout/reset
valid/establish
dialed number
connection
Invalid
Connected
number
Figure 2.2:
Hang up/reset State chart for the
dialed number telephone example
State charts offer a direct basis for generating test cases. The simplest of all state–
based test completeness criteria calls for covering all states at least once with a
test case. According to Fig. 2.2, this strategy does not guarantee a run–through of
all transitions. It is therefore possible to visit, for instance, all states without
testing the transition from the state “Dialing” to the state “Disconnected”, which
occurs when the receiver is hung up. Therefore, testing all transitions is a more
thorough test criterion than only testing all states. Fig. 2.2 shows that a run–
through of all transitions guarantees that all states are visited at least once. The
test of all state transitions is possible with the following test cases (states and
events are depicted in cursive and non–cursive script, respectively).
Disconnected, Unhook → Dialing, Hang up → Disconnected
Disconnected, Unhook → Dialing, Timeout → Timeout occurred, Hang up

→ Disconnected
Disconnected, Unhook → Dialing, Digit 0 … Digit 9 → Dialing, Digit 0

…Digit 9 → Dialing, Dialed number valid → Connected, Hang up →
Disconnected
Disconnected, Unhook → Dialing, Digit 0 … Digit 9 → Dialing, Digit 0

…Digit 9 → Dialing, Digit 0 … Digit 9 → Dialing, Digit 0 … Digit 9 → Dialing,
Dialed number invalid → Invalid number, Hang up → Disconnected
Disconnected, Unhook → Dialing, Digit 0 … Digit 9 → Dialing, Digit 0…Digit

9 → Dialing, Digit 0 … Digit 9 → Dialing, Digit 0 … Digit 9 → Dialing, Dialed
number invalid → Invalid number, Timeout → Timeout occurred, Hang up →
Disconnected
The state chart in Fig. 2.2 represents the desired behavior. The test cases derived
from the state chart are exclusively normal cases. It does not contain test cases for
error situations. These cases are rather overlooked by testing on the basis of state
charts. A “Failure” state may be added to the original state chart in order to
describe error handling (Fig. 2.3).
Timeout
occurred
Timeout/reset
dialed number Hang up
Hang up/reset
Digit 0, digit 1,
dialed number
…, digit 9/Add
digit to dialed Discon-
Dialing
number, validate nected
dialed number Unhook/reset
dialed number
Dialed number Hang up/reset

invalid dialed number,
reset connection
Dialed number
Timeout/reset
valid/establish
dialed number
connection
Invalid
Connected
number
Unhook, dialed Hang up/reset

number valid, dialed number
dialed number Unhook
Unhook, dialed invalid Unhook, timeout,
number valid, dialed number
dialed number valid, dialed
invalid number invalid
Hang up, timeout, Figure 2.3:
dialed number “Failure” state
valid, dialed
Failure number invalid added to state chart
of Fig. 2.2
Normally, one would like to differentiate three situations in a condition–based

system:
• Events that cause a transition and / or an action if they occur in a particular
state.
• Events that generate a failure when they occur in a certain state.
• Events that can occur in a particular state, but are ignored, i.e., they neither
cause a transition nor an action.
The first two categories are covered by test cases generated from the augmented
state chart in Fig 2.3. Based on this state diagram, no test cases are generated for
the third category. If necessary, at least some important test cases from this
category should be selected.
2.5.3 Evaluation of state–based testing

State–based tests can be used in unit and system testing. In practice, state charts
are extensively used and therefore form a basis for testing. State–based testing is
in widespread use particularly in technical applications such as industry
automation, avionics, or the automotive industry. Modern software development
methods often use state charts as a specification method. In addition to many of its
advantages, state–based testing also has a few disadvantages that arise from the
characteristics of state charts. In state charts of large systems, there tends to be an

explosion in the number of states, which leads to a considerable increase in
transitions. In such a situation, only minor coverage of the state charts is possible.
This can be attributed to being a result of inadequate design and not as an
insufficiency in state–based testing. It is advisable to involve a tester in the design
phase – at least in the audit of design documents – whose task would be to
observe the testability of the design. With respect to state–based testing, this
would require keeping the number of states small enough for testing. For testing
on the basis of state charts, normal and failure cases should be taken into account.
All transitions that result in an error state should be tested. Some tests should be
included that neither result in a change of state nor provoke any action. Ignoring
an event in a particular situation is also a functionality whose correct functioning
is in no way self–evident. In such cases, this calls for a selection because if one
were to consider all possible cases, the number of test cases would be too high.
2.6 Other function–oriented testing techniques

The number of function–oriented testing techniques is at least as large as the
number of different specification techniques. Testing techniques that assume the
application of a particular specification technique are, of course, only applicable if
these specification techniques are used. In contrast to these specific, function–
oriented testing techniques, the techniques represented in Sections 2.4 and 2.5 can
be applied to whole classes of specification systems. In the following, some
specific function–oriented testing techniques will be explained.
2.6.1 Transaction flow testing

According to /Beizer 90/, a transaction is a processing module from the viewpoint
of a system user. Transactions consist of a sequence of processing steps. Very
often, a transaction–flow–based system view exists in early development stages or
later test phases. Transaction–flow–based testing techniques go well with system
tests because they deal with the outer perception of the system.
There are different representation forms for the notation of transaction flow. In
/Beizer 90/, Beizer uses flow diagrams to denote transaction flow. Sequence
diagrams are another way for describing transactions. This diagrammatic form has
long been used for describing the behavior of data transfer protocols. In addition,
it is used as the message sequence chart (MSC) in the object–oriented language
UML (Fig. 2.4).
Pre-existing objects
Actor
firstObject
:Class3 :Class2 Newly created
object
Operation1()
new Class3() newObject
:Class3
Message
Operation5() Object sends

itself a message
Operation3()
Operation2() Bar length

Object is specifies the Figure 2.4:
deleted relative duration A message
of the activity sequence diagram
In sequence diagrams, the different component instances are written horizontally

next to each other. Time progresses vertically from top to bottom. On the one
hand, such a diagram is a good basis for generating test cases. It directly specifies
possible test cases. On the other hand, sequence diagrams display only one out of
many different options. Sequence diagrams are expected to describe the most
important situations that should essentially be tested.
2.6.2 Testing on the basis of decision tables or decision trees

Decision tables or decision trees can be used as a basis for function–oriented
testing. On the one hand, decision tables and decision trees guarantee a certain
level of test completeness by way of their methodological approach. On the other
hand, the size of this representation increases exponentially with the number of
conditions. Tab. 2.4 specifies the cases in which an e–commerce enterprise settles
orders by bill. Whether the payment of a bill is possible is determined by whether
the customer is a new customer, whether the order amount is greater than €1000,
and whether this is a private customer. The three conditions result in eight
combinations. These are represented in Tab. 2.4.
Conditions Customer = new customer N N N N Y Y Y Y Table 2.4:

Decision table
Order amount > €1000 N N Y Y N N Y Y
Customer type = private customer N Y N Y N Y N Y
Action Payment by bill is possible Y Y Y Y Y Y Y N

Basically, one can consider each of the eight columns as test cases. As the number
of columns is exponentially dependent on the number of conditions,
simplifications are generally necessary. This is often possible. So, Tab. 2.5
represents the same rules as Tab. 2.4, but in a more condensed way. Payment by
bill is not possible only if a new private customer places an order in the amount of
over €1000. In all other cases, bill payment is possible. Thus, the eight cases in
Tab. 2.4 can be reduced to four. Fig. 2.5 shows the corresponding decision tree.
Table 2.5: Conditions Customer = new customer N – – Y

Optimized decision
table Order amount > €1000 – – N Y
Customer type = private customer – N – Y
Action Payment by bill is possible Y Y Y N
Customer =
new customer
N Y
Bill is Order amount

possible > 1000 €
N Y
Customer type =
Bill is private cutsomer
possible N
Y
Bill is Bill is not

Figure 2.5: possible possible
Decision tree
Every path from the root to a leaf of the tree corresponds to a test case. Therefore,
there would be four test cases. The decision tree does not include the so–called
‘don’t cares’ in Tab. 2.5. Decision trees require a certain sequence of evaluation –
the root condition is evaluated first. This sequence of evaluation is not implied in
the decision table. In the decision tree, an IF–THEN–ELSEIF structure for the
evolution of the decision can be easily defined.
2.7 Evaluation of function–oriented testing

Systematic function–oriented test planning and execution is an essential
component of every acceptable testing procedure. This is demanded by all
standards of software quality assurance. Function–oriented tests can therefore be
viewed as indispensable components of every software development process. The
successful execution of systematic function–oriented planned test cases arbitrates

substantial trust in the quality of the software.
There are no rules for the selection of a particular function–oriented testing
technique. There are no accepted minimal criteria as there are for structure–
oriented testing techniques. The function–oriented testing process should be
selected considering its applicability. Different testing techniques can be
employed for the various testing phases – for example, unit test, integration test,
and system test. The most important requirement is that the testing technique fits
the situation. Normally, different testing techniques are chosen for testing object–
oriented software and for testing traditional software. Whereas in testing
traditional software, functional equivalence class analysis is a frequently used
technique for unit testing, state–based testing is the normal case for object–
oriented unit tests.

Equivalence
We want to test a customer management system. This system stores the following classes
information about a customer:
• Name (20 characters or less)
• Year of birth (between 1900 and 1990)
• ZIP (natural number with 5 characters or less including 0)
• Street (20 characters or less)
Determine the equivalence classes necessary to test the customer management
system given the data specification above.

Test cases
Considering the system described in problem 2.1:
Describe the inputs for the minimum number of test cases that are necessary
considering boundary value analysis. Each test case must refer to its
corresponding equivalence classes.
Chapter 3 – Control Flow Testing 33
3 Control Flow Testing

• understand the characteristics and objectives of control–flow–oriented
testing;
• describe and evaluate:
a) Statement coverage testing,
b) Branch coverage testing,
c) Condition coverage testing;
• classify condition coverage testing into the following categories:
a) Simple condition coverage testing,
b) Condition/decision coverage testing,
c) Minimal multiple condition coverage testing,
d) Modified condition/decision coverage testing,
e) Multiple condition coverage testing;
• use structured path and boundary interior testing to test loops;
• understand and evaluate path coverage tests.
3.2 Introduction
In this chapter, dynamic testing techniques will be described that assess the
completeness of testing using coverage of the source code. Therefore, they are
referred to as structure–oriented testing techniques. The described testing
techniques are based on the control structure or the control flow of the software to
be tested. As a result, we speak of control flow testing techniques. This group of
testing techniques has great practical significance, particularly regarding its
application in module testing. The control flow testing technique group is well
supported by test tool providers. Beyond this, there are accepted minimum criteria
in the area of control flow tests that should be observed. One minimal, i.e.,
essential, accepted test procedure is the so–called branch coverage test. In
particularly critical application areas, relevant standards demand the use of additional
testing techniques. A standard for software applications in aviation, for example,
recommends using a so–called condition coverage test. Particular control flow
testing techniques have such an elementary character that any review that fails to
apply these techniques in module testing must be considered to be inadequate.
34 Chapter 3 – Control Flow Testing
3.3 Characteristics and objectives of control–flow–oriented

testing
Control flow Control flow testing techniques belong to the group of structure–oriented testing
testing techniques techniques. As a result, they have the corresponding advantages and
define no
regulations for disadvantages. Test completeness is evaluated on the basis of coverage of the
test data control structure or control flow. For the evaluation of the output, a
generation corresponding specification is necessary. Like all structure–oriented testing
techniques, control flow testing techniques define no guidelines for the
generation of test cases. It is only important that the generated test cases evoke
corresponding coverage in the structure. This degree of freedom with respect to test
case generation is very important because it facilitates the combination of other
Main application testing techniques for test case generation with structural testing techniques. The
area: Module test most essential application area of control flow testing techniques is the module
test. Control flow testing techniques may also have a certain significance in
integration testing, whereas they are not employed in system tests. Control flow
testing techniques observe the code structure. The aspects of process logic, which
are represented in the software as statements, branches, conditions, loops, or
paths, are observed in detail. The disadvantage is that these procedures are blind
with respect to omission errors. Functions that are not implemented but specified
Test basis: are only detected by coincidence. The test basis is formed by the so–called
Control flow control flow graph. The control flow graph can be set up for every implemented
graph
program in an imperative programming language. As a result, control flow
testing techniques can be applied to all programs written in imperative languages.
Fig. 3.1 shows the control flow graph for the operation CountChar. Tools for the
support of control flow testing techniques generate such control flow graphs and
use them to represent test results.
Path
nstart void CountChar (int& VowelTotal, int& TotalCount)

{
char Ch;
n1 cin >> Ch;
branch, edge
statement, node while ((Ch >=’A’) && (Ch <= ’Z’) &&
n2 (TotalCount < INT_MAX))
{
TotalCount = TotalCount + 1;
n3 if ((Ch == ’A’) || (Ch == ’E’) || (Ch == ’I’) ||
(Ch == ’O’) || (Ch == ’U’))
{
n4 VowelCount = VowelCount + 1;
}
n5 cin >> Ch;
}
Figure 3.1:
nfinal }
Control flow graph
3.4 Statement coverage test
3.4.1 Characteristics and objectives of the statement

coverage test
The statement coverage test is the simplest control flow testing method. It is Degree of
abbreviated as C0–test. The objective of statement coverage is a minimal one–time statement
coverage
execution of all statements of the program, i.e., coverage of all nodes of the
control flow graph. The degree of statement coverage is defined as a test
measurement. It is the ratio of the executed statements to the total statements
existent in the tested code. Exhaustive statement coverage is achieved if all the
statements in the module are executed at least once with the provided test data.
Number of executed statements
C statement = (3.1)
Number of total statements
Statement coverage demands the coverage of all nodes of the control flow graph Example 3.1:
represented in Fig. 3.1. The test cases require that the corresponding program Control flow graph
paths contain all the nodes of the control flow graph. In the control flow graph
shown in Fig. 3.1, this is possible, for example, with the following test case:
Call of CountChar with: TotalCount = 0;
Characters read: ‘A’, ‘1’;
Execution path: (nstart, n1, n2, n3, n4, n5, n2, nfinal).
This test path contains all nodes. It does not contain all edges of the control flow
graph. The edge (n3, n5) that corresponds to the optional else–case is not
contained. In the example, as the else–case of the if–statement is not used, no
statements to be tested by the statement coverage are assigned to it.
3.4.2 Description of the statement coverage test

It is a clever strategy to test, at least once, all statements that a programmer uses in
a program. This ensures that the software contains no statements that have never
been executed. On the one hand, the execution of a statement is an essential
criterion for detecting errors contained in it. On the other hand, it is not a
sufficient criterion. The appearance of an error may require execution with certain
test data. Also, when a failure occurs, it is not guaranteed that it will be detected
by the person who is doing the testing. The erroneous situation needs to be
propagated to a place that can be observed externally.
Presume that a software module contains an erroneous condition (x > 5). Actually, Example 3.2:
the condition should be (x ≥ 5). The erroneous condition can be executed by any Erroneous
condition
value of x except the value 5, without the occurrence of a failure. The erroneous
and the correct condition behave identically for all values except the value 5. A
single execution of the erroneous location does not guarantee that a failure occurs
and can be detected.
Like all coverage tests, the statement coverage test endeavors to fulfill the
essential criterion of executing potentially erroneous locations in the software.
The execution of all statements is a component of almost all important test
procedures, which also observe other aspects in addition. The statement coverage
test occupies a subordinate position as a stand–alone test procedure and is directly
supported only by a few tools. Statement coverage provides the possibility to
detect non–executable statements (i.e., so–called dead code). The statement
coverage measure can be used to quantify the achieved test coverage.
3.4.3 Evaluation of the statement coverage test

The statement coverage test is seldom the main function of testing tools. It
The statement appears as a by–product of tools supporting branch coverage tests and others. The
coverage test statement coverage test appears to be too weak a criterion for meaningful test
appears too weak
implementation. It has a subordinate practical significance. The minimum
criterion of control flow testing techniques is the so–called branch coverage test,
which contains statement coverage. Therefore, the application of statement
coverage tests as the sole criterion for structured test completeness is not
advisable. /Girgis, Woodward 86/ implemented empirical studies concerning
some different testing approaches. The statement coverage test occupies the
lowest error identification quota, with 18%. The standard RTCA DO–178B for
aerospace software applications recommends using the statement coverage test for
software at level C. In the case of a failure, such software can generate a major
failure condition.
3.5 Branch coverage test
3.5.1 Characteristics and objectives of the branch coverage test

The branch coverage test is a stricter testing technique than the statement
coverage test. The statement coverage test is fully contained in the branch
coverage test. In other words: The branch coverage test subsumes the statement
The branch coverage test. In general, the branch coverage test appears as the minimum
coverage test is a criterion in the area of control flow testing. It is abbreviated as the C1–test. The
minimum
criterion
objective of the branch coverage test is to execute all branches of the program to
be tested. This requires running through all the edges of the control flow graph.
The ratio of the number of executed branches to the number of available branches
in the software is used as a simple measure for branch coverage.
3.5.2 Description of the branch coverage test

The branch coverage test is regarded as the minimum test criterion. The central
position occupied by the branch coverage test is justified by its position in test
procedures. As an essential testing technique, it is contained by most other test
procedures. The branch coverage test forms the largest common subset of all these
testing techniques. The branch coverage test makes it possible to detect program
branches that are not executed. This is the case if no test data can be generated
that has not caused the execution of a branch yet until that time. In particular,
frequently executed software components can be detected and optimized as
desired.
The branch coverage test recommends the coverage of all branches of the control Example 3.3:
flow graph in Fig. 3.1. This is possible if the following test case is used: Test case
Call of CountChar with: TotalCount = 0;

Character input: ‘A’, ‘B’, ‘1’;
Execution path: (nstart, n1, n2, n3, n4, n5, n2, n3, n5, n2, nfinal).
The test path contains all the edges. It particularly contains those edges that were
not necessarily executed by the statement coverage test.
In some cases, it can be difficult to execute all program branches, due to various
reasons. Often, non–executed branches are actually executable and these test cases
are difficult to generate, or the test cases are difficult to derive from the program.
Principally, branches can also be non–executable. The cause may be a design
error leading to a redundant branch.
3.5.3 Problems of the branch coverage test

It is true for the statement coverage test as well as for the branch coverage test that The branch
even software with a coverage rate of 100% is not guaranteed to be error–free. coverage test is
unsuitable for
Neither the combination of branches nor complicated conditions are taken into testing compound
consideration. In addition, loops are not sufficiently tested. In the context of decisions and
testing loops, it is particularly critical that a randomly selected number of loop loops
iterations should fulfill the branch coverage test. This is no acceptable attribute of
a criterion for testing loops. Simple branch coverage measurement proves to be
problematic. As all the branches are equally emphasized without considering the There is a non–
dependencies between them, there is no linear correlation between the achieved linear correlation
between coverage
coverage rate and the ratio between the number of required test cases and the total rate and test case
number of test cases, which is actually essential for 100% branch coverage (Fig. count
3.2). The estimated number of test cases needed to achieve 100% branch coverage
is much greater than the number of test cases that are normally executed for
testing the main functionality.
The executed test cases initially already cover a large number of branches. After a
few test runs, a proportionally high coverage rate is achieved. The remaining test
cases similarly cover these branches, but increase the coverage rate only through
the execution of branches not contained in the paths already tested. Therefore, at
the end of the test, a relatively large number of test cases is required to achieve a
slight increase in the coverage rate. Furthermore, the statement coverage of a
program by a test case set is usually higher than the branch coverage of the
program by the same test case set.
100%
Coverage rate
observed increase in
the coverage rate with
the number of test
cases
linear increase in the

coverage rate with the
number of test cases
Figure 3.2:
Coverage rate as a
test case number Current number of test cases
function Number of test cases for a 100% coverage rate
3.5.4 Evaluation of the branch coverage test

The branch coverage test is the minimum criterion of structural software tests.
The standard RTCA DO–178B for software applications in the aerospace field
prescribes a branch coverage test for software at level B /RTCADO–178B 92/. A
failure in such software can cause a severe failure condition of the aircraft. The
performance of the branch coverage test has been studied in several empirical
studies that arrive at different conclusions. Howden detected six out of 28 errors
in six programs with the aid of the branch coverage test /Howden 78 a, b, c/. In
/Gannon 79/, the error exposing ability was 25%. A previous study by /Gannon
78/ that supports branch test and static analysis with respect to eight smaller
programs obtained a quota of 70% of errors detected with the help of the branch
coverage test. In both studies, the success quota of the branch coverage is more
than that of static analysis. An observation of the type of errors detected
depending on the test procedure employed shows good efficiency of the branch
coverage test in handling logical errors and calculation errors. In the literature,
there exist details about the efficiency of the branch coverage test that are
There is very good scattered over wide areas. There are numerous supporting tools in various
tool support for programming languages for branch coverage tests. The tools analyze the control
branch coverage
tests
structure of software modules available in the source code, localize branches, and
insert additional statements that allow tracking the control flow.
3.6 Condition coverage test
3.6.1 Characteristics and objectives of the condition coverage

test
The condition coverage test handles the logical structure of the conditions in the The condition
software. There are various definitions: The least powerful condition coverage test coverage test
handles the
– the so–called simple condition coverage – does not subsume the statement and
decision structure
branch coverage tests. The so–called multiple condition coverage test subsumes
the branch coverage test, but contains many other shortcomings, which will be
dealt with later. A practical compromise is represented by the minimal multiple
condition coverage test and the so–called modified condition/decision coverage
test. The basic idea of the condition coverage test is the performance of a
thorough review of complicated decisions in the software. For an exhaustive
branch coverage test, it is sufficient to execute all decisions with a true and a false
value at least once. This strategy does not deal with complicated decisions, which
are often nested logical expressions composed of underlying conditions. Decisions
are often constructed from finer decisions that are connected by logical operators.
If a decision or a partial decision has no subordinate partial decisions, it is
described as an atomic decision.
The simple decision (x > 5) can be considered sufficiently tested provided both Example 3.4:
logical values occur during the test. The decision subdivides the possible test data Test cases
into two classes and requires at least one test data to be selected from each class.
A test of the complex decision (((u == 0) || (x > 5)) && ((y < 6) || (z == 0)))
against both logical values cannot be viewed as sufficient because the decision
structure was not taken into account.
An exhaustive branch coverage test can, for example, be achieved with the
following test cases:
Test case 1: u = 1, x = 4, y = 5, z = 0
Test case 2: u = 0, x = 6, y = 5, z = 0
We assume that the composite decision is tested from left to right and the test
ends when the logical value of the overall decision is known. This can be
described as the non–exhaustive evaluation of decisions. Test case 1 results in the
following situation: The value 1 of the variable u yields the logical value false for
the first partial decision of the OR combination. Therefore, the second partial
decision of the OR combination sets the logical value of the OR combination. The
value 4 for the variable x inserted in the first partial decision (x > 5) also yields
the logical value false. The combination of the first two decisions has the logical
value false. Based on the result of the following AND–operator and independent
of the logical value of the other partial decisions at this time, it is already known
that the overall decision possesses the logical value false.
In many cases, the logical values of the remaining partial decisions towards the
right are not tested further. Independent of whether they are evaluated or not, the
logical value false for the first component decision in an AND–combination

masks the value of all further component decisions. This test case is therefore
‘blind’ regarding errors in the remaining partial decisions.
Test case 2 results in the following situation: The value 0 for the variable u results
in the logical value true for the first partial decision (u == 0). Based on the OR–
combination of the first two partial decisions, it is guaranteed that the result of the
first OR combination is true. The second partial decision does not need to be
tested.
The test can directly progress with the first partial decision of the second OR–
combination. The logical value of the second OR–combination sets the overall
result. The value of y being set to 5 results in the logical value true for the partial
decision (y < 6). This partial decision is combined with the fourth partial decision
OR. Therefore, it is guaranteed at this point that the overall result true is
independent of the logical value of the fourth partial decision. Thus, this test case
is ‘blind’ to errors in the fourth decision.
The test cases 1 and 2 result in exhaustive branch coverage. Neither of the test
cases tests the fourth partial decision. Basically, in a decision evaluation done
from left to right, the further to the right the partial decision in the decision
combination is, the worse it is tested.
This characteristic of the branch coverage test regarding the testing of partial
decisions shows that the branch coverage test is unsuitable for this problem. An
exhaustive evaluation of the decisions, which can, for example, be chosen by
certain compilers does not eliminate this problem.
3.6.2 Simple condition coverage

Simple condition coverage requires the testing of all atomic component decisions
In general, simple against true and false. Broadly speaking, it cannot be guaranteed that simple
condition– condition coverage subsumes the branch coverage test. Whether the branch
coverage does not
subsume the
coverage test is subsumed or not is decided by how the decisions are evaluated. If
branch coverage the composite decisions from the compiler are converted such that a composite
test definition is non–exhaustively evaluated by the software execution, the branch
coverage test is contained in simple condition coverage (with the AND–, OR–,
and NOT–operations). This procedure of evaluating decisions is standard. As
composite decisions are only tested until the logical value of the overall decision
is known, this form of evaluating decisions results in a reduction of execution
time. With an exhaustive evaluation of the decisions, the branch coverage test is
not contained in simple condition coverage. Some compilers offer a choice of
evaluating the decisions as either exhaustive or non–exhaustive. As the branch
coverage test is an essential testing technique, and as it is definitely not acceptable
for the fulfillment of essential criteria to be dependent on the compiler or its
settings, simple condition coverage should be regarded as inadequate.
Example 3.5: The decision (((u == 0) || (x > 5)) && ((y < 6) || (z == 0) should be written in short
Test cases as ((A || B) && (C || D)). We wish to assume that there is no dependency between
the values of the variables u, x, y, and z. Then the partial decisions A, B, C, and D
can be independently true and false. Tab. 3.1 represents the 16 logical value
combinations that are possible with exhaustive evaluation of the decisions. A
simple condition coverage test, for example, can be achieved with the two test
cases 6 and 11. The four component decisions A to D are tested against true and
false, respectively. The component decisions (A || B) and (C || D) and the decision
((A || B) && (C || D)) are true in both cases. A branch coverage test is not
achieved with this test case.
If test cases 1 and 16 corresponding to Tab. 3.1 had been selected, exhaustive
branch coverage could have been achieved. As shown by the example, there are
also test cases that fulfill simple condition coverage without guaranteeing branch
coverage. Simple condition coverage with exhaustive evaluation of the decisions
does not assure branch coverage.
With a non–exhaustive evaluation of the decisions, there are only 7 possible
combinations in place of the 16 logical values. These are stated in Tab. 3.2. For
example, cases 1 to 4 in Tab. 3.1 are mapped to case I in Tab. 3.2 because after
the evaluation of the component decisions, A and B are set to false, so that the
overall decision also possesses the value false. Therefore, it is not essential to test
the component decisions C and D in this situation. As shown in Tab. 3.2, test case
11, which results in branch coverage with exhaustive evaluation of the decisions
when combined with test case 6, also results in another logical value (VII
corresponding to Tab. 3.2). After the execution of the two test cases, the partial
decisions A and C are tested against true and false, respectively. The partial
decisions B and D are only tested against true. In contrast to the exhaustive
evaluation of decisions, the two test cases do not result in exhaustive simple
condition coverage. To achieve that, further test cases must be executed.
With non–exhaustive evaluation of decisions, simple condition coverage
subsumes the branch coverage test. Simple condition coverage has the advantage
that the number of essential test cases has a linear correlation with the number of
atomic component decisions. The disadvantage is that the branch coverage test
can be guaranteed only in the special case of non–exhaustive evaluation of
decisions.
Table 3.1: Atomic Partial Composite Overall

Simple condition Decisions Decision
coverage u= =0 x>5 y<6 z= =0
A B C D A || B C || D (A || B) && (C || D)
1 f f f f f f f
2 f f f t f t f
3 f f t f f t f
4 f f t t f t f
5 f t f f t f f
6 f t f t t t t
7 f t t f t t t
8 f t t t t t t
9 t f f f t f f
10 t f f t t t t
11 t f t f t t t
12 t f t t t t t
13 t t f f t f f
14 t t f t t t t
15 t t t f t t t
16 t t t t t t t
Table 3.2: Test Case A B C D A || B C || D (A || B) && (C || D)

Logical value
combinations with I 1, 2, 3, 4 f f – – f – f
non–exhaustive
evaluation of II 5 f t f f t f f
decisions
III 6 f t f t t t t
IV 7, 8 f t t – t t t
V 9, 13 t – f f t f f
VI 10, 14 t – f t t t t
VII 11, 12, 15, 16 t – t – t t t

3.6.3 Condition/decision coverage test

The condition/decision coverage test guarantees exhaustive branch coverage in The
addition to simple condition coverage. It explicitly demands branch coverage to condition/decision
coverage test
be established in addition to simple condition coverage. This technique is subsumes the
significant only in the case of exhaustive evaluation of decisions. The execution branch coverage
of test cases 5 and 12 in Tab. 3.3 yields exhaustive condition/decision coverage, test and the
statement
as component decisions A, B, C, D and the overall decision at any given time are coverage test
evaluated as true and false. Tab. 3.3 shows that this is possible without testing the
composite partial decisions (A || B) and (C || D) against both logical values. The
partial decision (A || B) has the value true in both test cases. The
condition/decision coverage tests atomic partial decisions and overall decisions. It
largely ignores the logical partitioning of complicated decisions in many layers. The logical
Condition/decision coverage has no requirements for the testing of composite partitioning of
partial decisions beneath the overall decision. Condition/decision coverage decisions is
ignored
subsumes the branch coverage test and the statement coverage test.
A B C D A || B C || D (A || B) && (C || D) Table 3.3:

Condition/decision
1 f f f f f f f coverage
2 f f f t f t f
3 f f t f f t f
4 f f t t f t f
5 f t f f t f f
6 f t f t t t t
7 f t t f t t t
8 f t t t t t t
9 t f f f t f f
10 t f f t t t t
11 t f t f t t t
12 t f t t t t t
13 t t f f t f f
14 t t f t t t t
15 t t t f t t t
16 t t t t t t t
3.6.4 Minimal multiple condition coverage test

The minimal multiple condition coverage test demands that along with the atomic
partial decisions and the overall decisions, all composite decisions should be
tested against the values true and false. This technique subsumes
condition/decision coverage. As the decisions can be structured hierarchically, it
is advisable to observe this structure while testing. Minimal multiple condition
coverage demands that all decisions – irrespective of whether they are atomic or
non–atomic (i.e., composite) – should be tested against both logical values. This
form of condition coverage takes the decision structure into consideration better
than the technique represented above, as all the nested layers of a complicated
decision are noted equally.
The logical With exhaustive evaluation of decisions, the execution of test cases 1 and 16 in
partitioning of Tab. 3.4 yields exhaustive minimal multiple condition coverage. All partial
decisions is noted
with limitations
decisions A, B, C, D, (A || B), (C || D) and the decision ((A || B) && (C || D)) are
tested against both logical values. Upon close observation, it is evident that both
test cases do not actually test the logic of the decision structure. If the correct
version of the example above had been ((A && B) || (C && D)), both test cases
would not realize it, even if all operators were erroneous. The test case would
have put in identical logical values for all component decisions and for the overall
decision. The test cases are “blind” to this error.
Table 3.4: A B C D A || B C || D (A || B) && (C || D)

Minimal multiple
condition coverage 1 f f f f f f f
(exhaustive testing
of component 2 f f f t f t f
decisions)
3 f f t f f t f
4 f f t t f t f
5 f t f f t f f
6 f t f t t t t
7 f t t f t t t
8 f t t t t t t
9 t f f f t f f
10 t f f t t t t
11 t f t f t t t
12 t f t t t t t
13 t t f f t f f
14 t t f t t t t
15 t t t f t t t
16 t t t t t t t
3.6.5 Modified condition/decision coverage test

The modified condition/decision coverage test requires test cases that demonstrate
that every atomic partial decision can influence the logical value of the overall
decision, independent of the other partial decisions. The application of this
technique is required in avionics by the standard RTCA DO 178 B for flight–
critical software (level A) /RTCA DO 178 B 92/. Basically, the technique aims at
comprehensive testing of the logic of composite decisions with low test effort.
The correlation between the number of atomic partial decisions of a decision and
the number of necessary test cases is linear. For testing a decision with n partial
decisions, a minimum of n+1 and a maximum of 2n test cases are required.
Tab. 3.5 represents a sample test case of a modified condition/decision coverage
test with exhaustive evaluation of decisions. The left side of the table indicates
which pair of test cases tests which specific atomic decision. Test cases 2 and 6
show identical logical values for the partial decisions A, C, and D. They have
different values for the partial decision B. In test case 2, B has the logical value
false. In test case 6, B is true. Other than that, test case 2 delivers the overall
decision as false while in test case 6, the overall decision has the logical value
true. Hence, it has been proven that the atomic partial decisions influence the
logical value of the overall decision, independent of the other partial decisions. A
similar situation exists for test cases 2 and 10 with respect to A, 9 and 10 with
respect to D, and 9 and 11 with respect to C. The logical values of the other three
partial decisions at any given time are fixed. The logical value of the partial
decisions observed at any given time and the overall decisions keep changing.
Tab. 3.5 contains five test cases. According to the rule that n+1 test cases are
essential for testing a decision containing n atomic partial decisions, an exhaustive
test with less than five test cases is not possible as the sample decision comprises
four atomic partial decisions.
With non–exhaustive evaluation of decisions, it is essential to moderate the Altered rules for
requirements to retain the logical value of the non–tested atomic component non–exhaustive
evaluation of
decisions at any given time while the logical values of the tested atomic partial decisions
decisions and the overall decision change.
Table 3.5: A B C D A || B C || D (A || B) && (C || D)

Modified
condition/decision 1 f f f f f f f
coverage
(exhaustive 2 f f f t f t f
evaluation of
decisions) 3 f f t f f t f
B
4 f f t t f t f
5 f t f f t f f
A
6 f t f t t t t
7 f t t f t t t
8 f t t t t t t
D 9 t f f f t f f
10 t f f t t t t
C
11 t f t f t t t
12 t f t t t t t
13 t t f f t f f
14 t t f t t t t
15 t t t f t t t
16 t t t t t t t
3.6.6 Multiple condition coverage test

Multiple condition The multiple condition coverage test demands the testing of all logical
coverage is the combinations of the atomic partial decisions. This procedure doubtlessly yields a
most
comprehensive very comprehensive test of composite decisions. Beyond this, by considering all
condition possible combinations, it is guaranteed that both logical values are considered for
coverage the overall decision, independent of the combination of partial decisions.
technique
Therefore, in any case, the multiple condition coverage test subsumes the branch
coverage test.
Number of Table 3.6:

atomic Number of test
1 2 3 4 5 6 … n cases with multiple
component
condition coverage
decisions
Number of
21 = 2 22 = 4 23 = 8 24 = 16 25 = 32 26 = 64 … 2n
test cases
Multiple condition coverage is the most comprehensive condition coverage

technique. The disadvantage is its high testing effort. In a decision comprising n
partial decisions, there are always 2n test cases. This is an exponential increase in
testing effort. The relationship between the number of atomic partial decisions and
the number of test cases is given in Tab. 3.6. Such an exponential increase in the
number of test cases coupled with the testing effort is not acceptable. Tab. 3.7
shows the test cases necessary to achieve exhaustive multiple condition coverage
of decision ((A&&B)) || ((C&&D)). As the decision comprises four atomic partial
decisions, the table consists of 24, i.e., 16 test cases.
A B C D A || B C || D (A || B) && (C || D) Table 3.7:

Multiple condition
1 f f f f f f f coverage
(exhaustive
2 f f f t f t f evaluation of
decisions)
3 f f t f f t f
4 f f t t f t f
5 f t f f t f f
6 f t f t t t t
7 f t t f t t t
8 f t t t t t t
9 t f f f t f f
10 t f f t t t t
11 t f t f t t t
12 t f t t t t t
13 t t f f t f f
14 t t f t t t t
15 t t t f t t t
16 t t t t t t t
In a non–exhaustive evaluation of decisions, not all 16 test cases exist. Only those
existing in Tab. 3.8 can be generated. Fundamentally, test data can be generated
for all the 16 logical value combinations. But in the case of non–exhaustive
evaluation of decisions, there is no possibility to register other situations than the
seven given in Tab. 3.8 with a testing tool. Switching the compiler to exhaustive
evaluation should not be done because the test should show a minimum of
possible differences to the software to be released and delivered later. If a non–
exhaustive evaluation is planned, it should be the same during testing. Beyond
that, certain logical value combinations cannot be generated as a result of coupled
partial decisions. From the 32 (25) logical value combinations of the decision
((ch ==‘A’) || (ch ==‘E’) || (ch ==‘I’) || (ch ==‘O’) || (ch ==‘U’)) of the operation
CountChar, only six can be generated (Tab. 3.9). This does not indicate an error
in the programming but is one of the attributes of the variable used in the decision.
This hampers the definition of a strongly declared test measure. The objective of
test measures is to obtain a quantitative statement about the degree of the test.
Normally, this is the ratio of the number of tested objects (statements, branches,
and atomic partial decisions) to the number of specified objects to be tested.
Often, with multiple condition coverage, a part of the essential tests cannot be
executed; hence, a simple measurement of the described form is inhibited.
Table 3.8: ch A B C D A || B C || D (A || B) && (C || D)

Multiple condition
coverage (non– I f f – – f – f
exhaustive
evaluation of II f t f f t f f
decisions)
III f t f t t t t
IV f t t – t t t
V t – f f t f f
VI t – f t t t t
VII t – t – t t t
’A’ ’E’ ’I’ ’O’ ’U’ All other ? ? ? Table 3.9:

ch Multiple condition
characters
coverage (possible
ch == ’A’ t f f f f f t … t test cases)
ch == ’E’ f t f f f f t … t
ch == ’I’ f f t f f f f … t
ch == ’O’ f f f t f f f … t
ch == ’U’ f f f f t f f … t
(ch==‘A’) || t t t t t f t … t
(ch==‘E’) ||
(ch==‘I’) ||
(ch ==‘O’) ||
(ch==‘U’)
6 Possibilities generated 26 Possibilities

(cannot be generated)
32 Possibilities
3.6.7 Evaluation of condition coverage

Condition coverage is particularly interesting as a testing technique when Condition
complicated logic is involved, which results in complicated decision constructs. coverage is used
for testing
When considering the best compromise on efficiency and test effort, minimal
complicated
multiple condition coverage and modified condition/decision coverage are process logic
recommended. The application of condition coverage in certain critical cases is
obligatory.
3.7 Techniques for testing loops
3.7.1 Characteristics and objectives

Loops normally result in an extremely high number of program paths. The PROBLEM:
execution of these paths – so–called path coverage (see section 3.8) – is not Explosion in the
number of paths
possible in such cases. The procedures described below – structured path test and by loops
boundary interior coverage – offer a solution to this problem. They divide the
paths into equivalent classes and allow only the execution of suitable
replacements from these classes of paths. These two techniques are very similar.
The boundary interior test is a special case of structured path tests. On the one
hand, the techniques are not accurately described in primary publications and
hence, in complicated loop structures it is not clear which requirements need to be
fulfilled. On the other hand, the objective of these techniques is the definition of
an executable test criterion for loops with reasonable effort that comprises certain
rules. An exhaustive branch coverage test is necessary as a supplementary
condition, but we are on the lookout for a technique between the branch coverage
test and the path coverage test. According to fundamental definitions of the
procedures, there exist loop structures for which requirements are not fulfilled.
Heuristics for testing loops are found in /Beizer 90/. /Riedemann 97/ contains a
precise definition of structured path tests and boundary interior tests.
3.7.2 Structured path test and boundary interior path test
3.7.2.1 Description
Loops often, but The number of paths in a software module is often extremely high due to the
not always result presence of loops. This is not true for every type of loop. Loops with constant
in a high number
of paths iteration count do not present any problems in this regard.
If 32767 is the largest possible value of a variable of type int, the operation
CountChar consists of 232768 - 1 paths. That is about 1.41 * 109864 paths. If one
could test 1000 paths per second, the path coverage test would continue for
4.5 * 109853 years. As loops are the root cause of this problem, it is advisable to
define limitations for the testing of paths that contain loops. This is done by
combining paths outside a certain number of traversing loops into classes that are
regarded as sufficiently tested by selecting a test path from the class. Howden
states the following definition:
Definitions from /Howden 75/:
publications
In the boundary–interior approach to testing, it is assumed that a complete set of
tests must test alternative paths through the top level of a program, alternative
paths through loops, and alternative boundary tests of loops. A boundary test of a
loop is a test which causes the loop to be entered but not iterated.
An interior test causes a loop to be entered and then iterated at least once.
Experience indicates that both the boundary and interior conditions of the loop
should be tested.
The boundary–interior method separates paths into separate classes if they differ
other than in traversals of loops. If two paths P1 and P2 are the same except in
traversals of loops they are placed in separate classes if
1. one is a boundary and the other an interior test of a loop;
2. they enter or leave a loop along different loop entrance or loop exit
branches;
3. they are boundary tests of a loop and follow different paths through the
loop;
4. they are interior tests of a loop and follow different paths through the loop
on their first iteration of the loop.
/Howden 78a/:
In the structured testing approach all paths through a functional module which
require less than or equal to k (usually k >=2) iterations of loops are tested at
least once. Variations in this rule are necessary in special cases where this would
leave parts of a module untested because of complicated loop indexing operations
and dependencies between loop bounds.
/Howden 78b/:
Each path through a functional module which executes loops less than k times is
tested at least once.
/Tai 80/:
Since the number of distinct paths is usually very large or infinite, a limited
number of paths are selected by restricting the number of iterations of each loop
in a path (called structured testing).
Basically, in the structured path test, only paths till the k–th execution of the loop The structured
body are differentiated. By k, we do not mean the number of loop iterations, but path test with
k = 2 is identical
the number of executions of the loop body. The definition of the technique in the to the boundary–
original literature is not completely clear on this point. The structured path test interior test
with k = 2 is denoted as boundary–interior coverage. Boundary–interior coverage
distinguishes between three cases – no loop executions, single–loop executions,
and two–loop minimum executions. As a result of the probable dependencies
between variables before, inside, and after the loop, this distinction is meaningful.
In data flow anomaly, the same three cases need to be observed for loops that
demand a dynamic test for the boundary–interior coverage. In the boundary–
interior test with respect to loops in the software module, three classes of paths are
originated. The first class contains the paths that do not execute loops. In the case
of non–rejection loops, this class is empty because the body of this loop would be
executed at least once. An example of how to deal with such loops is found in
/Riedemann 97/. The second class contains all paths that enter the loop, but do not
iterate loops. The third class consists of all paths that execute the loop at least
twice. According to /Howden 75/, the test cases for such paths that enter the loop
but do not iterate them, are called boundary test. The test cases that cause at least
one more execution of the loop body are called interior tests. The structured path
test with k = 2 is identical to the boundary–interior test.
3.8 Path coverage test
3.8.1 Characteristics and objectives of the path coverage test

Path coverage is The path coverage test is a comprehensive control flow test. The path coverage
thorough, but can test requires the execution of all the different paths of a software module. A path p
often not be
implemented is a sequence of nodes (nstart, n1…nm, nfinal) of the control flow graph. Two paths
are identical only if the sequence of their nodes is identical. The path coverage test
subsumes all the testing techniques described in this chapter, except for the
condition coverage test. The path coverage test can often not be performed for real
software modules because these may consist of a huge number of paths /Howden
81/, /Tai 80/. But even when the number of paths is finite, it is often too high to
perform an economical test implementation. A large number of paths are normally
caused by loops that have no fixed number of iterations. Each iteration generates
a new path by adding the sub–path of the loop body.
3.8.2 Evaluation of the path coverage test

The path coverage test has no practical significance because of its extremely
limited feasibility.
3.9 Evaluation of control flow testing techniques

The control flow The control flow testing techniques are very important; so much so that the
test is unavoidable question of their performance sometimes becomes less important than the other
motivations for their use. Branch coverage is a widely accepted minimal criterion
of structural software tests, so that a test that does not attain branch coverage is
considered insufficient. In addition to this fundamental fact seen as “common
sense,” there are explicit demands on certain testing techniques in certain
application domains.

Java class
The following part of a Java class is given, which sorts a one–dimensional array
with the bubble sort algorithm.
public static int[] elements;
public static int length() {return elements.length;}
public static int get(int i) {return elements[i];}
public static void put(int i, int x) {elements[i]=x;}
public static void bubblesort(int length) {

int a0, a1, j;
int i=length-1;
while (i>=0) {
j=0;
while (j<i) {
a0=get(j);
a1=get(j+1);
if (a0>a1) {
put(j,a1);
put(j+1,a0);
}
j++;
}
i--;
}
}
a. Construct the control flow graph for the method bubblesort().

b. State the necessary paths for the statement coverage test.
c. State the necessary paths for the branch coverage test.

Java method
We have the following part of a Java method:
...
if ((a<0&&b>0)||(c==0&&d!=0))
...
a. State the logical values of the condition and those of its parts that are at least
necessary to achieve simple condition coverage with exhaustive evaluation
and non–exhaustive evaluation from left to right. Give examples of possible

input values that would trigger the corresponding logical values.
b. State the necessary combinations of logical values of the condition and its
parts to achieve modified condition/decision coverage with exhaustive
evaluation of decisions.
Chapter 4 – Data Flow Testing 55
4 Data Flow Testing

• understand how data flow techniques can be used to evaluate the
completeness of test cases;
• generate control flow graphs;
• categorize variable access as def, c–use, or p–use;
• define and understand:
a) the all p–uses criterion,
b) the all c–uses criterion,
c) the all c–uses/some p–uses criterion,
d) the all p–uses/some c–uses criterion,
e) the all–uses criterion,
f) the all du–path criterion;
• understand the subsumption relations.
4.2 Introduction
In this chapter, dynamic testing techniques will be described that evaluate the
completeness of a test based on the achieved coverage of the software source
code, just like the techniques presented in Chapter 3. In contrast to the control
flow testing techniques in Chapter 3, the techniques described here use the data
flow to evaluate the completeness of the test cases. The data flow test procedures
use the access to variables. Variables in a program can be accessed by “read
access“ or “write access“. So, the access to variables is used to test the software in
data flow tests. Every software module contains data that is processed and control
structures that control this processing. In contrast to control flow testing
techniques, data flow testing techniques shift data manipulation to the test focus.
The practical use of data flow test techniques is limited, due to a lack of
appropriate tools.
4.3 Characteristics and objectives of data–flow–oriented tests

As data flow testing techniques belong to the group of structure–oriented testing
techniques, they have all the corresponding advantages and disadvantages. Test
completeness is evaluated with regard to the coverage of data accesses. The
correctness of the generated output values is evaluated against the specification.
56 Chapter 4 – Data Flow Testing
Just like all structural testing techniques, data flow testing techniques also define
no rules for test case generation. This degree of freedom can be used to generate
test cases with functional testing techniques, using the data flow test only for
measuring test completeness. Data flow testing is based on an extended variation
of the control flow graph. After a variable is declared in the memory, only the
following two things can occur with it until it is destroyed (for example at the end
of the program):
• It can have write access – i.e., its value can be changed if necessary.
• It can have read access – i.e., its value is not changed.
Attention: Write access is also called definition (in short: def). Read access is called
definition ≠ reference (in short: r) or use (in short: u). In programming, there are two possible
declaration
reasons for read access to variables. Either the read variable value acts as input
c–use data for a calculation, or it serves to determine the logical value in a decision. The
p–use first form of access is called computational use (in short: c–use). The other form
of read access is called predicate use (in short: p–use).
Every variable access must therefore belong to one of these three categories:
• Definition (def)
• Computational use (c–use)
• Predicate use (p–use).
Import nodes Definitions (defs) of variables and computational uses (c–uses) of variables are
Export nodes assigned to nodes in the control flow graph. The statement y: = f(x1,…,xn)
contains c–uses of the variables x1, …, xn, followed by the definition of variable y.
The statement if p(x1,…,xn)… contains predicate uses (p–uses) of the variables x1,
…,xn. As decisions determine the control flow of a program, the corresponding p–
uses are assigned to edges. If the last statement of the node is a decision in which
the variables x1 to xn are used, all the edges of the control flow graph emerging
from these nodes would be assigned p–uses of x1,…,xn. In certain testing
techniques, the assignment of p–uses to edges ensures that branch coverage is
achieved as the accepted minimum criterion. The control flow graph extended to
these attributes is called data flow attributed control flow graph. It can be
automatically generated from the source code with the help of a tool. The module
under test here generally contains a parameter interface from which the data in the
module can be input or entered into the environment. This data flow should be
described similarly through corresponding extensions of the control flow graph.
This introduces the purpose of the additional nodes nin and nout in the data flow
representation of the control flow graph. They describe the import or export of
information via interface parameters or global variables. The definitions of all
imported variables are assigned to the node nin. The c–uses of all exported
variables are assigned to the node nout. Fig. 4.1 shows the control flow graph of
the operation CountChar with data flow attributes.
nstart
def (TotalCount) void CountChar (int& VowelTotal, int& TotalCount)

def (VowelCount) nin
{
char Ch;
def (Ch) n1
cin >> Ch;
while ((Ch >=’A’) && (Ch <= ’Z’) &&

p-use (Ch) n2 (TotalCount < INT_MAX))
p-use (TotalCount) {
c-use (TotalCount)
n3 TotalCount = TotalCount + 1;
def (TotalCount)
n4 if ((Ch == ’A’) || (Ch == ’E’) || (Ch == ’I’) ||

(Ch == ’O’) || (Ch == ’U’))
p-use (Ch)
{
c-use (VowelCount) n5 VowelCount = VowelCount + 1;
def (VowelCount) }
def (Ch) n6 cin >> Ch;
c-use (VowelCount) }
c-use (TotalCount) nout
} Figure 4.1:
Control flow graph
nfinal – data flow
representation
4.4 Defs/uses test

The defs/uses test criteria define test completeness on the basis of variable access.
They use the coverage of defs, c–uses, and p–uses by test cases as the test
completeness criterion.
void MinMax (int& Min, int& Max) Example 4.1:

{ Defs/uses test
int Temp;
if (Min > Max)

{
Temp = Min;
Min = Max;
Max = Temp;
}
}
The control flow graph shown with data flow attributes of the operation MinMax Global vs. local
variable access
is represented in Fig. 4.2.
nstart
nin Import of ’Min’ and ’Max’ def (Min), def (Max)
n1 if (Min > Max)
p-use (Min), p-use (Max)

Temp = Max; c-use (Max), def (Temp)
p-use (Min) n2 Max = Min; c-use (Min), def (Max)
p-use (Max) Min = Temp; c-use (Temp), def (Min)
nout Export of ’Min’ and ’Max’ c-use (Min), c-use (Max)
Figure 4.2:
Control flow graph nfinal
for MinMax
A c–use is global if no definition of the variable precedes the same block;

otherwise, it is local. C–uses are global with respect to Min and Max in the node
n2 of the control flow graph in Fig. 4.2, since the last immediately preceding
definition occurs in the node nin. The computed variable access for Temp is local
because of the preceding definition in the same block.
def–clear wrt x: A path p = (nin,…,nm) that contains no definition of variable x in (nin,…, nm) is
definition clear definition–clear with respect to x. The definition of a variable x in node ni reaches
with respect to x
a c–use of x in node nj if there exists a sub–path (ni)∙p∙(nj) and if p is def–clear
with respect to x. The definition of a variable x in node ni reaches a predicate use
of x in edge (nj, nk) if a sub–path (ni)∙p∙(nj , nk) exists and if p∙(nj) is definition–
clear with respect to x.
The definition of a variable x is global if it is the last definition of x in node ni and
if at least one definition–clear path exists concerning x from node ni to node nj,
which contains a global c–use of x, or an edge that contains a p–use of x. A
definition is local if a local c–use exists that follows the definition and no other
definition exists between the definition and the access. Definitions that are neither
local nor global are not used and indicate a programming error.
To clarify the procedure for testing, it is important to assign every node ni of the
control flow graph the set of globally defined variables (def(ni)) and the global
computational use (c–use(ni)). Every edge (ni, nj) is assigned the set of the
variables for which a predicate use (p–use (ni, nj)) occurs.
For a node ni and a variable x with x being an element of def(ni), dcu(x, ni) is the
set of all nodes nj for which x is an element of c–use(nj) and for which a
definition–clear path exists with respect to x from node ni to nj. The set dpu(x, ni)
is the set of all edges (nj, nk) for which x is an element of p–use (nj, nk) and for
which there exists a definition–clear path with respect to x from ni to (nj, nk).
Testing techniques can be defined on the basis of control flow graphs provided
with data flow attributes.
• The all–defs criterion requires a number of test paths so that for every All–defs criterion
node ni and for every variable x that is an element of def(ni), there is at
least one definition–clear path with respect to x from ni to an element of
dcu(x, ni) or dpu(x, ni). The criterion requires that the test cases should be
such that for every definition (all–defs) of all the variables, at least one c–
use or one p–use is tested. The sample program contains the definition of
the variables Min and Max in the nodes nin and n2, respectively. The test
path (nstart, nin, n1, n2, nout, nfinal) tests the program sufficiently in terms of
the all–defs criterion. The edge (n1, nout) is not executed. The all–defs
criterion subsumes neither branch coverage nor statement coverage.
• The all p–uses criterion is fulfilled if for every node ni and every variable x All p–uses
that is an element of def(ni), a definition–clear path is contained in the criterion
tested paths with respect to x from ni to all elements of dpu(x, ni). Every
combination of every variable definition with every p–use that is used by
this definition should be tested. In the sample program, the paths (nstart, nin,
n1, nout, nfinal) and (nstart, nin, n1, n2, nout, nfinal) should be exercised for the
fulfillment of this criterion in order to test the p–uses of the edges (n1, n2)
and (n1, nout). The all p–uses criterion subsumes branch coverage.
• The all c–uses criterion requires the execution of at least one definition– All c–uses
clear path with respect to x from ni to every element of dcu(x, ni) for every criterion
node ni and every variable x that is an element of def(ni). Every

combination of every variable definition with every c–use that is used by
this definition should be tested. The all c–uses criterion subsumes no other
defs/uses criterion and does not contain branch coverage.
• A number of test cases fulfill the all c–uses/some p–uses criterion if for All c–uses/some
every node ni and every variable x that is an element of def(ni), a p–uses criterion
definition–clear path with respect to x from ni to every element in dcu (x,

ni) is tested, or, if dcu (x, ni) is empty, at least one definition–clear path is
tested with respect to x from ni to one element of dpu(x, ni). All c–uses
should be tested with respect to every definition of the variable. If there is
no c–use for a definition, it must be used in at least one p–use. The process
thoroughly tests c–uses. The all c–uses/some p–uses criterion requires the
paths (nstart, nin, n1, n2, nout, nfinal) and (nstart, nin, n1, nout, nfinal) for testing the
sample program. The definition in nodes nin and n2 make it necessary to
test the c–uses in nodes n2 and nout. The existence of c–uses for a definition
renders the test of p–uses unimportant; however, since the predicate uses
are identified with branches, this criterion does not generally subsume the
branch coverage test. The all c–uses/some p–uses criterion subsumes the
all–defs criterion.
All p–uses/some • The all p–uses/some c–uses criterion is symmetrical to the all c–uses/some
c–uses criterion p–uses criterion. It requires a definition–clear path from every definition to
every predicate use, and, if it is not available, demands testing of a
corresponding path to a computational use. P–uses are intensively tested
by this procedure. The all p–uses/some c–uses criterion subsumes the all
p–uses criterion and the all–defs criterion and also the branch coverage.
The sample program is sufficiently tested through the execution of the
paths (nstart, nin, n1, n2, nout, nfinal) and (nstart, nin, n1, nout, nfinal).
All–uses criterion • The all–uses criterion subsumes the all p–uses/some c–uses criterion and
the all c–uses/some p–uses criterion. It requires testing of all definitions in
combination with all reachable p–uses and c–uses.
All du–path • If the requirement of the all–uses criterion is extended so that all the du–
criterion
paths should be tested with respect to the definitions of all variables, the
all du–paths criterion is obtained. A du–path is a path p = (ni, nj, ..., nk, nl)
with a global definition of x in ni for which the following holds true:
• p' = (nj, ..., nk) is def–clear with respect to x and p reaches a c–use in nl
AND the nodes ni, nj, ..., nk, nl are distinct (loop–free) or the nodes nj, ...,
nk are distinct and ni = nl (single loop traversal),
or
• p' = (nj, ..., nk) is def–clear with respect to x and p reaches a p–use in (nk,
nl) and the nodes ni, nj, ..., nk, nl are distinct (loop–free).
According to the specified definition, the limitation on the du–paths works
towards restricting the number of test paths, analogously to the boundary–
interior path test and the structured path test. It is to be noted that complete
test paths can contain many loop executions compared to the du–paths
/Bieman, Schultz 89/. The subsumption relations of the defs/uses criteria and
further testing techniques are represented in Fig. 4.3.
Figure 4.3:
Subsumption
relation of data
flow tests
4.5 Evaluation of data flow testing techniques

Nearly without exception, data flow testing techniques were published later than Data flow test
control flow testing techniques. One would expect that they are “more modern” tools are rarely
available
than controlflow–oriented techniques. However, in practice, their importance is
much lower than that of control flow testing techniques. This is particularly due to
the insufficient availability of data flow test tools. A few test tools are available The practical
for the defs/uses test. An application of these complicated data flow testing significance of
data flow tests
techniques to software is not advisable in practice without tool support. lags behind that of
Unfortunately, test tools for object–oriented software development predominantly control flow tests
support control flow testing techniques. Technically, data flow testing techniques
are better suited to object–oriented software development than control flow testing
techniques.

Bubble sort
algorithm The following part of a Java class is given, which sorts a one–dimensional
array with the bubble sort algorithm.
public static int[] elements;
public static int length() {return elements.length;}
public static int get(int i) {return elements[i];}
public static void put(int i, int x) {elements[i]=x;}
public static void bubblesort(int length) {

int a0, a1, j;
int i=length-1;
while (i>=0) {
j=0;
while (j<i) {
a0=get(j);
a1=get(j+1);
if (a0>a1) {
put(j,a1);
put(j+1,a0);
}
j++;
}
i--;
}
}
a. Construct the control flow graph for the method bubblesort().
b. Annotate it with the data flow attributes required for data flow testing.
Chapter 5 – Tool–Supported Static Code Analysis 63
5 Tool–Supported Static Code Analysis

• understand the characteristics, objectives, and significance of static code
analysis;
• give examples of a few types of syntactic programming conventions that
can be checked by tools;
• understand the importance of slicing;
• know how slices can be constructed from control flow graphs;
• learn how state automatons can be used by tools to perform data flow
anomaly analysis;
• use data flow tables to identify data anomalies;
• explain how data flow anomalies can be detected in loops.
5.2 Introduction
In this chapter, you will be introduced to methods that only require little time and
expenditure because they can be extensively automated using tools. Different
objectives can also be pursued with tool–supported static code analysis. Apart
from checking programming conventions, images or tables can be generated
automatically. It is even possible to reliably detect certain errors through
automated analysis, or to support debugging after a faulty behavior occurs, i.e., in
the case of dynamic testing. One common characteristic of the methods
introduced here is that with the exception of so–called dynamic slicing, they do
not require any execution of the software to be tested.
5.3 Characteristics and objectives of tool–supported static

code analysis
This chapter is called “Tool–supported Static Code Analysis” to emphasize that Tool support is
the methods described here have to be carried out with tool support. Basically, all absolutely
necessary
static analyses can also be done manually. However, manual execution clearly has
disadvantages. Apart from requiring a lot of time and apart from the expenditure
involved, the quality of the analysis results can be expected to be worse than in
results obtained automatically. The reason for this is the type of activity. Static
code analysis requires stubborn, monotonous use of relatively simple rules on
code that is often quite large. This task does not require any kind of creativity, but
a very large degree of oversight and control. Both cannot be guaranteed by human
beings, particularly when the object to be analyzed is extensive and complicated.
64 Chapter 5 – Tool–Supported Static Code Analysis
Therefore, static code analysis is intended to be automated by tools. These

The tools work in methods should be either applied with the support of tools or not be used at all.
a way similar to Static code analysis has many similarities to the functions of compilers. The
computers
supporting tools are therefore constructed in a way similar to compilers. Thus, for
example, an analysis of the structure of the code to be examined is required as a
first step. This so–called lexical and syntactic analysis essentially corresponds to
the steps that compilers also carry out. Specific static code analyses are thus
integrated into compilers. Moreover, separate tools are available on the market.
Static code analyses are partly integrated as components in other quality assurance
tools. Static code analyses are characterized by the following common properties:
• No program execution
• Analysis of the source code
• No proof of correctness
• Complete with respect to the property observed (for example, guaranteed
recognition of so–called data flow anomalies)
• Extensive automation capability
Tool–supported The methods described in this chapter are aimed at automating certain tests that
static code have a rigid system. This relieves the tester of specific responsibilities. The
analysis relieves
the tester of corresponding information is made available to him in a compressed form by the
responsibilities analysis tools. It is ensured that this information is complete. It is the job of the
tester to evaluate the automatically created information. Typical questions raised
are:
• Shall a recognized violation of a programming convention be permitted or
has the code to be modified correspondingly?
• Is the structure of an automatically generated control flow graph
acceptable or is it too complicated?
• How is a so–called “slice” examined for the error that triggers a
recognized faulty behavior?
• Shall a so–called dd data flow anomaly be removed or is this accepted?
Objective of static This can be summarized as follows: The objective of static code analysis is to
code analysis automate systematic tests as much as possible, to relieve the tester of this job and
thus to give him room to use his creativity for more important tasks.
5.4 Style analysis
5.4.1 Characteristics and objectives of style analysis

In many companies, there is a set of rules that programmers have to use when Company–specific
coding. These programming rules are also called programming conventions. programming
convention
Various objectives can be pursued when using programming conventions. For
instance, one tries to avoid constructs that favor errors in programming, or that
prevent the portability of the software or the comprehensiveness of the code.
Apart from limitations of the permitted syntax, however, additional requirements
usually need to be met. Thus, it is common to place an amendment history in the Assertion
module head. The obligation to indicate so–called assertions at the start and end of
an operation is another example. Apart from company–specific programming
conventions, there are rules that are generally true for certain programming Programming–
languages. This particularly applies to languages of the C family. The objective in language–specific
programming
the case of languages whose syntax contains parts that may affect the quality convention
negatively is to put them through a test that recognizes and identifies these
constructs. The first widely used tool for this task was “Lint” for C under Unix.
5.4.2 Programming conventions

Programming conventions can have syntactic or semantic contents. This Syntax = form
difference is important, since the former can be tested automatically using tools, Semantics =
whereas the latter require checking by humans. An example of syntactic content
programming conventions is the requirement that “goto” instructions must not be
Syntactic
used, or that pre–incrementing and post–incrementing as well as pre– programming
decrementing and post–decrementing instructions are not allowed. Checking these conventions can
rules requires a tool that examines the source code of the software for the symbols be checked
automatically
“goto”, “++” and “– –”. Therefore, checking syntactic programming conventions
does not take up much time. Examples of semantic programming conventions are
the requirements for giving clear, self–explanatory names – for example, for
variables – and for suitable comments. A tool cannot check such requirements. Semantic
Here, an interpretation by a human being is necessary. Whereas the testing of programming
conventions have
syntactic programming conventions is the purpose of tool–supported static code to be tested by
analysis, semantic programming conventions are often tested through code humans
inspections. For this purpose, the rules can be converted into a checklist that is
used during the inspection. Since manual tests are more time–consuming and
expensive than automatic tests, attention should be paid to making sure that only
semantic tests are done manually.
Only syntactic conventions will be presented and discussed in the following.
Extensive sets of rules have been published for the programming languages C,
C++, and Java in particular. These can also be found on the Internet.
In the following, a selection of syntactic programming conventions for C Programming
programs will be provided – i.e., rules that can be tested automatically: conventions for C
a) Control structures:
• The following should be obeyed in “switch” instructions:
• Each case that contains instructions is closed with a break instruction.
• Each case not containing instructions is not closed with a break
instruction (reference to the next case).
• It is necessary to have one – possibly empty – default case.
• Return instructions should only occur once per function.
• The ?–operator must not be used.
• Loops with multiple exits should not be used as far as possible. If they do
occur, care should be taken that they are broken off at the end of the loop.
b) Operations:
• Operators consisting of a combination of operation and assignment must
be avoided. Instead of x + = y; , x = x + y; must be written.
• Assignments in expressions are not allowed.
• Multiple assignments are not permitted, for example,
x=y + (z = 2);
• Post–incrementing (x++) and pre–incrementing (++x) as well as post–
decrementing and pre–decrementing operations have to be avoided. The
control of for–loops is an exception.
• Attention should be paid to the prevalence of congruence between
operations and data types (for example, use of the operation mod only on
positive integer values).
c) Data types, variables, and indicators:
• Type conversions must be carried out explicitly.
• Variable names cannot start with the symbol underscore (‘_’).
• Variables always have to be initialized.
• Invalid pointers must be marked as nil (or null) pointers.
Assertions in C Apart from such limiting rules, expanding rules frequently also exist. An example
of these is the obligation to use input and output assertions. Assertions are
particularly suitable for checking whether input conditions have been fulfilled.
They are realized in C by the macro “void assert (int expression)“. If “expression”
has the value “false” during execution of the macro, then this violation of the
assertion is reported and the program execution is stopped. Checking for valid
parameter values, for example, is a part of this. On the output side, it is advisable
to use assertions for checking the results. Therefore, the formulation of input and
output assertions makes sense. Whether there is an input and an output assertion
for each function can be checked automatically.
5.4.3 Evaluation of style analysis

Checking programming conventions is an established procedure in the area of There are
quality assurance. There are standards that require the use of programming standards that
require the use of
conventions. If programming languages that have a restrictive syntax are used (for programming
example, Pascal), style analysis can be rejected. However, this is only applicable conventions
with respect to the checking of limiting rules. Checking expanding rules (for
example, the use of assertions) also makes sense in programming languages such
as Ada and Pascal. Relevant tools are available on the market at moderate prices.
It is absolutely necessary to make extensive use of the corresponding tests of the
compiler.
5.5 Slicing
5.5.1 Characteristics and objectives of slicing

Slicing aims at the largely automatic identification of interdependencies between Slicing has
statements of a program. The way this works is comparable to manual debugging similarities to
manual debugging
after the occurrence of faulty behavior. Debugging is also the typical area of use
for automatic slicing. The term program slice, as used here, describes which
instructions influence an observed value and how. A slice is therefore always
defined with respect to an observed variable at a specific point of the program.
Strictly speaking, this is the definition of the so–called backward slice.
There is also forward slicing. This type of slicing indicates which instructions are Backward slicing
influenced by a specific variable and how this influence takes place. Backward vs. forward slicing
slicing corresponds to the analysis that is carried out in the manual search for the
cause of an observed faulty behavior. First, one checks whether the faulty
behavior was caused by the currently observed instruction. If this is not the case,
the instructions that provide data for the instruction or control their execution are
checked. This procedure can be continued until no predecessors exist anymore,
i.e., until input instructions are reached. The slice created in this way includes all
potential causes of the faulty value at the observed point. The following
information is available in compressed form through slicing:
• Instructions that can influence the value of an observed variable at a
specific point
• Participating variables and data flows
• Control influences
All instructions that are not a component of the slice do not have to be checked in Slicing reduces
the search for errors. The time consumed by searching for errors can therefore be expenditure in the
search for errors
reduced by automatic slicing. A distinction is made between static and dynamic
slicing /Korel, Laski 88/. Static slicing can be carried out as in static analysis. It is
not necessary to run the program or to make assumptions on the values of
variables. The limitation is critical if dependencies are only a result of the
concrete value of a date. Such situations occur in connection with the use of
pointers or arrays, for example. Dynamic slicing is of help here. We will focus on
static slicing.
Starting point: A suitable starting point for the representation of static slicing is the control flow
control flow graph, which can be created using static analysis and can be extended by data flow
graph with data
flow attributes attributes, and which is also used as a basis for data flow testing. The instructions
(nodes) of the control flow graphs are allocated to data flow attributes, which
describe the type of data access included in the instructions. A difference is made
between write access and read access. Write access is called definition (def, d).
Read access is known as reference. If a read access takes place in a decision, we
speak of a predicative reference (p–use, predicate use, p). A read access in a
calculation is described as a calculating reference (c–use, computational use, c).
The control flow graph in Fig. 5.1a refers to an example from /Korel 87/. Often,
the slice is also noted graphically. The instructions are indicated as nodes with the
same numbering as in the control flow graph. Interwoven nodes represent a data
dependency. Control dependencies are indicated by dashed edges. The following
definitions apply to both edge types:
Control edges of a • Control edges point from instructions containing a predicative reference
slice (p–use) to the directly controlled instructions, i.e., to those instructions that
are only executed when the predicate has a specific value. Control edges
are only drawn between the controlling instruction and the directly
controlled instructions. If a further control level is nested in a controlled
unit, then no control edges are drawn that cross over more than one level.
Data flow edges of • Data flow edges point from instructions in which a variable is defined to
a slice instructions in which this variable is referenced. The variable must not be
redefined between the definition and the reference. This is known as the
definition–clear path with respect to this variable.
The backward slice is determined by searching the control flow graph against the
direction of the edges, starting at the statement containing the reference to the
observed variable, for definitions of this observed variable. If computational
references exist for the definitions, the procedure is continued recursively, until no
additional nodes are found anymore. The dependencies between instructions
found in this way are data dependencies. If an observed node is found in a unit
whose execution is directly controlled by a decision, this is a control dependency.
Nodes with corresponding definitions, i.e., data flow dependencies, are searched
recursively for the predicative references of the variables that are a part of the
decision and that possibly possess further control dependencies.
Figure 5.1 a:
Control flow graph
with data flow
attributes
Figure 5.1 b:
Slice to max
Figure 5.1 c:
Slice to avr
5.5.2 Evaluation of slicing

Slicing is a particularly suitable means of analysis for complicated, unclear Slicing supports
programs. Slicing can primarily support the search for errors. the search for
errors
5.6 Data flow anomaly analysis
5.6.1 Characteristics and objectives of data flow anomaly

analysis
Certain errors in software can be recognized without program execution, purely Static error analy-
statically – by looking at them carefully. These methods are described as static sis: finding errors
without program
error analysis. A certain part of these static analyses is already carried out by execution
compilers. This applies to the checking of operation interfaces for consistency Programming lan-
between formal and current parameters, for example. Automatic, static error guages enable the
recognition is supported by the properties of modern programming languages. recognition of
errors through
Declaring procedures and functions with formal parameters permits detecting redundancy
interface anomalies, i.e., an inconsistency between the type, the number, or the Anomaly =
use of formal and current parameters. An anomaly is each deviation in specific deviation
properties of a program or module from the correct version of these properties. An
anomaly is always a suspicious place in a program. A form of anomaly checking Data flows
that is also carried out by some compilers is data flow anomaly analysis /Taylor,
Osterwell 80/, /Wilson, Osterwell 85/. Data flows were already introduced in the
chapter on data flow testing. Data flows are formed by the conversion of input
values into output values through interim results. The input data are processed to
determine interim results that are written into the memory to then be re–read and
to finally be converted into output values. The data almost “flows” through the
software, from inputs to outputs. Not every possible sequence of data access is
correct. Incorrect data flows are called data flow anomalies. Not all data flow
anomalies necessarily cause failures.
5.6.2 Performing data flow anomaly analysis

Data flow is always observed along an execution path for a specific variable under
consideration. All data is undefined at the beginning and at the end of a program’s
execution. It does not exist in the memory. The initial variables are allocated in
the memory when the program starts and as the program is executed, further
variables are added, for example local variables in functions. A variable that exists IMPORTANT:
in the memory as a consequence of the processing of its declaration, however, is The terms
declaration and
usually still undefined. An example of this situation is a pointer that has already definition are
been filed in the memory but still does not have any valid value. It is undefined. A partly also used in
variable is only defined after the initial value assignment, for example its other ways
initialization. For this reason, it is definitely an error to read the value of a
variable existing in the memory and to process it further before it has been
initialized. This is a so–called “ur” data flow anomaly.
The following data flow attributes can be allocated to the nodes of the control
flow graph with reference to the variable x:
Definition • x is defined: The variable x is assigned a value (for example, x = 5;).
abbreviation = d
Reference • x is referenced: The value of the variable x is read in a calculation or a
abbreviation = r decision, i.e., the value of x is not modified (for example, y = x + 1; or if
(x == 0)…).
Undefined; • x is undefined: The value of the variable x is destroyed (for example,
abbreviation = u destruction of local variables when ending a function). At the beginning of
a program, all variables are found in an undefined state.
• x is not used (empty): The instruction of the node does not influence the
variable x. x is not defined, referenced, or undefined.
Data flow attributes are often abbreviated as d (defined), r (referenced), u
(undefined), and e (empty). The empty action (e) can be neglected, since it does
not affect the data flow. If one looks at the execution of a program, the data flow
can be represented with respect to a specific variable by a sequence made up of
State automaton definitions, references, and undefinitions. The state automaton shown in Fig. 5.2
for data flow
describes the process of data flow anomaly analysis. Beginning with the start state
anomaly analysis
S, a variable is initially undefined. The variable must be defined before it can be
referenced. A correct data flow remains in the states “defined” and “referenced”
so that it can be ended with an undefinition in the state “undefined”. Each correct
data flow starts and ends in the state “undefined” and obeys certain rules, as
describd by the state automaton in Fig. 5.2. If the state “data flow anomaly” is
reached or the state “undefined” has not been reached at the end of a data flow
anomaly analysis, a data flow anomaly has been recognized. The state automaton
defines a so–called regular grammar. These grammars serve as a basis for the
lexical analysis performed in compilers. Therefore, data flow anomaly analysis
can be carried out by tools similar to compilers or may be integrated into
compilers.
Example 5.1: Let us look at the following two access sequences for a variable (u: undefined, d:
Two sequences definition, r: reference):
1. u r d r u
2. u d d r d u
Three types of data flow anomalies can be found: ur (reference of a variable with
an undefined value), dd (two sequential variable definitions), and du (variable
value definition is never referenced).
Sequence 1 starts with ur. The variable under consideration has a random value at
the time of the reference since it was not defined beforehand. There is a data flow
anomaly of the type ur; the reference of a variable with an undefined, random
value.
Sequence 2 includes two sequential variable definitions. The first definition does
not have any effect since the value is always overwritten by the second definition.
The data flow anomaly is of the type dd.
Sequence 2 ends with a definition followed by an undefinition. The value
assigned by the definition is not used since it is destroyed immediately afterwards.
This data flow anomaly is of the type du.
Figure 5.2:
State automaton for
data flow anomaly
analysis
void MinMax (int& Min, int& Max) Example 5.2:

Data flow
{ anomalies
int Temp;
if (Min > Max)

{
Max = Temp;
Max = Min;
Temp = Min;
}
}
The operation MinMax in the example contains data flow anomalies. Fig. 5.3 Import and
shows the control flow graph with the data flow attributes for MinMax. Since the export nodes
values of the variables Min and Max are imported into the operations through the
interface, the data flow form of the control flow graph must be used. Definitions Definite
of Min and Max must be assigned to the node nin. The export of Min and Max recognition of
data flow
when exiting the operation is represented by the assignment of the corresponding anomalies
variable references to the node nout. In the nodes nstart and nfinal all variable values
are set to undefined. The operation MinMax contains two paths. Table 5.1
represents the data flow attributes of the variables through the paths. The
sequences of the data flow attributes can now be examined for the two paths
regarding the data flow anomalies ur, dd, and du. Since this search for all
variables is carried out on all paths, anomalies are sure to be recognized. The
sequence udrddru for the variable Max contains two definitions directly following
one another – i.e., an anomaly of the form dd. The sequence urdu for “Temp”
starts with an ur–anomaly – the reference of an undefined variable – and ends
with a du–anomaly, i.e., a definition that becomes ineffective through subsequent
direct destruction of the value.
Figure 5.3:
Control flow graph
for the operation
MinMax
Table 5.1: Path nstart nin n1 n2 n3 n4 nout nfinal nstart nin n1 nout nfinal
Data flow table of Variable
the operation Min u d r r r r u u d r r u
MinMax Max u d r d d r u u d r r u
Temp u r d u u u
u – undefine d – define r – reference
The operation MinMax gets two numbers through an interface that need to be
sorted according to size and returned. The local variable “Temp” is undefined at
first. If Min is larger than Max, the value of “Temp” is assigned to the variable
Max. However, the value of “Temp” is random since it is neither initialized nor
has it been transferred through the interface.
This part of the operation is therefore definitely faulty. There is an ur–anomaly.
The two repeated value assignments on the variable Max are also suspicious. The
first assignment is always overwritten by the second assignment so that the first
one does not make sense. This is a dd–anomaly. Since “Temp” is not output but is
destroyed at the end of the operation, the previous assignment of a value to this
variable is a du–anomaly.
Fig. 5.4 provides the control flow graph of the corrected version of the operation.
The data flows in Table 5.2 do not contain any anomalies.
Path nstart nin n1 n2 n3 n4 nout nfinal nstart nin n1 nout nfinal Table 5.2:
Variable Data flow table of
Min u d r r d r u u d r r u the corrected
Max u d r r d r u u d r r u operation MinMax
Temp u d r u u u
u – undefine d – define r – reference
Figure 5.4:
Control flow graph
of the corrected
operation MinMax
5.6.3 Possible problems in data flow anomaly analysis

Until now it has been assumed that data flow anomaly analysis has to be carried Data flow
out for all paths. Fortunately, this is not correct. Otherwise the process could not anomaly analysis
also works with
even be used for small software components because the number of paths can an unrestricted
already be unmanageably large in quite simple programs. The explosion in the number of paths
number of paths is caused by loops. It is sufficient in data flow anomaly analysis

to analyze the paths until the second loop execution. If no data flow anomalies
have occurred until then, it is ensured that no anomalies will occur on the paths
with a higher number of loop iterations either. These facts are clarified in the
following.
An operation for determining the square root using the approximation procedure
of Newton's iteration will be used as an example. The operation shall determine
the square root for non–negative inputs. The value 0.0 shall be returned for
negative inputs. Due to the approximation process, it is difficult to estimate the
maximum number of loop iterations. This causes a potentially infinite number of
paths.
Example 5.3: double Sqrt (double X)

Sqrt operation {
double ReturnValue;
if (X > 0.0)
{
double W;
while (ABS (W * W – X) > 0.01)
{
W = W – ((W * W – X) / (2.0 * W);
}
ReturnValue = W;
}
else
{
ReturnValue = 0.0;
}
return (ReturnValue);
}
Fig. 5.5 shows the control flow graph with data flow attributes of the operation
Sqrt. The analysis of the path that is run through for non–positive input values is
quite simple. The following data flows are obtained, which do not have any
anomalies:
• X: u d r u
• W: u u
• ReturnValue: u d r u
nstart
u (X), u (W), u (ReturnValue)
nin d (X)
n1 if (X > 0.0) r (X)
n2 while (ABS ((W * W) – X) > 0.01) r (X), r (W)
n3 W = W ((W * W – X) / (2.0 * W)); r (X), r (W), d(W)
n4 ReturnValue = W; r (W), d (ReturnValue)
n5 else ReturnValue = 0.0; d (ReturnValue)
n6 return ReturnValue; r (ReturnValue)
nout
Figure 5.5:
Control flow graph
nfinal u (X), u (W), u (ReturnValue) for the operation
Sqrt
The iterative approximation process is carried out for positive inputs. The data
flow for the variable X starts with the sequence udrr up to the loop decision. If the
loop is not entered, the sequence u follows directly. If the loop is entered, the
sequence rr follows in between. This sequence is repeated with each further loop
execution. Therefore, the data flows on these paths can be indicated in closed
form. The data flow u d r r (r r)nu with n ≥0 applies to the variable X. The value n
indicates the number of loop executions. Closed expressions for the data flows are
also obtained for the variables W and ReturnValue.
• X: u d r r (r r)n u with n ≥0
• W: u r (r d r)n r u with n ≥0
• ReturnValue: udru
It can easily be seen that, if no data flow anomalies have occurred on the paths up
to the second loop execution, there will also be no more occurrences.
The operation Sqrt contains a data flow anomaly for the variable W. The data Data flow
anomalies are not
flow u r (r d r)n r u, n ≥ 0 starts with a “u r”–anomaly. The value of the variable W definitely found
has not yet been initialized when it is read for the first time. However, the by dynamic
operation works correctly for initial random values of W that happen to be testing
positive, so that a dynamic test does not recognize the error for sure. The negative
root is determined for negative initial values of W. If W is initially zero by
coincidence, the program “crashes”. Whereas this error can only be found in an
unreliable way using dynamic testing, it can definitely be recognized by a data
flow anomaly analysis. Fig. 5.6 shows the corrected version of the control flow
graph.
Data flow Frequently, a part of the theoretical paths that can be constructed cannot be
anomalies on executed since there are no corresponding input data or operational situations.
paths that cannot
be executed Data flow anomalies on this type of path that cannot be executed are not unusual,
although they can be avoided. They do not occur during program execution and
therefore cannot cause faulty behavior either. Since it is sometimes hard to
determine whether a certain path is executable, such anomalies must also be
removed if there are any doubts.
nstart
u (X), u (W), u (ReturnValue)
nin d (X)
n1 if (X > 0.0) r (X)
n2 W = 1.0; d (W)
n3 while (ABS ((W * W) – X) > 0.01) r (X), r (W)
n4 W = W ((W * W – X) / (2.0 * W)); r (X), r (W), d(W)
n5 ReturnValue = W; r (W), d (ReturnValue)
n6 else ReturnValue = 0.0; d (ReturnValue)
n7 return ReturnValue; r (ReturnValue)
nout
Figure 5.6:
Control flow graph
for the corrected nfinal u (X), u (W), u (ReturnValue)
operation Sqrt
5.6.4 Evaluation of data flow anomaly analysis

The analysis of data flow anomalies is useful for identifying only a limited type of
errors that cannot be identified by other quality assurance techniques, such as
dynamic testing. But these errors are sure to be revealed. Due to the low time
consumption compared to dynamic tests and the direct localization of errors, data
flow anomaly analysis is an extremely interesting technique in practice. The
optimal tool support for this is a data flow anomaly analysis component integrated
into the compiler. Some compilers already support this.
5.6.5 Evaluation of tool–supported static code analysis

In many application areas, certain static code analyses are required by standards.
But even if this is not required, the obligation to use state–of–the–art techniques
requires using the described methods. This particularly applies with respect to
their low time consumption due to their extensive level of automation.

Style analysis
Why is style analysis for programming languages such as Pascal not necessary? In
what form is it still recommended?

Data flow anomaly
The following Matlab code is given, which computes the maximum likelihood analysis
estimates and the 100(1–alpha) percent confidence intervals for uniformly
distributed data.
function unifit(x)
ahat = min(x);
ahat = 0; % for debugging purposes only
bhat = max(x);
tmp = (bhat - ahat) ./ alpha.^(1./length(x));
aci = [bhat - tmp; ahat];
bci = [bhat; ahat + tmp];
disp(sprintf('A_Hat= %f B_Hat= %f \n CI-A (lb)=%f \n
CI-A (ub)=%f \n CI-B(lb)=%f \n CI-B
(ub)=%f',ahat,bhat,aci(1),aci(2),bci(1),bci(2)));
end
What anomalies can be detected using data flow anomaly analysis? How can the
problems be solved?
Chapter 6 – Software Inspections and Reviews 81
6 Software Inspections and Reviews

• understand the significance of manual testing;
• differentiate between the three types of review techniques:
a) formal inspection techniques,
b) informal review techniques with review meetings, also called
“structured walkthrough”,
c) informal review techniques without review meetings;
• explain the procedure for carrying out formal inspections and describe
each phase in the inspection;
• understand the responsibilities of different inspectors in an inspection
team;
• understand how the inspection team should work to achieve efficient
testing;
• identify the advantages and disadvantages of formal inspections;
• know the characteristics of structured walkthroughs and informal reviews
without review meetings;
• identify important criteria to be kept in mind while carrying out structured
walkthroughs and informal reviews without review meetings.
6.2 Introduction
Manual testing of documents and codes is a commonly used method in practice.
Numerous ways exist of performing such analyses, including inspections, reviews,
peer reviews, and structured walkthroughs. In this chapter, the three main forms
of manual analyses will be presented. Formal inspections are a particularly
effective means for finding errors. They will therefore be discussed in detail in the
following. Since formal inspections are usually quite time–consuming, however,
we cannot reject simpler review methods completely. Manual analyses are
particularly important in the early phases of software development, like the
checking of design documents. They can also be used as a supplement to dynamic
tests of the source code.
82 Chapter 6 – Software Inspections and Reviews
6.3 Characteristics and objectives of software inspections and

reviews
The three variants Three main review techniques exist. Two of these require a review team to meet
of manual quality in order to go through documents together. A team review that obeys certain
assurance differ
in terms of formal rules is called a formal inspection; otherwise it is called a structured
efficiency and walkthrough. The third category does not require a review meeting. These
time consumption techniques are called informal review techniques without review meetings. The
documents are usually sent by email. Every reviewer can respond and report
faults, ambiguities, etc.
Regarding informal reviews with and without review meetings, various techniques
and classifications can be found in the literature. Some authors differentiate
among reviews, peer reviews, and structured walkthroughs. A structured
walkthrough is often seen as “less careful” compared to a formal inspection.
In the following, we will limit ourselves to the three main types of review
techniques mentioned above. These three review types have different levels of
performance and cause different amounts of time to be consumed. The most
powerful method is formal inspection. Formal inspection (Fagan inspection) will
be discussed in detail. It is particularly thorough, but unfortunately it is time–
consuming and expensive. Therefore, usually not all parts of a piece of software
are subjected to a formal inspection. Informal reviews with review meetings are in
the middle. Fewer resources are needed for these than for formal inspection
methods. The reduction in time consumed, however, goes hand in hand with
reduced performance. Doing reviews without a review meeting is the simplest
version. It is uncomplicated, can be carried out quickly and flexibly, and does not
consume much time. The limited performance of this method, however, is risky.
Therefore, reviews done with this technique are only used in uncritical tasks.
Semantics can All review techniques have similar characteristics and objectives. One positive
only be examined characteristic is their ability to consider semantic aspects. The contents of
manually
documents can thus be evaluated using the expertise of the review team members.
Many quality characteristics – e.g., understandability, maintainability – can only
be checked manually. These checks cannot be automated using tools. Empirical
examinations (e.g., /Moeller 96/) demonstrate that errors that occurred in an early
phase of software development, e.g., in the definition or design phase, cause very
high correction costs. It therefore makes sense to check early documents for faults
in order to fix them before they can cause major problems. Syntactic tests can be
carried out by CASE tools. Semantic tests require manual judgment by human
beings. Here, inspection and review techniques play a significant role. It is
advisable to use an inspection or a review at the end of every early development
phase as a milestone. This ensures that a verified result is transferred to the
Principle of subsequent software development phase. Since the participation of persons other
external quality than the author of the document to be inspected is required in all review and
control
inspection methods, these methods realize the principle of external quality control.
This principle requires quality control by persons who were not directly
concerned with the preparation of the document under inspection. The inclusion
of several people in inspection and review teams has another positive aspect. The
expertise regarding the common product is spread out into the development team.
Moreover, people learn more details about the work methods from their
colleagues, if necessary. These can possibly be taken over into one’s own work.
Moreover, the responsibility for the quality of the product is borne by the entire
development team. If serious problems occur in a product after an inspection has
been carried out, the responsibility for this cannot be given to the author alone.
The inspection team as a whole bears the responsibility. The awareness for quality
no longer focuses on every tiny work product but rather on the whole product
produced by the development team.
Experience shows that authors make an effort to use comprehensible forms of
expression when writing documents since several people have to assess the
products. This improves readability. An easily understood and explained style is
preferable. Critical components are identified early on by the timely use of
inspection and review methods during development. This information is important
with respect to appropriate risk management.
In the past, there were discussions about whether reviews or dynamic testing was
the better means for detecting errors. Experience shows that these two different
solutions are not alternatives. Reviews or inspections are a supplement to tool–
supported tests. Neither tool–supported static analyses nor dynamic testing can be
replaced by review and inspection methods. In cases where tool–supported static Inspections and
analysis is possible, it can be carried out, as it takes less time and delivers higher reviews
supplement other
quality than a manual analysis. Tool–supported data flow anomaly analysis (see testing methods
the previous chapter) can be seen as an example. Manual tracing of data flow
anomalies during a code inspection is possible, but does not make sense, since
tool–supported data flow anomaly analysis works more reliably and creates less
expenditure. With dynamic testing, only errors that lead to faulty behavior are
found. Deviations from the standards – e.g., violation of programming
conventions – are not recognized if they do not lead directly to errors. Such errors
are detected in reviews. In the case of reviews, the focus is on “local” errors.
Faulty behavior that occurs due to the interaction of remote parts of the code is
hardly ever found by a source code review. Reviews of architectural models can
handle this issue. Additionally, dynamic testing is more reliable in revealing this
type of faulty behavior. Therefore, dynamic testing methods, tool–supported static
analyses, and review and inspection techniques complement one another.
Empirical examinations show that formal inspections, in particular, are a very
effective and efficient means for detecting errors.
6.4 Formal inspections
6.4.1 Characteristics and objectives of formal inspections

Formal inspection Formal inspections were suggested in 1976 and 1986 by Michael Fagan /Fagan
= Fagan 76, 86/. In /Gilb, Graham 93/, they are described extensively. They are therefore
inspection
also known as Fagan inspections or simply as software inspections. The objective
of formal inspections is to increase the effectiveness and efficiency of the error–
Inspections
require planning revealing process by adhering to a precisely defined inspection process. It is
characteristic of formal inspection methods that they are carried out during exactly
defined phases with defined input and output criteria, a defined inspection rate,
and predefined goals to be reached, as well as with distributed roles. The time
consumed in inspections is high. Therefore, the necessary time has to be firmly
planned for in the project plan. Inspections require trained participants. The
strictly regulated process of the execution of an inspection requires a
correspondingly well–structured framework. Successful inspections need to be
embedded in a defined software development process that runs in a controlled
way. Moreover, there has to be a quality management process with defined quality
objectives from which the objectives of the inspections can be derived.
6.4.2 Description of the formal inspection

Inspections are As described above, the main difference between formal inspection methods and
based on a formal other review methods is their execution as a formal process. Inspections are
process
characterized by the following properties:
• Fixed input and output criteria
• Defined inspection phases
• Trained participants with fixed, delegated roles
• Collection and analysis of inspection data including feedback on the
inspection process
• Explicit documentation of the revealed errors
• Guidelines for the time required for preparation and the speed of
inspections
• Objectives for results
Inspections were originally intended as a testing method for designs and for
source code. In the meantime, however, inspections are used in all phases of the
software development process. There are inspections of requirements documents,
design, source code, and test cases.
Inspections are carried out in six phases: Phases in an

inspection
1. Planning: Organizational preparation
2. Overview: Distribution of information about the product
3. Preparation: Each inspector does the preparation on his or her own,
separately from the other inspectors
4. Inspection meeting
5. Subsequent work: Correcting faults
6. Follow–up: Checking of the corrections and preparation of inspection
reports
The planning phase serves the purpose of organizational preparation for the 1. Planning phase
inspection. As with all other activities in software development, a plan is also
prepared for inspections at the start of the project, including aspects of time,
expenditure, and resources. When the planned time has arrived and the author has
completed the product to be inspected, he reports this to the moderator. The
moderator checks whether the product fulfills the input criteria: If the product
does not fulfill the input criteria, the moderator informs the author about the
required modifications or additions. Input criteria typically concern syntactic rules
that can be easily checked. Thus, for example, the input criterion required for a
code inspection would be that the code has to be free of syntax errors. If
development documents are to be inspected that were compiled using a computer,
i.e., with so–called CASE tools, the automatic tests of the tools must not reveal
any errors. If the product fulfills the input criteria, it can be inspected as intended
in the project plan.
The overview serves the purpose of informing the inspectors about the product. 2. Overview
This can be useful for the following reasons:
• The product is extensive, complicated, or is related to numerous other
product parts.
• The development method used is new, e.g., the first project in a new
programming language.
• The product originated in a “one–man project”. The other inspectors
therefore require background knowledge.
As a rule, an overview session should take no longer than two to three hours. If
errors are already detected during the overview session, they must be corrected
before the material is distributed to the inspectors. As a rule, the overview session
is carried out by the author of the document to be inspected. In special cases,
however, it makes sense to let the overview session be conducted by another
person. If the overview session is conducted because of a newly used method, for
example, an internal methods consultant could be considered as the person chosen
to conduct it. If every inspector is already familiar with the work product, the
overview session is unnecessary, and then it can and should be left out. This
diminishes the costs for the inspection and therefore increases its efficiency.
3. Preparation The inspectors must prepare themselves for the inspection meeting. For this
purpose, each inspector receives a complete set of the required documents. These
documents cannot be modified until the inspection has been carried out. This
prevents the discussion of an old state of the product in the inspection meeting.
Since the continuation of work on the product is stopped until the inspection
meeting, the inspection has to be carried out with high priority. The inspectors all
prepare on their own, individually, for the inspection meeting based on the
documents. They note down all the errors they find and the points that are not
clear. There are guidelines on the time to be spent on preparation, based on which
the preparation time is planned. If too little time is assigned for preparation, the
inspectors will have little knowledge during the inspection meeting. They would
be almost unprepared and only get to know the product during the actual
inspection meeting. Therefore, they would find relatively few mistakes. This
would reduce the efficiency of the inspection. If too much time is used for
preparation, efficiency is also reduced, since preparation time is added to the
inspection effort. Efficiency can be expressed as a quotient of the number of
errors found and the effort consumed for finding them. There has to be a medium,
optimal value for the preparation time. This value may be different in each
organization and for the various types of inspection. However, there are
guidelines in the literature that are given below. The main objective of preparing
for the inspection is to gain an understanding of the product. The objective is not,
first and foremost, the detection of errors. If errors are found, this is a desired side
effect. However, the crucial factor is that each inspector has a good understanding
of the function of the product to be inspected after preparation.
4. Inspection Conducting the inspection meeting is the central phase of a formal inspection. The
meeting participants in the inspection meeting are assigned the following roles:
• Moderator
• Author
• Reader
• Recording clerk
• Further inspectors
The moderator The moderator must be a recognized specialist with special training as a
should have moderator of inspections. The main literature on the subject requires the
completed special
training moderator to be technically competent. This would mean that the moderator of a
code inspection, for example, must understand the programming language used.
This is definitely desirable, but not absolutely necessary. What is more important
than specialist technical knowledge is the ability to moderate. The moderator
conducts the meeting and guarantees that the inspection is carried out in the
planned manner. He/she must specifically see to it that all inspectors work
according to their competencies. The moderator must prevent individual

inspectors from dominating the inspection team and other inspectors from not
saying a word. The main job of the moderator is to create synergy. He/she must
see to it that the inspection team “gets going”. Experience shows that this
synergetic cooperation between inspectors is an important condition for high
efficiency. The moderator must ensure that all inspectors focus on the recognition
of errors. Discussions are only permitted about errors and types of errors. Based
on experience, inspections have a tendency to sidetrack into a discussion on
solutions after a few errors have been found. This is undesirable since it reduces
the efficiency of the inspection. The moderator must prevent this.
The author is the creator of the product to be inspected. He/she is responsible for The author does
correcting the errors found during inspection. As a specialist for the technical not play any
further role
realization of the product, he answers the questions of the other inspectors. The
author can never be the moderator, reader, or recording clerk.
The reader guides the inspection team through the meeting. He/she reads out the The reader guides
technical content, explaining it at the same time. The reader must therefore be in a the meeting with
respect to
position to describe the different parts of the work. Unlike the moderator, He/she technical matters
must therefore be a technical specialist. The reader is never the same person as the
author. This rule achieves two positive effects. First, it prevents the author, who
knows the details of his work very well, from dominating the inspection team by
forcing his technical views on the inspection team at his own specific reading
speed. Second, the technical reading by another person permits the author to view
his own work from an almost external point of view. This offers him the chance to
recognize specific errors in his own product. What applies to the preparation
speed also applies to the reading speed – the so–called inspection rate. If the
reading speed is too high, the result is a superficial inspection. If it is too low, the
time spent on the inspection is too high and thus efficiency is too low. Statements
on the optimal reading speed can be found in the literature. They are given below.
The recording clerk of the meeting writes down and classifies all errors found The recording
during the inspection. Moreover, he assists the moderator in preparing the clerk documents
the results
inspection reports.
All members of the inspection team – including the moderator, author, reader, and All team members
recording clerk – are inspectors, whose objective must be to detect errors. The inspect
following persons can also be considered as inspectors:

• Project staff from the same project
• Methods consultants (adherence to standards)
• Systems specialists
• Persons appointed for data protection
The inspection team should be as small as possible. As a rule, there should be Small inspection
between three and seven people in it. It only has to cover the necessary inspection teams increase
efficiency
expertise in a team. Small inspection teams increase the efficiency of the
inspection since the number of members of the inspection team affects the
inspection effort in a linear fashion. In order to be equally efficient, a team made
up of six people should therefore detect twice as many errors in the same time as a
team made up of three persons. This cannot be expected. The minimum number of
participants during inspections is three. If only three persons form an inspection
team, the moderator also takes on the role of the recording clerk. The remaining
two persons are the author and the reader.
IMPORTANT: It is extremely important that the results of inspections (as well as other review
Inspection results methods) are not used for assessing staff members. Carrying out an inspection is
should not be used
for assessing staff sometimes an unpleasant situation for the author of a product. A group of
members colleagues criticizes his work. However, experience from inspections teaches us
that the results of the inspection help to improve one’s own work and to reduce
problems later on. Nevertheless, a prerequisite for this is that inspections are
understood as a purely technical instrument. The personal value given to staff
must be fully detached from the inspection results. If the number of errors found
in the inspection is used as an indicator of the staff’s efficiency, inspections are
not used as an instrument for assuring technical quality, but as an assessment
procedure for each person’s own performance. This leads to the author attempting
to deny as many mistakes as possible, which reduces the efficiency of the
inspection and makes the method almost useless. A simple rule that helps to
prevent personal assessment of the author in the inspection is the requirement to
let only people with the same rank (peers) work together during inspections.
Checklists are The errors found during the inspection meeting must be classified to the extent
meaningful possible. It is advisable to have the recording clerk use a given classification
scheme for this purpose. It also makes sense to use checklists. On the one hand,
this ensures that no tests are left out. On the other hand, working through a
checklist helps to recognize whether the inspection is progressing continuously or
is going around in circles. This is an indicator for the moderator to defer the
current point in the checklist and to proceed with the next point.
A goal–oriented inspection performed according to this procedure is strenuous.
Synergetic work cannot be maintained over a long period of time without feeling
exhausted. Experience shows that the maximum duration for an inspection should
not exceed two to three hours. At the end of the inspection meeting, a decision is
made on whether the product is accepted, conditionally accepted, or whether a re–
inspection is necessary.
5. Subsequent Next, subsequent work on the inspection must be carried out. The author works on
work the errors noted in the inspection records. In the simplest case this means
correcting the errors. However, it is also possible that errors cannot be corrected
immediately. The error must then be introduced into change request procedures to
enable an appropriate decision to be made about its correction. An example of this
type of situation is the recognition of a design error during a code inspection. The
design documents are already under configuration control and cannot be modified.
The correction of errors must therefore take place through change management.
Moreover, the correction of errors can show that a presumably erroneous point is,
in fact, correct, contrary to the view expressed during the inspection. In such
cases, the author must put forward his point of view in the so–called follow–up.
After the author has worked on the list of errors, the product can be brought under
configuration control if it has been accepted in the inspection meeting. Usually,
products containing only a few small errors are accepted under the condition that
these errors are corrected by the author. Checking the correction of errors is not
necessary in this case. If the product was accepted conditionally during the
inspection meeting or if re–inspection is required, further steps are necessary.
These steps take place during the last inspection phase, the follow–up. If the 6. Follow–up
product was conditionally accepted during inspection, the moderator can check
the corrections together with the author. However, if the moderator is not a
technical specialist, He/she will not be a suitable partner for executing the test. In
this case, it makes more sense to appoint another person from the inspection team
to check the correction of errors. As a rule, the reader is selected due to the
required technical competence. It is recommended that in the case of a conditional
acceptance of the product, the correction of errors should be checked by the
author and the reader together.
If, during the inspection, the decision is made that a re–inspection is necessary, an
inspection meeting must take place again. The same inspectors take part in this
meeting. The meeting focuses on the errors found in the previous inspection
meeting. The record of errors from the inspection meeting is used and the
modifications of the product are checked. As a rule, re–inspections are critical
because they are not taken into consideration in the project plan and therefore
often lead to delays. Pending inspection reports are prepared in the follow–up.
These concern technical reports on detected errors, in particular, on expenditure,
and on the preparation and inspection time.
Numerous publications show that formal software inspections are both a very Inspections are
effective and a very efficient means of detecting errors. Relevant empirical data efficient and
effective but time–
can be found, for example, in /Thaler, Utesch 96/ and in /Ebert 00/. Further data is consuming
included in /Fagan 76/ and /Fagan 86/. The number of errors detected per
document size is called effectiveness. Efficiency is the quotient of the number of
errors detected and the effort required for finding them. Therefore, efficiency is a
yardstick for economic viability.
Thaler and Utesch compared the efficiency and effectiveness of software
inspections with conventional review methods and with dynamic tests. The results
are shown in Tab. 6.1. On the one hand, software inspections are superior to
conventional review methods (i.e., walkthroughs) with respect to efficiency and
effectiveness. On the other hand, Tab. 6.1 also shows the principal problem of
formal inspection methods: They are very time–consuming. Therefore, they can
only be used on relatively small parts of a product. In spite of their high efficiency
and good economic viability, formal inspections are usually too time–consuming
and expensive to apply to a complete project. Further statements on the efficiency
of inspection methods can be found in /Russel 91/, /Schnurrer 88/ and /Kosman,
Restivo 92/. Due to the relatively large amount of time consumed by formal
inspection methods, conventional review methods cannot be outright rejected.
The publications mentioned above do not compare the different types of errors
that can be revealed by specific quality assurance techniques.
Empirical data In /Fagan 86/, Fagan indicates a rate of 500 net source code lines per hour for the
overview session. Net source code lines can be understood as source code lines
without comments. Fagan indicates 125 net source code lines for the preparation
rate per hour. The inspection speed should be 90 net source code lines per hour.
The maximum inspection rate should not exceed 125 net source code lines per
hour. Fig. 6.1 shows empirical results from /Thaler, Utesch 96/. It can be clearly
recognized that the effectiveness of the inspection increases as the inspection rate
decreases. Still, one should not aim at an extremely low empirical data inspection
rate purely for economic reasons. Fig. 6.2 shows empirical data from /Ebert 00/.
As shown in the upper diagram, effectiveness also increases here as the inspection
rate decreases. The reverse value of efficiency shown in the lower diagram,
however, shows a minimum. To put it differently: Efficiency increases with a
decrease in the inspection rate until a maximum value is reached and then drops
again if the inspection rate decreases further. According to /Ebert 00/, the
optimum lies at about 90 instructions per staff member hour.
Table 6.1:
Efficiency and
effectiveness
comparison of
testing methods
from /Thaler,
Utesch 96 /
Figure 6.1:
Empirical data on
the inspection rate
from /Thaler,
Utesch 96/
Figure 6.2:
Empirical data on
the inspection rate
from /Ebert 00/
6.5 Informal review techniques with review meetings:

structured walkthrough
Informal reviews with review meetings take the middle path between formal Informal reviews:
inspection methods and even simpler approaches with respect to efficiency, a compromise
between time
economic viability, and use of resources. On the one hand, they offer less consumed and
efficiency and effectiveness than formal inspections. On the other hand, the time efficiency
consumed, the costs, and the use of resources are also reduced compared to formal
inspection methods. Reviews with review meetings and inspections require a
dedicated inspection meeting to be conducted. The important difference between
formal inspections and reviews is that the reviews are not carried out according to
a formal procedure and there are no defined roles for the team participants.
Moreover, review data is often not recorded and analyzed. Furthermore, neither
quantitative objectives nor input and output criteria are defined. Whereas formal
inspections are aimed only at detecting errors, conventional reviews can also be
used as an instrument for making decisions.
Conventional review methods can be found under a number of designations with
partly unclear demarcations and often only a few differences. Typical descriptions
are peer review and structured walkthrough. As shown in relevant empirical
examinations, these methods are widespread in practice. In spite of their low
efficiency compared to formal inspection methods, people still want to use them.
Apart from the already described aspect of a reduction in effort, their many Informal reviews
possibilities for use and their high flexibility play an important role. Principally, are flexible
the same basic conditions apply for the execution of conventional reviews as in
the case of formal inspection methods. Particular care must be taken that teams in
informal reviews are rather small and team members lack the necessary
experience to efficiently conduct the reviews. It is especially important to create a
specialized atmosphere for conventional reviews as well. Conventional review
methods are described in detail in /Yourdon 89/. Empirical data on the efficiency
of conventional review methods are included in /Thaler, Utesch 96/.
6.6 Informal review techniques without review meetings

Reviews in These techniques form the lower end of the scale in manual analysis methods with
commentary respect to efficiency and effectiveness. This disadvantage is diametrically opposed
techniques cause
less time to the advantage of low effort consumption. It is characteristic of these reviews
consumption that no meeting is required. The documents to be checked are distributed to the
reviewers. In the past, this was typically done by sending them through internal
mail. Nowadays email or Intranet is used. Since there is no uniform procedure, it
is particularly advisable here to use checklists. These checklists serve the purpose
of determining the characteristics to be checked. They should therefore be
enclosed when sending the documents. A deadline is usually set by which the
review has to be conducted. As a rule, it is assumed that exceeding the deadline
corresponds to implicit agreement. Therefore, there is a risk when using these
reviews that, for example due to work overload, review deadlines pass and some
of the documents have not even been checked. Such reviews are therefore only
used for relatively uncritical product parts. These could be slightly modified
peripheral software modules, for example, where no major problems are expected.
6.7 Evaluation of software inspections and reviews

Reviews are an appropriate complement to dynamic testing and tool–supported
static analyses when checking code. As methods for checking documents in early
development phases, they are at least equally important. The technical content of
analysis and design documents cannot be checked with tool support. Here,
inspection and review methods are the only sensible approaches. Reviews and
inspection methods are not an alternative to other solutions such as dynamic
testing, static analyses, or formal methods, but rather an efficient complement.

Inspection meeting
List the different roles assigned in an inspection meeting.

Inspection phases
What are the different phases of an inspection?
Solutions to the Problems

General remarks: In many cases, there are several possible answers to the
problems. Most important are explanatory statements for the solutions. In the
following, possible answers are given. In many cases, there will be correct
answers that differ from these solutions.
Solutions to the problems for Chapter 1

Solution to problem 1.1: Solution to
problem 1.1
Reliability: Measure for the ability of an item to remain functional, expressed
by the probability that the required function is executed failure–free
under given working conditions during a given time period.
→ Continuity of correct service
Availability: Measure for the ability of an item to be functional at a given time
→ Readiness for correct service

problem 1.2
Failure
of a system: Inconsistent behavior wrt. specified behavior during the execution
Fault/defect: Statically existent cause of a failure, i.e., a “bug”
Error: Basic cause of the fault (e.g., misunderstanding of a particular
statement of the programming language)

problem 1.3
true false
Correctness has a binary character.  

Even if there are no defects,  
the program might not be correct.
An artifact is not consistent with  
its specification if it is not correct.
Robustness has a binary character.  
A correct system can have low robustness.  
Robustness is a property only of the  
implementation.
94 Solutions to the Problems
Correctness
• Correctness has a binary character, i.e., an item is either correct or
incorrect.
• A fault–free realization is correct.
• An artifact is correct if it is consistent with its specification.
• If no specification exists for an artifact, correctness is not defined.
Robustness
• The property to deliver an acceptable behavior also in exceptional
situations (e.g., ability of a software to detect hardware failures).
• A correct system – as measured by the specification – can actually have
low robustness.
• Accordingly, robustness is rather a property of the specification than of the
implementation.
• A robust program is the result of the correct implementation of a good and
complete specification.
• Robustness has a gradual character.

Solution to Solution to problem 2.1:
problem 2.1
#<identifier > – stands for the number of characters of the speficied identifier
(e.g., #Name is the number of characters a Name can have).
Valid equivalence classes Invalid equivalent classes
Name 0 <= # Name <= 20 1) #Name > 20 2)
Year of birth (YOB) 1900 <= YOB <= 1990 3) YOB < 1900 4)
YOB> 1990 5)
ZIP 0 <= ZIP <= 99999 6) ZIP < 0 7)

ZIP>99999 8)
Street 0 <= #Street <= 20 9) #Street > 20 10)


problem 2.2
Testcase Equ. classes Name YOB ZIP Street
1 1,3,6,9 F. Walter 1954 67663 Fifth Ave
2 2,3,6,9 ABC…T 1954 67663 Fifth Ave
5 1,3,7,9 F. Walter 1954 –1 Fifth Ave
7 1,3,6,10 F. Walter 1954 67663 ABC…T
Test case 1 represents the combination of valid equivalence classes

Test cases 2 to 7 contain invalid cases. The variables with invalid values are
marked in gray.

problem 3.1
a)
nstart
n in
n0 int a0;
c
n1 int a1;
d
n2
int j;
e
n3 int i = length() -1;

f
n4 while (i>=0)
g
n5 j=0;
h
n6 while (j<i )
i
n7 a0=get(j);
j
n8 a1=get(j+1);
s k
q r
n9 p if (a0>a1)
l
n 10 put(j,a1);
put
o m
n 11 put(j+1,a0);
n
n 12 j++;
n 13 i--;
nout
t
nfinal
b) The goal of statement coverage is to execute each statement at least once, i.e.,
to execute all nodes of the control flow graph. This can be done with:
abcdefghijklmnpqrst.
c) Branch coverage is a stricter testing technique than statement coverage.
Statement coverage is fully contained in branch coverage. Branch coverage
subsumes statement coverage. It aims at executing all branches of the program to
be tested. This requires the execution of all edges of the control flow graph:
abcdefghijklmnpijkopqrst

problem 3.2
a)
Exhaustive evaluation
A B C D
a<0 b>0 c==0 d!=0 Total
1 T F T F F
a=-1 b=0 c=0 d=0
2 F T F T F
a=0 b=1 c=1 d=1
Non–exhaustive evaluation
A B C D
a<0 b>0 c==0 d!=0 Total
1 T F T F F
a=-1 b=0 c=0 d=0
2 F - F - F
a=0 b=? c=1 d=?
3 T T - - T
a=-1 b=1 c=? d=?
4 F - T T T
a=0 b=? c=0 d=1
b)
A B C D E F Total
a<0 b>0 c==0 d!=0 A&&B C&&D E||F
1 T F T F F F F
a=-1 b=0 c=0 d=0
D B 2 T T T F T F T
a=-1 b=1 c=0 d=0
A 3 T F T T F T T
a=-1 b=0 c=0 d=1
4 F T T F F F F
C a=0 b=1 c=0 d=0
5 T F F T F F F
a=-1 b=0 c=1 d=1

problem 4.1
nstart
a
nin
n0 int a0;
c
n1 int a1;
d
n2 int j;
e
def(i) n3 int i = length()-1;

f
p-use(i) p-use(i)
n4 while (i>=0)
g
def(j) n5 j=0;
h p-use(i),
p-use(i), p-use(j)
n6
p-use(j) while (j<i)
i
c-use(j),
n7 a0=get(j);
def(a0)
j
c-use(j), n8 a1=get(j+1);
def(a1)
s k
q r
p-use(a0), n9 p if (a0>a1)
p-use(a1) p-use(a0),
l
p-use(a1)
c-use(j), n10 put(j,a1);
c-use(a1) o m
c-use(j),
n11 put(j+1,a0);
c-use(ao)
n
c-use(j), n12 j++;

def(j)
c-use(i),
def(i) n13 i--;
nout
t
nfinal

problem 5.1
If programming languages with a restrictive syntax are used (for example, Pascal),
style analysis can be rejected. However, this is only applicable with respect to the
checking of limiting rules. Checking expanding rules (for example, the use of
assertions) also makes sense in programming languages such as Ada and Pascal.

problem 5.2
Variables:
x: u d r r r u
ahat: u d d r r r r u ( dd anomaly)
bhat: u d r r r r u
tmp: u d r r u
alpha: u r u (ur anomaly)
aci: u d r u
bci: u d r u
We have problems with the variables ahat and alpha. The first definition of ahat
has no effect; it is always overwritten by a second definition. Alpha is referenced
before it is defined. As a solution, we remove the second definition of ahat and set
alpha as a function parameter:
function unifit(x, alpha)

if nargin < 2
alpha=0.05;
end
ahat = min(x);
bhat = max(x);
tmp = (bhat - ahat) ./ alpha.^(1./length(x));
aci = [bhat - tmp; ahat];
bci = [bhat; ahat + tmp];
disp(sprintf('A_Hat= %f B_Hat= %f \n CI-A (lb)=%f \n CI-
A (ub)=%f \n CI-B(lb)=%f \n CI-B (ub)=%f',ahat,bhat...
,aci(1),aci(2),bci(1),bci(2)));
end

problem 6.1
• Moderator
• Author
• Reader
• Recording clerk
• Further inspectors

problem 6.2
• Planning: Organizational preparation
• Overview: Distribution of information about the product
• Preparation: Each inspector prepares on his or her own, separately from
the other inspectors
• Inspection meeting
• Subsequent work: Correcting faults
• Follow–up: Checking of the corrections and preparation of inspection
reports
Index 103
Index
defs/uses test ............................................... 57, 61
A
dpu(x, ni)................................................ vi, 58, 59
all c–uses criterion ........................................ v, 59 du–path .............................................................. vi
all c–uses/some p–uses criterion ................... v, 59 dynamic techniques .......................................... 10
all du–paths criterion .................................... v, 60 dynamic testing........................................... vii, 12
all p–uses criterion........................................ v, 59
all p-uses/some c-uses criterion ........................ 60 E
all p–uses/some c–uses criterion ......................... v equivalence class testing ................................... vii
all uses criterion............................................ v, 60 equivalence classes ............................... 19, 22, 24
all–def criterion .................................................. v error .......................................... vii, 4, 6, 9, 10, 12
author.................................................v, 86, 87, 88
availability .............................................. v, 4, 5, 7 F
failure ...................... vii, 4, 6, 8, 26, 35, 36, 38, 71
B
fault .......................................................... vii, 4, 6
backward slicing ........................................... v, 67 formal inspection ................. 81, 82, 84, 86, 89, 91
boundary interior coverage ............................... 49 formal inspection methods.............. vii, 84, 90, 91
boundary interior path test .......................... 50, 60 formal verification ............................................ 14
boundary interior testing..................................... v forward slicing ............................................ vii, 67
branch coverage test ........................ vi, 13, 36, 37 functional equivalence class construction ... 19, 24
functional equivalence class testing .................. vii
C
functional testing .................................. vii, 12, 13
condition coverage ........................................... 49 function–oriented test techniques ......... 17, 18, 28
condition coverage test ...............vi, 33, 39, 41, 52 function–oriented testing ...........17, 18, 28, 29, 30
condition/decision coverage test .................. vi, 43
control flow graph ... vi, 34, 35, 36, 37, 52, 56, 57, I
58, 68, 69, 72, 73, 74, 75, 76, 77, 78, 97 informal review techniques................... vii, 91, 92
control flow test ................................... viii, 33, 52 inspections ............... 82, 83, 84, 85, 88, 89, 91, 92
control flow–oriented testing ............................ 34 Integrated Quality Assurance ...................... viii, 2
control flow–oriented testing techniques .... vi, 11,
33, 34 L
correctness ............ vi, 2, 4, 5, 6, 12, 13, 17, 55, 64
loops ........................ 34, 37, 49, 50, 51, 52, 66, 76
D M
data flow anomalies .. vi, 14, 71, 73, 76, 77, 78, 83
minimal multiple condition coverage test .. vii, 39,
data flow anomaly analysis. 14, 71, 72, 73, 75, 78,
44
83
moderator............................ vii, 85, 86, 87, 88, 89
data flow attributed control flow graph ....... vi, 56
modified condition/decision coverage test. vii, 39,
data flow–oriented test .......................... vi, 55, 61
45
dcu(x, ni) ................................................ vi, 58, 59
multiple condition coverage test .......... viii, 39, 46
decision tables ............................................. vi, 29
decision trees ......................................... vi, 29, 30
def(ni) .................................................... vi, 58, 59
104 Index
P security ............................................................ 5, 7
simple condition coverage .. viii, 39, 40, 41, 42, 43
path coverage test ................................ viii, 50, 52
slicing .................................................. viii, 67, 71
Principle of Integrated Quality Assuranceviii, 2, 4
Software Quality Assurance ............................ viii
Q state–based test .................................... viii, 25, 27
state–based testing ............................... viii, 25, 27
quality ......................................................... 2, 4, 5
statement coverage test..................... ix, 13, 35, 36
quality characteristic ......................viii, 3, 5, 8, 82
static analysis .............................................. ix, 13
quality measure ....................................... viii, 4, 6
static code analysis .................................ix, 63, 64
quality requirement ..........................viii, 1, 3, 4, 5
static techniques ................................................10
quality target specification ...................... viii, 1, 3
structured path test .......................... 49, 50, 51, 60
R structured testing ......................................... ix, 51

structured walkthrough ..................... ix, 81, 82, 91
reader ....................................... viii, 86, 87, 88, 89
structure–oriented testing techniques ix, 12, 33, 34
recording clerk ...............................viii, 86, 87, 88
style analysis .................................... ix, 14, 65, 67
reliability .................................... viii, 2, 4, 5, 7, 13
symbolic testing ................................................14
review techniques ............................ vii, 14, 82, 92
reviews ............................................ 82, 83, 91, 92 T
robustness ............................................ viii, 4, 5, 8
tool–supported static code analysis . 63, 64, 65, 79
S transaction flow testing ............................... ix, 28
safety ................................................... viii, 4, 5, 7

Textbook SQA+ (Liggesmeyer) P2

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Textbook SQA+ (Liggesmeyer) P2

Caricato da

Copyright:

Formati disponibili

POSTGRADUALE FERNSTUDIENGÄNGE

SCIENCE & ENGINEERING

TEXTBOOK V-M.1.1 SOFTWARE ENGINEERING FOR EMBEDDED SYSTEMS

4th edition 2017

3.4.3 Evaluation of the statement coverage test 36

5.6.3 Possible problems in data flow anomaly analysis 75

About the Author

Möller, K.–H.: Ausgangsdaten für Qualitätsmetriken – Eine Fundgrube für

References for Chapter 2

References for Chapter 3

References for Chapter 4

References for Chapter 5

References for Chapter 6

Objectives of the Textbook

1.1 Learning objectives of this chapter

required, comprising both construction techniques and analytical examination. It

Analysis Errors Analysis Analysis Errors

- Testing of UML tools

- Testing of UML tools

- Compiler syntax testing

1.4 Definition of terms

practice to classify errors as tolerable and non–tolerable. Different error priorities

1.5 Software quality assurance

From the cost perspective, early quality assurance is especially important. In a

1.6 Classification of testing, analysis, and verification

static verifiable formal Assertion techniques

control flow-oriented Statement coverage test

data flow-oriented Defs-/Uses-criterion

function-oriented Functional equivalence class construction

diversifying Back to back test

Range test Path domain test

1.6.1 Dynamic testing

The classification of dynamic tests according to Fig. 1.4 is essentially based on

1.6.2 Static analysis

Even though a static analysis can be performed manually, tool support is

1.6.3 Formal techniques: symbolic testing and formal

1.7 Problems for Chapter 1

Problem 1.2: Problem 1.2:

Problem 1.3: Problem 1.3:

Even if there are no defects,  

An artifact is not consistent with  

Robustness has a binary character.  

A correct system can have low robustness.  

Robustness is a property only of the  

2.1 Learning objectives of this chapter

2.3 Characteristics and aims of function–oriented tests

evaluating the software responses. Because of their derivation from the

The concrete implementation of the software is irrelevant in function–oriented

2.4 Functional equivalence class construction

2.4.1 Characteristics and objectives of functional equivalence

2.4.2 Building equivalence classes

can be further differentiated as electronic or acoustic piano, which need to be

Table 2.1: Input Valid equivalence classes Invalid equivalence classes

Length 5) 100 ≤ length ≤ 500 6) Length < 100

7) Length > 500

Number of 8) 1 ≤ number ≤ 9999 9) number < 1

Order number 11) First character is W 12) First character is not W

(additionally) 1) 2) 3) 4) 6) 7) 9) 10) 12)

Length 200 300 300 300 50 1000 200 200 200

Number of 100 200 100 100 100 100 0 50000 100

Length 100 500 300 300 99 501 200 200 200

Number of 1 9999 100 100 100 100 0 10000 100

2.4.3 Evaluating functional equivalence class construction