3/18/2013

Model & Specification-based Software Engineering

Specifications

MODEL & SPECIFICATION-BASED SOFTWARE TESTING

Specifications = Requirements Specification: Precise and detailed description of the systems functionality and constraints. Intended to communicate what is required to system developers and serve as the basis of a contract. Written for technical audience.

Instructor: Nasir Minhas

nasirminhas@uaar.edu.pk

Specification-based Testing

Specifications

Specification-based testing uses the specification of the program as the point of reference for test data selection and adequacy. What specification ?

A specification can be any one (or more) of the following: A written document A collection of user scenarios (use-cases) A set of models A formal, mathematical description A prototype

Specifications as Written Document

Specifications as Written Document

Command: lookup Syntax: lookup <pattern> <file> Function: The lookup command is used to locate one or more instances of a given pattern in a text file. All lines in the file that contain the pattern are written to standard output. A line containing the pattern is written only once, regardless of the number of times the pattern occurs in it. The pattern is any sequence of characters whose length does not exceed the maximum length of a line in the file. To include a blank in the pattern , the entire pattern must be enclosed in quotes (). To include a quotation mark in the pattern , two quotes in a row () must be used.

Examples: lookup john myfile

displays lines in the file myfile which contain john

lookup john smith myfile

displays lines in the file myfile which contain john smith

lookup john smith myfile

displays lines in the file myfile which contain john smith

3/18/2013

Specification as Written Document

Typically written in natural language. Readable by all, but... Lack of clarity. Precision is difficult without making the document difficult to read.

Modeling

A model is an abstraction of the system being studied. Different models present the system from different perspectives Various semi-formal and formal notations are available for modeling

Requirements confusion. Functional and nonfunctional requirements tend to be mixed-up.

Diagrams, tables, structured language, etc.

Mostly visual, capture structure and some semantics Facilitate communication with different audiences Precise semantics, reasoning possible Detailed models

Requirements amalgamation. Several

different requirements may be expressed together.

Finite State Machines (FSMs), grammars, set theory, ...

Modeling Techniques

Specification-based Testing

Modeling Information & Behavior

E-R (Information modeling) SADT (Structured Analysis) UML (Object Oriented Analysis) Z, Larch, VDM, Pre/Post Conditions (Formal Methods)

Modeling System Qualities

Timed Petri Nets (Performance) Task Models (Usability) Probabilistic MTTF (Reliability)

The multitude of specification flavors, coupled with different degrees of conduciveness for analysis have made life difficult for people looking to develop encompassing technology for specification-based testing. There has been attention from the research community, however small, fragmented steps have been accomplished in producing techniques for the practitioner. In general, when testing industrial software systems, testers may need to rely on more than one approach to test to the specification.

Formal Specification

In traditional engineering disciplines, such as electrical and civil engineering, progress has usually involved the development of better mathematical techniques. Mathematical analysis is a routine part of the process of developing and validating a product design. However, software engineering has not followed the same path. Although there has now been more than 30 years of research into the use of mathematical techniques in the software process

FORMAL SPECIFICATION

3/18/2013

Formal Specification

Formal Specification

The term formal methods is used to refer to any activities that rely on mathematical representations of software including formal system specification, specification analysis and proof, transformational development, and program verification. A formal software specification is a specification expressed in a language whose vocabulary, syntax and semantics are formally defined.

Formal specifications use mathematical notation to describe in a precise way the properties which an information system must have, without unduly constraining the way in which these properties are achieved. They describe what the system must do without saying how it is to be done. This abstraction makes formal specifications useful in the process of developing a computer system

Specification in the Software Process

Specification and design are intermingled. Architectural design is essential to structure a specification. Formal specifications are expressed in a mathematical notation with precisely defined vocabulary, syntax and semantics.

In the 1980s, many software engineering researchers proposed that using formal development methods was the best way to improve software quality. They argued that the rigor and detailed analysis that are an essential part of formal methods would lead to programs with fewer errors and which were more suited to users needs. They predicted that, by the 21st century, a large proportion of software would be developed using formal methods. This prediction has not come true (Why is this the case?)

Why Arent Formal Methods Used?

Why Arent Formal Methods Used?

Inherent management conservatism. Many software engineers lack the training in discrete math necessary for formal specification. System customers may be unwilling to fund specification activities. Some classes of software (particularly interactive systems and concurrent systems) are difficult to specify using current techniques.

There is widespread ignorance of the applicability of formal specifications. There is little tool support available for formal notations. Some computer scientists who are familiar with formal methods lack knowledge of the real-world problems to which these may be applied and therefore oversell the technique.

3/18/2013

Advantages of Formal Specification

Advantages of Formal Specification

It provides insights into the software requirements and the design. Formal specifications may be analyzed mathematically for consistency. It may be possible to prove that the implementation satisfies the specification.

Formal specifications may be used to guide the tester of the component in identifying appropriate test cases. Formal specifications may be processed using software tools. It may be possible to animate the specification to provide a software prototype.

Approaches to Formal Specifications

Formal Specification Languages

Two fundamental approaches to formal specification have been used to write detailed specifications for industrial software systems. These are:
An algebraic approach where the system is described in terms of operations and their relationships. A model-based approach where a model of the system is built using mathematical constructs such as sets and sequences and the system operations are defined by how they modify the system state.

Different languages in these families have been developed to specify sequential and concurrent systems.

Our Concern would be with Model Based Sequential Languages we will discuss Z and probably VDM

Seven Myths of Formal Methods

Seven Myths of Formal Methods

Perfect software results from formal methods

Nonsense - the formal specification is a model of the realworld and may incorporate misunderstandings, errors and omissions. Formally specifying a system is valuable without formal program verification as it forces a detailed analysis early in the development process.

Formal methods are for mathematicians

Nonsense - only simple math is needed. Not proven. However, formal methods definitely push development costs towards the front-end of the life cycle. They can if they are paraphrased in natural language. There are now many published examples of experience with formal methods for non-trivial software systems.

Formal methods increase development costs

Formal methods means program proving

Clients cannot understand formal specifications

Formal methods can only be justified for safety-critical systems

Formal methods have only been used for trivial systems

Industrial experience suggests that the development costs for all classes of system are reduced by using formal specification.

3/18/2013

The Verdict!

Use of Formal Methods

The reasons put forward for not using formal specifications and methods are weak. However, there are good reasons why these methods are not used:
The move to interactive systems. Formal specification techniques cannot cope effectively with graphical user interface specification. Successful software engineering. Investing in other software engineering techniques may be more costeffective

These methods are unlikely to be widely used in the foreseeable future. Nor are they likely to be costeffective for most classes of system. They will become the normal approach to the development of safety critical systems and standards. This changes the expenditure profile through the software process.

Development Costs with Formal Specification

MODEL BASED TESTING

Model Based Testing

Advantages

Model-based testing is software testing in which test cases are derived in whole or in part from a model that describes some (usually functional) aspects of the system under test (SUT).

Model-based testing is easy to understand from both the business and developer communities Model-based testing separates (business-) logic from testing code Model-based testing increases the test coverage with the same effort as Classic test automation Model-based testing is the fastest way to get use of automated test

3/18/2013

Advantages

Modelling

Model-based testing enables us to switch testing tool if needed or support multiple platforms using the same model Model-based testing focuses on requirement coverage, not how many test cases you executed last week etc. Model-based testing proved to positively affect the maintenance problem, the nemesis of all test automation.

What should the model represent? I suggest that you start with the model representing the end usage/user functionality of the system. That will usually resemble your traditional functional test case approach. There are several other approaches for you to explore later in you new MBT career