User Acceptance Testing

The objective of software development is to develop

the software that meets the true needs of the user, not
just the system specifications.

To accomplish this, testers should work with the users

early in a project to clearly define the criteria that
would make the software acceptable in meeting the
user needs.
 Acceptance Testing Concepts
 Software acceptance is an incremental process of approving or rejecting software
systems during development or maintenance, according to how well the software
satisfies predefined criteria.
 Acceptance decisions occur at pre-specified times when processes, support tools,
interim documentation, segments of the software, and finally the total software
system must meet predefined criteria for acceptance.
 The final acceptance decision occurs with verification that the delivered
documentation is adequate and consistent with the executable system and that the
complete software system meets all buyer requirements.
 Formal final software acceptance testing must occur at the end of the development
process. It consists of tests to determine whether the developed system meets
predetermined functionality, performance, quality, and interface criteria.
 Acceptance testing involves procedures for identifying acceptance criteria for interim
life cycle products and for accepting them.
 As a life cycle process, software acceptance enables:


Early detection of software problems (and time for
the customer or user to plan for possible late
delivery)
 Preparation of appropriate test facilities
 Early consideration of the user’s needs during
software development
The Customer or User's Responsibilities
in Acceptance Testing

Ensure user involvement in developing system requirements and
acceptance criteria.
 Identify interim and final products for acceptance, their
acceptance criteria, and schedule.
 Plan how and by whom each acceptance activity will be
performed.
 Plan resources for providing information on which to base
acceptance decisions.
 Schedule adequate time for buyer staff to receive and examine
products and evaluations prior to acceptance review.
The Customer or User's Responsibilities
in Acceptance Testing (Continued)
 Prepare the Acceptance Plan.
 Respond to the analyses of project entities before accepting or
rejecting.
 Approve the various interim software products against quantified
criteria at interim points.
 Perform the final acceptance activities, including formal
acceptance testing, at delivery.
 Make an acceptance decision for each product
 Acceptance Testing Measures the
Software's “Fit”

 Is the software “fit” for use?

 Does it “fit” into the customer's business process?
The Four Components of Fit
 The Four Components of “Fit”

Data
The reliability, timeliness, consistency, and usefulness of the data included in the
automated application.
 People
People should have the skills, training, aptitude, and desire to properly use and
interact with the automated application.
 Structure
The structure is the proper development of application systems to optimize
technology and satisfy requirements.
 Rules
The rules are the procedures to follow in processing the data.
 Acceptance Testing vs. System Testing


System Testing
 Performed by developers and/or software testers.
 Focuses on determining whether or not the software specifications have
been implemented as specified.
 Should be performed before acceptance testing.
 Acceptance Testing
 Performed by user personnel with possible the assistance of software
testers.
 Focuses on input processing, use of the software in the user organization,
and whether or not the specifications meet the true processing needs of
the user.
Three Reasons Why the Software Specified Might not Meet the
User’s True Needs.

 The users do not specify the skill level of the people who will be
using the system.
 Processing may be specified but turnaround time might not be
specified,
 The users may not know that they have to specify the
maintainability attributes of the software.
 Roles in Acceptance Testing
 User's Roles:
 Determine whether acceptance
testing will or will not occur.
 Take primary responsibility for
planning and conducting
acceptance testing.
 Software Tester takes
one of these 3 Roles:
 No involvement at all.
 Advisor to the user.
 Active participant in testing up
to the point where the results of
acceptance testing are
documented. (Can't define
acceptance criteria, nor make the
final acceptance decisions.)
 At a minimum the user will have the following roles in acceptance testing:

Defining acceptance criteria in a testable format.



Providing the use cases that will be used in acceptance testing.



Training user personnel in using the new software system.



Providing the necessary resources, primarily user staff personnel, for acceptance
testing.

Comparing the actual acceptance testing results with the desired acceptance testing
results (NOTE: This may be performed using testing software).

Making decisions as to whether additional work is needed prior to placing the software
in operation, whether the software can be placed in operation with additional work to be
done, or whether the software is fully acceptable and can be placed into production as is.
 Acceptance Criteria
In preparation for developing the acceptance criteria,
the user should:
 Acquire full knowledge of the application for which the system is intended.

 Become fully acquainted with the application as it is currently implemented by

the user’s organization.

Understand the risks and benefits of the development methodology that is to be

used in correcting the software system.


Fully understand the consequences of adding new functions to enhance the
system.
 Acceptance Criteria that “Should” be in
the Requirements Specifications

Functionality Requirements

These requirements relate to the business rules that the system must execute.

Performance Requirements

These requirements relate to operational aspects, such as time or resource constraints.

Interface Quality Requirements

These requirements relate to connections from one component to another component of processing
(e.g., human-machine, machine-module).

Overall Software Quality Requirements

These requirements specify limits for factors or attributes such as reliability,

testability, correctness, and usability.
 Determining the Criticality of Requirements

 All safety criteria are critical


Certain security requirements – as required by law - are critical

Other factors affecting criticality:

 Importance of the system to organization or industry


Consequence of failure

Complexity of the project

Technology risk
 Complexity of the user environment
 Developing the Acceptance Criteria


Should contain pass or fail criteria – the
Acceptance Requirement.


Should cover the parts of the software product in
detail.


Specifying acceptable numeric values or ranges
of values is useful.
Example of Required Information to
Document Acceptance Criteria
Acceptance Criteria Example
 Acceptance Test Plan
Example Table of Contents for an Acceptance Test Plan

Project Description

Type of system; life cycle methodology; user community of delivered

system; major tasks system must satisfy; major external interfaces of the system;
expected normal usage; potential misuse; risks; constraints; standards and practices.

User Responsibilities

Organization and responsibilities for acceptance activities; resource and schedule

requirements; facility requirements; requirements for
automated support; special data, and training; standards, practices, and conventions;
updates and reviews of acceptance plans and related products.
 Example Table of Contents for an Acceptance Test Plan (Cont.)

Administrative Procedures

Anomaly reports; change control; record-keeping; communication

between developer and manager organizations.

Acceptance Description

Objectives for entire project; summary of acceptance criteria; major

acceptance activities and reviews; information requirements; types of
acceptance decisions; responsibility for acceptance decisions.
 Use Case Test Data

A use case is a test case which represents how the software will be used
in operation. A use case is built on a business transaction and can be test
data or a test script.

When use cases are developed by clerical people intimately familiar with
the system, they tend to know the type of problems that are typical in the
business system. Thus, they can simulate those unusual events through
use cases that may not be developed during normal system testing.

􀂃
 An individual use case consists of:


Preconditions that set the stage for the series of events that
should occur for the use case

Post-conditions that state the expected outcomes of the
above process

Sequential narrative of the execution of the use case

See “Skill Category 5, Executing the Test Plan,” for
information on the process for preparing use cases.
http://www.agilemodeling.com/artifacts/
useCaseDiagram.htm
 Scott W. Ambler
UML 2 Use Case Diagrams

Using System boundary boxes to indicate releases:
Executing the Acceptance Test Plan

Use one or both of these techniques:


Reviewing partially developed deliverables during
development.


Testing the executable software system.
 Acceptance Decisions
Typical acceptance decisions include:

Required changes are accepted before progressing to the

next activity.

Some changes must be made and accepted before further

development of that section of the product; other changes
may be made and accepted at the next major review.

Progress may continue and changes may be accepted at the

next review.

 No changes are required and progress may continue.

 Final Acceptance


Occurs when the user has made the decision to accept
the software.


Usually, means that the the software project is
completed and the developer has no further
obligations.


Usually, some criteria will not be completely
satisfied.