Matakuliah : 0604M / Testing dan

Implementasi

Introduction
Session 1

Outline
Students can explain testing and implementation
steps in software development life cycle
Student can make a test plan
Students can create a test plan team

Bina Nusantara

Text Book
Black, Rex. (2009). Managing The Testing Process. 3rd
Ed., Microsoft Press, Redmond, Washington 98052-6399
ISBN 0-7356-0584-X

Bina Nusantara

Other References
www.rexblackconsulting.com

Bina Nusantara

Topics
1.
2.
3.
4.
5.
6.
7.
8.
9.

Bina Nusantara

Introduction & Ch. 1 Session. 1

Software quality (Ch. 1) Session. 2 & 3
Test Plan (Ch. 2) Session 4
Test System Architecture, Cases, & Coverage (Ch. 3)
Session 5
The Bug Tracking Databases (Ch. 4) Session 6 & 7
Test Tracking Spreadsheet (Ch. 5) Session 8
Managing the Dynamic (Ch. 6) - Session 9
Staffing & Managing Test Team (Ch. 8) Session 10 & 11
Implementation Project (Ch. 14) Sessions 12 &13

Introductions

Bina Nusantara

The Foundation for Testing Project

(Software Testing)

Bina Nusantara

Pengujian (Testing)
Software testing is a process to quantify the
quality of the software development
software quality make some measurement the
quality of design quality of conformance
International Standard that we use for evaluation
the quality of software is ISO 9126

Bina Nusantara

Why test software?

Software bugs, or errors... ...cost the

U.S. economy an estimated $59.5
billion annually...more than a third of
these costs, or an estimated $22.2
billion ... could be eliminated by an
improved testing ...
NIST (National Institute of Standards and Technology), 2002

Bina Nusantara

Why find bugs in software early?

A software defect left unfixed until

late in the development cycle
costs 80 to 1000 times more to fix
than it would if it was dealt with
earlier
Gartner Inc.

Bina Nusantara

Purpose of software testing

Software Testing is the process of executing a
program or system with the intent of finding
errors *
or
Any activity aimed at evaluating an attribute or
capability of a program or system and
determining that it meets its required results **

* Glenford Myers, The art of software testing

** William Hetzel, The Complete Guide to Software Testing
Bina Nusantara

When to stop testing?

Complete testing of a system is generally
unfeasible (too expensive)
Testing normally stops:
when resources are exhausted or
when the software meets the requirements or
when the benefit of continued testing cannot
justify the added testing costs
So, how do we know this?

Bina Nusantara

System Implementation

Bina Nusantara

System Implementation
System Implementation or Adoption deals with
the transfer (conversion) between an old system
to a target system in an organization. So if a
company works with an old software system, it
may want to use a new system which is more
efficient, has more work capacity etc. So then a
new system needs to be adopted, where after it
can be used.

Bina Nusantara

(Testing & Implementation Phases in

SDLC)

Bina Nusantara

Traditional SDLC
(1) Systems Investigation
(2) Systems Analysis
(3) Systems Design
(4) Programming

An eight-stage
systems
development life
cycle (SDLC)

(5) Testing
(6) Implementation
(7) Operation
(8) Maintenance
Go Back to a previous Stage or Stop
Bina Nusantara

16

Testing & Implementation Phases in

SDLC

Testing (Pengujian)

Checks to see if the information system will produce the

expected and desired results under certain conditions

Implementation (Implementasi)
The process of converting from the old system to the new
system
Four major conversion strategies
Parallel conversion: the old and new systems operate
simultaneously for a period of time
Direct conversion: the old system is cut off and the new
systems is turned on at a certain point in time
Pilot conversion: introduces the new system in one part of
the organization
Phased conversion: introduces components of the new
system in stages
Bina Nusantara

Purpose of test management

To devise and apply an effective and economic
test strategy
Key elements:
Test planning
Test specifications
Test execution
Test monitoring and reporting
System evaluation

Bina Nusantara

The Tools You Need

A thorough test plan:

meramalkan dan menghindari

masalah yg potensial.

A
A
A
A

well-engineered test system

state-based bug tracking database
comprehensive test tracking spreadsheet
simple change management database

Diperlukan untuk pengelolaan proses pengujian.

Bina Nusantara

The Resources You Need

A practical test lab

Test Engineers and Technician
Contractors and Consultants
External test Labs and Vendors

Diperlukan untuk menjalankan proses pengujian.

Bina Nusantara

Dasar Sistem Pengujian

Suatu pengujian yang akan dilakukan harus
mempunyai fokus yang jelas.
Ada 3 pertanyaan yang harus dijawab sebelum
melakukan suatu pengujian:
What you might test ?
What you should test ?
What you can test ?
Jawaban -> membantu menentukan fokus
pengujian yang dilakukan.
Fokus pengujian yang jelas akan menentukan
kualitas dari sistem yang diuji.
Bina Nusantara

1. What you might test?

(The Extended Test Effort)

Bina Nusantara

Usaha Pengujian yang Diperluas

On my favorite development projects, testing was
pervasive. By this, I mean that a lot of testing
went on outside the independent test team.
In addition, testing started early.
This arrangement not only made sense
technically, but also kept my teams workload
manageable
There are 2 types of testing :
1. Test Granularity : level of focus of a test
2. Test Phases : Steps of testing process .

Bina Nusantara

Test Granularity
Test granularity refers to the fineness or
coarseness of a tests focus. A fine-grained test
case allows the tester to check low-level details,
often internal to the system; a coarsegrained test
case provides the tester with information about
general system behavior.
Test granularity can be thought of as running
along a spectrum ranging from structural (whitebox) to behavioral (black-box and live) tests

Bina Nusantara

Test Granularity
Refers to the fineness or coarseness of a tests focus
Can be thought of as running along a spectrum ranging
from structural to behavioural test

Bina Nusantara

IS THE WHITE-BOX/BLACK-BOX
MODEL WRONG?

Bina Nusantara

The white-box/black-box model is widespread. Glenford Myers contrasts white-box

and black-box approaches in The Art of Software Testing, a pioneering book. Cem Kaner,
Jack Falk, and Hung Nguyen refer to test cases as following a glass-box or black-box
The model is also handy. With my clients, I have found that the use of the white-box
and black-box models to explain the type of testing used in particular projects or phases
helps make communication easier. The concepts are quite intuitive.
However, the model is not ubiquitous. Bill Hetzel, in The Complete Guide to Software
Testing, describes six types of test cases: requirements-based, design-based, code-based,
randomized (especially in terms of the underlying data), extracted (from live data), and
abnormal (or extreme). Of these, he does point out that requirement-based tests are black
box, design- and code-based tests are white box, and extracted tests are live. However,
the index contains neither the phrase black-box nor white-box.
Some argue that the model is an oversimplification, and a dangerous one at that. Boris
Beizer, who wrote a book called Black Box Testing, has had second thoughts about the
phrase, which he describes in his essay, The Black Box Vampire. He argues that the better
model is to think in terms of the structural and behavioral spectrum, with a fault model
(i.e., how the bugs were created) providing an orthogonal testing dimension as well. He
argues that the white-box/black-box model makes testing look simpler than it is,
encourages a possibly negative division of test work between programmers and testers,
feeds into the mindset that testers are lower skilled than programmers, and fosters a false
belief that black-box testing is about demonstrating compliance to requirements.
Whos right? To me, the issue is the usefulness of the abstraction or simplification of
the rich set of techniques available. I find the abstractions intuitive and clarifying,
although I dont get too hung up on the matter. I prefer to think in terms of quality risks,
and then let the choice of critical quality risks drive the selection of test techniques.

Structural Test
Structural test find bugs in low level operations
such as those that occur down at the levels of
lines of code, database schemas, chips,
subassemblies, and interfaces.
Structural tests are based on how a system
operates

Bina Nusantara

Who did the structural test?

Expert Programmers can be good structural
testing
Testers develop most structural test by looking at
the code and data structures

Bina Nusantara

Behavioural Test
Behavioural tests are often used to find bugs in high-level
operations, at the level of features, operational profiles, and
customer scenarios
They are functional tests based on what a system should do
Behavioral testing involves a detailed understanding of the
application domain, the business problem being solved, and
the mission the system serves

Bina Nusantara

Who did the behavioral test?

Testers who understand the design of the system, at least
at a high level, so that they can effectively find bugs
common to that type of design.
In addition to understand some of the technological issues
surrounding the system under test, behavioral testers must
understand the special behavioral test techniques that are
most effective at finding such bugs

Bina Nusantara

What is Good Behavioral Test?

Structured
Methodical
Often repeatable sequences of tester-created
conditions

Behavioral testing is the primary test technique for

most independent test organizations

Bina Nusantara

Live Testing
Live Test involve putting customers, content
experts, early adopters and other end users in
front of the system
In some cases, we encourage the testers to try to
break the system
Well-known form: Beta Testing

Bina Nusantara

Test Phases

Unit Testing
Component or Subsystem Testing
Integration or Product Testing
String Testing
System Testing
Acceptance or User Acceptance Testing
Pilot Testing

Bina Nusantara

Benefit of Phased Test Approach

Structural Testing can build product stability

Structural testing using scaffolding or stubs can start early
You can detect bugs earlier and more efficiently
You can gather better metrics and use best-practice
techniques in your testing effort
Phases provide real and psychological milestones against
which you can gauge the "doneness" of testing and thus
the project

Bina Nusantara

Test Phase Sequencing

Bina Nusantara

2.What you should test?

Bina Nusantara

Software Quality
According to session 1, the purpose of software
testing is to get a good quality software and
quality of conformance
In other word, software testing is a way to
determine the software quality standard

Bina Nusantara

Defining Quality
"features [that] are decisive as to product
performance and as to 'product satisfaction' ...
freedom from deficiencies... [that] result in
complaints, claims, returns, rework and other
damage

Bina Nusantara

Defining Quality (cont.)

the users and customers become the arbiters of
quality when they experience product
dissatisfaction - and then make complaints, return
merchandise, or call technical support.
Testing looks for situation in which a product fails
to meet customers' or users' reasonable
expectations in specific areas.

Bina Nusantara

Software Standard
International standard that we use for evaluating
the software quality is ISO 9126

Bina Nusantara

Quality Characteristics of Software

Bina Nusantara

The Perils of Divergent Experiences of

Quality
Customers Expectation
vs
Test Teams's Quality Definition

Bina Nusantara

A High Fidelity Test System

Bina Nusantara

A Low Fidelity Test System

Bina Nusantara

Two Scenarios Testing System

Bina Nusantara

Risk Analysis of Quality

Bina Nusantara

Resiko Kualitas
Bug or failure can be exist we call it a quality
risk
Some simtomps caused by bug which we can
feel, we call it failure mode
Two methods that we can use for judjement and
analyze the risk of quality :Informal method and
formal method

Bina Nusantara

Informal Risks Analysis Techniques

Goals: Address as many of quality risks as
possible, developing tests in an order consistent
with customer priorities.
Breaking down the test process into the classic
phases of component testing, integration testing
and system testing

Bina Nusantara

Component Testing

States
Transactions
Code Coverage
Data Flow Coverage
Functionality
User Interface
Mechanical Life
Signal Quality

Bina Nusantara

Integration Testing

Component or subsystem Interfaces

Functionality
Capacity and volume
Error/disaster handling
Data quality
Performance
User Interface

Bina Nusantara

System and Acceptance Testing

Functionality
User Interface
Operations
Capacity and Volume
Reliability, Availability, and
stability
Error/disaster handling and
recovery
Stress
Performance
Date and Time Handling
Localization
Network and Distributed
environments

Bina Nusantara

Configuration option and

compatibility
Standard compliance
Security
Environment
Power Input, consumption and
output
Shock, vibration and drop
Installation, cut-over, setup and
initial configuration
Documentation and Packaging
Maintainability
Alpha, beta and other live tests

Checking and Completing Your List

Peer Review
Internal Expert
External Sources
Proposing The
quality of Risk

Bina Nusantara

Failure Mode and Effect Analysis

A Formal Method for Understanding Quality Risks
FMEA is a technique for understanding and
prioritizing possible failure modes (or quality
risks) in system functions, features, attributes,
behaviors, components and interfaces

Bina Nusantara

Bina Nusantara

Column Detail
System Function and Feature : Description of function that we
want to test
Potential Failure Modes-Quality Risk
Potential Effect of Failure
Critical ?: Apakah efeknya critical bagi user ?
Severity : 1. Loss of Data, hardware damage or a safety issues
2. Loss of functionality with no workaround
3. Loss of functionality with a workaround
4. Partial Loss of functionality
5. Cosmetic or trivial

Bina Nusantara

Column Detail
Potential failure
Priority :
1. Complete loss of system value
2. Unacceptable loss of system value
3. Possibly acceptable reduction in system value
4. Acceptable reduction is system value
5. Negligible reduction in system value
Detection Method
Likelihood : Defect for client
1. Certain to affect all users
2. Likely to impact some users
3. Possible impact on some users
4. Limited impact to few users
5. Unimaginable in actual usage

Bina Nusantara

Column Detail
RPN( Risk Potential Number) : From 1 (Most
dangerous quality risk) to 125 ( Least dangerous
quality risk)
Recommended Action
Who/When
Reference : Seperti spesifikasi produk dan
dokumen yang dibutuhkan
Action Results

Bina Nusantara

3. What you can test?

Bina Nusantara

Schedule, Resource, Budget

Bina Nusantara

Testing Steps

Planning
Configuration
Development
Execution

Bina Nusantara

Fitting a Test Schedule into the Project

Bina Nusantara