Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Kapsch CarrierComm
Relationship TCS-KAPSCH
Mohan Raghunath Sabat
Telecom
mohan.sabat@tcs.com
Confidentiality Statement
Include the confidentiality statement within the box provided. This has to be legally
approved
Confidentiality and Non-Disclosure Notice
The information contained in this document is confidential and proprietary to TATA
Consultancy Services. This information may not be disclosed, duplicated or used for any
other purposes. The information contained in this document may not be released in
whole or in part outside TCS for any purpose without the express written permission of
TATA Consultancy Services.
Table of Content
1.
Introduction .............................................................................................................................................................. 5
1.1
2.
3.
4.
5.
6.
2.2
2.3
2.4
2.5
2.6
2.7
3.2
3.3
3.4
Static testing.................................................................................................................................................... 14
4.2
Review Process................................................................................................................................................ 14
4.3
4.4
Static Analysis.................................................................................................................................................. 15
5.2
5.3
5.4
5.5
5.6
6.2
7.
8.
6.3
6.4
Test Estimation................................................................................................................................................ 25
6.5
6.6
6.7
Risks in testing................................................................................................................................................. 25
7.2
7.3
7.4
7.5
7.6
7.7
Reference ................................................................................................................................................................ 31
1. Introduction
1.1
What is ISTQB?
The ISTQB is responsible for the "ISTQB Certified Tester", which is an international qualification scheme and
the qualifications in the scheme are based on a syllabus.
Indian Testing Board (ITB) is the International Software Testing Qualifications Board (ISTQB) approved
National board for India .ITB is responsible for the "ISTQB-Certified-Tester" Certification in India.
ISTQB consists of various levels of certification exams Foundation level and Advanced level. Here we will
be dealing with only Foundation level.
ISTQB- Foundation Level Certification syllabus consists of 6 chapters, with different levels assigned to each
section of the chapters.
The exam consists of 40 Questions and passing criteria is at least 65% (i.e. 26 out of 40).
The different Levels are as follows:
K1 Remember
K2 Understand
K3 Apply and Analyse
Note:
There is a misconception that memorizing the dump questions is enough to pass the exam. This is absolutely wrong
because there are no real exam questions in any of the Sample papers. Those papers are only for practice. The ISTQB
question papers are taken away after the exam, so none of the sample papers are real exam papers. The questions asked
in exams are very tricky.If you try to read some books directly for the exam it would be very boring due to large number
of pages. I have prepared this document as I saw, read and cracked the exam, hence the name, From My eyes. You can
go through this document once and then start reading some good books (I have mentioned one book in the references
section) so that you do not take much time understanding those difficult terms and sentences. I have explained the
Sums asked in paper (5-6 sums are asked every paper) which are difficult to understand from books. Also, you can use
this document as a revision material before the exam.
2.
Fundamentals of Testing
In this section, we will see the fundamentals of testing: why testing is needed; what is testing; its limitations,
objectives and purpose; the principles behind testing; and the process that testers follow. By reading this section
you'll gain an understanding of the fundamentals of testing.
2.1
2.2
Testing is necessary to check everything and anything that we produce because things can always go
wrong- humans make mistakes all the time - it is what we do best.
Some of those mistakes are unimportant, but some of them are expensive or dangerous.
Software systems are an increasing part of life, from business applications (e.g. banking) to consumer
products (e.g. cars).
Software that does not work correctly can lead to many problems, including loss of money, time or
business reputation, and could even cause injury or death.
Human beings tend to commit mistakes (errors), which produces defects (bugs).When this defect/s is executed in
software or system it causes a failure. Defects may be caused due to time pressure, complex code, changed
technologies, complex infrastructure, etc. Defects in software, systems or documents may result in failures, but not
all defects do so.
2.3
What is testing?
Testing is a process rather than a single activity - there are a series of activities involved. A common
perception of testing is that it only consists of running tests, i.e. executing the software. This is a part of
testing, but not all of the testing activities.
Test activities exist before and after test execution.
Testing activities include the following:
Planning and Control
Test Analysis and Design
Implementation and Execution
Evaluating exit criteria and reporting
6
2.4
Objectives of Testing
2.5
To Find Defects.
To Prevent Defects.
To gain confidence about quality of software and give information for decision making.
Testing Principles
2.6
This section describes the fundamental test process and activities. These start with test planning and continue
through to test closure. We would be discussing the main tasks involved in each part of the test process.
Planning and control
7
Planning:
-
Control:
2.7
Evaluate how testing went and lessons learned for future releases and projects.
Level of risk.
Time and budget of the software.
3.
This section discusses the most commonly applied software development models, test levels and test types.
The four test levels used in this syllabus are:
3.1
Component(unit) Testing
Integration Testing
System Testing
Acceptance Testing
User and System requirements >> Global Design >> Detailed Design >> Implementation >> Testing
V-Model
The V-Model demonstrates the relationships between each phase of the development life cycle and its
associated phase of testing i.e. testing happens simultaneously at every stage. So, here we have advantage
of finding early defects. Although variants of the V-model exist, a common type of V-model uses four test
levels, corresponding to the four development levels. There may be more, fewer or different levels of
development and testing depending on the project and the software product.
E.g.: There may be component integration testing after Component testing or System Integration testing
after System testing.
10
3.2
Test Levels
This section looks in detail at the various test levels. The key characteristics for each test level are discussed and
defined to be able to more clearly separate the various test levels.
Component Testing
Component testing searches for defects in and verifies the functioning of, software modules, programs,
objects, classes, etc., that are separately testable.
Stubs, drivers and simulators may be used.
Test cases are derived from work products such as a specification of the component, the
software design or the data model.
Component testing usually involves the programmer who wrote the code.
Integration Testing
Integration testing interfaces between components, interactions with different parts of system.
Component integration testing is done after component testing and tests the interactions between software
components. System integration testing may be done after system testing and tests the interactions
between different systems.
Consider we have two components A and B(as shown in diagram below). As per the control flow A comes
before than B.
Top-down testing - While integrating these components, A is tested first with a simulator used in
place of B. The simulator here is called as Stub.
Bottom-up testing - While integrating these components is tested first with a simulator used in
place of A. The simulator here is called as Driver.
11
System Testing
System Testing is concerned with the behaviour of a whole system.
Acceptance Testing
Acceptance testing is often the responsibility of the customers or users of a system. Stakeholders may be
involved as well. Finding defects is not the main focus of acceptance testing. Goal is to establish confidence
in the system. Before a product is put on sale commercially, it has to get feedback from potential or existing
customers.
Often this type of system undergoes two types of acceptance test.
Alpha testing: It is a pre-release testing carried out at the developing organization's site but not
by the developers. The testers could be a cross-section of potential users or an independent
test team.
Beta testing: It is carried out by potential users or customers at their own location. The system is
sent to a cross-section of users who install it and use it under real-world working conditions. It is
also called as field testing.
3.3
Test types
A test type is focused on a particular test objective. The test types are described below in short.
Testing of Function:
The function that a system or component performs may be documented in the functional
specification or any other document.
The functions are what the system does.
Specification-based techniques (black box testing), may be used for functional testing.
12
3.4
Regression test suites are run many times so is a strong candidate for automation.
Maintenance testing
13
4.
Static Techniques
Static test techniques provide a powerful way to improve the quality and productivity of software development.
This section describes static test technique review process, types of review.
4.1
Static testing
4.2
Review Process
14
4.3
Types of Reviews
A single document may be the subject of more than one review. For example, an informal review may be carried
out before a technical review, or an inspection may be carried out on a requirements specification before a
walkthrough with customers. The different types of reviews serve different purposes at different stages in the life
cycle of a document.
The main review types and their characteristics are described below.
Walkthrough:
Meeting is led by Author.
Sessions can be formal or informal.
It is not time boxed.
Primary purpose is learning, understanding and finding defects.
Technical Review:
Meeting is led by trained moderator and not author.
It may be performed as a peer review without management participation.
Pre-meeting preparation by reviewers is needed.
Solutions sought after review is completed.
Primary purpose: Finding defects, solving technical problems.
Inspection
-
4.4
Static Analysis
5.
This section looks at dynamic testing, where the software we are interested in is run by executing tests on the
running code.
5.1
In this section we are looking at three things: test conditions, test cases and test procedures (or scripts). Each is
specified in its own document, according to the Test Documentation Standard [IEEE829].
Let us understand the above terms with the help of an example on ATM machine and ATM card.
Test Basis
It is a document or anything (Source document), looking at which we can derive test information.
E.g.: The Manual which is provided to the tester which provides the functionality of the ATM Machine or card
can be considered as a test basis. A tester will derive test information based on these documents.
Test Condition
We look at the test basis in order to see what could be tested. A test condition is something that we could
test. For example: Checking functionality of an ATM card. This could be one of the test conditions for which
multiple test cases would be derived.
Test conditions are documented in the IEEE 829 document called a Test Design Specification.
Contents of Test Design Specification Template:
Test design specification identifier
Features to be tested approach
Refinements test identification feature
Pass/Fail criteria
Test Cases
A test case is a set of conditions or variables under which a tester will determine whether an application,
software system or one of its features is working as it was originally established for it to do. One or more
test cases could be derived from a test condition. For example: Withdrawing money from the machine
with your ATM card, Entering PIN number, Checking balance, etc. are a set of test cases derived for above
the test condition.
Test Cases are documented in the IEEE 829 document called a Test Case Specification.
Contents of Test Case Specification Template:
Test case specification identifier
Output specifications
Test items
Environmental needs
16
5.2
Input specifications
Special procedural requirements
Inter-case dependencies
Test Procedure
The test procedure specifies the sequence of actions for the execution of a test.
e.g.: The steps to be followed for withdrawing money.
1. Enter your ATM card in the slot.
2. Enter your PIN number
3. Select Cash Withdrawal
4. Select Yes.
5. Collect cash.
Contents of Test Procedure Specification Template:
Test procedure specification identifier
Purpose
Special Requirements
Procedure Steps
Testing Techniques
There are two main categories of testing technique: Dynamic and Static Technique.
As shown in below diagram, Dynamic Techniques are sub-divided into 3 more categories:
Specification-based(Black box)
Structure-based(White box)
Experience-based
Figure below shows the test techniques explained in the syllabus for ISTQB Foundation level.
17
5.3
Specification-based techniques are also known as 'black-box' testing techniques because they view the software
as a black-box with inputs and outputs, but they have no knowledge of how the system or component is
structured inside the box. In essence, the tester is concentrating on what the software does, not how it does it.
In this section we will have a look at four specification-based or black-box techniques.
Equivalence Partitioning(EP)
The idea behind the technique is to divide (i.e. to partition) a set of test conditions into groups or sets that
can be considered the same (i.e. the system should handle them equivalently), hence 'equivalence
partitioning'.
Consider the following example, input and output conditions from a Keyboard.
Input
Output
10 20
A
21 30
B
31 40
C
We can derive test cases to test all the values, since the values are not large. But the same result can also be
achieved if we derive one Test case from each class. 10 20 is one class. Similarly, 21- 30 and 31 40. Hence
we can derive a Test case with following values to achieve 100% coverage:
Values: 5, 14, 25, 34, 48 (One value from each class).
5 and 48 also need to be considered since both of these will be invalid data.
18
From the above diagram, we see the change in transitions depending upon the previous states.
If we see the state 1st try, from this point we have two states, depending on what input was given during
1st try.
If 1st try = PIN OK, output = Access to account
If 1st try = PIN not OK, output = 2nd try.
Use Case Testing
Use case testing technique helps us identifying test cases that exercise the whole system on a transaction by
transaction basis from start to finish. Use case is description of a particular use of a system by the user of the
system (actor). It describes what actor sees and does, rather what input system expects and what system
outputs. It often uses language and terms of business rather than technical terms.
5.4
Structure-based techniques serve two purposes: test coverage measurement and structural test case design.
Types of White box technique:
19
5.5
In order to see how coverage actually works, we will use some code-level examples. These examples are asked in
exam as well. So let us see the method to solve them 100% correctly.
Example 1:
1. Read A
2. Read B
3. IF A=B
4. Print C
5. End if
Note: I have numbered the lines for explanation purpose only. In exams it would not be numbered.
Statement Coverage:
For Statement coverage we need to cover each and every statement from 1 to 5.
Let us consider Test case 1, A=10 and B=10.
If we see for this test case, each and every statement from 1 to 5 is covered, that means 100 % Statement coverage.
Hence we can say that, minimum of one Test case is needed in this case for 100% Statement Coverage.
In the above example we are considering only True condition and not the false one, because in the example we do
not see ELSE written anywhere. Only IF is given.
We will be seeing another example later to have a clearer view.
Decision Coverage:
For Decision coverage we need to cover each and every decision that comes in between lines 1 to 5.
Here, IF condition on line number 3 shows two conditions True and False. So we need to cover both the
conditions for 100% decision coverage.
Let us consider Test case 1, A=10 and B=10.
If we see for this test case, from line number 3 only the TRUE condition is satisfied.
Total number of conditions = 2
20
21
Hence we can say that, in this case at least 2 are needed for 100% Statement Coverage.
Decision Coverage:
For Decision coverage we need to cover all the conditions are present between lines 1 to 7.
Let us consider Test case 1, A=10 and B=10.
If we see for this test case, only TRUE condition will be executed.
Test case 2, A=10 and B=11
If we see for this test case, only FALSE condition will be executed.
But if we add both the above, all decisions coming between lines 1 to 7 are covered at least once.
Hence we can say that, in this case at least 2 are needed for 100% Decision Coverage.
5.6
Experience-based technique
Some defects are hard to find using more systematic approaches. For such scenarios we have experience based
testing. Experienced-based testing is where tests are derived from the testers skill and intuition and their
experience with similar applications and technologies.
Types of experience-based techniques: Error Guessing and exploratory testing.
Error Guessing:
The tester is encouraged to think of situations in which the software may not be able to cope.
22
A structured approach to the error-guessing technique is to list possible defects or failures and
to design tests that attempt to produce them. These defect and failure lists can be built based
on the tester's own experience.
Exploratory testing:
As its name implies, exploratory testing is about exploring, finding out about the software,
what it does, what it doesn't do, what works and what doesn't work.
Exploratory testing is a hands-on approach in which testers are involved in minimum planning
and maximum test execution.
This is an approach that is most useful when there are no or poor specifications and when time
is severely limited.
-
23
6.
Test Management
Testing usually accounts for a substantial proportion of the overall project budget. Therefore, we must understand
how we should manage the testing we do. In this section, we will cover essential topics for test management.
6.1
Independent testing
A certain degree of independence often makes the tester more effective. Several levels of independence can be
defined as shown here from low to high Tests designed by person who wrote the software under test - Lowest level.
Test designed by another person within team Low level.
Test designed by a person from different organizational group or company Highest level.
6.2
Tester
Following are the tasks of a tester,
Review and contribute to test plans.
Identifying test conditions.
Creating test designs test procedure specifications and test data.
Setting up test environment.
Executing and logging the tests.
Evaluating results and document the problems found.
6.3
Entry Criteria
Entry criteria define when to start testing such as at the beginning of a test level or when a set of tests is
ready for execution.
Typically entry criteria may cover the following:
Test environment availability and readiness
Test tool readiness in the test environment
Testable code availability
Test data availability
24
6.4
Exit Criteria
Exit criteria define when to stop testing such as at the end of a test level or when a set of tests has
achieved specific goal.
Thoroughness measures, such as coverage of code, functionality or risk:
Estimates of defect density or reliability measures
Cost
Residual risks, such as defects not fixed or lack of test coverage in certain areas.
Test Estimation
6.5
Metrics based approach - Estimating the test effort based on metrics of former or similar projects.
Expert based approach - Estimating the tasks based on estimates made by the owner of tasks tests is ready
for execution.
Test Approach
6.6
The test approach is the implementation of the test strategy for a specific project. The test approach is
defined and refined in the test plans and test designs. It typically includes the decisions made based on
the (test) projects goal and risk assessment.
Typical approaches include:
Analytical approaches
Model based approaches
Methodical approaches
Process or standard compliant approaches
Dynamic approaches
Consultative approaches
Test Monitoring
The purpose of test monitoring is to provide feedback and visibility about test activities. Monitoring
percentage of work done and test coverage of requirements, risks or code.
Test Control
Test control describes any guiding or corrective actions taken as a result of information and metrics
gathered and reported.
Note: Some other topics like test planning, test execution, etc. needed in this Section are already covered in
Section Fundamentals of testing.
6.7
Risks in testing
This section covers a topic that we believe is critical to testing: risk. We'll see that there are risks related to the
product and risks related to the project, and look at typical risks in both categories.
25
Project Risk
Project risks are the risks that surround the projects capability to deliver its objectives, like
Skill, training and staff shortages
Personnel issues
Problems in defining the right requirements
The extent to which requirements cannot be met given existing constraints
Test environment not ready on time
Failure of a third party
Contractual issues
Product Risk
Potential failure areas in the software or system are known as product risks as they are a risk to the quality
of the product, like
Failure-prone software delivered
The potential that the software/hardware could cause harm to an individual or company
Poor software characteristics (e.g., functionality, reliability, usability and performance)
Poor data integrity and quality (e.g., data migration issues, data conversion problems, data
transport problems, violation of data standards)
Software that does not perform its intended functions.
26
7.
In this section we will see that the tools are grouped by the testing activities or areas that are supported by a set of
tools. For example, tools that support management activities, tools to support static testing, etc.
7.1
Tools Classification:
7.2
The tools in this broad category provide support for the "management of tests" as well as "managing the testing
process". The management of testing applies over the whole of the software development life cycle, so a test
management tool could be among the first to be used in a project. A test management tool may also manage the
tests, which would begin early in the project and would then continue to be used throughout the project and also
after the system had been released.
Tools used in this category are mentioned below.
7.3
The tools described in this section support the testing activities described in Section Static Techniques.
28
7.4
Modelling tools
Modeling tools help to validate models of the system or software. These tools are normally used by
developers. The features provided by Modelling tools include those listed below
Identifying inconsistencies and defects within the model
Helping to identify and prioritize areas of the model for testing
7.5
Test comparators
The essence of testing is to check whether the software produces the correct result, and to do that, we
must compare what the software produces to what it should produce. A test comparator helps to
automate aspects of that comparison.
7.6
Security tool
Security testing tools can be used to test security by trying to break into a system, whether or not it is
protected by a security tool.
The features provided by security tool include those listed below
Identifying viruses.
Detecting intrusions such as denial of service attacks.
Identifying weaknesses in password files and passwords.
Security checks during operation. E.g. for checking integrity of files.
The tools described in this section support testing that can be carried out on a system when it is operational, i.e.
while it is running.
Monitoring tools
Monitoring tools are used to continuously keep track of the status of the system in use, in order to have
the earliest warning of problems and to improve service.
7.7
30
8.
Reference
Foundations of Software Testing ISTQB Certification by Dorothy Graham, Erik Veenendaal, Isabel Evans,
Rex Black
The book is also available for download for free at the below url:
http://api.ning.com/files/IPWzwvTv3Hu3RchV7iA9H64BC1i06bZfNKpuGuPSBonklDmaVgsD6VExvHEQWryd2nDINaCTGS6f1M6X6acz5V6JFPdCocd/istqb_foundations_of_software_test
ing.pdf
31
Thank You
Contact
For more information, contact DEG Core KM Grp
IT Services
Business Solutions
Consulting
All content / information present here is the exclusive property of Tata Consultancy Services Limited (TCS). The content / information
contained here is correct at the time of publishing. No material from here may be copied, modified, reproduced, republished,
uploaded, transmitted, posted or distributed in any form without prior written permission from TCS. Unauthorized use of the content /
information appearing here may violate copyright, trademark and other applicable laws, and could result in criminal or civil penalties.
Copyright 2011 Tata Consultancy Services Limited