Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
The two major categories of software are system software and application software.
System software is made up of control programs. Application software is any program that
processes data for the user (spreadsheet, word processor, payroll, etc.).
Application software can once again break up into Web, Window and mobile as of now.
A software product should only be released after it has gone through a proper process of
development, testing and bug fixing.
It is important to note that software is mainly tested to see that it meets the customers’ needs
and that it conforms to the standards.
It is a usual norm that software is considered of good quality if it meets the user requirements.
Now, everyone would be confused that only the software tester and developer may perform the test.
But that’s not the case. The following are some who performs testing at various instances
Software Tester
Software Developer (Perform Unit testing at the time of developing the software)
Project Lead/Manager (Performs an overall check when the product is ready to ship. They would
check if everything is working fine and upto the standards)
End User (Involved in Beta testing, that is pre-release testing. To check for any bugs available
and explore new features)
Different companies have different designations for people who test the software on the basis of their
experience and knowledge such as Software Tester, Software Quality Assurance Engineer, QA
Analyst, etc.
In Software Development Life Cycle (SDLC), testing can be started from the Requirements
Gathering phase and continued till the deployment of the software. It also depends on the development
model that is being used. For example, in the Waterfall model, formal testing is conducted in the
testing phase; but in the incremental model, testing is performed at the end of every
increment/iteration and the whole application is tested at the end.
1
Requirements Gathering - During the requirement gathering phase, the analysis and verification of
requirements are also considered as testing.
Design Phase - Reviewing the design in the design phase with the intent to improve the design is also
considered as testing.
Testing Phase - Execution of the test cases that is prepared. Testing , debugging and tracking the bug to
closure.
Maintenance Phase – Once the product is deployed, Testing carried in the production environment for
retesting the bug fixes and updates in the software.
It is difficult to determine when to stop testing, as testing is a never-ending process and no one
can claim that a software is 100% tested. The following aspects are to be considered for stopping the
testing process:
Testing Principles
A number of testing principles have been suggested over the past 40 years and offer general guidelines
common f or all testing.
Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces
the probability of undiscovered defects remaining in the software but, eve n if no defects are found, it is
not a proof of correctness.
Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases.
Instead of exhaustive testing, risk analysis and priorities should be used to focus testing efforts.
2
To find defects early, testing activities shall be started as early as possible in the software or
system development life cycle, and shall be focused on defined objectives.
Testing effort shall be focused proportionally to the expected and later observed defect density of
modules. A small number of modules usually contains most of the defects discovered during pre-
release testing, or is responsible for most of the operational failures.
During software testing, as defects are found, analysis of defects can give surprising results!
Defect Clustering in Software Testing means that the majority of the defects are caused by a small
number of modules, i.e. the distribution of defects are not across the application but rather
centralized in limited sections of the application.
Defect Clustering in Software Testing is based on the Pareto principle, also known as the 80-20
rule, where it is stated that approximately 80% of the problems are caused by 20% of the
modules.
This can give a good indication that when a defect is found in one area of the application, chances
are there are more defects in that particular area, so it is worth investing more time to test that
particular area of the application to find as many defects as possible.
However, testers should not ignore to test the rest of application as well as there may be other
defects scattered around.
Defect aggregation or defect clustering in software testing can also indicate which area of the
application needs more regression testing to ensure related features are not broken.
If the same tests are repeated over and over again, eventually the same set of test cases will no
longer find any new defects. To overcome this “pesticide paradox”, test cases need to be regularly
reviewed and revised, and new and different tests need t o be written to exercise different parts
of the software or system to find potentially more defects.
3
Let’s say you are testing a application. You have written a set of test cases.
Now you run one cycle of testing. You find few bugs and report them to the development team.
Development team fixes the bugs and reverts to you with the updated code. You again execute
the same set of test cases. This time you find that few of the bug were still not fixed and you
report that back to the development team. They work of it and send an update to you. Once
again, you execute the same set of test cases and don't find any bugs.
Now in a new release some changes were made in the application. You run the same set of test
cases and they all pass. But, what you miss here is the new bugs that may have introduced when
the fix and new changes were applied. The old sets of test cases are incapable of identifying these
new bugs.
This is called Pesticide Paradox. To avoid this you need to update your test cases with each cycle
and add new cases to the old set.
Testing is done differently in different contexts. For example, safety-critical software is tested
differently from an e-commerce site.
Finding and fixing defects does not help if the system built is unusable and does not fulfill the
users’ needs and expectations.
Verification VS Validation
Verification Validation
1. Verification is a static Activity of verifying 1. Validation is a dynamic activity of
documents, design, code and program. validating and testing the actual product.
2. Done by developers. 2. Done by Testers.
3. Ensures that the software system 3. Ensures that the functionalities
meets all the functionality. meet the intended behavior.
4. Verification addresses the concern: "Are 4. Validation addresses the concern:
you building it right?" "Are you building the right thing?"
5. It is human based checking of documents 5. It is computer based execution of
and files. program.
6. For Example: Building a weight machine 6. Once this weight machine is built it’s the
which weighs object and displays the weight Tester who needs to validate if the weight
only upon inserting one rupee coin. Now this machine displays the weight only for 1Rs
is the requirement of the Customer. coin or it displays the weight for "2" and "5"
Rupee coin. If machine displays the weight
for "2" and "5" rupee coin then it’s a bug and
it need to be reported.
4
Quality Assurance VS Quality Control
Audit means the inspection, examination or verification of a person, organization, system, process,
enterprise, project or product and to recommend necessary changes in controls, policies, or procedures.
An audit can be done anytime. It is a systematic process to determine how the actual testing process is
conducted within an organization or a team. Types of audit include Legal Compliance Audit, Internal
Audit, and System Audit
Inspection indicates that the regulatory authorities are checking documents, records, facilities and any
other resources to verify a certain set of standards. It is a formal technique that involves formal or
informal technical reviews of any artifact by identifying any error or gap. inspection is a formal
evaluation technique in which software requirements, designs, or codes are examined in detail by
a person or a group other than the author to detect faults, violations of development standards, and
other problems.
A common perception of testing is that it only consists of running test s, i.e., executing the soft ware.
This is part of testing, but not all of the testing activities. Test activities exist before and after test
execution. These activities include planning and control, choosing test conditions, designing and
executing test cases, checking results, evaluating exit criteria, reporting on the testing process and
system under test, and finalizing or completing closure activities after a test phase has been completed.
Debugging is the development activity that finds, analyzes and removes the cause of the failure.
Subsequent re-testing by a tester ensures that the fix doe s indeed re solve the failure. The responsibility
for these activities is usually testers test and developers debug.
5
Types of Testing:
Manual testing is the process of manually testing software for defects. i.e., without using any
automated tool or any script. It requires a tester to play the role of an end user and use most of all
features of the application to ensure correct behavior.
Testers use test plans, test cases, or test scenarios to test a software to ensure the completeness of
testing. Manual testing also includes exploratory testing, as testers explore the software to identify
errors in it.
Automation testing, which is also known as Test Automation, is when the tester writes scripts and
uses another software to test the product. Automation Testing saves time, cost and manpower. Once
recorded, it's easier to run an automated test suite when compared to manual testing which will require
skilled labor.
Any type of application can be tested manually but automated testing is recommended only for stable
systems and is mostly used for regression testing. Also, certain testing types like ad-hoc and monkey
testing are more suited for manual execution.
What to Automate?
It is not possible to automate everything in software. The following are the some cases that one can
perform the automation
The following are the some cases that one cannot perform the automation
How to Automate?
There are many tools available that can be used to write automation scripts. Before mentioning the
tools, let us identify the process that can be used to automate the testing process:
6
Automation Software Testing Tools
The following tools can be used for automation testing:
HP Quick Test Professional - Now UFT (Unified Functional testing)
Selenium
IBM Rational Functional Tester
SilkTest
TestComplete
Testing Anywhere
WinRunner
LoadRunner
Visual Studio Test Professional
Virtual Users
Appium (Selenium Mobile testing tool)
Testing Methods
There are different methods that can be used for software testing.
Black-Box Testing
White-Box Testing
Grey-Box Testing
Black-Box Testing:
The technique of testing without having any knowledge of the interior workings of the application is
called black-box testing. The tester is oblivious to the system architecture and does not have access
to the source code. Typically, while performing a black-box test, a tester will interact with the
system's user interface by providing inputs and examining outputs without knowing how and
where the inputs are worked upon.
The following table lists the advantages and disadvantages of black-box testing.
Advantages Disadvantages
Well suited and efficient for large code segments. Limited coverage, since only a selected number of
test scenarios is actually performed.
Large numbers of moderately skilled testers can Inefficient testing, due to the fact that the tester
test the application with no knowledge of only has limited knowledge about an application.
implementation, programming language, or
operating systems.
Clearly separates user's perspective from the Blind coverage, since the tester cannot target
developer's perspective through visibly defined specific code segments or error-prone areas.
roles.
Code access is not required. The test cases are difficult to design.
7
White-Box Testing:
White-box testing is the detailed investigation of internal logic and structure of the code. White-box
testing is also called glass testing or open-box testing. In order to perform white-box testing on an
application, a tester needs to know the internal workings of the code. The tester needs to have a look
inside the source code and find out which unit/chunk of the code is behaving inappropriately.
The following table lists the advantages and disadvantages of white-box testing.
Advantages Disadvantages
As the tester has knowledge of the source code, it Due to the fact that a skilled tester is needed to
becomes very easy to find out which type of data perform white-box testing, the costs are increased.
can help in testing the application effectively.
It helps in optimizing the code. Sometimes it is impossible to look into every nook
and corner to find out hidden errors that may
create problems, as many paths will go untested.
Extra lines of code can be removed which can It is difficult to maintain white-box testing, as it
bring in hidden defects. requires specialized tools like code analyzers and
debugging tools.
Grey-Box Testing:
Grey-box testing is a technique to test the application with having a limited knowledge of the
internal workings of an application.
Mastering the domain of a system always gives the tester an edge over someone with limited domain
knowledge. Unlike black-box testing, where the tester only tests the application's user interface; in grey-
box testing, the tester has access to design documents and the database. Having this knowledge, a
tester can prepare better test data and test scenarios while making a test plan.
Advantages Disadvantages
Offers combined benefits of black-box and white- Testing every possible input stream is unrealistic
box testing wherever possible. because it would take an unreasonable amount of
time; therefore, many program paths will go
untested.
Based on the limited information available, a grey- Since the access to source code is not available,
box tester can design excellent test scenarios. the ability to go over the code and test coverage is
limited.
The test is done from the point of
view of the user and not the designer
8
Test Levels
Test cases are derived from work products such as a specification of the component, the
software design or the data model.
Typically, component testing occurs with access to the code being tested and with the
support of a development environment, such as a unit test framework or debugging tool.
In practice, component testing usually involves the programmer who wrote the code.
Defects are typically fixed as soon as they are found, without formally managing these
defects.
One app roach to component testing is to prepare and automate test cases before coding.
This is called a test-first app roach or test-driven development.
This approach is highly iterative and is based on cycles of developing test cases, then building
and integrating small pieces of code, and executing the component tests correcting any issues
and iterating until they pass.
Integration Testing:
Integration testing tests interfaces between components, interactions with different parts of
a system, such as the operating system, file system and hardware.
The greater the scope of integration, the more difficult it becomes to isolate defects to a
specific component or system, which may lead to increased risk and addition al time for
troubleshooting.
Systematic integration strategies may be based on the system architecture (such as top-down
and bottom-u p), function al tasks, transaction processing sequences.
9
In order to ease fault isolation and detect defects early, integration should normally be
incremental rather than “big bang” (performed as a whole).
At each stage of integration, testers concentrate solely on the integration itself. For example,
if they are integrating module A with module B they are interested in testing the
communication between the modules, not the functionality of the individual module as that
was done during component testing.
Ideally, testers should understand the architecture and influence integration planning.
System Testing
System testing is concerned with the behavior of a whole system/product.
In system testing, the test environment should correspond to the final target or production
environment as much as possible in order to minimize the risk of environment-specific
failures not being found in testing.
System testing should investigate functional and non-functional requirements of the system.
Testers also need to deal with incomplete or undocumented requirements.
System testing of functional requirements starts by using the most appropriate specification-
based (black-box) techniques for the aspect of the system to be tested. Structure-based
techniques (white-box) may then be used to assess the thoroughness of the testing with
respect to a structural element.
Acceptance Testing
Acceptance testing is often the responsibility of the customers or users of a system; other
stakeholders may be involved as well.
The goal in acceptance testing is to establish confidence in the system, parts of the system or
specific non-functional characteristics of the system.
Finding defects is not the main focus in acceptance testing. Acceptance testing may assess
the system’s readiness for deployment and use
Acceptance testing may occur at various times in the life cycle, for example:
A COTS (Commercial off the Shelf)software product m ay be acceptance tested when
it is installed or integrated.
10
Acceptance testing of the usability of a component may be done during component
testing.
Acceptance testing of a new functional enhancement may come before system
testing.
Alpha testing is performed at the developing organization’s site but not by the developing team,
but still within the company. Unit testing, integration testing and system testing when
combined together is known as alpha testing.
Organizations may use other terms as well, such as factory acceptance testing and site
acceptance testing for systems t hat are tested before and after being moved to a customer’s
site.
Test Types
Levels of testing include different methodologies that can be used while conducting software
testing. The main levels of software testing are:
Functional Testing
Non-functional Testing
Structural Testing
Re-testing and Regression.
11
Functional Testing
The functions that a system, subsystem or component are to perform may be described in
work products such as a requirements specification, use cases , or a functional specification,
or they may be undocumented.
T he functions are “what” the system does. Function al tests are based on functions and
features (described in documents or understood by the testers) and their interoperability
with specific systems. Functional testing considers the external behavior of the software
(black-box testing).
A type of functional testing, security testing, investigates the functions (e.g., a fire wall) relating
to detection of threats, such as viruses, from malicious outsiders. Another type of functional
testing, interoperability testing, evaluates the capability of the software product to interact with
one or more specified component s or systems.
Non-functional Testing
Non-functional testing includes, but is not limited to, performance testing, load testing, stress
testing, usability testing, maintain ability testing, reliability testing and portability testing.
It is the testing of “how” the system works. Non-functional testing may be performed at all test
levels.
The term non-functional testing describes the tests required to measure characteristics of
systems and software that can be quantified on a varying scale, such as response times for
performance testing.
Non-function al testing considers the external behavior of the software and in most cases uses
black-box test design techniques to accomplish that.
Structural Testing
Structural (white-box) testing may be performed at all test levels. Structural techniques are best
used after specification-based techniques, in order to help measure the thoroughness of testing
through assessment of coverage of a type of structure.
Coverage is the extent that a structure has been exercised by a test suite, expressed as a
percentage of the items being covered. If coverage is not 100%, then more tests may be
designed to test those items that were missed to increase coverage.
At all test levels, but especially in component testing and component integration testing, tools
can be used to measure the code coverage of elements, such as statements or decisions.
Structural testing approaches can also be applied at system, system integration or acceptance
testing levels.
12
statement and branch are covered.
Regression testing is the repeated testing of an already tested program, after modification, to
discover any defects introduced or uncovered as a result of the change(s). These defects ma y
be either in the software being tested, or in another related o r unrelated software component.
It is performed when the software, or its environment, is changed. The extent of regression
testing is based on the risk of not finding defects in software that was working previously.
Dynamic testing is performed in a runtime environment, with security analysis carried out whilst
software is in operation. With a given input, the software’s actual output is compared to its expected
output. This allows developers to analyze the functional behavior of a piece of software, and monitor its
interaction with system memory, CPU function and overall system performance. Dynamic testing is
often referred to as validation: the evaluation of a finished product.
13
What is Smoke Testing?
Smoke Testing is a testing technique that is inspired from hardware testing, which checks for the smoke
from the hardware components once the hardware's power is switched on. Similarly in Software testing
context, smoke testing refers to testing the basic functionality of the build.
If the Test fails, build is declared as unstable and it is NOT tested anymore until the smoke test of the
build passes.
Smoke Testing - Features:
Identifying the business critical functionalities that a product must satisfy.
Designing and executing the basic functionalities of the application.
Ensuring that the smoke test passes each and every build in order to proceed with the testing.
Smoke Tests enables uncovering obvious errors which saves time and effort of test team.
Smoke Tests can be manual or automated.
14
What is Adhoc testing?
When a software testing performed without proper planning and documentation, it is said to be Adhoc
Testing. Such kind of tests is executed only once unless we uncover the defects.
Adhoc Tests are done after formal testing is performed on the application.
Adhoc methods are the least formal type of testing as it is NOT a structured approach. Hence, defects
found using this method are hard to replicate as there are no test cases aligned for those scenarios.
Testing is carried out with the knowledge of the tester about the application and the tester tests
randomly without following the specifications/requirements.
Hence the success of Adhoc testing depends upon the capability of the tester, who carries out the test.
The tester has to find defects without any proper planning and documentation, solely based on tester's
intuition.
Buddy Testing: Two buddies, one from development team and one from test team mutually
work on identifying defects in the same module. Buddy testing helps the testers develop better
test cases while development team can also make design changes early. This kind of testing
happens usually after completing the unit testing.
Pair Testing: Two testers are assigned the same modules and they share ideas and work on the
same systems to find defects. One tester executes the tests while another tester records the
notes on their findings.
Monkey Testing: Testing is performed randomly without any test cases in order to break the
system.
State Transition testing, a black box testing technique, in which outputs are triggered by changes to the
input conditions or changes to 'state' of the system. In other words, tests are designed to execute valid
and invalid state transitions.
Any system where you get a different output for the same input, depending on what has happened
before, is a finite state system.
15
The above diagram shows an example of entering a Personal Identity Number (PIN) to a bank account.
The states are shown as circles, the transitions as lines with arrows and the events as the text near the
transitions. (We have not shown the actions explicitly on this diagram, but they would be a message to
the customer saying things such as ‘Please enter your PIN’.)
The state diagram shows seven states but only four possible events (Card inserted, Enter PIN, PIN OK
and PIN not OK). We have not specified all of the possible transitions here – there would also be a time-
out from ‘wait for PIN’ and from the three tries which would go back to the start state after the time had
elapsed and would probably eject the card. There would also be a transition from the ‘eat card’ state
back to the start state. We have not specified all the possible events either – there would be a ‘cancel’
option from ‘wait for PIN’ and from the three tries, which would also go back to the start state and eject
the card.
Type of software testing performed to check newer version of the software can work successfully
installed over previous version of the software and newer version of the software works as fine with
table structure, data structures, files that were created by previous version of the software.
Boundary Value Testing is a testing technique that is based on concept “error aggregates at boundaries”.
In this testing technique, testing is done extensively to check for defects at boundary conditions. If a
field accepts value 1 to 100 then testing is done for values 0, 1, 2, 99, 100 and 101.
Equivalence partitioning technique is used in black box and grey box testing types. Equivalence
partitioning classifies test data into Equivalence classes as positive Equivalence classes and negative
Equivalence classes, such classification ensures both positive and negative conditions are tested.
16
For example, if a program accepts integer values only from 1 to 10. The possible test cases for such a
program would be the range of all integers. In such a program, all integers up to 0 and above 10 will
cause an error. So, it is reasonable to assume that if 11 will fail, all values above it will fail and vice versa.
If an input condition is a range of values, let one valid equivalence class be the range (0 or 10 in this
example). Let the values below and above the range be two respective invalid equivalence values (i.e. -1
and 11). Therefore, the above three partition values can be used as test cases for the above example.
Localization testing a type of software testing performed by software testers, in this type of testing,
software is expected to adapt to a particular locale, it should support a particular locale/language in
terms of display, accepting input in that particular locale, display, font, date time, currency etc., related
to a particular locale. For e.g. many web applications allow choice of locale like English, French, German
or Japanese. So once locale is defined or set in the configuration of software, software is expected to
work as expected with a set language/locale.
It is mostly used to identify any bottlenecks or performance issues rather than finding bugs in software.
There are different causes that contribute in lowering the performance of software.
Network delay
Client-side processing
Database transaction processing
Load balancing between servers
Data rendering
Performance testing can be either qualitative or quantitative and can be divided into different sub-types
such as Load testing and Stress testing.
It is a process of testing the behavior of a software by applying maximum load in terms of software
accessing and manipulating large input data. It can be done at both normal and peak load conditions.
This type of testing identifies the maximum capacity of software and its behavior at peak time.
Most of the time, load testing is performed with the help of automated tools such as Load Runner,
AppLoader, IBM Rational Performance Tester, Apache JMeter, Silk Performer, Visual Studio Load
Test, etc.
Virtual users (VUsers) are defined in the automated testing tool and the script is executed to verify the
load testing for the software. The number of users can be increased or decreased concurrently or
incrementally based upon the requirements.
Stress testing includes testing the behavior of software under abnormal conditions. For example, it may
include taking away some resources or applying a load beyond the actual load limit.
17
The aim of stress testing is to test the software by applying the load to the system and taking
over the resources used by the software to identify the breaking point. This testing can be
performed by testing different scenarios such as:
What is UI Testing?
UI testing involves testing the Graphical User Interface of the Software. UI testing ensures that the GUI
functions according to the requirements and tested in terms of color, alignment, size, and other
properties.
Security testing involves testing the software in order to identify any flaws and gaps from security and
vulnerability point of view. Listed below are the main aspects that security testing should ensure:
Confidentiality
Integrity
Authentication
Availability
Authorization
Non-repudiation
Software is secure against known and unknown vulnerabilities
Software data is secure
Software is according to all security regulations
Input checking and validation
SQL insertion attacks
Injection flaws
Session management issues
Cross-site scripting attacks
Buffer overflows vulnerabilities
Directory traversal attacks
Portability testing includes testing a software with the aim to ensure its reusability and that it can
be moved from another software as well. Following are the strategies that can be used for portability
testing.
18
Transferring installed software from one computer to another.
Building executable (.exe) to run the software on different platforms.
Portability testing can be considered as one of the sub-parts of system testing, as this testing type
includes overall testing of a software with respect to its usage over different environments.
Computer hardware, operating systems, and browsers are the major focus of portability testing.
Testing how well a system recovers from crashes, hardware failures, or other catastrophic problems.
It is a type of non-functional testing technique. Recovery testing is the forced failure of the software to
verify if the recovery is successful.
Penetration testing is testing how well the system is protected against unauthorized internal or external
access, or willful damage. This type of testing usually requires sophisticated testing techniques.
It is a type of security testing. Penetration testing is done to tests how secure software and its
environments (Hardware, Operating system, and network) are when subject to attack by an external or
internal intruder.
An intruder can be a human/hacker or malicious programs. Pentest uses methods to forcibly intrude (by
brute force attack) or by using a weakness (vulnerability) to gain access to a software or data or
hardware with an intent to expose ways to steal, manipulate or corrupt data, software files or
configuration.
Penetration Testing is a way of ethical hacking, an experienced Penetration tester will use the same
methods and tools that a hacker would use but the intention of Penetration tester is to identify
vulnerability and get them fixed before a real hacker or malicious program exploits it.
Type of testing which checks for memory leaks or other problems that may occur with prolonged
execution. It is usually performed by performance engineers.
19
Black box testing technique performed without planning and documentation. It is usually performed by
manual testers.
Testing type that verifies the program under test stores data files in the correct directories and that it
reserves sufficient space to prevent unexpected termination resulting from lack of space. It is usually
performed by the testing team.
Bottom up approach for testing i.e continuous testing of an application as new functionality is added;
Application functionality and modules should be independent enough to test separately. done by
programmers or by testers.
Bottom-up integration testing is an integration testing approaches where testing starts with smaller
pieces or sub-systems of the software till all the way up covering entire software system. Bottom-up
integration testing begins with small portions of the software and eventually scales up in terms of size,
complexity, and completeness.
• Bug: A software bug may be defined as a coding error that causes an unexpected defect, fault or flaw.
In other words, if a program does not perform as intended, it is most likely a bug.
• Error: A mismatch between the program and its specification is an error in the program.
• Defect: Defect is the variance from a desired product attribute (it can be a wrong, missing or extra
data). It can be of two types – Defect from the product or a variance from customer/user expectations.
It is a flaw in the software system and has no impact until it affects the user/customer and operational
system. 90% of all the defects can be caused by process problems.
• Failure: A defect that causes an error in operation or negatively impacts a user/ customer.
20
1. New: When a defect is logged and posted for the first time. It’s state is given as new.
2. Assigned: After the tester has posted the bug, the lead of the tester approves that the bug is
genuine and he assigns the bug to corresponding developer and the developer team. It’s state
given as assigned.
3. Open: At this state the developer has started analyzing and working on the defect fix.
4. Fixed: When developer makes necessary code changes and verifies the changes then he/she
can make bug status as ‘Fixed’ and the bug is passed to testing team.
5. Pending retest: After fixing the defect the developer has given that particular code for retesting
to the tester. Here the testing is pending on the testers end. Hence its status is pending retest.
6. Retest: At this stage the tester do the retesting of the changed code which developer has given
to him to check whether the defect got fixed or not.
7. Verified: The tester tests the bug again after it got fixed by the developer. If the bug is not
present in the software, he approves that the bug is fixed and changes the status to “verified”.
8. Reopen: If the bug still exists even after the bug is fixed by the developer, the tester changes
the status to “reopened”. The bug goes through the life cycle once again.
9. Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer
exists in the software, he changes the status of the bug to “closed”. This state means that the
bug is fixed, tested and approved.
10. Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug,
then one bug status is changed to “duplicate“.
11. Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of
the bug is changed to “rejected”.
12. Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next
releases. The reasons for changing the bug to this state have many factors. Some of them are
priority of the bug may be low, lack of time for the release or the bug may not have major effect
on the software.
13. Not a bug: The state given as “Not a bug” if there is no change in the functionality of the
application. For an example: If customer asks for some change in the look and field of the
21
application like change of color of some text then it is not a bug but just some change in the
looks of the application.
Software Testing is not a just a single activity. It consists of series of activities carried out
methodologically to help certify your software product. These activities (stages) constitute the Software
Testing Life Cycle (STLC).
The different stages in Software Test Life Cycle -
Requirement Analysis
During this phase, test team studies the requirements from a testing point of view to identify the
testable requirements. The QA team may interact with various stakeholders (Client, Business Analyst,
Technical Leads, System Architects etc) to understand the requirements in detail. Requirements could
be either Functional (defining what the software must do) or Non Functional (defining system
performance /securityavailability ) .Automation feasibility for the given testing project is also done in
this stage.
Activities
Test Planning
This phase is also called Test Strategy phase. Typically , in this stage, a Senior QA manager will
determine effort and cost estimates for the project and would prepare and finalize the Test Plan.
22
Activities
Deliverables
This phase involves creation, verification and rework of test cases & test scripts. Test data , is
identified/created and is reviewed and then reworked as well.
Activities
Deliverables
Test cases/scripts
Test data
Test environment decides the software and hardware conditions under which a work product is tested.
Test environment set-up is one of the critical aspects of testing process and can be done in parallel with
Test Case Development Stage. Test team may not be involved in this activity if the
customer/development team provides the test environment in which case the test team is required to
do a readiness check (smoke testing) of the given environment.
Activities
Understand the required architecture, environment set-up and prepare hardware and software
requirement list for the Test Environment.
Setup test Environment and test data
23
Perform smoke test on the build
Deliverables
Test Execution
During this phase test team will carry out the testing based on the test plans and the test cases
prepared. Bugs will be reported back to the development team for correction and retesting will be
performed.
Activities
Deliverables
Testing team will meet , discuss and analyze testing artifacts to identify strategies that have to be
implemented in future, taking lessons from the current test cycle. The idea is to remove the
process bottlenecks for future test cycles and share best practices for any similar projects in
future.
Activities
Evaluate cycle completion criteria based on Time, Test coverage, Cost, Software, Critical
Business Objectives , Quality
Prepare test metrics based on the above parameters.
Document the learning out of the project
Prepare Test closure report
Qualitative and quantitative reporting of quality of the work product to the customer.
Test result analysis to find out the defect distribution by type and severity.
24
What is Waterfall model- advantages, disadvantages and when to use it?
The Waterfall Model was first Process Model to be introduced. It is also referred to as a linear-
sequential life cycle model. It is very simple to understand and use. In a waterfall model, each
phase must be completed fully before the next phase can begin. This type of model is basically
used for the for the project which is small and there are no uncertain requirements. At the end of
each phase, a review takes place to determine if the project is on the right path and whether or
not to continue or discard the project. In this model the testing starts only after the development
is complete. In waterfall model phases do not overlap.
Diagram of Waterfall-model:
25
Disadvantages of waterfall model:
Once an application is in the testing stage, it is very difficult to go back and change something
that was not well-thought out in the concept stage.
No working software is produced until late during the life cycle.
High amounts of risk and uncertainty.
Not a good model for complex and object-oriented projects.
Poor model for long and ongoing projects.
Not suitable for the projects where requirements are at a moderate to high risk of changing.
This model is used only when the requirements are very well known, clear and fixed.
Product definition is stable.
Technology is understood.
There are no ambiguous requirements
Ample resources with required expertise are available freely
The project is short.
Very less customer enter action is involved during the development of the product. Once the
product is ready then only it can be demoed to the end users. Once the product is developed and
if any failure occurs then the cost of fixing such issues are very high, because we need to update
everywhere from document till the logic.
V- model means Verification and Validation model. Just like the waterfall model, the V-Shaped
life cycle is a sequential path of execution of processes. Each phase must be completed before
the next phase begins. Testing of the product is planned in parallel with a corresponding phase
of development in V-model.
Diagram of V-model:
26
The various phases of the V-model are as follows:
Requirements like BRS and SRS begin the life cycle model just like the waterfall model. But, in
this model before development is started, a system test plan is created. The test plan focuses on
meeting the functionality specified in the requirements gathering.
The high-level design (HLD) phase focuses on system architecture and design. It provide
overview of solution, platform, system, product and service/process. An integration test plan is
created in this phase as well in order to test the pieces of the software systems ability to work
together.
The low-level design (LLD) phase is where the actual software components are designed. It
defines the actual logic for each and every component of the system. Class diagram with all the
methods and relation between classes comes under LLD. Component tests are created in this
phase as well.
The implementation phase is, again, where all coding takes place. Once coding is complete, the
path of execution continues up the right side of the V where the test plans developed earlier are
now put to use.
Coding: This is at the bottom of the V-Shape model. Module design is converted into code by
developers.
Advantages of V-model:
27
Simple and easy to use.
Testing activities like planning, test designing happens well before coding. This saves a lot of
time. Hence higher chance of success over the waterfall model.
Proactive defect tracking – that is defects are found at early stage.
Avoids the downward flow of the defects.
Works well for small projects where requirements are easily understood.
Disadvantages of V-model:
The V-shaped model should be used for small to medium sized projects where requirements are
clearly defined and fixed.
The V-Shaped model should be chosen when ample technical resources are available with
needed technical expertise.
High confidence of customer is required for choosing the V-Shaped model approach. Since, no
prototypes are produced, there is a very high risk involved in meeting customer expectations.
In incremental model the whole requirement is divided into various builds. Multiple
development cycles take place here, making the life cycle a “multi-waterfall” cycle. Cycles are
divided up into smaller, more easily managed modules. Each module passes through the
requirements, design, implementation and testing phases. A working version of software is
produced during the first module, so you have working software early on during the software life
cycle. Each subsequent release of the module adds function to the previous release. The process
continues till the complete system is achieved.
For example:
28
application. Then he started building it and in the first iteration the first module of the application
or product is totally ready and can be demoed to the customers. Likewise in the second iteration
the other module is ready and integrated with the first module. Similarly, in the third iteration the
whole product is ready and integrated. Hence, the product got ready step by step.
Generates working software quickly and early during the software life cycle.
This model is more flexible – less costly to change scope and requirements.
It is easier to test and debug during a smaller iteration.
In this model customer can respond to each built.
Lowers initial delivery cost.
Easier to manage risk because risky pieces are identified and handled during it’d iteration.
This model can be used when the requirements of the complete system are clearly defined and
understood.
Major requirements must be defined; however, some details can evolve with time.
29
There is a need to get a product to the market early.
A new technology is being used
Resources with needed skill set are not available
There are some high risk features and goals.
In case of some software deliverables, especially the large ones, it is difficult to assess the effort
required at the beginning of the software development life cycle.
There is lack of emphasis on necessary designing and documentation.
30
The project can easily get taken off track if the customer representative is not clear what final
outcome that they want.
Only senior programmers are capable of taking the kind of decisions required during the
development process. Hence it has no place for newbie programmers, unless combined with
experienced resources.
When new changes are needed to be implemented. The freedom agile gives to change is very
important. New changes can be implemented at very little cost because of the frequency of new
increments that are produced.
To implement a new feature the developers need to lose only the work of a few days, or even
only hours, to roll back and implement it.
Unlike the waterfall model in agile model very limited planning is required to get started with
the project. Agile assumes that the end users’ needs are ever changing in a dynamic business
and IT world. Changes can be discussed and features can be newly effected or removed based
on feedback. This effectively gives the customer the finished system they want or need.
Both system developers and stakeholders alike, find they also get more freedom of time and
options than if the software was developed in a more rigid sequential way. Having options gives
them the ability to leave important decisions until more or better data or even entire hosting
programs are available; meaning the project can continue to move forward without fear of
reaching a sudden standstill.
The spiral model is similar to the incremental model, with more emphasis placed on risk
analysis. The spiral model has four phases: Planning, Risk Analysis, Engineering and
Evaluation. A software project repeatedly passes through these phases in iterations (called
Spirals in this model). The baseline spiral, starting in the planning phase, requirements are
gathered and risk is assessed. Each subsequent spirals builds on the baseline spiral.
Planning Phase: Requirements are gathered during the planning phase. Requirements like
‘BRS’ that is ‘Bussiness Requirement Specifications’ and ‘SRS’ that is ‘System Requirement
specifications’.
Risk Analysis: In the risk analysis phase, a process is undertaken to identify risk and alternate
solutions. A prototype is produced at the end of the risk analysis phase. If any risk is found
during the risk analysis then alternate solutions are suggested and implemented.
Engineering Phase: In this phase software is developed, along with testing at the end of the
phase. Hence in this phase the development and testing is done.
31
Evaluation phase: This phase allows the customer to evaluate the output of the project to date
before the project continues to the next spiral.
32
Significant changes are expected (research and exploration)
The basic idea here is that instead of freezing the requirements before a design or coding can
proceed, a throwaway prototype is built to understand the requirements. This prototype is
developed based on the currently known requirements. By using this prototype, the client can get
an “actual feel” of the system, since the interactions with prototype can enable the client to better
understand the requirements of the desired system. Prototyping is an attractive idea for
complicated and large systems for which there is no manual process or existing system to help
determining the requirements.
The prototype are usually not complete systems and many of the details are not built in the
prototype. The goal is to provide a system with overall functionality.
33
• Leads to implementing and then repairing way of building systems.
• Practically, this methodology may increase the complexity of the system as scope of the
system may expand beyond original plans.
• Incomplete application may cause application not to be used as the full system was
designed
Prototype model should be used when the desired system needs to have a lot of interaction
with the end users.
Typically, online systems, web interfaces have a very high amount of interaction with end users,
are best suited for Prototype model. It might take a while for a system to be built that allows
ease of use and needs minimal training for the end user.
Prototyping ensures that the end users constantly work with the system and provide a feedback
which is incorporated in the prototype to result in a useable system. They are excellent for
designing good human computer interface systems.
Diagram of RAD-Model:
34
The phases in the rapid application development (RAD) model are:
Business modeling: The information flow is identified between various business functions.
Data modeling: Information gathered from business modeling is used to define data objects that
are needed for the business.
Process modeling: Data objects defined in data modeling are converted to achieve the business
information flow to achieve some specific business objective. Description are identified and
created for CRUD of data objects.
Application generation: Automated tools are used to convert process models into code and the
actual system.
Testing and turnover: Test new components and all the interfaces.
35
Increases reusability of components
Quick initial reviews occur
Encourages customer feedback
Integration from very beginning solves a lot of integration issues.
Depends on strong team and individual performances for identifying business requirements.
Only system that can be modularized can be built using RAD
Requires highly skilled developers/designers.
High dependency on modeling skills
Inapplicable to cheaper projects as cost of modeling and automated code generation is very
high.
RAD should be used when there is a need to create a system that can be modularized in 2-3
months of time.
It should be used if there’s high availability of designers for modeling and the budget is high
enough to afford their cost along with the cost of automated code generating tools.
RAD SDLC model should be chosen only if resources with high business knowledge are available
and there is a need to produce the system in a short span of time (2-3 months).
There are two key things in defects of the software testing. They are:
1) Severity
2) Priority
1) Severity:
It is the extent to which the defect can affect the software. In other words it defines the impact
that a given defect has on the system. For example: If an application or web page crashes when
a remote link is clicked, in this case clicking the remote link by an user is rare but the impact of
application crashing is severe. So the severity is high but priority is low.
36
Critical: The defect that results in the termination of the complete system or one or more
component of the system and causes extensive corruption of the data. The failed function is
unusable and there is no acceptable alternative method to achieve the required results then the
severity will be stated as critical.
Major: The defect that results in the termination of the complete system or one or more
component of the system and causes extensive corruption of the data. The failed function is
unusable but there exists an acceptable alternative method to achieve the required results then
the severity will be stated as major.
Moderate: The defect that does not result in the termination, but causes the system to produce
incorrect, incomplete or inconsistent results then the severity will be stated as moderate.
Minor: The defect that does not result in the termination and does not damage the usability of
the system and the desired results can be easily obtained by working around the defects then
the severity is stated as minor.
Cosmetic: The defect that is related to the enhancement of the system where the changes are
related to the look and field of the application then the severity is stated as cosmetic.
2) Priority:
Priority defines the order in which we should resolve a defect. Should we fix it now, or can it
wait? This priority status is set by the tester to the developer mentioning the time frame to fix the
defect. If high priority is mentioned then the developer has to fix it at the earliest. The priority
status is set based on the customer requirements. For example: If the company name is
misspelled in the home page of the website, then the priority is high and severity is low to fix it.
Low: The defect is an irritant which should be repaired, but repair can be deferred until after
more serious defect have been fixed.
Medium: The defect should be resolved in the normal course of development activities. It can
wait until a new build or version is created.
High: The defect must be resolved as soon as possible because the defect is affecting the
application or the product severely. The system cannot be used until the repair has been done.
Few very important scenarios related to the severity and priority which are asked during
the interview:
High Priority & High Severity: An error which occurs on the basic functionality of the
application and will not allow the user to use the system. (Eg. A site maintaining the student
details, on saving record if it, doesn’t allow to save the record then this is high priority and high
severity bug.)
High Priority & Low Severity: The spelling mistakes that happens on the cover page or
heading or title of an application.
High Severity & Low Priority: An error which occurs on the functionality of the application
(for which there is no workaround) and will not allow the user to use the system but on click of
link which is rarely used by the end user.
37
Low Priority and Low Severity: Any cosmetic or spelling issues which is within a paragraph or
in the report (Not on cover page, heading, title).
By design we mean to create a plan for how to implement an idea and technique is a method or
way for performing a task. So, Test Design is creating a set of inputs for given software that will
provide a set of expected outputs. The idea is to ensure that the system is working good enough
and it can be released with as few problems as possible for the average user.
Broadly speaking there are two main categories of Test Design Techniques. They are:
1. Static Techniques
2. Dynamic Techniques
3. Below is the tree structure of the testing techniques:
Walkthrough:
38
It is not a formal process/review
It is led by the authors
Author guide the participants through the document according to his or her thought process to
achieve a common understanding and to gather feedback.
Useful for the people if they are not from the software discipline, who are not used to or cannot
easily understand software development process.
Is especially useful for higher level documents like requirement specification, etc.
i. To present the documents both within and outside the software discipline in order to gather the
information regarding the topic under documentation.
ii. To explain or do the knowledge transfer and evaluate the contents of the document
iii. To achieve a common understanding and to gather feedback.
iv. To examine and discuss the validity of the proposed solutions
Performed on requirement design or code without actually executing the software or before the
code is actually run.
Goal of static analysis is to find the defects whether or not they may cause failure.
Static analysis find defects rather than failures.
It is typically used by the developers before and sometimes during component and integration
testing.
The various features of static analysis tools are discussed below with a special focus on static
code analysis tools because they are the most common in day to day practice.
Static code analysis tools are as follows:
39
people spend time checking coding standards in reviews that will distract them from other
defects that might otherwise find and makesing the review process less effective.
2. Code metrics: Code metrics is basically the measurement of depth of nesting, cyclomatic
number and number of lines of code. This information can be computed not only as the design
and code are being created but also during the changes that are made to the system, to see if the
design or code is becoming bigger, more complex and more difficult to understand and maintain.
The measurement also helps us to decide between several design alternatives. There are many
different types of structural measures. One of them is Cyclomatic complexity metric. The
Cyclomatic complexity metrics based on the number of decisions in a program. It is important to
tester because it provides an indication of the amount of testing. There are many ways to
calculate cyclomatic complexity but the easiest way is to sum the number of binary decision
statements (e.g. if, while, for, etc.) and add 1 to it.
In the program mentioned above has 2 IF conditions. Thus just add 1 to it and the cyclomatic
complexity is 2+1=3.
We can also calculate the cyclomatic complexity using the control flow.
In the control flow shown below there are 7 nodes (shapes) and 8 edges (lines). Thus by formula
((no. of edges-no. of nodes)+2) that is (8-7)+2 = 1+2 = 3.
40
3. Code structure: Code structure tells us about the effort required to write the code in the first
place, to understand the code while making the change, or to test the code using particular tools
or techniques. There are several aspect of code structure to consider:
Control flow structure: It addresses the sequence in which the instructions are executed.
Data flow structure: It follows the track of the data item as it is accessed and modified by the
code.
Data structure: It refers to the organization of the data itself, independent of the program.
Test conditions should be able to be linked back to their sources in the test basis, this is known as
traceability. Traceability can be horizontal through all the test documentation for a given test
level (e.g. system testing, from test conditions through test cases to test scripts) or it can be
vertical through the layers of development documentation (e.g. from requirements to
components).
Now, the question may arise is that Why is traceability important? So, let’s have a look on the
following examples:
The requirements for a given function or feature have changed. Some of the fields now have
different ranges that can be entered. Which tests were looking at those boundaries? They now
need to be changed. How many tests will actually be affected by this change in the
requirements? These questions can be answered easily if the requirements can easily be traced
to the tests.
A set of tests that has run OK in the past has now started creating serious problems. What
functionality do these tests actually exercise? Traceability between the tests and the
requirement being tested enables the functions or features affected to be identified more
easily.
Before delivering a new release, we want to know whether or not we have tested all of the
specified requirements in the requirements specification. We have the list of the tests that have
passed – was every requirement tested?
41