Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Programming
To understand the concept of agile testing, first let's understand-
What is Agile?
AGILE is a methodology that promotes continuous iteration of development and testing
throughout the software development life cycle of the project. Both development and testing
activities are concurrent unlike the Waterfall method.
I hope we got an idea of Agile!!! Now, we can step on to Agile Testing.
Scrum
SCRUM is an agile development method which concentrates particularly on how to manage tasks
within a team based development environment. Basically, Scrum is derived from activity that
occurs during rugby match. Scrum believes in empowering the development team and advocates
working in small teams (say- 7 to 9 members). It consists of three roles and their responsibilities
are explained as follows:
Scrum Master
o
Master is responsible for setting up the team, sprint meeting and removes obstacles to
progress
Product owner
o
The Product Owner creates product backlog, prioritizes the backlog and is responsiblefor
the delivery of the functionality at each iteration
Scrum Team
o
Team manages its own work and organizes the work to complete the sprint or cycle
Product Backlog
This is repository where requirements are tracked with details on the no of requirements to be
completed for each release. It should be maintained and prioritized by scrum master and it should
be distributed to the scrum team. Team can also request for new requirement addition or
modification or deletion
Test Design
o
User stories are used for development of test cases .User stories are written by the product
owner. User stories are short description of functionalities of the System Under Test.
Example for Insurance Provider is - Premium can be paid using online system
Test Cases are developed based on user stories which should be approved by business
analysts and the customers
Test Execution
o
Test execution is carried out in a lab where both tester and developer work hand in hand.
Defect are logged in Defect Management tool which are tracked on a daily basis. Defects
can be discussed during scrum meeting
Test Automation
o
Test Automation can be achieved by utilizing various opensource or paid tools available
in the market .
This proves effective in ensuring that everything that needs to be tested was covered.
Sufficient Test coverage can be achieved with the close interaction with the team.
Managing
o
Sprint(Cycle) meeting is held at the beginning to decide the scope of sprint, understand
the product backlog, task estimates and assignments
Daily 15 minutes meeting will be held where participants answer three questions.
Scrum Practices
Practices are described in detailed:
Product backlog is a list where all details are entered to get end product
During each Sprint, top items of Product backlog are selected and turned into Sprint backlog
Planning
Infrastructure Requirements
Analysis
Design
Execution
Coding
Unit Testing
Wrapping
Small Releases
Regression Testing
Closure
Pilot Launch
Training
Production Launch
Production Support
There are two story boards available to track the work on a daily basis and those are listed below
for reference.
Story Cardboard
o
This is traditional way of collecting all the stories in a board in the form of stick notes to
track daily XP activities. As this manual activity involves more effort and time, it is better
to switch to online form.
Online tool Storyboard can be used to store the stories .This can be used by several teams
for different purposes.
XP Testing
Scrum Testing
features
Scrum and eXtreme are the popular Agile methodologies. Below listed methodologies are used
less frequently
Dynamic System Development Method (DSDM)
This is an Iterative and incremental approach that emphasizes on the continuous user
involvement.
This is a technique which has short iterations where new test cases covering the desired
improvement or new functionality are written first.
XBreed
Agile enterprise previously known as Xbreed .It is agile way of managing, architecting and
monitoring the enterprise.
Crystal
Agile metrics:
Metrics that can be collected for effective usage of Agile is:
Drag Factor
o
Drag factor can be improved by reducing number of shared resources, reducing the
amount of non-contributing work
New estimates can be increased by percentage of drag factor -New estimate = (Old
estimate+drag factor)
Velocity
o
Conclusion:
Agile testing involves testing as early as possible in software development life cycle. It demands
high customer involvement and testing code as soon as it becomes available. The code should be
stable enough to take it to system testing. Extensive regression testing can be done to make sure
that the bugs are fixed and tested. Mainly, Communication between the teams makes agile testing
success!!!
Waterfall model
What is it?
In the waterfall model ,software development progress through various phases like Requirements
Analysis , Design etc - sequentially.
In this model, next phase begins only when the earlier phase is completed.
What Is The Testing Approach?
The first phase in waterfall model is the requirements phase in which all the project requirements
are completely defined before starting the testing. During this phase , the test team brainstorms
the scope of testing , test strategy and drafts a detailed test plan.
Only once the design of software is complete, the team will move on to execution of the test
cases to ensure that the developed software behaves as it expected.
In this methodology, the testing team proceeds to the next phase only when the previous phase is
completed.
Advantages
This model is very simple to plan and manage. Hence, projects where requirements are clearly
defined and stated beforehand can be easily tested using waterfall model.
Disadvantages
In the waterfall model , you can begin with the next phase only once the previous phase is
completed. Hence , this model cannot accommodate unplanned events and uncertainty.
This methodology is not suitable for projects where the requirements change frequently.
Iterative development
What is it?
In this model , a big project is divided into small parts , and each part is subjected to multiple
iterations of the waterfall model. At the end of iteration, a new module is developed or an existing
module is enhanced. This module is integrated into the software architecture and the entire
system is tested all together
What is the testing Approach?
As soon as iteration is completed, the entire system is subjected to testing. Feedback from testing
is immediately available and is incorporated in next cycle. The testing time required in successive
iteration can be reduced based on the experience gained from past iterations.
Advantages
The main advantage of iterative development is the test feedback is immediately available at the
end of each cycle.
Disadvantages
This model increases communication overheads significantly since at the end of each cycle,
feedback about deliverables , effort etc must be given.
Agile methodology
What is it ?
Traditional software development methodologies work on the premise that software
requirements remain constant throughout the project. But with increase in complexity , the
requirements undergo numerous changes and continuously evolve. At times, the customer himself
is not sure what he wants. Though iterative model addresses this issue, it's still based on the
waterfall model.
In Agile methodology , software is developed in incremental, rapid cycles. Interactions
amongst customers, developers and client are emphasized rather than processes and tools. Agile
methodology focuses on responding to change rather than extensive planning.
Advantages
It is possible to make changes in the project at any time to comply with the requirements.
This incremental testing minimizes risks.
Disadvantages
Constant client interaction means added time pressure on all stake holders including the client
themselves , software development and test teams .
Extreme programming
What is it?
Extreme programming is a type of agile methodology which believes in short development
cycles. A project is divided into simple engineering tasks. Programmers code a simple piece of
software and get back to customer for feedback. Review points from the customer are
incorporated and the developers proceed with the next task.
Scheduling
Realistic scheduling is the key to the implementation of successful testing methodology and the
schedule should meet the needs of every member of the team.
Defined deliverables
In order to keep all the members of the team on the same page, well defined deliverables should
be provided. The deliverables should contain direct content without any ambiguity.
Test approach
Once scheduling is complete and defined deliverables are made available, the testing team should
be able to formulate the right test approach. Definition documents and developer meetings should
indicate the team about the best test approach that can be used for the project.
Reporting
Transparent reporting is very difficult to achieve, but this step determines the effectiveness of the
testing approach used in the project.
Cyclomatic complexity is a software metric used to measure the complexity of a program. These
metric, measures independent paths through program source code.
Independent path is defined as a path that has atleast one edge which has not been traversed
before in any other paths.
Cyclomatic complexity can be calculated with respect to functions, modules, methods or classes
within a program.
This metric was developed by Thomas J. McCabe in 1976 and it is based on a control flow
representation of the program. Control flow depicts a program as a graph which consists of Nodes
and Edges.
In the graph, Nodes represent processing tasks while edges represent control flow between the
nodes.
Mathematical representation:
Mathematically, it is set of independent paths through the graph diagram. The complexity of the
program can be defined as V(G) = E - N + 2
Where,
E - Number of edges
N - Number of Nodes
V (G) = P + 1
Where P = Number of predicate nodes (node that contains condition)
Example ?
1
i = 0;
2
3
4
5
6
while (i<n-1) do
j = i + 1;
8
9
while (j<n) do
10
11 if
A[i]<A[j] then
12
13 swap(A[i],
A[j]);
14
15 end
do;
16
17 i=i+1;
18
19 end
do;
Computing mathematically,
V(G) = 9 - 7 + 2 = 4
1, 7
1, 2, 6, 1, 7
1, 2, 3, 4, 5, 2, 6, 1, 7
1, 2, 3, 5, 2, 6, 1, 7
If (Condition 1)
2
3
Statement 1
4
5
Else
6
7
Statement 2
8
9
If (Condition 2)
10
11 Statement
12
13 Else
14
15 Statement
Steps to be followed:
The following steps should be followed for computing Cyclomatic complexity and test cases
design.
Step 1 - Construction of graph with nodes and edges from the code
Step 2 - Identification of independent paths
Step 3 - Cyclomatic Complexity Calculation
Step 4 - Design of Test Cases
Once the basic set is formed, TEST CASES should be written to execute all the paths.
More on V (G):
Cyclomatic complexity can be calculated manually if the program is small. Automated tools need
to be used if the program is very complex as this involves more flow graphs. Based on
complexity number, team can conclude on the actions that need to be taken for measure.
Following table gives overview on the complexity number and corresponding meaning of v (G):
Complexity Number
Meaning
Structured and well written code
1-10
High Testability
Cost and Effort is less
Complex Code
10-20
Medium Testability
Cost and effort is Medium
Very complex Code
20-40
Low Testability
Developers can assure that all the paths have been tested atleast once
Using these metrics early in the cycle reduces more risk of the program
Conclusion:
Cyclomatic Complexity is software metric useful for structured or white box testing .It is mainly
used to evaluate complexity of a program. If the decision points are more, then complexity of the
program is more. If program has high complexity number, then probability of error is high with
increased time for maintenance and trouble shoot.
What is assurance?
Assurance is nothing but a positive declaration on a product or service ,which gives confidence. It
is certainty of a product or a service, which it will work well. It gives a guarantee that the product
will work without any problems as per the expectations or requirements.
Plan
Do
Check
Act
These above steps are repeated to ensure that processes followed in the organization are evaluated
and improved on a periodic basis. Let's look into the above steps in detail
Plan - Organization should plan and establish the process related objectives and determine
the processes that are required to deliver a high Quality end product.
Do - Development and testing of Processes and also "do" changes in the processes
Check - Monitoring of processes, modify the processes, and check whether it meets the
predetermined objectives
Act - Implement actions that are necessary to achieve improvements in the processes
An organization must use quality assurance to ensure that the product is designed and
implemented with correct procedures. This helps reduce problems and errors , in the final
product.
Quality control popularly abbreviated as QC, is a process used to ensure quality in a product or a
service.It does not deal with the processes used to create a product, rather It examines the quality
of the "end products" and the final outcome .
Main aim of Quality control is to check whether the products meet the specifications and
requirements of the customer. If an issue or problem is identified, it needs to be fixed before
delivery to the customer.
QC also evaluates people on their quality level skill sets and imparts training and certifications.
This evaluation is required for the service based organization, and helps provide "perfect" service
to the customers.
Walkthrough
Quality Audit
Testing
Defining Process
Inspection
Checkpoint review
The above activities are concerned with QA and QC of any product and not essentially
software. With respect to software
SQA
Software Testing
Process focused
Product focused
Preventive technique
Corrective technique
Proactive measure
Reactive measure
The scope of SQA applied to all products that will The scope of Software Testing applies to a
be created by the organization
ISO 9000
This standard was first established in 1987, and it is related to Quality Management Systems.
This helps the organization ensure quality to their customers and other stakeholders. An
organization who wishes to be certified as ISO 9000 is audited based on their functions, products,
services and their processes.The main objective is to review and verify whether the organization
is following the process as expected and check whether existing processes needs improvement.
This certification helps
CMMI level
The Capability Maturity Model Integrated (CMMI) is a process improvement approach
developed specially for the software process improvement. It is based on the process maturity
framework and used as general aid in business processes in Software Industry. This model is
highly regarded and widely used in Software Development Organizations.
CMMI has 5 levels. An organizations is certified at CMMI level 1 to 5 based on the maturity of
their Quality Assurance Mechanisms.
Level 1 - Initial: In this stage quality environment is unstable. Simply, no processes has
been followed or documented
Level 2 - Repeatable : Some processes are followed which are repeatable. This level
ensures processes are followed at the project level.
Level 3 - Defined : Set of processes are defined and documented at the organizational level.
Those defined processes are subject to some degree of improvement.
Level 4 - Managed : This level uses process metrics and effectively controls the processes
that are followed.
Level 5 - Optimizing : This level focuses on the continuous improvements of the processes
through learning & innovation.
Level 1 - Initial: There is no quality standard followed for testing processes and only ad-hoc
methods are used at this level
Level 2 - Definition: Defined process .Preparation of test strategy, plans, test cases are done.
Level 3 - Integration: Testing is carried out throughout the software development life cycle
(SDLC) - which is nothing but integration with the development activities E.g. V- Model.
Level 4 - Management and Measurement: Review of requirements and designs takes place
during this level and criteria has been set for each level of testing
Level 5 - Optimization: Many preventive techniques are used for testing processes and tool
support(Automation) is used to improve the testing standards and processes.
Conclusion:
Quality Assurance is to check whether the product developed is fit for use. For that, Organization
should have processes and standards to be followed which need to be improved on a periodic
basis. It concentrates mainly on the quality of product / service that we are providing to the
customers during or after implementation of software.
Manually
Typically test data should be generated before you begin test execution since in many testing
environments creating test data takes many pre-steps or test environment
configurations which is very time consuming. If test data generation is done while you are in
test execution phase you may exceed your testing deadline.
Below are described several testing types together with some suggestions regarding their testing
data needs.
It is desirable to cover as many branches as possible; testing data can be generated such that
all branches in the program source code are tested at least once
Path testing: all paths in the program source code are tested at least once - test data can be
designed to cover as many cases as possible
Testing data may contain invalid parameter types used to call different methods
Testing data may consist in invalid combination's of arguments which are used to call
the program's methods
Integrity: Determine that the information provided by the system is correct. To design
suitable test data you can start by taking an in depth look at the design, code, databases and
file structures.
Authentication: Represents the process of establishing the identity of a user. Testing data
can be designed as different combination of usernames and passwords and its purpose is to
check that only the authorized people are able to access the software system.
Authorization: Tells what are the rights of a specific user. Testing data may contain
different combination of users, roles and operations in order to check only users with
sufficient privileges are able to perform a particular operation.
Valid data : Check system response when Valid test data is submitted
Invalid data :Check system response when InValid test data is submitted
Illegal data format: Check system response when test data is in invalid format
Boundary Condition Data set: Test data meeting bounding value conditions
Equivalence Partition Data Set : Test data qualifying your equivalence partitions.
Decision Table Data Set: Test data qualifying your decision table testing strategy
State Transition Test Data Set: Test data meeting your state transition testing strategy
Use Case Test Data: Test Data in-sync with your use cases.
Note: Depending on the software application to be tested, you may use some or all
of the above test data creation
Protect data privacy by creating a clone of the existing data and masking confidential values
Test Data generator by DTM, is a fully customizable utility that generates data, tables (views,
procedures etc) for database testing (performance testing, QA testing, load testing or usability
testing) purposes.
Datatect by Banner Software, generates a variety of realistic test data in ASCII flat files or
directly generates test data for RDBMS including Oracle, Sybase, SQL Server, and Informi.
In conclusion, well-designed testing data allows you to identify and correct serious flaws in
functionality. Choice of test data selected must be reevaluated in every phase of a multiphase product development cycle. So, always keep an eye on it.