An Introduction to

Agile Software Testing

1
Course Objectives
After this seminar, you will:

 Understand what an Agile project

 Know the differences between Kanban and Scrum
 Understand what and how a tester performs the
testing in Scrum process
 Understand what testing techniques required for a
tester in Agile
 Know what challenges of Agile Software Testing

2
Course Content

 What is an Agile Project?

 What are Scrum and Kanban?
 What are Scrum & Kanban Processes?
 What are testing Activities in Scrum?
 What are Agile Testing Techniques?

3
What is an Agile Project?
All projects are “agile”, but some agile projects
are more agile than others

In practice, “Agile” labeled

How to “agile” a project to projects that its delivery of
An Agile purist would point to agile manifesto and each increment taking no
its principles
longer 4 weeks
Individuals and interactions over process and tools
Is it met level of agility?
 Working SW over comprehensive documents
 Customer collaboration over contract negotiation
 Responding to change over following a plan 4
Agile Project
An
introduction
of Scrum and
Kanban

5
What is Scrum?
Scrum in 100 words
• Scrum is an agile process that allows us to focus on
delivering the highest business value in the shortest
time.
• It allows us to rapidly and repeatedly inspect actual
working software (every two weeks to one month).
• The business sets the priorities. Teams self-organize to
determine the best way to deliver the highest priority
features.
• Every two weeks to a month anyone can see real
working software and decide to release it as is or
continue to enhance it for another sprint.
6
What is Scrum? - Principles
9 Scrum principles
Scrum contains sets of principles (some of them
required while others are optional) and
predefined roles
(1)Demo
(2)Sit together
(3)Daily Scrum
(4)Retrospective
(5)Sprint backlog
(6)Product Backlog
(7)Sprint planning meetings
(8)Time-boxed iterations
(9)Definition of Done 7
What is Kanban?
Kanban in 100 words
Visualize the workflow
Split the work into pieces, write each item on a card and put on
the wall.
Use named columns to illustrate where each item is in the
workflow..
Limit Work In Progress (WIP) – assign explicit limits to how many
items may be in progress at each workflow state
Measure the lead time (average time to complete one item,
sometimes called “cycle time”), optimize the process to make
lead time as small and predictable as possible

8
Kanban vs Scrum Board
Kanban and Scrum Board

V.S.

Scrum and Kanban over time

V.S.

Scrum resists change within V.S. Kanban limits number of items

an Iteration in workflow state
9
 An
introduction
of Scrum
Process

10
 What is Scrum process?
Scrum Roles Key Artifacts Ceremonies Process
Sprint planning
Product Backlog • Hosted by Scrum Master Product
• Requirements – user stories • Pick highest priority items in Backlog
• Desired work Product backlog and the team turns
• Prioritized by Product Owner the items into Sprint Backlog
• Anybody can add to it
Product Owner • Estimate sprint backlog in hours Sprint – 2-4 weeks
• Work breakdown
Sprint Goal Sprint Planning Sprint Goal
• Summary of focused work in • Declare Sprint Goal
sprint Daily Scrum Sprint
• Declared by Product Owner
• Accepted by team • Hosted by Scrum Master Daily Scrum Backlog
• 15 minutes - same time every day
Scrum Master Blocks
Sprint Backlog • Not for problem solving
• Team signs up for work of • 3 questions in meeting: (1) What Daily Work
their own – work never Product
did you do? (2) What will you do?
assigned (3) What’s in your way?
• Owned/managed by the team
• Estimated work remaining is • Team updates sprint backlog
updated daily Sprint Review/
Sprint Review
Blocks List • Hosted by Scrum Master – 2-4 hours Retrospective
Scrum Team • List of blocks & unmade • Accomplishments
decisions • Whole team participants
(max – 9) • Owned by Scrum Master
• Updated daily • Take form of demo for features
Product
Sprint Retrospective
Burndown chart Backlog
• Hosted by SM – 15-30 minutes
• Effort spent over period
• Stories/ features completed • Discuss on “Start doing”, “Continue 11
doing” and “Stop doing”
What are roles in Scrum?

Product Owner (PO) is a Scrum Master represents The Team

client’s representation
Define features of product
Decide Release Date and
content
Prioritize features
according to market value
Be responsible for the
profitability of product
 Accept or reject work
item result
Management 5-9 members
Enacting Scrum values Including: developers,
testers, designers,…
Ensure team’s productivity
Full time
Prioritize features according
to market value Work Self-organizing
 Membership should be
Corporate across all roles
and functions changed each sprint
 Shield team from external
interferences

A practice for prioritizing features (or User Story)

Priority = Max (5 (or 6) x priority that assigned by PO, sum
(priority that assigned by team members) 12
 What are roles of tester in Scrum?
There is NO tester in formal Scrum Process
Testing is carried out by developer with Unit Test
•Test Driven Development means testing
executed before development and refactoring
performed to optimize development
•Testing coverage

 Testing is carried out by Product Owner/ Client

•Frequently testing by Product Owner – each

sprint
•Testing Acceptance Criteria

Isn’t it really necessary tester position in Scrum?

What happens without tester?
No future for tester? 13
What are roles of tester in Scrum ? (cont.)
Are testers obsolete?
Questions:
• Will developers do acceptance
testing well?
•Will developers test their work
product well?
•Is that enough with only one
kind of testing – Unit Test?
• How to test non-functional
requirements?

Benefits when have

dedicated test team:
•Focus on customer usage over
technical implementation
•Focus on uncovering flaws over
confirming completeness

14
What are roles of tester in Scrum? (cont.)
• Participant in Release/ Sprint
Planning
• Support developers in Unit Testing
– say them about testing view instead
of development view
• Test User Story when completed,
last gate to confirm testing
completeness
• Collaborate with customer and
Product Owner to define acceptance
criteria
• Provide positive feedback to
customer
• Develop automation testing

15
Testing Activity
in Scrum &
Kaban Process

16
What are Testing Activities in Scrum Process?
Process Tester must know what model of cadence used
Product

cadence
Backlog

single
Sprint – 2-4 weeks

cadence
Three
Sprint Planning Sprint Goal
Sprint
Daily Scrum Backlog
Blocks

Driven
Daily Work
Product event

In Sprint Planning:

Sprint Review/
Retrospective • As a tester, he/she says what should be picked from
Product Backlog for this Sprint to bring value to customer
& to create high performance

Product •As a tester, he/she says how many hours should have
Backlog to complete testing for each of selected User Stories
•As a tester, he/she must understand what sprint goals
• As a tester, involves into the prioritizing process17
What are Testing Activities …? – estimate effort
Effort Estimate techniques:
Heuristic (expert judgment based) Parametric (calculation based)
Comparison to similar systems  Function Points
 Expert Judgment  Use Case Points
 Activity Based (top down)  User Story Points
 Task Based (bottom up)
 Three-point estimate
 50/90 estimate

In Agile, testers are empowered to make decision. It means they

are given authority to estimate effort for each work item that
stated in Story Card for Product Owner reviews
As a result, testers are responsible for all testing activities that
they estimated.

18
What are Testing Activities …? – 50/90 estimate

50/90 Estimation
Worst case: number of hours (cost) in the worse-case scenario
 Most -likely: number of hours (cost) in the Most -likely scenario

Total Hours = Most Likely

Task variance = (Worst Case (W) – Most Likely (A))
Deviation = sqrt (sum((Wi-Ai)^2))

19
What are Testing Activities …? – 3-point estimate

3 Points Estimate (PERT)

Opt: indicates number of hours (cost) in the best-case scenario

Pess: indicates number of hours (cost) in the worse-case scenario
 Most -likely: indicates number of hours (cost) in the Most -likely scenario

PERT = (Opt + 4 x Most-Likely + Pess) / 6

Variance = (Pess – Opt) / 6

20
What are Testing Activities …? – User Story Points
User Story points (USP)
 Story points are a unit of measure for expressing the overall size of a
user story, feature, or other piece of work
 The raw values assigned are unimportant
 Image a story with 2 USP(s) will 2 times bigger than 1-USP one
 Two estimation scales or or
Velocity is a measure of a team’s rate of progress – Ideal Time

The beauty of this is that estimating in story points completely separates

the estimation of effort from the estimation of duration
If we find that a particular work unit is twice the size we thought.
The estimate remains the same, but rate of progress through it
will be slower

21
What are Testing Activities in Scrum Process?
Process
Product
Backlog

Sprint – 2-4 weeks

Sprint Planning Sprint Goal

Sprint
Daily Scrum Backlog In Daily Scrum:
Daily Work
Blocks
• As a tester, he/she joins all daily meeting (standup)
Product to speak up
1. what was tested and did yesterday/ progress;

Sprint Review/ 2. What will test and do today?

Retrospective 3. Any in testing progress
•As a tester, he/she can bring any backlog item that
cannot be completed in current sprint and put to next
Product
sprint – EX: I don’t have enough time to execute
Backlog
testing for item #4. it will be tested in next sprint.

22
What are Testing Activities in Scrum Process?
Process
Product
Backlog

Sprint – 2-4 weeks In Daily Work:

Sprint Planning Sprint Goal • As a tester, he/she is responsible for develop
Sprint automation scripts (for automation testing
Daily Scrum Backlog •As a tester, he/she schedules automation testing
Blocks with Continuous Integration (CI) System.
Daily Work
Product •As a tester, he/she reviews & reports test results to
stakeholders via CI System or by himself/herself.
•As a tester, he/she executes non-functional testing
Sprint Review/ with approved user stories
Retrospective
Some organizations, Product Owner does not have
enough energy to do acceptance testing each sprint.
Tester is hired for this case
Product
Backlog •As a tester, in end of Sprint, he/she executes UAT
and confirms testing completeness for current sprint.
23
What are Testing Activities in Scrum Process?
In Scrum, Software is integrated daily, build and test
each day. Therefore, CI System is deployed to most
of Agile Project. A CI System & test components can
be identified as below
• Hudson / CruiseControl – CI Framework
• Junit (for Java)/ Nunit (.Net) – Unit Test
• Cobertura (Java)/ NCoverage – Testing
Coverage
• Ant (for Java)/ Nant (.Net) – Automation Build
• SVN – Version Control
•Selenium integrated with Hudson - Functional
testing

In Daily Work, tester will responsible for develop

automation scripts, integrate testing automation
testing tools/scripts with CI for regression test and
UAT
As a part of duty, tester is playing role supporter in
helping developers create Unit Tests effectively and
efficient 24
What are Testing Activities in Scrum Process?

In Daily Work:

As a practice, “Done” of sprint can be considered when
Code checked in.
 Builds ready
 Unit Test complete successfully Tester’s
supports
 80% code branch coverage on UT
 Non-functional testing completed
(confirmed by testers)
 Finish deploying to Production

25
What are Testing Activities in Scrum Process?
Process
Product
Backlog

Sprint – 2-4 weeks In Review & Retrospective:

Sprint Planning Sprint Goal • As a tester, he/she identifies what went right and
Sprint what went wrong in current sprint
Daily Scrum Backlog •As a tester, he/she identifies lesson learned and
Blocks best practices.
Daily Work
Product •As a tester, he/she is encouraged to write user
stories that support testing.
EX: As a tester, I need interface with inputs – A, B, C
Sprint Review/ to execute testing of feature X.
Retrospective
•As a tester, he/she is encouraged to write user
stories that bring value to customer.
Product EX: As a user, I need to be able to sort items.
Backlog All these user stories are put to Product Backlog
for review and Approved
26
What are Testing Activities in Scrum Process?
In Review & Retrospective:
• Any items remaining will be
•Review why tester could NOT complete in
current sprint
Problem may be identified by lacking of
experience in estimate or any roadblock
prevented it completed. Whatever reason
identified will be noted as lesson learned
•Put to next sprint for continuing testing
• Any roadblock comes from lacking of support of
Product Owner, should be pointed out for Scrum
Master’s follow-up

To bring effectiveness and efficiency to review sessions,

Scrum Master should request participants to prepare list of
items/ issues at least 1 day prior to meeting date

27
What are Testing Activities in Kanban Process?
Kanban is less prescriptive than Scrum
All testing activities are similar with Scrum

No required time-boxed iteration

•Testing is not limited by time duration
•Accept changes at anytime – Scrum protects changing backlog items
in sprint while Kanban still accepts removing and adding new items
• Testing item in workflow state can be added or removed to get the
most effectiveness and efficiency.

Kanban limits number of items for each workflow state

•Testing items should be picked logically to put to workflow stated

28
What are Testing Activities in Kanban Process?
An Example of Managing Tasks/ Work Item
1. Backlog: state all work items/ user stories translated into
tasks - As a characteristic of Backlog may be limited
number of items
2. Ready: List of work items that ready for executing
3. Specifics: Identify what criteria to confirm work item
completeness
4. Execute: Work Item in executing

Tuan completed work item -
testing for “Sorting Items”. He
reviews and confirms that it
satisfied criteria state. He moves
this item to “Done”
Next, he moves another work
item – testing for “Filter Items”
from ready to “execute” to
start new work
He also moves another work
item from Backlog as a
pipeline item.
29
What are Testing Activities in Product Development?
Testing against Testing in live
user story environment - UAT

Non-functional
Unit Testing
testing/ confirms
Integration Test
story is met

In Agile, testing is focused on

Tester is also responsible for
1. Unit Test that developers need support of testers to testing user story that cannot be
create effective set of tests and
verified by UT to confirm testing
2. Acceptance Test that the responsibility is belonged
to tester completed before demo and
release to production.
Integration & regression test are executed daily by
automation. 30
What are Testing Activities in Product Development?
Testing against Testing in live
user story environment - UAT

Non-functional
Unit Testing
testing/ confirms
Integration Test
story is met

 Story Exploration Communication within an agile team to explore story

understanding
 Estimation
 Consider flows of between Product Owner and team on story
 Story Planning
 Testers also speak up with questions to clarify any vagueness
 Story Progression in the story
 Story Acceptance
What are Testing Activities in Product Development?
EX: For User story: “As a user, I can
reset my account password if I forgot
my password”
Test may have questions relate to this
story as follows
1. What does it mean for an email
address to be ‘unknown’?
2. What does it mean to ‘require
confirmation’ of the password?

And then s/he temporarily states their

understanding as next picture

 Story Exploration Estimation is a part of iteration where tester is using appropriate

techniques
 Estimation
Story Planning occurs after a story has been estimated and be
 Story Planning placed in iteration. Details of test tasks defined as
 Story Progression 1. Prepare test data 2. Extend acceptance test
 Story Acceptance 3. Execute manual 4. Automate CI tests
What are Testing Activities in Product Development?
 If acceptance criteria are not in
place, agile teams may collaborate
on writing acceptance tests during
story planning

 Tester can select acceptance

tests for automation. Not all
acceptance tests will be automated

Ex: for Forgot Password story, team may

have scenario as next picture for acceptance
test

 Story Exploration Story Progression: uncover additional tests that reply on

 Estimation
 Story Planning
 Story Progression
 Story Acceptance
exploratory testing and communication. Conducting testing while
monitor results from CI system
Story acceptance: “done” in test may be understood as all tests
pass or level of test automation is met
What are documents of testing in Agile Project?
Agile in common, or Scrum/ Kanban in
particular, is focused on how to deliver
As a part of process, testing in Agile has
value to customer by involving customer lightweight documentation with:
(that presented by Product Owner) in
whole process; transferring product to
live (production) environment to bring Use reusable checklists to
benefits to business soon as possible suggest tests
and expect changes to maximize

Focus on essential of test

customer business value. While Agile
reduces documentation overhead by
simplifying process and number of
artifacts created rather than the incidental details

 Capture test ideas for

Exploratory Testing
In case test-cases required, they should be
created at high level
If any, leverage one artifact for Manual and
Automation Tests

34
How to track bugs in Agile Process?
 Do or Do Not need Bug Tracking.
No bug Tracking is still working well when testers perform their tests
in Dev environment

However, in Test environment, testers need to track bugs because they

don’t see fixes for a while

With Bug Tracking, any bugs which are identified (opened) by

developers also will be regress tested by testers. Then test scripts will
be updated to cover the case

Ultimately, bugs can be prioritized with stories

Keep in mind, in Agile, testing is not concrete phase. It is a part of

development

Development Development &

Testing
Testing 35
Agile Testing
Techniques

36
What are Agile Testing Techniques? Quadrants
Automation &
Business Facing Manual
Manual Test Test
Exploratory Testing
Functional Testing
Scenario Test
Story Test
Usability Test
Prototypes Test
Support the team

Critique Product
User Acceptance Test

Unit Testing Performance & Load Testing

Unit tests combined with Performance Test

Test coverage Load Test
Security Test
Special Tool &
Automation
Automation
Test
Technology Facing Test
37
What are Agile Testing Techniques?
 As typical testing, black box testing is used.
 Agile testing is focus on using
Automation testing techniques
Test driven development, auto builds and continuous regression test
daily all reply on automation and will not work without it. And
acceptance test is carried out each Sprint that automation also has
benefits

Exploratory testing technique

This technique simultaneous learning, test design and test execution.
While the software is being tested, the tester learns things that
together with experience and creativity generates new good tests to
run.

Scenario testing technique

Use real-world scenarios that exercise multiple stories

Don’t think of Exploratory is Agile Testing – it is only one of Agile

testing approaches 38
What are Agile Testing Techniques? Exploratory
 An example of Exploratory testing
technique.
User Story
As an online shopper, I want to be able to delete
items from my shopping cart

Exploratory is kind of creativity techniques

We start with thinking of some personas or
characters and devise exploratory scenarios which
role may get into. Ex:

1. Senior citizen who never shopped on web

2. Hacker looking for troubles
3. Internet-savvy teen

For each role, behaviors will be different to have

cases of testing generated and tested

39
References

Scrum and Kaban at InfoQ

http://www.infoq.com/

Testing Experience
http://www.testingexperience.com/

Mountain Goat Software

http://www.mountaingoatsoftware.com/

 Agile Estimating and Planning – Mike Cohn

 User Stories Applied: For Agile Software Development

40
Recommended useful tool for Agile Team
- the ultimate screenshot capture software designed specifically for
testers that:
- Snap a single screenshot or record a sequence of screens
- Automatically generate intelligent step-by-step narration (defect report)
- Auto submit tickets to defect tracking and ALM systems

- calling for BETA user - Full featured Test Management System to

give testers a simple yet productive working environment. Some of the qTest
features and modules include:
- Test Planning
- Requirement Management
- Test Case Management
- Test Execution
- Defect Tracking
- Dashboards and Reports
- Built in workflows and collaboration

Get the tools as free download at http://www.qasymphony.com/

41
Thank You

Welcome your questions

Agile
Project 1
Scrum & 2
Kanban

Agile 3
Testing
Others 4

42