Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Interview with
Magazine for Software Testers
Dan Ashby
www.TestingCircus.com
Coming soon
www.testing.news
www.doranjones.com!
Testing Circus
Volume 7 - Edition 4 - April 2016
Table of Contents
Topic
Author
Alan Richardson
Quadrants of Context
Jyothi Rangaiah
10
Leanne Howard
15
Communicating Context
Erik Davis
17
Srinivas Kadiyala
19
Adam Knight
27
34
Mohit Verma
38
Alexander Podelko
43
Page #
www.TestingCircus.com
April 2016
- 04 -
Feedback please!
team@testingcircus.com
www.TestingCircus.com
April 2016
- 05 -
Join the
Anti-Test Automation
Brigade
- Alan Richardson
In this article Im going to show you a very quick way to
fix all your "Test Automation" problems. This will be
especially useful to you if you have ever asked either of
the following questions:
"What test automation should we do?" and "How much
test automation should we have?".
Both of these are questions that I am asked at
conferences, when Im onsite consulting with clients
and via email. They seem to be common questions in the
testing industry. My answer to both questions has
changed over the years.
I used to find these questions hard to answer because I
wanted my answers to fit the process and approach of
the person asking the question. Inevitably I would have
to ask them follow on questions like: "Are you using an
Agile process?", "What technology are you working
with?", "Do you have programming skills?".
But I dont need to ask those clarifying questions any
more. My answers are simple. And, Ive reached the
point now, where my answers are the same:
"None. You cant."
Over the past few years my attitude to "Test
Automation" has changed. It has now changed such that
I am vehemently anti-"Test Automation".
Immediately, some people reading this will be nodding
their heads in agreement. At this point it would be
presumptuous for you to do so because I have not
explained why I now take this stance, or what I meant.
Those of you nodding may have interpreted my
statement as:
we can not automate testing
testers should not program
www.TestingCircus.com
April 2016
- 06 -
"Test" is a verb. "I will Test", "I Test". A verb that has
multiple meanings and we rightly argue about its
meaning in the test community since it seems that it
could describe the essence of what we are all about.
"Test" is also a noun. "I will review this Test", "I have
written a Test", "I will execute this Test". When people
say these statements, I dont know what they mean,
because "Test" is ambiguous. Im sure there are
standards with definitions of the term. Im also sure that
when people use the word "Test" they dont always
mean those definitions, and I know that I didnt when I
used the word.
"Test" is too ambiguous. I try to limit my usage of the
word.
Is "Automation" a verb or a noun? Is it a verb? "I will
automation", "I automation". Not really, but we can
make it part of a verb phrase - we tend to say "I will do
automation", "I am doing automation". We do use it to
imply action. We do use it as a verb.
www.TestingCircus.com
April 2016
- 07 -
--------------------Additional Reading
For other opinions on this subject you might want to
checkout the following resources. In some ways I am
expressing similar sentiments to, but approaching the
topic slightly differently from, Richard Bradshaw with
his desire to promote the phrase ["Automation in
Testing"] and James Bach and Michael Bolton in their
paper ["A Context Driven Approach to Automation in
Testing"]. I also have presentations on this on my
[conference talks page]
www.TestingCircus.com
April 2016
- 08 -
www.TestingCircus.com
April 2016
- 09 -
Quadrants
of
Context
- Jyothi Rangaiah
translates
to
being
effective
www.TestingCircus.com
April 2016
- 10 -
Diversifying
If we choose to work solo at a point in time and not
mingling so much with the team because of the reason
that we like to work solo, then start by sharing the
solution with others in the team when its ready.
Together brainstorm several solutions, then implement
one feasible solution that suits the context which we
have defined together as a team and know that this too
is mutable.
Try to be flexible when you become aware of another
element that adds on to the existing context.
Iteration 1
Take notes
picturized?
on
what/how/who/when/where
www.TestingCircus.com
April 2016
- 11 -
you
to
Whenshouldwebegintotest?
Whatandhowshouldwetest?
www.TestingCircus.com
April 2016
- 12 -
my
Shouldwetestalready?
Whoshouldbetestingthis?
Shouldwestoptestingnowbasedonacontextweas
a team have defined for this sprint?
Answer and proceed to define Agile testing for yourself.
important
in
the
Conclusion
Know this
Expandyourhorizonsandknowwhentostop
Letusbefairtoourselveswhendecisionmaking,so
we can be fair to others
Yesandremembertoagreetodisagreewhenwebuild
and demolish the boundaries when defining the context
Rememberifwedonotdefinethecontext,thereaders
construct a context of their own as they read the
requirements
Define context, there are chances of being highly
misunderstood if we do not define it
www.TestingCircus.com
April 2016
- 13 -
Applicability
Further reading
www.TestingCircus.com
April 2016
- 14 -
How Much
Documentation should
Teams Produce?
- Leanne Howard
Knowing how to create documentation is something I
am often asked about, having worked with many teams.
It is also one of the most challenging aspects of Agile
that teams struggle with, particularly coming from
traditional projects. The answer to the question of how
much documentation to create is it depends. This
article will not give you the magic formula, but
hopefully provides some guidance which can be
applied with a bit of good old common sense.
Starting at the high level, here are some dependencies to
consider regarding the amount of documentation
required:
More documentation
Newly formed teams may be required if levels
of trust have not yet been built
Compliance driven project - need to provide
high levels of traceability to individual
compliance requirement
High turnover of staff - conversations cannot be
transferred efficiently to new staff without
providing it in writing
Non collocated - sharing any information face to
face can be a challenge
Less documentation
Well established teams collaboration patterns
established
No requirement to meet external standards informal conversations are good enough
Stable teams - The IP is retained in teams heads
(although this may be a separate issue)
Collocated - All information is shared across the
team
www.TestingCircus.com
April 2016
- 15 -
April 2016
- 16 -
Communicating
Context
- Erik Davis
No, this is not another article about the importance of
context to software testing. Well not exactly. There are
plenty of those out there already and I dont need to
attempt to rehash what others have said already.
www.TestingCircus.com
April 2016
- 17 -
www.TestingCircus.com
April 2016
- 18 -
Interview with
Testers
DAN ASHBY
AstraZeneca
Software Quality and Testing Capability Lead
London
Dan is in the Senior Leadership team at AstraZeneca.
Hes been testing for over a decade, working on a wide variety of
products from printer software/hardware/firmware, to web and
mobile apps and sites of all different shapes and sizes.
Dan is passionate about context-driven testing and is currently focused
on testing web-based software while coaching/training people in
software testing and agile. Dan is involved in the testing community
and regularly speaks at conferences. He also blogs (danashby.co.uk), is
the co-host of the Testing in the Pub podcast series
(testinginthepub.co.uk) and runs he Software Testing Clinic
workshops within London (softwaretestingclinic.com).
responsibilities
engagements?
different
from
previous
I have recently moved to AstraZeneca into a GlobalHead /Senior Leadership position. AstraZeneca are
a top pharmaceutical company. Ive worked for a
pharma company in the past so its not an entirely
new domain for me, but I'm sure it will certainly
bring new challenges, new responsibilities and new
experiences.
Coaching is something I have been doing for a good
few years now. Ive worked in many domains too:
finance, medical/pharma, ecommerce, government,
energy, and on hardware, software and firmware
for a big printer manufacturer. I still like getting
hands on though, either by getting involved in
projects or through pairing as part of the coaching
I'm doing.
www.TestingCircus.com
April 2016
- 19 -
DAN ASHBY
www.TestingCircus.com
April 2016
- 20 -
DAN ASHBY
www.TestingCircus.com
April 2016
- 21 -
DAN ASHBY
www.TestingCircus.com
April 2016
- 22 -
DAN ASHBY
www.TestingCircus.com
April 2016
- 23 -
DAN ASHBY
www.TestingCircus.com
April 2016
- 24 -
DAN ASHBY
www.TestingCircus.com
April 2016
- 25 -
https://www.testingcircus.com/advertise
Preparation over
Planning
An Agile Test Strategy
- Adam Knight
When I started out in software testing, test plans and
strategy documents played a central role in the testing
process. On larger projects the test plan and test strategy
were separate pieces targeted at different levels, but on
many of the smaller projects and teams that I worked on
these terms were ubiquitous and related to a single
document. Despite the bad press that such documents
may now receive, at the time the creation of a test
strategy document wasnt seen as a chore, or a necessary
evil. The creation of a test strategy was in fact a highly
prized responsibility for those working in software
testing. Creating the test strategy was the preserve of the
test team lead or test manager, a mark of status on a
testing project. Due to the long duration of many
development projects, the opportunity to create a test
strategy did not often arise, yet it was the very length of
these projects that really inspired the desire to create
them. In reality it was often the case that the strategy
was not referred to once the development had
commenced. The strategy often contained a lot of
standard boilerplate content and it was common for
the actual testing that was done often to deviate
significantly from that which was defined in it.
Inevitably any detailed plan was rarely maintained with
the decisions that were made as the project progressed
and the need for such decisions emerged. All of these
eventualities undermined the validity of the creation of
an up-front strategy, however the fact remained that a
significant commitment of testing time and resources
was about to be made to a long term project, and the
creation of a strategy provided the people making that
commitment with a sense of confidence in
understanding how those resources were to be used.
If we skip forward to today, much of the software
development landscape looks very different. Many
organisations have, in some form or another, adopted
www.TestingCircus.com
April 2016
- 27 -
items they are working on, such as user story, and also
the higher level features and products that this is part of.
For me this is a very real risk in Agile teams. The narrow
focus on the testing of thinly scoped, low-level user
stories can lead to emergent, performance or
architectural risks being overlooked simply because
they werent obviously part of a specific user story.
Whereas these activities might historically have been
pre-defined as hierarchical elements of the testing
strategy or plan, in Agile development a more
appropriate approach would be to consider these as
elements of a testing mission at the relevant level. The
teams and individuals involved in the testing may not
know specifically which products or features they will
be testing in advance, instead the parameters of this
mission will emerge and evolve as the product develops.
In exploratory testing approaches, the idea of a testing
charter is a commonly used concept to scope a testing
activity which is not-predetermined but has clear scope
and focus. In her excellent writing on Exploratory
Testing3, Elizabeth Hendrickson describes the
characteristics of an exploratory charter as fitting the
following pattern:
Explore an Area
With Resources
to discover Some Information
of three stone-cutters2:
An old story tells of three stonecutters who were
asked what they were doing. The first replied, I
am making a living. The second kept on
hammering while he said, I am doing the best job
of stonecutting in the entire country. The third
one looked up with a visionary gleam in his eyes
and said, I am building a cathedral.
In order to define an appropriate testing
approach those responsible for testing need
to understand at any point in time what their
responsibilities are towards testing both the low level
www.TestingCircus.com
April 2016
- 28 -
www.TestingCircus.com
April 2016
- 29 -
www.TestingCircus.com
April 2016
- 30 -
www.TestingCircus.com
April 2016
- 31 -
References
1. 2016 State of Testing Survey
https://www.practitest.com/resources/state-ot-testing2016/?utmsource=colaborators&utmmedium=sharing&
utm_campaign=state%20of%20testing%202016
https://staroversky.com/blog/the-parable-of-the-threestonecutters
3. Exploratory Testing in an Agile Context, Hendrickson
http://www.qualitytree.com/ebooks/et.pdf
4. T-Shaped Tester, Square-Shaped team - Adam Knight
http://thesocialtester.co.uk/t-shaped-tester-squareshaped-team/
www.TestingCircus.com
April 2016
- 32 -
www.99tests.com
#TestRelatedAccounts
EuroSTAR Conferences
Official Twitter of EuroSTAR Software Testing Community.
Europe's #1 software testing network.
https://twitter.com/esconfs
SoftwareTestPro
Serving the global software testing & quality assurance community, providing information, education & professional networking.
https://twitter.com/SoftwareTestPro
SoftwareTestingClub
A professional software testing community. Come join us!
https://twitter.com/testingclub
Let'sTest Conference
A Kick-Ass Conference on Context-Driven Testing - for Testers, by
Testers
https://twitter.com/LetsTest_Conf
https://Twitter.com/TestingCircus
www.TestingCircus.com
April 2016
- 34 -
s
ster
e
t
e
wa r
so f t
fo r
ine ting
z
a
ag
tes
e m 010. #
g
a
2
er
ngu
h la ptemb
s
i
l
Eng nce Se
us
ng
i
c
si
d
r
s
a
i
n
e
s l
itio
ngC
i
orld thly ed
t
s
w
a
/Te
s is s. Mon
u
m
c
t
r
o
Ci
sias
er.c
t
ting enthu
s
t
e
i
T
test
://tw
s
and
p
t
ht
s at
u
low
F ol
g
n
i
Test
s
u
c
Cir
O
s
U
w
lo
l
o
#F
r
e
t
t
wi
T
n
r.co
e
t
it
w
T
://
s
p
htt
us
c
r
Ci
g
n
sti
e
T
m/
www.TestingCircus.com
April 2016
- 35 -
wall too.
http://www.facebook.com/TestingCircus
Tutorial 4
Automation with
Selenium
- Learn Selenium with Mohit Verma
Automating Radio Buttons using Selenium Webdriver
In last tutorial we learned how to automate the dropdowns using Selenium Webdriver. In continuation to automate Web UI, this tutorial will explain how to handle the radio buttons in a web application.
For practice purpose, we have come up with this simple webpage having 3 radio buttons in it. This page displays
as follows:
Following code can be used to create the webpage shown as above and save it as RadioButton.html.
<!DOCTYPE html>
<html>
<body>
<form>
<input type="radio" name="gender" value="Male" checked="Checked"> Male<br>
<input type="radio" name="gender" value="Female"> Female<br>
<input type="radio" name="gender" value="Other"> Other
</form>
</body>
</html>
Scenario to be automated:
1. Launch Firefox browser and Open RadioButton.html page
www.TestingCircus.com
April 2016
- 38 -
Code Snippet:
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
public class RadioBtn1 {
/**
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
WebDriver driver = new FirefoxDriver();
String url = "C:\\Users\\Mohit\\Desktop\\Selenium\\Article 4\\RadioButton.html";
//Open RadioButton Html page
driver.get(url);
//Locate and select "Female" radio button
driver.findElement(By.xpath("html/body/form/input[2]")).click();
}
}
www.TestingCircus.com
April 2016
- 39 -
Selecting the Radio Button by Value: Imagine a scenario, where the Xpath for radio button is changing with
every new build and you cant use it for automation. In such case, you would like to locate the radio button with
an attribute with consistent value such as value of radio button. In the following code snippet, we would select
the Female radio button by value.
Code Snippet:
import java.util.List;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
In few scenarios, you might need to find the text of selected radio button. The following scenario prints the value
of selected option
Scenario to be automated:
1. Launch the Firefox browser and open RadioButton page
2. Select the Other option
3. Print the value of selected option
4. Close the browser
To automate the above scenario we would need to perform following actions:
www.TestingCircus.com
April 2016
- 40 -
www.TestingCircus.com
April 2016
- 41 -
Summary: Radio buttons are commonly used in the applications and they are simple to automate. This tutorial
explained how radio buttons can be automated using Selenium Webdriver. We have covered most common scenarios associated with usage of radio buttons, the readers shall practice to automate the other scenarios.
www.TestingCircus.com
April 2016
- 42 -
Is Performance
Testing just a
Testing?
Performance testing is an interesting area based on two
larger and rather independent disciplines: performance
analysis and testing. The statement sounds rather trivial
and it can be easily induced from the name - but actually
has important consequences that are often not well
understood and sometimes lead to serious issues if not
factored in. Performance component requires such a
different set of skills and knowledge, maybe even
somewhat different way of thinking, so it makes rather
misleading to approach performance testing just as a
variation of testing.
Let us first define terminology as it is rather vague.
There are different (and sometimes conflicting)
definitions of almost every term here. I'd suggest to
consider every testing involving measurement of
performance in any way to be performance testing. And
every testing requiring application of a multi-user,
synthetic load to be load testing. Many different names
may be used for multi-user testing (such as concurrency,
stress, scalability, endurance, longevity, soak, stability,
or reliability testing) depending on test goals and
design, but in most cases they use the same approach:
applying multi-user, synthetic workloads to the system.
So the term "load testing" is used here for all multi-user
performance testing, which is the main subject of this
article and differs most from functional testing
(comparing, for example, with single-user performance
testing).
This article discusses load testing and highlights points
that are often missed by people moving into
performance testing from functional testing or
development. Applying the best practices and metrics of
functional testing to load testing quite often results in
disappointments, unrealistic expectations, sub-optimal
test planning and design, and misleading results.
I got interested in the topic attending the first Workshop
on Performance and Reliability (WOPR) back in 2003
and was elaborating it since using periodic
opportunities to discuss the subject with experienced
- Alexander Podelko
www.TestingCircus.com
April 2016
- 43 -
www.TestingCircus.com
April 2016
- 44 -
users are running the same queries, but one per hour,
the throughput is 500 queries per hour. So with the same
500 users we have a 60-fold difference between loads
and, probably, at least 60-fold difference in hardware
needed.
The intensity of load can be controlled by adding delays
(often referred as think time) between actions in
scripts or harness code. So one approach is to start with
the total throughput the system should handle, then
find the number of concurrent users, get the number of
transactions per user for the test, and then try to set
think times to ensure the proper number of transactions
per user.
Finding the number of concurrent users for a new
system can be tricky too. Usually information about real
usage of similar systems can help to make an initial
estimate. Another approach may be to assume what
share of named (registered in the system) users are
active (logged on). So if that share is 10%, 1,000 named
users results in 100 active users. These numbers, of
course, depend greatly on the nature of the system.
Workload Implementation
If we work with a new system and have never run a load
test against it before, the first question is how to create
load. Are we going to generate it manually, use a load
testing tool, or create a test harness?
Manual testing could sometimes work if we want to
simulate a small number of users. However, even if it is
well organized, manual testing will introduce some
variation in each test, making the test difficult to
reproduce. Workload implementation using a tool
(software or hardware) is quite straightforward when
the system has a pure HTML interface, but even if there
is an applet on the client side, it may become a serious
research task, not to mention dealing with proprietary
protocols. Creating a test harness requires more
knowledge about the system (for example, an API) and
some programming skills. Each choice requires different
skills, resources, and investments. Therefore, when
starting a new load-testing project, the first thing to do
is to decide how the workload will be implemented and
to check that this way will really work. After we decide
how to create the workload, we need to find a way to
verify that the workload is really being applied.
Workload Verification
Unfortunately, an absence of error messages during a
load test does not mean that the system works correctly.
www.TestingCircus.com
April 2016
- 45 -
www.TestingCircus.com
April 2016
- 46 -
www.TestingCircus.com
April 2016
- 47 -
www.TestingCircus.com
April 2016
- 48 -
e
b
i
r
c
s
b
u
To S
e
r
e
H
k
c
i
l
C
Testing Circus
www.testingcircus.com
Still relying on
reading
Testing Circus
from tweets
& facebook
updates?
Subscribe
with your
email id and
get the
magazine
delivered to
your email
every month,
free!
www.TalentPlusPlus.com
www.TalentPlusPlus.com