2011 5th Malaysian Conference in Software Engineering (MySEC)
Agile Testing with Selenium
Rosnisa Abdull Razak
Product Quality and Reliability Engineering
Mimos Berhad
Malaysia
rosnisa.razak@mimos.my
A good tool for testing is the ability to be part of
automation and its capabilities to enhance the testing
performance. This paper will explain on how Selenium tool
will accelerate the test execution process and cut the cost of
the whole test execution process. In this paper, we will
emphasize test as early as possible with less cost and the focus
should be against integration which is the main ingredients in
agile. Automated testing can be made to be cost-effective,
however, if some common sense is applied to the process.
Keywords-component: Agile Testing, functional testing,
automation script, Selenium, cost-effective.
I. INTRODUCTION – AGILE TESTING
Agile testing is a software testing practice that follows the principles of
agile software development [1]. Agile testing involves testing from the
customer perspective as early as possible, testing early and often as code
becomes available and stable enough, since working increments of the
software are released often in agile software development. This is commonly
done by using automated script to accelerate the test execution process and
cut the cost of the whole test execution process.
II. INTRODUCING SELENIUM
Selenium is set of different software tools each with a different approach to
supporting test automation. Most Selenium QA Engineers focus on the one or
two tools that most meet the needs of their project. The entire suite of tools
results in a rich set of testing functions specifically geared to the needs of
testing of web applications of all types. These operations are highly flexible,
allowing many options for locating UI elements and comparing expected test
results against actual application behavior. One of Selenium’s key features is
the support for executing one’s tests on multiple browser platforms. Selenium
is composed of multiple software tools. Each has a specific role [2].
Selenium2 (aka. Selenium Webdriver) is the future direction of the project
and the newest addition to the Selenium toolkit. This brand new automation
tool provides all sorts of awesome features, including a more cohesive and
objects oriented API as well as an answer to the limitations of the old
implementation. Selenium 2.0 is the product of that effort. It supports the
WebDriver API and underlying technology, along with the Selenium 1
technology underneath the WebDriver API for maximum flexibility in porting
your tests. In addition, Selenium 2 still runs Selenium 1’s Selenium RC
interface for backwards compatibility.
Selenium 1 (aka. Selenium RC or Remote Control) is still actively
supported (mostly in maintenance mode) and provides some features that may
not be available in Selenium 2 for a while, including support for several
languages (Java, Javascript, PRuby, HP, Python, Perl and C#) and support for
almost every browser out there.
Selenium IDE (Integrated Development Environment) is a prototyping tool
for building test scripts. It is a Firefox plugin and provides an easy-to-use
interface for developing automated tests. Selenium IDE has a recording feature,
which records user actions as they are performed and then exports them as a
reusable script in one of many programming languages that can be later
executed.
978-1-4577-1531-0/11/$26.00 ©2011 IEEE
Fairul Rizal Fahrurazi
Product Quality and Reliability Engineering
Mimos Berhad
Malaysia
fairul.fahrurazi@mimos.my
Selenium-Grid allows the Selenium RC solution to scale for large test
suites and for test suites that must be run in multiple environments.
III. PYTESTFRAMEWORK
In this section, we will focus on details how we use the framework [3] that
we developed by ourselves by combining all the selenium models and provide
the additional following features.
1. consolidates all selenium 2.0 python test case and run them using a
nosetests against a configurable parameters
o Browsers type – IE and Firefox and Chrome etc
o Selenium Hub Server
o Selenium Hub Server Port
o Base URL
o Test Report filename in Junit xml format
2. Running all test cases in parallel per browser using nosetests
multiprocessing feature. *Note there is a work in progress to fix
running test cases in parallel regardless of browser type, as of now
the implementation is - test cases will run in parallel for one type
of browser first i.e. Firefox before moving to the next browser type
or version.
3. Provide a Junit xml report.
A. PyTestFramework in details
There are 4 files available when you extracting .zip /.tar.gz code
1- config.ini
o Configurable paramaters are located here.
o Selenium 2.0 webdriver remote will be using the parameters from
the following when initializing up .
seleniumHubserver=<localhost>
port=<5555>
browser=internet explorer,firefox,chrome # You can speficy as many browser
you may want to test by field separator \“,” .
base_url="http://www.google.com" testResult=Mimos-pqre-khtp-TestReport
# Test Result Report pre-fix's name in xml format.
Figure1. Inside Config.ini
2- functionR.py
The file that stored re-usable unit test functions and acts as function library
.Pre-defined unit test setUp is available by default and being used to launch
webdriver.remote based on browser manually defined in config.ini file.
If any test operations that you feel will be heavily used by most of the test
case, it’s good to make it as a function and re-use it (i.e login operation).
3- test_google.py
This is a test case template. The test case filename should have keyword “test”
as nosetests will pick whatever file with keyword test to be run.
217
 from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException
import unittest 1-Launch Jenkins and Click at “New Job”.Choose a free-style software
#You need to import functionR.py file as at least setUp function is project option.
located there. 2- Fill in the the Job name. In this example PythonTestFramework1.0-
import function Example is specified in the field.
3- Click on OK button.
#Specify Your pre-Test Case title as part of the class

class GoogleSearchJohnLennon(unittest.TestCase):

#Dont change setUp over here.If needed to change ,do it at function.py -

setup method.

def setUp(self):
functionR.setUp(self) #This is where setUP function from function.py
is called.

# Your Selenium 2.0/webdriver Python test cases should start here

,Below is the example#

def test_google_search_john_lennon(self):
driver = self.driver
driver.get("http://www.google.com.my/")
driver.find_element_by_id("lst-ib").clear()
driver.find_element_by_id("lst-ib").send_keys("John Lennon") Figure4. Step 4 and 5
driver.find_element_by_css_selector("#imagebox_bigimages > h3.r >
a > em").click() 4-If you need to run on a specific Jenkins node here is (in "Restrict
where this project can be run") where you specify the node name/IP
5- Depending on your source code management, you need to specify you
def is_element_present(self, how, what): code’s location ,in my case I am using subversion.
try: self.driver.find_element(by=how, value=what)
except NoSuchElementException, e: return False
return True

def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
#Test Case completed#

if __name__ == "__main__":
unittest.main()
Figure2. An example of test cases

4- Runx.py
This is the runner that initiates a nosetests module to run against the available
test cases in the current folder and generate a test reports. There report will be Figure5. Step 6, 7, 8
generated against per browser type in Junit xml forma
6- I also want it to be run periodically, in this example "Build periodically" is
B. Integrate with Continous Integration server ticked and add it to run everyday once at 0005
7. At Build->Execute Shell or in windows environment you may be able to
choose Build->Execute Windows command.
In our environment, we tend to integrate with Continuous Integration tool i.e.
Jenkins so it can be set to be auto run continuously in an effort to test
integration as early as possible with less cost [4]. You need to execute the run script - Runx.py
Example:

`python Runx.py`

8-Finally, for test report, "Publish Junit test result report" is ticked and
specifies the related xml file .In my case below I want all report in .xml
format to be displayed in Jenkins. The important thing which is important in
agile is the report is accessible to everyone hence give a transparent result and
management should have a same amount of information against others.

Figure3. Main Page of Jenkins job

218
 Generate Code Library (“driver”)
Python NA
support
Generate Code Library (“driver”)
Ruby NA
support

Testing Frameworks
Selenium Selenium
Framework Selenium RC
IDE Core
Comes
Manipulate browser,
with
Bromine check assertions via NA
template to
custom driver
add to IDE
Out-of-the- Manipulate browser,
JUnit box code check assertions via NA
generation Java driver
Out-of-the- Manipulate browser,
NUnit box code check assertions via NA
generation .NET driver
Out-of-the- Manipulate browser,
RSpec (Ruby) box code check assertions via NA
Figure6. Example of Test Report generation Ruby driver
Out-of-the- Manipulate browser,
Test::Unit(Ruby) box code check assertions via NA
generation Ruby driver
Out-of-the- Manipulate browser,
IV. TESTING SUPPORTED TestNG (Java) box code check assertions via NA
generation Java driver
Browsers Out-of-the- Manipulate browser,
Browser Selenium IDE Selenium RC Selenium Core unittest (Python) box code check assertions via NA
Record & Start browser, run generation Python driver
Firefox 3 Run tests
Playback Tests tests
Record & Start browser, run
Firefox 2 Run tests V.
Playback Tests tests CONCLUSIONS
Start browser, run We have no doubts about the positive effect of automated testing. It is
IE 8 Not supported Run tests
tests clear to us that creating a good and comprehensive automated test with
Start browser, run integration of good testing tool are a demanding time-consuming process. It is
IE 7 Not supported Run tests
tests cost effective when you need to perform a certain amount of testing in limited
Start browser, run time but with automate testing tools, the time wastage can be reduce. The
Safari 3 Not supported Run tests evolution of automate tools from code-based record/playback to data-driven
tests
Start browser, run and key/action word frameworks reflects the realization that the less code there
Safari 2 Not supported Run tests is to be developed and maintained the more efficient and effective test
tests
Start browser, run automation becomes[5].
Opera 9 Not supported Run tests
tests
REFERENCES
Start browser, run
Opera 8 Not supported Run tests [1] Michael Bolton,”Rapid Software Testing”,Agile Conference ,Berlin
tests
,Germany, March ,2010.
Programming Languages [2] http://seleniumhq.org/docs/01_introducing_selenium.html
Language Selenium IDE Selenium RC Selenium Core [3] https://github.com/fairul82/PythonTestFramework1.0/wiki
Library (“driver”) [4] Titus Brown,”Agile Development and Testing in
C# Generate Code NA
support Python”,PythonConference Addison,Texas,2006.
Generate Code Library (“driver”)
Java NA [5] Titus Brown ,Grig Gheorghiu, “Practical Applications of Agile Web
support
Testing Tools”,Python Conference Chicago,2008.
Generate Code Library (“driver”)
Perl NA
support
Generate Code Library (“driver”)
PHP NA
support

219