A Case Study on Web Application Security Testing

with Tools and Manual Testing

LaShanda Dukes, Xiaohong Yuan, Francis Akowuah
Department of Computer Science
North Carolina Agricultural & Technical State University
Greensboro, United States
ldukes@aggies.ncat.edu, xhyuan@ncat.edu, fakowua@aggies.ncat.edu



AbstractWeb application security has become a big issue
because of common vulnerabilities found in web applications.
This paper illustrates a case study on conducting security testing
on an example application, Tunestore. The example application
was tested using a number of tools such as Paros, WebScarab,
JBroFuzz, Acunetix, and Fortify. Manual testing was also
conducted. The testing results of different tools and manual
testing are compared and discussed. Our case study shows
manual testing is very important since some vulnerability types
can only be found through manual testing and testers
observations, and it is important to utilize a variety of tools as
well as conduct careful manual testing in order to find the most
number of vulnerabilities in a web application. Based on this case
study, hands-on labs can be developed for teaching web security,
software security testing, and other topics.
I. INTRODUCTION
Web applications today are highly functional, and rely
upon a two-way flow of information between the server and
browser. Security becomes a big issue because no one wants to
use a web application if they believe their information will be
disclosed to unauthorized parties [1]. Vulnerabilities
commonly found in web applications include injection, cross-
site scripting, cross-site request forgery, security
misconfiguration, broken authentication and session
management, and more.
Penetration testing and static code analysis may be used to
assess the vulnerabilities of web applications. Penetration
testing is a method of security testing through the simulation of
an attack. Static code analysis, also known as source code
analysis is a code review process that examines the softwares
source code for common coding errors and defects without
execution [3].
This case study uses an example web application,
Tunestore to conduct security testing. It illustrates web
application security testing using tools and manual testing. The
tools used to conduct testing on Tunestore are Paros,
WebScarab, JBroFuzz, Fortify, and Acunetix. Our case study
shows manual testing is very important since some
vulnerability types can only be found through manual testing
and testers observations, and it is important to utilize a variety
of tools as well as conduct careful manual testing in order to
find the most number of vulnerabilities in a web application.
Based on this case study, hands-on labs can be developed to
teach about web application vulnerabilities, software security
testing, and other topics.
This paper is organized as follows: Section II discusses
some common web application vulnerabilities. Section III
describes the tools we used for web application security
testing, Section IV gives a description of the example
application used to conduct security testing, and Section V and
VI discuss the results from testing the example application
using tools as well as through manual testing. Section VII
provides further discussion on the testing results and Section
VIII concludes the paper.
II. COMMON WEB APPLICATION VULNERABILITIES
This section discusses common vulnerabilities of web
applications. They include SQL injection, cross-site scripting,
cross-site request forgery, broken authentication and session
management. These vulnerabilities were listed in OWASPs
Top 10 Project as among the Top 10 critical web application
security risks [4]. They also appear significantly in the Web
Hacking Incident Database (WHID) [5]. A brief discussion is
provided here. A more detailed discussion can be found in [6].
SQL injection is a vulnerability type where an application
accepts malicious scripts that are later parsed and executed.
The fundamental way of attack involves the direct insertion of
code into user-input variables that are concatenated with SQL
commands and executed. A rare and indirect way of attack is
where the malicious code is injected into strings that are
destined for storage in a table or as metadata. Subsequently
when the stored string is concatenated into a dynamic SQL
command the malicious code is executed. An attacker can
anonymously read and modify data contained in a database as
well as take full control of the server by exploiting the SQL
vulnerability.
An application is vulnerable to cross-site scripting (XSS)
vulnerabilities when it takes untrusted data and sends it to a
web browser without proper validation and escaping. This
allows attackers to embed malicious JavaScript, VBScript,
ActiveX, HTML or Flash into a dynamic page to trick the user,
executing the script on his machine in order to gather data,
hijack sessions, redirect users to a malicious site or deface the
website. Furthermore, a successful attack may result in the
compromising of private information, manipulation or theft of
cookies, creation of requests that can be mistaken for those of a
This work is partially supported by Department of Education under grant
P120A090049 and NSF under grant HRD-1137516. Any opinions, findings,
and conclusions or recommendations expressed in this material are those of
the author(s) and do not necessarily reflect the views of the National Science
Foundation and Department of Education.































978-1-4799-0053-4/13/$31.00 2013 IEEE
valid user. It might also cause the execution of malicious code
on the end user systems. Three types of XSS exist. The first
one is Reflected XSS, in which dynamic web pages take text
submitted by a client or user and simply renders this text back
to user within its response. The second type is Stored XSS. It
occurs where some malicious user-submitted data is stored in a
database to be used in the creation of pages that will be served
to other users later. Thus, visitors of the web page fall victim
for this attack. Lastly, Local XSS exploit targets vulnerabilities
that exist within the webpage code itself. Usually, this happens
when Document Object Model (DOM) is wrongly used in
JavaScript. As a result, opening another web page with
malicious JavaScript code in it at the same time might actually
alter the code in the first page on the local system.
Broken authentication and session management is the
vulnerability whereby the application does not implement its
authentication and session management functions correctly.
The attacker uses leaks or flaws such as exposed accounts,
session IDs and passwords in the authentication or session
management function to impersonate users. When successful,
an attacker is able to do anything the victim could do. The
situation worsens when the administrator is the victim.
Cross-site request forgery usually occurs where HTTP
cookies are primarily used to transmit session tokens. Where
precaution is not taken against the misuse of the token, an
attacker can trick an authenticated user to perform actions of
his (attacker) choosing. Victims are forced to change data and
perform functions they are allowed or authorized to do
Insecure direct object reference is a common web
application vulnerability regarding access control. This occurs
when an application wrongfully exposes a reference to an
internal implementation object. An attacker, usually an
authorized user, exploits this vulnerability by simply changing
a parameter value that references a system object. This enables
him to access unauthorized objects.
III. TOOLS USED IN THE CASE STUDY
This section briefly describes the tools used in this case study
which are: Paros, WebScarab, JBroFuzz, Acunetix and
Fortify.
A. Paros
Paros [7] is a free web application vulnerability assessment
tool written in Java by ProofSecure.com. It can be used as a
web spider, vulnerability scanner, proxy, and a fuzzer. The web
spider function is used to discover the content of a website by
parsing a webpage for links to other content, requesting these
pages, and continuing this process recursively. The scanner
function scans the web application to identify common
vulnerabilities such as cross-site scripting, SQL injection,
forms with autocomplete enabled, old versions of files, etc.
Paros can also trap HTTP and HTTPS requests and responses
so they can be modified manually [1, 8].
B. WebScarab
WebScarab [9] is an open-source web application security
testing tool written in Java. The tool is a part of the Open Web
Application Security Project (OWASP) [10]. It can be used for
analyzing applications that communicate with HTTP and
HTTPS protocols. WebScarab operates as an intercepting
proxy, in which the user is allowed to review and modify
intercepted requests that are created by the browser and
responses from the server. Similar to Paros, it can be used as a
web spider to discover the content of a website. The fuzzer
feature of WebScarab allows a user to test a web page by
taking a list of test strings and trying them in the input fields
automatically [11].
C. JBroFuzz
JBroFuzz is a web application fuzzer written in Java and is
a part of the OWASP Project. The fuzzer is for requests made
over HTTP and HTTPS protocols [12]. The fuzzing feature is
used to create a HTTP request to send to the web application.
A fuzzer can be added to a parameter field to add attack
payloads. The payload feature has attack strings of several
vulnerability categories such as Injection, XSS, Integer
Overflow, XPATH Injection, SQL Injection, and more. HTTP
responses are displayed in an output window showing the
payload and the HTTP response status code. The responses of
the web application to the attack strings are also shown. The
graphing feature can generate different types of graphs to
visualize the fuzzing results. There are six types of
graphs/charts: Status Code Pie Chart, Response Time Bar
Chart, Response Size Bar Chart, Jaccard Index, Hamming
Distance, and Response Header.
D. Acunetix
Acunetix is a product of the Acunetix Company [13]. It is a
web vulnerability scanner created to replicate hackers
methodologies to find vulnerabilities in web applications.
Acunetix crawls through the websites to analyze the content
including flash content, SOAP and AJAX. The scanner
includes features such as advanced and in-depth SQL injection
and cross-site scripting testing, penetration tools which
include HTTP Editor and HTTP Fuzzer, and others. The cost
of the tool ranges from $1,000 - $12,995. It has a free trial
version that can only scan for cross-site scripting
vulnerabilities in a web application. In this case study, the free
trial version was used to test the example web application.
E. Fortify
Fortify is a static analysis tool used to find root causes of
security vulnerabilities in source code. Fortify supports a
range of languages, platforms, build environments and
software component APIs. Fortify can detect more than 480
types of software security vulnerabilities among 20 different
languages. The tool displays results sorted by severity of risk
and guidance on how to mitigate the risks [14]. The
demonstration version of the software that comes with the
book Software Security: Building Security In [3] only scans
for SQL injection, buffer overflow, system information leak,
and HTTP response splitting vulnerabilities. In this case study,
the demonstration version of Fortify was used to test the
example web application.
Table 1 summarizes the automated tools used for the case
study.
IV. THE EXAMPLE APPLICATION
The application used to conduct security testing is the
Tunestore web application developed by Dr. Bei-Tseng Bill
Chus project team from the University of North Carolina at
Charlotte. The Tunestore web application is a .war file installed
on a server on an Ubuntu Virtual Machine to conduct security
testing. Tunestore is an online retail store of songs. A user has
to create an account to gain access to the store and its database
of songs, although users may view the list of songs without
being a registered user. Registered users are able to select
several options within the store: add balance, friends,
profile, CDs, and log out. The add balance option
allows the user to add money to their account by using a credit
card to purchase songs. The friends option allows the user to
add and view friends, as well as accept friend requests. The
profile option provides the user with their username and their
current account balance. The user may also change their
password under the profile option. The CDs option
displays a list of songs the user can buy, send as a gift, or make
a comment under the CD. This option also shows the users
purchased CDs. The log out option logs the user out of their
profile and redirects them to the homepage of the store.
V. SECURITY TESTING WITH TOOLS
The Tunestore web application was tested using the five
tools described in Section III. This section describes the
testing results of using these tools.
A. Testing Results with Paros
Using the scanner feature of Paros, 37 potential
vulnerabilities were discovered in the Tunestore web
application. These vulnerabilities are listed below. The
number following the vulnerability refers to the number of
occurrences of the vulnerability.

Obsolete File (7): It indicates that miscellaneous
items including files, backup, unused, or obsolete
files exist. If these files contain program source,
information such as server logic or ODBC/JDBC
user IDs and passwords may be revealed since these
file extensions may not be processed by the web
server.
Obsolete File Extended Check (7): It also indicates
that miscellaneous items including files, backup,
unused, or obsolete files exist.
Password Autocomplete in Browser (7): This
vulnerability means the AUTOCOMPLETE attribute
is not disabled in the HTML FORM/INPUT element
containing password type input. Passwords may be
stored in browsers and retrieved.
Session ID in URL rewrite (7): URL rewrite is used
to track user session IDs. The session ID may be
disclosed in the referer header. Besides, the session
ID can be stored in the browsers history or server
logs.
SQL Injection (4): SQL injection is possible. User
parameters submitted will be formulated into a SQL
query for database processing. If the query is built by
simple string concatenation, it is possible to modify
the meaning of the query by carefully crafting the
parameter.
SQL Injection Fingerprinting (4): SQL injection may
be possible.
Tool Functionalities Cost Operating System
Paros [7] Web proxy
Web spider
Automated vulnerability scanner
Manual request with proxy, fuzzer
Free Linux, Apple Mac OS X, Microsoft Windows
WebScarab [8] Web proxy
Web spider
Manual fuzzer
Manual vulnerability scanner
Manual request with web proxy, spider,
fuzzer, history
Free Linux, Apple Mac OS X, Microsoft Windows
JBroFuzz [11] Automated fuzzing
Graphing
Built-in attack payloads
Free Linux, Apple Mac OS X, Microsoft Windows
Acunetix [12] Automatic client script analyzer
SQL Injection and Cross-site scripting
testing
Fast scanner crawls
Port scans on web servers
Commercial
$1,000 -
$12,995
Microsoft Windows, Microsoft Windows
Server, Microsoft Windows 2008 Server
Fortify [13] Automated vulnerability scanner
Source code analyzer
Commercial Linux, Microsoft Windows, Mac OS, Oracle
Solaris, HP-UX
Table 1. Automated Tools used in the Case Study
Cross-Site Scripting (1): Malicious scripts may be
injected into the browser which can appear to be
genuine content from the original site. These scripts
can be used to execute arbitrary code or steal
customers sensitive information such as user
passwords or cookies.
B. Testing Results with WebScarab
Using the fuzzer and the proxy features of WebScarab, we
discovered potential SQL injection and cross-site scripting
vulnerabilities:
SQL Injection: An input attack file was created
which included SQL injection attack strings. This
attack file was submitted to the WebScarab fuzzer
to test the login page of the Tunestore web
application. Some of the attacks were successful.
Reflected Cross-Site Scripting: Using
WebScarabs fuzzer feature, an input attack file
was created which included cross-site scripting
attack strings to test the login page of the
Tunestore web application. The scripts were
executed showing that the login page has XSS
vulnerabilities.
Stored Cross-Site Scripting: Using WebScarabs
proxy feature, comments submitted by the user
was intercepted and changed to malicious scripts.
The malicious scripts were then saved on the
server, and would be activated when someone
views the comment on the CD.
Other input fields such as those related to the registration,
friend, and add balance options could also be tested using
WebScarab. However, in this case study, those fields were
tested using the manual approach instead of WebScarabs
fuzzer feature.
C. Testing Results with Fortify (the Demonstration Version)
Using Fortify Source Code Analyzer, 30 potential
vulnerabilities were discovered. These vulnerabilities are
listed below. The number following the vulnerability refers to
the number of occurrences of the vulnerability.
SQL Injection (27).
System Information Leak (2): It means system
data or debugging information is revealed, which
can help attackers learn about the system and plan
attacks.
HTTP Response Splitting (1): It means
unvalidated data in HTTP response headers can
enable cache-poisoning, cross-site scripting,
cross-user defacement or hijacking attacks.
D. Testing Results with JBroFuzz and Acunetix
JBroFuzz was first used to test for SQL injection and XSS
vulnerabilities. However, a bug in the tool was discovered. For
some valid SQL injection attack strings, JBroFuzz generated
invalid requests to the web application, and thus could not
identify the SQL injection vulnerability. Because of this bug,
JBroFuzz was no longer used to test for other vulnerabilities in
the Tunestore web application. However, the payloads
embedded in JBroFuzz were used in manual testing to
discover some of the vulnerabilities in Tunestore.
Using Acunetix (free edition), cross-site scripting
vulnerabilities were discovered.
VI. MANUAL TESTING
Using manual testing, authentication, access control,
injection, cross-site scripting, cross-site request forgery, and
integer overflow vulnerabilities were discovered. Some of the
vulnerabilities confirm those found through automated tools.
Others were not reported by the tools.
The vulnerabilities found through manual testing which
confirm those reported by the automated tools are the
following:
SQL Injection This allows illegitimate users to
login as a legitimate user.
Reflected Cross-Site Scripting Malicious scripts
may be injected into input fields of the login page
and be executed.
Stored Cross-Site Scripting Using WebScarabs
proxy feature, a malicious script can be stored in
the comment field.
Using manual testing, we were able to confirm some of the
vulnerabilities reported by some of the tool. Acunetix indicated
cross-site scripting vulnerabilities in the login page of the
Tunestore web application. Although, the tool reported true
positives, manual testing also revealed cross-site scripting
vulnerabilities on the comment page. Malicious scripts were
stored into the comment field and stored in the database. This
indicates a typical case of stored cross-site scripting (XSS)
vulnerability.
The vulnerabilities found through manual testing which are
not reported by the automated tools are the following:
The application allows users to register with weak
passwords. It does not have a minimum or
maximum length for passwords. It does not give
users feedback on password strength. This
vulnerability can cause an attacker to carry out
brute force attacks and enumerate existing users
on the system.
The application does not have a lockout policy for
failed login attempts. This makes the application
have a high risk of brute force attacks.
Unencrypted credentials are transmitted over
HTTP protocol.
Forgot password feature was not implemented.
Registered users will not be able to access the
system if they do not remember their passwords.
The application implements a stay logged in
feature. This makes the application have a high
risk of session hijacking.
Insecure storage of credentials. Username and
passwords stored in the database are not
encrypted. As a result, an attacker can eavesdrop
on the network and gain access users
unauthorized information. Thus, an attacker is
able to gain access to the database to view users
data.
Horizontal privilege escalation The application
allows users to view friends CDs and view
comments on CDs without logging into the
application.
Cross-Site Request Forgery Malicious scripts
may be injected into the comment field of
Tunestore which includes a crafted link. When a
victim clicks on the link, it causes some actions to
occur, such as logging the victim out, etc.
Integer Overflow The submission of large
numbered values will overflow the balance field
on the account. The add balance option will no
longer operate correctly.
An attacker can add an amount to the add
balance feature in their profile by using SQL
injection techniques. Hence, an attacker can
successfully create a new user account with a
substantial amount of money added to their
account without providing valid credit card
information. As a result, the attacker is able to
purchase CDs for free. This vulnerability was
discovered through examining the source code.
Insecure Direct Object Reference. Requests for
protected resources, in this case the music files,
are made directly to the static resources, which
are located within the web root of the server. The
music filenames are predictable. The last names
of the artists are used. Changing the values in the
download link permits a user to download any
song from Tunestore without purchasing them.
VII. DISCUSSION
Our case study shows that Tunestore is a good example
application for teaching students about web security. It can be
used to demonstrate various web application vulnerabilities to
the students. Limited by our academic environment, we do not
have the full versions of Fortify and Acunetix. Therefore the
testing results from the demonstration or the trial version of
the tools are very limited in this study. Universities can apply
for the Hewlett-Packard Company Educational Software
License Agreement to receive the full version of Fortify for
free for educational purposes. However, we could not get the
full version of Fortify because our university did not agree
with the user agreement for the license for the Fortify
software.
The web spidering and proxy features of Paros and
WebScarab are very useful for web application testing. The
web spidering functions allow testers to understand the
content and structure of the web application. The proxy allows
testers to intercept and modify requests and responses
transmitted between the client and server to conduct attacks.
The scanner feature of Paros is quite limited. It does report
some main vulnerabilities such as SQL injection, cross-site
scripting, and session ID in URL rewrite, etc. WebScarabs
fuzzer feature can be very useful because a large number of
attack strings can be submitted to test the application
automatically. Some of the manual testing conducted in this
case study, such as testing for buffer overflow, integer
overflow vulnerabilities, could have been done using
WebScarabs fuzzer feature. JBroFuzz does not have good
documentation on how to use the tool. From our use of the
tool, we discovered a bug in the tool, and did not think the tool
worked properly as it is supposed to. However, the attack
types and payloads embedded in the tool can be useful. The
payloads can be used for manual testing, or testing with
WebScarabs fuzzer feature.
Manual testing discovered most of the vulnerabilities
especially the design flaws. Some of them are the same as
those found using Paros, Fortify, Acunetix, for example, SQL
injection, and cross-site scripting vulnerabilities. Most of the
authentication and access control vulnerabilities can only be
found through manual testing based on testers observations.
Therefore it is important to utilize a variety of tools as well as
conduct careful manual testing in order to find the most
number of vulnerabilities in a web application.
VIII. CONCLUSION
This paper describes a case study of testing for security
vulnerabilities of an example application Tunestore. It
provides an overview of web application security testing tools
such as Paros, WebScarab, JBroFuzz, Acunetix, and Fortify. It
presents the security vulnerabilities found within an
application by using these tools, and by manual testing. While
commercial tools are not easily available in an academic
environment, open source tools can be used to conduct
security testing with limited results. Our case study shows
manual testing is very important since some vulnerability
types can only be found through manual testing and testers
observations. It is important to utilize a variety of tools as well
as conduct careful manual testing in order to find the most
number of vulnerabilities in a web application. Based on this
case study, hands-on labs can be developed for teaching web
security, software security testing, and other topics.
REFERENCES
[1] Studdard, D., & Pinto, M. The Web Application Hackers
Handbook: Discovering and Exploiting Security Flaws,
Wiley Publishing: Indianapolis, Indiana, 2008.
[2] The Open Web Application Security Project, Testing:
Introduction and Objectives, 2012
https://www.owasp.org/index.php/Testing:_Introduction_
and_objectives, retrieved on November 6, 2012.
[3] McGraw, G. Software Security: Building Security In.
Pearson Education: Boston, Massachusetts, 2006.
[4] The Open Web Application Security Project, OWASP,
Top Ten 2010 Main, 2010,
www.owasp.org/index.php/Top_10_2010-Main, retrieved
on January 13, 2013.
[5] Web-Hacking-Incident-Database. Retrieved on Jan 6,
2013
http://projects.webappsec.org/w/page/13246995/Web-
Hacking-Incident-Database
[6] Top 10 2010. Retrieved on January 10, 2013:
https://www.owasp.org/index.php/Top_10_2010-Main
[7] Paros, Paros for web application security assessment,
2004, http://www.parosproxy.org/, retrieved on
November 6, 2012.
[8] Paros, User Guide for Paros v2.x, 2003,
http://www.parosproxy.org/paros_user_guide.pdf,
retrieved on November 6, 2012.
[9] The Open Web Application Security Project, Category:
OWASP WebScarab Project, 2012,
https://www.owasp.org/index.php/Category:OWASP_We
bScarab_Project, retrieved on November 6, 2012.
[10] The Open Web Application Security Project, OWASP,
2012, www.owasp.org, retrieved on November 6, 2012.
[11] Dustin, E., Nelson, L., Wysopal, C., Zovi, D. The Art of
Software Security Testing: Identifying Software Security
Flaws. Symantec Corporation: Upper Saddle River, New
Jersey, 2007.
[12] The Open Web Application Security Project, JBroFuzz,
2012, https://www.owasp.org/index.php/JBroFuzz,
retrieved on November 6, 2012.
[13] Acunetix Ltd., Acunetix Web Application Security,
www.acunetix.com/vulnerability-scanner/, retrieved on
November 6, 2012.
[14] HP Enterprise Security, HP Fortify Static Code Analyzer,
2012, http://www.hpenterprisesecurity.com/products/hp-
fortify-software-security-center/hp-fortify-static-code-
analyzer/, retrieved on November 6, 2012.