Web Application Scanning

Quick Requests

• Phones on mute
• Exam

2 Qualys, Inc. Corporate Presentation

 Agenda

§ WAS Overview (LAB 1)

§ Basic Web Application Setup and Discovery (LAB 2)
§ Advanced Web Application Setup and Scanning (LAB 3)
§ WAS Reporting (LAB 4)
§ Tagging and Users (LABS 5 & 6)
§ 3rd Party Integration
§ MD Integration (LAB 7)

3 Qualys, Inc. Corporate Presentation

qualys.com/learning

• LAB Exercises
• Presentation Slides
• Selenium Scripts
• BURP Results
• Certification Exam
• Click “Forgot your password?”
link to reset your password
Qualys Student Trial Account

• The Learning Management System (LMS) will not send trial accounts to
public email domains (e.g., gmail.com, yahoo.com, hotmail.com, etc...).
• Check your email inbox (and SPAM folder) for your student trial account
credentials.
• The WAS lab exercise document provides instructions for activating and
setting up your student trial account.
 Web Application Scanning Overview

6 Qualys, Inc. Corporate Presentation

 WAS Overview

Automated Testing (Fault Injection)

• Submit “specially crafted” characters
• Observe the server’s response
• This represents 80 – 85% of Web app vulnerabilities

Manual Testing (BURP Integration)

• Automated tools effectively detect Web application bugs (SQL
execution inside user input)

• Human beings are much better at discovering program design flaws

7 Qualys, Inc. Corporate Presentation

 What Do Automated Tools Miss?

Logic Errors: Point of authentication vs. point of

authorization
• Forced Browsing Links - user forces access to
unauthorized link.
Permission Errors: File system permissions have a
significant impact on application security.
• Public file share that has employee payroll and medical
records.
These typically require manual testing and detection.
8 Qualys, Inc. Corporate Presentation
Qualys WAS Lifecycle

1. Define the
Application

4. Report 2. Discovery
Scan

3.
Vulnerability
Scan
Qualys Cloud Platform
IaaS Providers

Web App

QUALYS
Internal Scanner
PLATFORM
ü Strong Data
Encryption
ü Firewalls
ü IDS
ü TLS communications

Internal External Scanner Pool

Web App

External
Web App

Qualys User

Corporate Environment
 Qualys Browser Recorder and Chrome

During this course you will need to install the Qualys

Browser Recorder extension for Chrome.

11 Qualys, Inc. Corporate Presentation

 KnowledgeBase and Search Lists

12 Qualys, Inc. Corporate Presentation

What do we check for?
Web App Vulnerabilities
 Search Lists Overview

User-defined Groups of QIDs

• Static search list - Manually defined
• Dynamic search list - Criteria-based

Benefits
• Dynamic List updates when new QIDs meet the
search criteria
• No limitation to the number of QIDs in search list

15 Qualys, Inc. Corporate Presentation

 Search Lists Overview

Search lists allow you to modify the

vulnerabilities for which you are:
• Scanning
• Reporting

Example:
Run a scan for only SQLi
Exclude a vulnerability from a scan
Build a report for only XSS

16 Qualys, Inc. Corporate Presentation

 Lab 1

Account Setup

17 Qualys, Inc. Corporate Presentation

 Basic Application Setup and Discovery

18 Qualys, Inc. Corporate Presentation

 The Qualys definition of a website

Start with the link as it appears in a web browser

Valid URL:
protocol://hostname[:port]/[path/]file
Ex. http://www.example.com/products

Decompose the link into three pieces

1. Fully Qualified Domain Name or IP Address
2. Starting port
80 is the default for http
443 is the default for https
3. Starting directory (path and query components)
/ (/ often redirects to a start page like /index.php, as in this example)
19 Qualys, Inc. Corporate Presentation
 Defining an Application

An application is:
• A business function typically requiring login
• Running unique code
• Typically supported by a single or team of developers

20 Qualys, Inc. Corporate Presentation

 Defining Applications – Unique Business Process

Example site:
http://site/admin/
http://site/hr/
http://site/finance/

Scenario 1: Scenario 2:
• Each directory is • Authentication
part of a single credentials are
app if they are different for each,
part of an Intranet with different
Portal business functions
• (1 app total) • (3 apps total)

21 Qualys, Inc. Corporate Presentation

 Defining Applications – Different ports
Example site:
E-commerce site that
authenticates over https, allows
browsing over http a catalog
Scenario:
https://e-commerce:443/login.cgi • WAS users only need to define the
http://e-commerce:80/browse.cgi starting port.
• The scanner will discover all ports in
other links.
• (1 app total)

22 Qualys, Inc. Corporate Presentation

 Defining Applications – Different Ports

Example site:
http://intranet:80/index.cgi
http://intranet:8080/index.cgi

Scenario 1: Scenario 2:
• If the app on port • If app on port 80
80 has links to app doesn’t have links
on port 8080 to port 8080
• Links are same • Links are different
business function business functions
(1 app total) • (2 apps total)

23 Qualys, Inc. Corporate Presentation

 Defining Applications – Different hostnames

Example site:
http://production.domain:80/
http://qa.domain:80/

Generally considered 2 applications

because they are separate
hostnames

24 Qualys, Inc. Corporate Presentation

 Web Applications - Filtering

Filter your apps by:

• URL
• Tags
• Scan information
• Last Scan Date
• Last Scan Status
• Scanner Appliance
• Scanner Appliance Tags
• Authentication Record
• Custom Attribute
• Creation Date

25 Qualys, Inc. Corporate Presentation

 Web Applications – Bulk Edit

Bulk Edit:
• Owner
• Scope
• Option Profile
• Scanner Appliance
• Header Injection
• Authentication Record

26 Qualys, Inc. Corporate Presentation

 Removing Web Applications

Used to remove retired

Applications

Similar to VM, does a full purge for

that web app within Qualys

27 Qualys, Inc. Corporate Presentation

 Crawl Scope

28 Qualys, Inc. Corporate Presentation

 Scope – Limit to URL hostname

Select this to crawl the hostname within the URL using http or https and
any port
Example: http://www.example.org/new
• All links in the http://www.example.org domain will be crawled
http://www.example.org/support
http://www.example.org:8080/news
• Links from www.example.org will not be followed
http://video.www.example.org
http://cdn.example.org

29 Qualys, Inc. Corporate Presentation

 Scope – Limit to content located at or below Sub-
directories
We can limit crawling to the starting URI and its sub-
directories.

30 Qualys, Inc. Corporate Presentation

 Scope – Limit to URL hostname and specified sub-
domain
We can limit it to crawl only sub-domains

31 Qualys, Inc. Corporate Presentation

 Scope – Limit to URL hostname and specified
domains
We can crawl the starting URL, and the additional domains

32 Qualys, Inc. Corporate Presentation

 Scanning

33 Qualys, Inc. Corporate Presentation

 Scan Types

Discovery Scan
• Validate Scope settings
• Crawl and ensure right coverage

Faster than Vulnerability Scan

• Vulnerability Scan
• Should happen after at least one Discovery Scan
• Tests the web application for vulnerabilities

34 Qualys, Inc. Corporate Presentation

 Discovery Scan

36 Qualys, Inc. Corporate Presentation

 Discovery

1. Scan begins at starting URL identified in the application

definition
2. Using the Scope Options identified in the application
definition, the scan traverses links to discover pages
and content
3. Configuration data is collected from the target app and
its host
4. Vulnerability testing is not performed

37 Qualys, Inc. Corporate Presentation

 The Crawl

Crawl Searches for: HTML-based links

Links via JavaScript

38 Qualys, Inc. Corporate Presentation

 QID 150009 - Links Crawled

This list may contain less

links than the maximum
threshold
Maximum links to crawl
includes:
• Links in the list
• Requests for the same
link made as an
anonymous user and
authenticated user
• Requests made via
html forms

39 Qualys, Inc. Corporate Presentation

 Discovery - Forms

What is a form?
Forms are used to pass data to a server**

40 Qualys, Inc. Corporate Presentation

 Web Application Sitemap

View Web Application or

Scan Sitemap To:
• View Pages Crawled and
Vulnerability Statistics
• Create New Web Apps
• Add URLs to Black List
• Add URLs to White List

42 Qualys, Inc. Corporate Presentation

 Lab 2

Basic Web Application Setup and Scanning

43 Qualys, Inc. Corporate Presentation

 Advanced Application Setup and Scanning

44 Qualys, Inc. Corporate Presentation

 Option Profile – Scan Parameters
• Modify Form submission for GET, POST,
GET&POST, None

• Change User agent

• Create Parameter sets

• Ignore common binary files

• SmartScan Support

• Change Behavior and Performance

settings

• Modify Bruteforcing settings

45 Qualys, Inc. Corporate Presentation

 Option Profile - Crawling

Crawl stops when:

• Max number of links threshold is met

• No new links are discovered

• Scan time-out is reached

46 Qualys, Inc. Corporate Presentation

 Option Profile - SmartScan

• Used for enhanced AJAX or Single Page Applications (SPA)

• Supports sites using AngularJS and bootstrap
• View QID 150148 to see links crawled – this will be your hint to
verify SmartScan is working

47 Qualys, Inc. Corporate Presentation

 Option Profile – Behavior Settings
Behavior settings:
Timeout Error: Network connectivity or someone reboots a
server
Unexpected Error: Web app returning 500/Internal Server
Errors
If a threshold is met, your scan will give you a “Service Errors
Detected” status

48 Qualys, Inc. Corporate Presentation

 Option Profile – Bruteforcing

• Performed when Form Authentication is used

• Make sure you include QID 150049

• Use Qualys list or import your own

49 Qualys, Inc. Corporate Presentation

 Web Application - Explicit URLs to Crawl

• Specify URLs you

want the service to
crawl

• Useful for pages not

linked to other pages
in the application
 Web Application - Progressive Scanning

Works best with Frequently Scheduled Scans

51 Qualys, Inc. Corporate Presentation

 Web Application - Progressive Scanning

• Performs ‘look back’ at previous scans

• Prioritizes pages not previously crawled
• Prioritizes new functionality
• Includes vulnerable pages detected previously
• Enhances flexibility in scheduling

52 Qualys, Inc. Corporate Presentation

 Web Application - Redundant Links

Specify fully customizable patterns of redundant links so that

the scan may not spend time crawling the similar links.
 Web Application - Authentication

Form Records
• Standard Login
• Custom
• Selenium Script / Qualys Browser Recorder

Server Records
• Basic
• Digest
• NTLM

54 Qualys, Inc. Corporate Presentation

 Web Application - Form Authentication

Crawl form-authentication behavior

• The crawl can take supplied form-authentication

credentials and create a session

• The session is maintained and if lost will re-authenticate and

continue

• Session behaviors are monitored and tested

55 Qualys, Inc. Corporate Presentation

 Exclusions

White list
- Crawl specific directories or pages (within application scope).
- Content outside of ‘white-list’ is black-listed by default.
- Target a specific area of modified/updated code.
Black list
- Prevent WAS from crawling sensitive or protected locations.
Post Data Black List
- Prevent WAS from posting HTTP forms on sensitive pages (i.e.,
Contact Us page).
Logout Regular Expression
- WAS scanner will not crawl to specified ’logout’ links.

56 Qualys, Inc. Corporate Presentation

 Advanced Options - DNS Override
PRODUCTION 2

www.yourwebapp.com: 64.39.106.246

64.39.106.249
QUALYS CLOUD PLATFORM

PRODUCTION 1
External Scanner Pool

64.39.106.246
57 Qualys, Inc. Corporate Presentation
 DNS Override Settings

DNS Override:
• Configure if DNS not yet
configured for your app that’s
currently in Dev or QA
• Tag to manage assignment of
DNS Override
 Web Application - Form Training

• This is a way for us to tell WAS what data to submit in a

form, to follow a certain workflow.
• Similar to how Qualys Recorder works. Works with just
about any browser.
 Web Application - Path Fuzzing

Use case: For testing sites that use URL re-writing (asp.net MVC)

Example: Let us consider sports web page

http://www.abc.com/issue/17/section/sports/article/28

However, the web server will read this URL as

http://www.abc.com/search.php?issue=17&section=sports&article=28

The path fuzzing rule would be:

http://www.abc.com/issue/{issue}/section/{section}/article/{article}
60 Qualys, Inc. Corporate Presentation
Manage Authentication Records

The Authentication tab provides a convenient place for managing both Form
and Server authentication records.
 No Web Service

The scan will give “No Web Service” status if the scanner:
• Cannot get a DNS lookup on the site
• Cannot reach the target because of routing
• Cannot get a web service to respond to a GET request

62 Qualys, Inc. Corporate Presentation

 Web Application Testing

63 Qualys, Inc. Corporate Presentation

 Web Application Scanning

WASC www.webappsec.org
divides Web vulnerabilities into
six categories
Authentication
Authorization
Client-side Attacks
Command Execution
Information Disclosure
Logical Attacks

64 Qualys, Inc. Corporate Presentation

OWASP
 Stored XSS
1

<script>…</script>

https://forum.corp.com
GET /page1.html
2
<script>…</script>

https://drop.hacker.com
1 Hacker injects code into database
2 User requests page, stored hacker code sent
3 Client executes code

66 Qualys, Inc. Corporate Presentation

 Reflected XSS
1

/q?=<script>…</script>
“Error: <script>…</script>”

https://forum.corp.com
GET /q?=<script>…</script>
2

“Query Error: <script>…</script>”

1 Hacker find vulnerable parameter. https://drop.hacker.com

2 User requests page, hacker code reflected back.
3 Client executes code.

67 Qualys, Inc. Corporate Presentation

 SQL injection

johndoe ` OR 1 = 1; /*

qualys */--

select * from users where username=‘johndoe’ and password = ‘qualys’

select * from users where username=‘` OR 1 = 1; /* ‘and password = ‘ */--’

68 Qualys, Inc. Corporate Presentation

 Blind SQLi
True False

GET /article.php?id=1 and 1 = 1 GET /article.php?id=1 and 1 = 2

• By asking the server true and false questions and getting different results
we are able to determine if vulnerability.

69 Qualys, Inc. Corporate Presentation

 Cross-Site Request Forgery (CSRF)
1

/login

/transfer?from=mine&to=yours 3

https://account.mybank.com
4

1 User logs into MyBank. https://drop.hacker.com

2 User is sent a phishing email with false link.
3 Clinking on the link a request is sent to the banks website using the users authentication token.
4 User money is transferred to the hackers account.
70 Qualys, Inc. Corporate Presentation
 Directory and Path Traversal

Directory discovery
Discovered directories are analyzed
Pages are read for mime types and content

Common default pages

Default pages like /admin are attempted

71 Qualys, Inc. Corporate Presentation

Scan Results
 Web Application Scanning

See scan details

• Where did the scan
originate?
• What was the target?
• Who launched the scan
and when?
• What were the option
profile settings when the
scan was run?
73 Qualys, Inc. Corporate Presentation
 Lab 3

Advanced Web Application Setup and Scanning

74 Qualys, Inc. Corporate Presentation

 Reporting

75 Qualys, Inc. Corporate Presentation

Dashboard
 WAS Reporting

• Results listed by
vulnerability, link, type,
app
• Redundant results are
condensed to a base
cause
• Create Templates to save
report formats
• Four report types
• Schedule Reports to run
when you need them
77 Qualys, Inc. Corporate Presentation
 Web Application Report

• Normalized data of all

scans on the web
application
• Choose tags or
applications for report
targets
• Vulnerability Status
included (New, Active,
Re-opened, Fixed)
• History of vulnerability
• Retest for vulnerability
78 Qualys, Inc. Corporate Presentation
 Scan Report

• Raw Scan Results

• Pick the specific scan
results you’d like to
view in a report
• View Threat, Impact,
and Solution for
vulnerabilities

79 Qualys, Inc. Corporate Presentation

 Scorecard Report

• Statistics on all
applications tagged in
UI
• Top 10 most vulnerable
applications
• OWASP breakdowns

80 Qualys, Inc. Corporate Presentation

 Catalog Report

• Lists web apps as

New, Approved,
Rogue, or Ignored
• Number of entries
added over time
• Number of entries
by status

81 Qualys, Inc. Corporate Presentation

 Report Management

• Create, download,
run reports
• Filter existing
reports
• Add tags to
reports

82 Qualys, Inc. Corporate Presentation

 QID 150021 – Scan Diagnostics

The scan
diagnostics data
provides technical
details about the
crawler's
performance and
behavior.

83 Qualys, Inc. Corporate Presentation

 QID 150100 – Selenium Diagnostics

Troubleshoot
Selenium script

See which parts of

the script ran

84 Qualys, Inc. Corporate Presentation

 Lab 4

WAS Reporting

85 Qualys, Inc. Corporate Presentation

 Tags and Users

86 Qualys, Inc. Corporate Presentation

 Tag Management

Add and remove tags to:

• Users
• Web Applications
• Reports
• Option Profiles
• Brute Force Lists
• Search Lists
• Scanners
• Parameter Sets
• Authentication Records

87 Qualys, Inc. Corporate Presentation

 User Roles

• User roles provide privileges

to access tagged assets
• Set granular permissions
• Grant QA or Developers
access
Customize a role
 Lab 5 and 6

Tagging and User Management

90 Qualys, Inc. Corporate Presentation

 3rd Party Integration

91 Qualys, Inc. Corporate Presentation

 WAS Integration

• Centralized location for

vulnerability details.
Burp Suite Professional Integration
 Bugcrowd Integration

• Qualys WAS and

Bugcrowd can now
bi-directionally import
and export findings
 Malware Detection

95 Qualys, Inc. Corporate Presentation

 Malware Detection

Protect site from distributing malware

Automated Alerts
Automated Reports
Malware Detection
1. Enter URL
2. MDS does a breadth crawl URL
You plug in your URL (we stay in the domain).
3. MDS runs both behavioral and
static analysis.
4. Qualys will email user if Malware
is found.

Qualys Virtual
Machine Farm
Malware Detection
 Lab 7

Malware Detection

99 Qualys, Inc. Corporate Presentation

 Exam Tips and CPE

• You have five attempts to pass

• The test is linear, no going back to an older question
• Passing score: 75% and above
• No negative marking
• Test can be taken anytime
• 30 questions (Multiple choice included)
• You may use presentation slides, lab exercises, Qualys Community,
and you may have an active Qualys session open while attempting
the exam.
• No set time limit (please start a new LMS session, before launching
the exam.
• A CPE credit is earned for each hour of attendance.

100 Qualys, Inc. Corporate Presentation

 Thank You

training@qualys.com

101 Qualys, Inc. Corporate Presentation