1402-Planning Application Deployments With ACE™

Session 1402
Planning Application Deployments

with ACE™
Application and System Performance
CONFIDENTIAL ─ RESTRICTED ACCESS: This information may not be disclosed, copied, or transmitted in any
format without the prior written consent of OPNET Technologies, Inc.
© 2007 OPNET Technologies, Inc.
1402 Planning Application Deployments with ACE™
Motivation for This Session
ACE has two main functions

Application troubleshooting
Pre-deployment analysis
Application troubleshooting
Reactive: “The network is slow!”
Server problem? Network problem? Application problem? Best way to
improve performance?
Pre-deployment analysis
Proactive: “We’re going to roll out this new application.”
Sufficient network capacity? Application efficient? Response times
acceptable? Application architecture sound?
Different questions—different answers—different way of thinking
CONFIDENTIAL ─ RESTRICTED ACCESS: This information may not be disclosed, copied, or transmitted in any format without the prior written consent of OPNET Technologies, Inc. © 2007 OPNET Technologies, Inc.
2
OPNET Solutions
ACE
IT Guru® with:
ACE
ACE™ Decode Module (ADM)
Advanced ACE Capabilities (AAC)
Required for Multi-User QuickPredict®
Helpful for advanced studies:
IT Guru® Network Planner / SP Guru® Network
Planner
IT Guru / OPNET Modeler® with:
XDI or VNE Server®
Flow Analysis™
3
Course Agenda
Application Network Readiness Assessment

Methodology: Pre-deployment Analysis in ACE
Terminology
Analyzing a single application task
Lab 1: First Import
Dealing with multiple application tasks
Lab 2: Quick Analysis of Many Transactions
Capture Strategies
Comparative analysis
Lab 3: Examine Problematic Step in Detail
Capacity planning
Lab 4: Multi-User Quick Predict
Tips for improving the testing process
4
Application Network Readiness

Assessment
Motivation
Definition
Challenges
How ACE fits in development cycle
5
Common Scenario
New application rolled out to users

in all branch offices
Help desk phones ringing off the
hook
Network team is called in to fix the
“network problem”
ACE Results:
Application is chatty
Application is sending huge files
Resolution:
Citrix / remote desktop
Costly modifications to application
Costly network upgrades
Users have to live with slow
performance
6
Other Common Scenarios
Servers moved to new data

center
Application slows to a crawl or is
unusable
Prolonged outage while servers are
moved back
New version of application
deployed to all users
Remote users suddenly have
performance problems
Remote users rolled back to old
version while problem is
investigated
7
Application Network Readiness

Assessment
Goal: look at application from a network perspective, evaluating
efficiency, capacity, and risk
Load testing: not just for servers anymore!
Overlooked part of testing
Answer the following questions:
How will this application perform once deployed?
Does the network have sufficient capacity?
Are there inefficiencies in the application design that cause unnecessary
network or server load?
What is the best way to solve any problems identified?
8
Benefits of Application Network Readiness

Assessment
Reduce costs
Lower application development lifecycle costs
Identify performance issues quickly and fix them before deployment
Develop network-efficient applications
Optimize bandwidth usage and quantify cost savings
Ensure achievable service levels
Accurately predict response times and capacities before deployment
Set expectations!
Increase performance stability
Expose a common view of application / network dynamics
Identify dominant performance factors
Trade-off application performance and link capacity
Prevent future application performance problems!

9
System Development Lifecycle (Modified

Waterfall)
Applicability of ACE
Operations
10
Challenges
Management doesn’t understand value of study

Delayed development projects reduce the time available for testing
Hard to bridge gap between network and application knowledge
Physical
Application TCP IP Data
Link
Link
Application Network
Development Engineering
Build and maintain
Develop functionality
communications
to meet business and
infrastructure
service requirements.
for service delivery.
11
Methodology: Pre-Deployment Analysis in

ACE
12
Methodology: Capture Key Workflows
Work with testers, users to

understand workflows
Use existing test plans
Try to understand context of tasks
Look for “interesting” tasks
Capture as many tasks as you can!
13
Methodology: Import Multiple Tasks,

Create Comparison Reports
Workflow:
First, import one task
Validate architecture/filters
Save Import Configuration
Create QuickPredict and Comparison
Report templates
Batch Import remaining tasks
Generate Comparison Report
Goals:
Quickly analyze a large number of capture
files
Save ACE files for baselining and capacity
planning
Look for the problem areas
14
Methodology: Identify Red Flags
Focus on these statistics:

Number of application turns
Network data (total)
Network data (over WAN)
Predicted response times
Look for two things:
Inefficient behavior
Network-heavy tasks
15
Methodology: Analyze Problem Tasks
Examine each problem task in ACE

Tree view
Decodes
AppDoctor
QuickPredict Bar Charts
Perform additional captures
Study impact of changes
Determine best solution
16
Methodology: Capacity Planning
Goals
Determine whether network can support
planned number of users
Predict response times with many users
Scope
Network load testing only
Server performance held static
Tools:
Multi-User QuickPredict
Requires AAC or ACE Plus
Discrete Event or Hybrid Simulation
Requires ACE Plus or IT Guru +
ACE
Flow Analysis
Requires IT Guru Network Planner or
IT Guru + Flow Analysis + Import
17
Terminology
Application Task
Workflow
User Think Time
Duration
Response Time
18
Application Task
What is a task?
A basic unit of user activity
Workflow can be made up of:
A single task
A series of tasks
ACE file of a series of tasks
19
Application Task
Open_Form
Click_OK
Enter data
Click_UpdateandQuit
Click_Submit
20
User Think Time
New in 14.0
ACE detects User Think Time
Default = 1 second of
contiguous client delay
Client = first talker
Click Setup… to change
threshold
Click Apply to OK this
assumption
21
User Think Time
New statistic: Duration

Duration is true duration of capture including user think time
Response time = Duration – User Think Time
Two views of Pie Chart:
22
Analyzing a Single Application Task
1. Validate what you know about the application

2. Examine key metrics
3. Create groundwork for future batch imports by creating templates
23
Validate
Too often, communications patterns between tiers are unknown or

misunderstood
Start with architecture diagram

Who talks to whom
IP address of all servers
Ports/protocols in use
Take initial captures

Default filter
Multiple capture points
24
Validate
Import into ACE Summary

Give friendly names to known tier pairs
Validate tiers that communicate
Validate protocols and ports
Look for unexpected packets
Account for anomalies
Filter
Analyze Included in ACE
Continue through import
25
Examine Key Metrics
Examine AppDoctor Statistics

Application Turns
Network Data (total)
Network Data (across WAN)
Network Packets
Connection Resets
Protocol-specific metrics
Tree view
Decodes
26
Examine Key Metrics
QuickPredict Bar Charts

Predict response time for different
users
Understand sensitivity to network
conditions
Assess server colocation
Create a template
Save for future use
27
Create Templates
Once you are confident of which traffic to include, create filter

Filter will be reused for subsequent imports
Be prepared to re-validate:
Transactions known to hit different servers
If application architecture changes
28
Create Templates
Import Configuration file

saves:
Filters
Tier name mappings
Capture location vs. tier
location
Bandwidth and latency
during capture
from ACE File Menu
Only available after import
Create and use filter first
29
Create Templates
Comparison Reports
Short
Spreadsheet/table
Multiple Transaction Reports
Transaction Summary
Table of Contents
Comparison Report
Normal ACE Word Report per
transaction
How to create templates:
Choose report type
Choose tasks (doesn’t matter)
Choose what to include
Point to QuickPredict Bar Chart
templates
Save template
30
Create Templates
Helpful statistics to include:

Response Time
Application Turns
Network Packets
Network Data (total)
Network Data (WAN)
QuickPredict results
31
Lab 1: First Import
Objectives:
Import one task into ACE Plus, exploring the contents of the capture file as
you go
Create a Quick Predict bar chart to use for other tasks
32
Lab 1: First Import
Conclusions
Login response time is 8.2 seconds on LAN
It’s not going to get any faster on the WAN
Worst case response time: Wireless Card, 15 seconds
QuickPredict Bar Chart template saved for Lab 2
33
Importing Many Transactions into ACE
First, import one task

Validate architecture/filters
Create QuickPredict and Comparison
Report template
Batch Import remaining tasks
Generate Comparison Report
Look for red flags!
34
Batch Import
Used to import many captures at once

Use when importing each file is tedious
12-step test plan
Separate capture file(s) for each task
Methodology
Import the file(s) for one task normally
Create and save filters
Save import configuration
Save report settings
Import remaining sets using Batch Import
35
Batch Import
Select the directory where capture files are stored

Set Import Configuration
Set Output Directory
Set Report Template
36
Batch Import
Single point of capture

Save all captures to a directory
Click Add Multiple Transactions…
Choose Select a directory
Choose Add one transaction per capture file
37
Batch Import
Multiple capture points, using ACE Agents

Naming convention is consistent!
Choose Add one transaction for each set of capture files with the same
<name>@...@...appcapture
38
Batch Import
Multiple capture points, using variety of capture technologies

Naming convention may not be consistent
Put each task’s capture files in a separate directory
Check Include subdirectories
Choose Add one transaction for each directory that contains capture files
39
Comparison Report
Run during/after Batch Import

Focus on key metrics
Look for red flags
40
Comparison Report—QuickPredict
41
Multiple Transaction Reports
Useful for documentation and

baselining
Tips:
Keep reports as short as possible
Use Table of Contents
Only include Per-Transaction

Details relevant to predicting
performance
AppDoctor Statistics
QuickPredict Results
Protocol-specific statistics
42
Lab 2: Quick Analysis of Many

Transactions
Create a filter for the packet captures
Do a batch import of the remaining tasks
Examine the report for problems
43
Lab 2: Quick Analysis of Many

Transactions
Conclusion
Step 1
Lots of server delay (as noted in Lab 1)
Slow predicted response time
Step 6
Largest number of application turns
Largest amount of network data
Anomalous results compared to other steps
Slow predicted response time
Other tasks
Small amounts of network data
Small number of application turns
No red flags
44
Capture Strategies
What to capture
How to capture
When to capture
How to organize what you’ve captured
45
Capture Strategies—Legacy Application
How do users do this today?

How long does it take?
Complaints about the current
application?
Expectations about the new
application?
46
Capture Strategies—What to Capture
Try to understand context of tasks

How often is task performed?
Are there specific time constraints?
Is task interactive?
Be on the lookout for “interesting” tasks
Direct communications with database
Large database entries
Transmission/creation of huge documents
Tasks that are slow in test
Tasks developers/testers are concerned about
Tasks with many dependencies
47
Capture Strategies—How to Capture
Think in terms of workflows

Workflows contain many tasks
Use test plans as starting point
Work with testers and users
Workflow capture options
Capture all tasks in workflow in one file, or
Capture each task in a separate file
48
Workflow Capture Options
Option 1—Capture all tasks in workflow in one file

Good for Flow Analysis
Easier to capture and import
ACE can remove Think Time
Separate into tasks later
Transaction Extraction Utility (ping-slicing)
Manually edit in ACE QuickView or ACE
Recapture key tasks
Option 2—Capture each task in a separate file
More time consuming to capture
Easier to analyze data later
49
Transaction Extraction Utility
Splits single packet capture file into many files
Methodology
Start capture(s)
Use the application as normal
Send pings between each transaction
Import entire file(s) into ACE
Use Transaction Extraction Utility to chop up the file(s)
Use Batch Import to import each set of short capture files
50
Capturing During Load Tests
Good for seeing real server response times

Hard to isolate backend traffic
51
Capturing During Load Tests
Tips for success:

Capture easily isolated traffic only (client <-> server)
Focus on QuickPredict Response Times
52
Capturing with No Load
Tester is only user hitting app

Best case response time – least server delay
Easy to isolate traffic
QuickPredict Response Times only reflect network delays
True response times may be longer
53
Capturing with Minimal Load
Most common scenario: several testers hitting application

Backend traffic relatively easy to isolate
Validation/filtering is necessary after Batch Import!
Servers under low load, same as no load
54
Organize Files—Example Structure
Capture Files
Bob
New patient entry
Send bill
Linda
Adding a donation
Looking up product code
ACE Files
Filtered and validated
New patient entry
Send bill
Batch Import
Test
Filters and Templates
Reports
55
Comparative Analysis
Compare two versions of same task

Upgrading to new version of existing app
Migration from one app to another
Testing a fix to an application problem
Comparison Report
Difference Report
Eyeball Data Exchange Chart
56
Changing Application Behavior
QuickRecode
ACE™ Whiteboard
57
Objectives
Find out the root cause of the slow performance of a task
Evaluate an alternative approach
Compare the two traces and determine which is better
58
Conclusions
PDF file is better across the board
Less data
Fewer application turns
Fewer packets
Faster response times
59
Full Scale Capacity Planning Study
Goals
Determine whether network can support planned number of users
Predict response times when multiple users are using application
Scope
Network load testing only
Server performance held static
Tools:
Multi-User QuickPredict
Requires AAC license or ACE Plus
Discrete Event or Hybrid Simulation
Requires ACE Plus or IT Guru + ACE
Flow Analysis
Requires IT Guru Network Planner or IT Guru + Flow Analysis + Import
60
Multi-User Quick Predict (MUQP)
MUQP vs. QuickPredict

MUQP performs capacity planning to the edge
Results:
Predicted response times (taking network load into account)
Utilization of WAN links
61
Define the environment

User workloads
Application tasks
SLAs
Client sites
Data centers
Save as a template
62
Report on Average and/or Nth percentile

95th percentile = worst case
Everyone logs in at once
Determine whether results vary widely
63
View results by site, link, or transaction

Navigate using drilldowns and breadcrumbs
Generate report
64
Capacity Planning with Flow Analysis
Capacity planning through network core

Capacity planning to user locations
Workflow:
Build network model
Add historic traffic (optional)
Convert ACE file(s) to traffic flows
Run Flow Analysis
View link utilization
Session 1350
65
Create Network Topology
Manually create using GUI

For small scale or “greenfield” networks
For incremental changes to topology, configuration
Partial automation—importing data from multiple

sources
Using eXpress Data Import (XDI)
Model is created through multiple imports
Full automation—importing near-real-time

integrated network data
Using VNE Server
Intelligently fuses data from disparate sources VNE Server
Continuously maintains unified network data
Model is created through “single click”
import operation
66
Import Traffic into OPNET
Traffic flows
Cisco NetFlow
NetScout nGenius
Cflowd
Fluke Network OptiView
Spreadsheet
Text files
ACE
Link loads
CA eHealth
MRTG
InfoVista
HP OpenView Insight
Text files
67
ACE to Flows
Start with an ACE file

Single task
Workflow including Think Time
Overlay ACE file onto network model
68
ACE to Flows
Convert ACE model to application flows
Specify ACE file(s) to include
69
ACE to Flows
Assign nodes in network

model to ACE tiers
Clients
Servers
Configure client node

Number of users
Transactions per hour
70
ACE to Flows
Generate the flow elements
Traffic Center View of derived

flows
71
Flow Analysis Results
72
Capacity Planning with Discrete Event

Simulation
Get response times and link utilization
Model more complex scenarios than MUQP
QoS impact on response time
Multiple load-balanced links to a site
Workflow:
Create topology
Model historic traffic
Convert ACE file(s) to Discrete Traffic
Run simulation
View results
73
Flow Analysis vs Discrete Event Simulation
Topology + Traffic
Discrete Event Simulation
Event-based simulation
Link Utilization
Application response time
QoS Impact
Flow Analysis
Analytical modeling/simulation
Link Utilization
No response times
Quick performance evaluation
74
ACE to Discrete Traffic
Find out response time for each task over simulated network
Server delay held constant
Works with DES only
75
Discrete Event Simulation Results
76
Objectives
Perform a Multi-User QuickPredict on worst case task from worst case
location
If network can handle this scenario, you are done!
Perform a Multi-User QuickPredict for all transactions for users in:
Four branch offices
Person working from home
Person using wireless card
Determine which transactions and scenarios do not meet SLAs
77
Conclusion
Paris office needed more bandwidth to support Step 6
Wireless card had worst performance
Even after upgrade, Paris office was not speedy
Most transactions ran fine in all scenarios
Steps 1 and 6 were the problem transactions (predicted in Lab 2!)
78
Improving the Testing Process
Challenges revisited….
Management doesn’t understand value of study
Delayed development projects reduce the time available for testing
Hard to bridge gap between network and application knowledge
Mandate for Application Network Readiness Assessment is essential
How can you improve your organization’s current process?
79
Characterizing Application Performance

Management
Chaotic
No monitoring
No tools
No documentation
No process!
80

Management (cont.)
Reactive
Some monitoring, tools, documentation
When problem occurs, respond
Wide range of reactive models
81

Management (cont.)
Proactive
Applications tested prior to deployment
Plan is in place in case something goes wrong
Baselining of current applications occurs on an ongoing basis
Changes are tested before being deployed
82

Management (cont.)
Strategic
Network team involved from initial design meeting through project lifecycle
Communication between groups allows concerns to be addressed earlier
Teams work toward business goal
83

Management (cont.)
How would you categorize your organization’s process for application
performance management?
Chaotic
Reactive
Proactive
Strategic
What are the benefits to being more proactive and strategic?
84
Improving the Troubleshooting Process
Shorten the time needed to set up a capture

Deploy agents on all servers and workstations
Deploy, document, allow access to hardware probes
Create a SWAT team
Across divisions
Authority and knowledge to
Get data
Find problem
Propose/implement solution
Profile and baseline applications before problems are reported
Pre-deployment testing
Profile key applications now to understand behavior
85
Perform Pre-Deployment Studies Rather

Than Troubleshooting
Troubleshooting is expensive
Solving problems earlier in lifecycle is easier / cheaper / faster

Pre-deployment planning takes time, but:
It will get things working and prevent unwanted surprises
Net time savings
Better to design well than to live with poor design
86
Pre-Deployment Testing
Pre-deployment application testing should include network

performance testing
Encourages developers to be aware of the impact the network will have on
their applications
Increases communication between application and network teams
Eliminates “surprises” from application deployments
Reduces the amount of time and energy spent fighting fires later
Tips for making ACE a part of testing process

Get visibility by testing one critical application
Get “grassroots” buy-in to value of performance testing
Get application developers involved and trained
Do some PR—advertise application profiling service to upper management
87
Adding ACE to Pre-Deployment Planning
When is the best time to advocate:

Being more proactive?
Getting yourself involved earlier in the lifecycle?
Predicting application performance before deployment?
88
Adding ACE to Pre-Deployment Planning
When is the best time to advocate:

Being more proactive?
Getting yourself involved earlier in the lifecycle?
Predicting application performance before deployment?
Answer based on our experience:

When you just solved a production performance issue!
89
Example Production Diagnosis
Example:
Application tested in lab 6 months ago – 2 second response time
When deployed to NY office – 41 second response time!
ACE quickly pin-points the source of the performance problem
90
Example Production Diagnosis (cont.)
Add “retroactive” prediction to production diagnosis report

Show how diagnosed problem could have been avoided
Using the production trace file, show how ACE:
Could predict what happened in the lab
Could have predicted the observed problem
Could have predicted other potential problems
“I could have told you…if only you’d asked!”
91
Sample Addition to Production Diagnosis

Report
Not only is ACE useful for identifying application performance problems, but ACE is also used to
proactively identify production performance problems before an application is rolled into
production.
For example, we would have predicted this exact production problem in the NYC office if we had
used ACE on this application before it was rolled out.
Below on the right is a prediction of the Lab, NYC office, and London office performance. Note
how QuickPredict’s prediction matches AppDoctor’s diagnosis of the production NYC problem.
Production Diagnosis “Retroactive” Prediction
92
Documentation References
FAQs
1254: How does QuickPredict work? How can I validate QuickPredict? What
might be a reason QuickPredict results differ from my expectations?
951: Why might my results for ACE QuickPredict differ from real world or
simulation? What does QuickPredict model?:
1048: How do I simulate a network trace of my application using ACE?
Product Documentation
ACE Documentation: Predicting Application Performance
Methodologies & Case Studies:
Application Networkability Assessment
Application Impact Assessment
Application Performance Troubleshooting and Analysis
93
Related OPNETWORK Sessions
1442 Application Capture and Import Strategies with ACE™—Introduction

1415 Application Capture and Import Strategies with ACE™—Advanced
1452 Troubleshooting Application Performance with ACE™—Advanced
1350 Network Capacity Planning for Enterprises
1302 Traffic Modeling Techniques
1432 Modeling Applications with the ACE™ Whiteboard
94
Take-Away Points
It is always better to ask the question

What will happen?
rather than
What happened?
Application Network Readiness Assessment is a critical part of testing

Capture as much as possible
Get a high level view of many tasks first
Use comparison reports!
Study problem tasks in more detail
Network and application teams should look at ACE together
95

1402-Planning Application Deployments With ACE™

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

1402-Planning Application Deployments With ACE™

Caricato da

Copyright:

Formati disponibili

Session 1402

Planning Application Deployments

Motivation for This Session

 ACE has two main functions

 Application Network Readiness Assessment

Application Network Readiness

 New application rolled out to users

Other Common Scenarios

 Servers moved to new data

Application Network Readiness

Benefits of Application Network Readiness

 Prevent future application performance problems!

System Development Lifecycle (Modified

 Management doesn’t understand value of study

Methodology: Pre-Deployment Analysis in

Methodology: Capture Key Workflows

Work with testers, users to

Methodology: Import Multiple Tasks,

Methodology: Identify Red Flags

 Focus on these statistics:

Methodology: Analyze Problem Tasks

 Examine each problem task in ACE

Methodology: Capacity Planning

ACE file of a series of tasks

User Think Time

User Think Time

 New statistic: Duration

Analyzing a Single Application Task

1. Validate what you know about the application

 Too often, communications patterns between tiers are unknown or

 Start with architecture diagram

 Take initial captures

 Import into ACE Summary

Examine Key Metrics

 Examine AppDoctor Statistics

Examine Key Metrics

 QuickPredict Bar Charts

 Once you are confident of which traffic to include, create filter

 Import Configuration file

 Helpful statistics to include:

Lab 1: First Import

Lab 1: First Import

Importing Many Transactions into ACE

 First, import one task

 Used to import many captures at once

 Select the directory where capture files are stored

 Single point of capture

 Multiple capture points, using ACE Agents

 Multiple capture points, using variety of capture technologies

 Run during/after Batch Import

Multiple Transaction Reports

 Useful for documentation and

 Only include Per-Transaction

Lab 2: Quick Analysis of Many

Lab 2: Quick Analysis of Many

Capture Strategies—Legacy Application

 How do users do this today?

Capture Strategies—What to Capture

Try to understand context of tasks

Capture Strategies—How to Capture

 Think in terms of workflows

Workflow Capture Options

ACE has two main functions

Application Network Readiness Assessment

New application rolled out to users

Servers moved to new data

Prevent future application performance problems!

Management doesn’t understand value of study

Work with testers, users to

Focus on these statistics:

Examine each problem task in ACE

New statistic: Duration

Too often, communications patterns between tiers are unknown or

Start with architecture diagram

Take initial captures

Import into ACE Summary

Examine AppDoctor Statistics

QuickPredict Bar Charts

Once you are confident of which traffic to include, create filter

Import Configuration file

Helpful statistics to include:

First, import one task

Used to import many captures at once

Select the directory where capture files are stored

Single point of capture

Multiple capture points, using ACE Agents

Multiple capture points, using variety of capture technologies

Run during/after Batch Import

Useful for documentation and

Only include Per-Transaction

How do users do this today?

Try to understand context of tasks

Think in terms of workflows

Option 1—Capture all tasks in workflow in one file

Splits single packet capture file into many files

Good for seeing real server response times

Tips for success:

Tester is only user hitting app

Most common scenario: several testers hitting application

Compare two versions of same task

MUQP vs. QuickPredict

Define the environment

Report on Average and/or Nth percentile

View results by site, link, or transaction

Capacity planning through network core

Manually create using GUI

Partial automation—importing data from multiple

Full automation—importing near-real-time

Start with an ACE file

Convert ACE model to application flows

Specify ACE file(s) to include

Assign nodes in network

Configure client node

Generate the flow elements

Traffic Center View of derived

What are the benefits to being more proactive and strategic?

Shorten the time needed to set up a capture

Solving problems earlier in lifecycle is easier / cheaper / faster

Pre-deployment application testing should include network

Tips for making ACE a part of testing process

When is the best time to advocate:

When is the best time to advocate:

Answer based on our experience:

Add “retroactive” prediction to production diagnosis report