Performance Testing for TM & D An Overview

Company confidential

Agenda

Introduction & Performance Engineering Coverage on

TM&D Consultation & Performance Engineering Phases Sample Model for S&D Reporting Data collection and Analysis Post test Identification Deliverables

Company confidential

Objective

To have a high level understanding of Performance Engineering Engagements for TM & D

Company confidential

Practical conversations on performance

I think we need Performance Testing, but what is it exactly? I think around 300 users will use the system, they will do all kinds of activity, so can we determine performance ? Houston, this application is slow do we need high-end server? What you will do with production data in performance? Post Test charts looks nice, let me know the performance is good or bad Does that mean were done? Can we release?

And a latest conversation two days back, one client called me and asked I am not winning the on-line bidding, can I send 20-30 bidding requests at a time, and increase my bidding chances?
Company confidential

Introduction
Performance Testing has three dimensions (Keeping the resource as constant): Number of Users

Amount of Data
Amount of Activity Any performance related issues can be mapped using these dimensions

Performance is dependent on the following:

Resources that are used by the application Computing Resources, Network Resources and Storage Resources Bottlenecks and wait times The number of operations involved in performing the computations and the time taken to complete these operations
Company confidential

Performance Coverage on TM & D Systems

Outlet Data Management System

High Level Scope of Performance Testing

S&D Strategy, Planning and Evaluation

TM&D
Business Performance and Process Assessment

Testing integrated portal access Customer data query and updation Report generation (weekly/monthly) - S&D - ODMS (outlet specific reports) - Key Accounts - Performance and Process Offline data access and update Report Publishing in central system Performance team will identify the other crucial Transactions during System Study Level

Key Accounts/ HoReCa

Company confidential

Our High Level Process Consultation

Company confidential

Major Steps in Performance Testing

Confirm Performance Requirements Factor/Metric

Develop Performance Strategy/test plan

Load Performance Stress testing Reliability Testing Security testing Usability testing Compatibility testing

Develop Scripts, test data Set up Production mirror image

Issue Management System Product Maturity Analysis

Test design Test Execution

Continuous Analysis Feedback to Dev team Report issue/Concern

Post Test Analysis Report Development

Company confidential

Tool: Performance Automation

Solves the resource limitations Replace Testers with Virtual Users Runs many Vusers on a few machines Controller manages the virtual users Analyzes results with graph & report tools Repeats tests with scripted actions

Controller

Analysis

Vuser host

Portal Server/s

Database server

Load Generation

TM&D System Under Test

Speed - Does the application respond quickly enough for the intended users? Scalability Will the application handle the expected user load and beyond? Stability Is the application stable under expected and unexpected user loads? Confidence Are you sure that users will have a positive experience on go-live day?
9

Company confidential

Sample Transaction

Ability for the AE to generate the Forecast for the next quarter on all of the Plan components
60
No of Users in System

50 40 30 20 10 0 0 5 1 2 4 10 20

50 45 40 35 30

35 User Load 25 15 10 5 6 8 10

Time in Biz Hour

Slow Ramp up, Collect Client side, Server Monitor mean data
Company confidential

Peak Hour Ramp up, Collect Client side, Server Monitor mean data

Ramp Down, Collect Client side, Server Monitor mean data

10

User Model

For each Transaction, User Model is created based on

Authentication and Authorization of users Feature Usage Criteria/Transaction Type Total Number of Users in System User random activities on a transaction Transactions/unit time Transaction MIX

Company confidential

11

Online Monitoring

Some common data for online monitoring

Error & Exception Response time graph Throughput graph Transaction passed failed Page/component download time graph Keep checking server logs Capture the slow SQLs Server Resource

Company confidential

12

Performance Measurement

Workload Data Data Characteristics Execution Characteristics Path characteristics Software resource usage Processing overhead Scenario Response Time Scenario Throughput

Software resources: CPU Usage Memory Usage SQL Queries File I/O Messages Logging to files or databases Calls to Middleware functions Calls to software in a different process, thread or processor Application Cache and Buffers OS Handles like threads, memory, sockets network I/O and type of network resource access,

Computer System Usage

Key System Resource Usage

Resource Utilization Server throughput

number of connections to the databases etc

Company confidential

13

Assignment Deliverables
Pre-Test Performance Strategy for TM&D Performance Modeling for major sections of applications Resource Engagement outline and Effort estimation Delivery Process Methodology Performance Entry Criteria On-Test Detailed Test Plan Test Scripts/Templates, Test design Test Configuration Map (Data/Setup) Test Tool, monitoring agent setup and PoC Report (LoadRunner) Performance Exit Criteria Test Results, Review log Post Test Data Risk/Issue log

Post-Test System/sub-system performance analysis Report Bottleneck identification

Company confidential

14

Performance findings and Related Tuning Code optimization Caching strategy Load balancing Distributed Computing Logic SQL Query Profiling Usage of database indexing Removal of Normalization Resource configuration (RAM, Network) Client software version (OS, Browser) Identification of performance pattern

Company confidential

15

 Most often performance is thought of as something related to tuning the code this is perhaps the single most reason why performance failures occur
80% performance issues belong to architecture and business definition it is impossible to get more than 10% benefit by tuning code
Company confidential

16

Your Questions are welcome Thank you Bangalore TEAM

Company confidential

17