1

The top 20 tricks for

obtaining faster SAP
NetWeaver BI response time

Dr. Bjarne Berg

MyITgroup Ltd.
A VIP consulting company
2008 Wellesley Information Services. All rights reserved.
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs

EarlyWatch Reports
Wrap-up

3
 Introduction

In this session we will cover the top 20 must-do technical

performance tricks to help you optimize SAP NetWeaver BI
reporting for your end users.

We will look at performance modeling of InfoCubes, how

to improve memory utilization by caching and how to use
diagnostics to analyze performance issues.

We will also explore best practices on how to develop and

manage aggregates and MultiProviders, and see what the
BI- Accelerator (BIA) can do for your organization.

Finally, we will look at how to analyze EarlyWatch reports

from Solution Manager 4.0 so they become actionable.
4
 Capacity and Scalability Is the Top Concern for Your
CxO
Dont under size your global BI system
Spend adequate funding on hardware, memory, processing power and disk
space
AAsurvey
surveyofof353
353top
topC-level
C-levelofficers
officersininlarge
largecompanies,
companies,
reported
reportedthat
thatthe
thetop
topBI
BIconcern
concernwas
wasthe thescalability
scalabilityofoftheir
their
solutions.
solutions.

Source: Intel, SAP & Business Week

"Seizing the BI Opportunity" 2006.
5
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs

E arlyWatch Reports
Wrap-up

6
 Tip 1: MultiProviders and Hints

Problem: To reduce data volume in each InfoCube,

data is partitioned by Time period.

A query now have to search in all InfoProviders to find

the data (i.e. billing docs from 2007). This is very slow.

Solution: We can add hints to guide the query execution. In the

RRKMULTIPROVHINT table, you can specify one or several
characteristics for each MultiProvider which are then used to
partition the MultiProvider into BasicCubes.

If a query has restrictions on this characteristic, the OLAP processor is

already checked to see which part cubes can return data for the
query. The data manager can then completely ignore the
remaining cubes.

An entry in RRKMULTIPROVHINT only makes sense if a few attributes of this

characteristic (that is, only a few data slices) are affected in the majority of, or
the most important, queries (SAP Notes: 911939. See also: 954889 and 1156681). 7
Tip 2: The Secret about MultiProviders & Parallel
Processing
To avoid an overflow of the memory, parallel processing is
cancelled as soon as the collected result contains 30,000 rows or
more and there is at least one incomplete sub process

The MultiProvider query is then restarted automatically and processed

sequentially
What appears to be parallel processing corresponds to sequential
processing plus the preceding phase of parallel processing up to the
termination

Generally, its recommended that you keep the number of

InfoProviders of a MultiProvider to no more than 10. However,
even at 4-5 large InfoProviders you may experience performance
degradation.

8
MultiProviders and Parallel Processing (cont.)
Consider deactivating parallel processing for those queries that
are MultiProvider queries and have large result sets (and hints
cannot be used).

With SAP BW 3.0B SP14 (SAP BW 3.1 SP8 and later versions, you can
change the default value of 30,000 rows refer to SAP Notes 629541,
622841, 607164, and 630500.

A larger number of base InfoProviders is likely to result in a

scenario where there are many more base InfoProviders than
available dialog processes, resulting in limited parallel
processing and many pipelined sub-queries

You can also change the number of dialogs (increase the use of parallel
processing) in RSADMIN by changing the settings for QUERY_MAX_WP_DIAG.
9
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs

E arlyWatch Reports
Wrap-up

10
 Aggregates

Aggregates are much less used by the SAP installation base than
training and common sense should dictate.
The interface to build the summary tables (aggregates) are
intuitive and easy to master, but few are taking real advantage of
them.
Even among those that are using aggregates, many have poorly
defined solutions & seldom monitor the usage, thereby limiting the
benefits of this simple technology.

To avoid poor definition and usage, aggregates should

be developed after the system has been in production
for a while and real user statistics are captured.
11
Tip 3: Building aggregates is easy Propose from
statistics
This example shows how to
build aggregates by using
system statistics to
generate proposals
Note: To make this work, the BW
statistics must be captured.

Select the run time of queries

to be analyzed (e.g., 20 sec)
Select time period to
be analyzed
Only those queries executed in
this time period will be reviewed
to create the proposal
12
Correct Aggregates Are Easy to Build Propose from
Query
We can also create proposals
from the Query user statistics.

To make this work, a

representative number of
queries must be executed to
gather the statistics to
optimize from.

We can also create

proposals for aggregates
based on individual queries
that are performing poorly.

13
 Tip 4: Reduce the number of overlapping Proposals
We reduce the overlapping proposals
by optimizing them.
This may reduce the proposals from
99 to less than a dozen

High valuation and high usage is what we are looking for. This indicates high
reduction
of records in aggregate and high benefits to users .
When using 3 rd party query tools and ODBC to query directly into .
the DSO, you are bypassing the OLAP Processor. Therefore, you
cannot accurately performance tune the system using aggregates
(statistics), nor will the 3 rd party tool benefit from aggregates.
14
 Activate the aggregate

The process of turning 'on' the

aggregates is simple
Fill aggregate with summary data
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs

E arlyWatch Reports
Wrap-up

17
 Tip 5: Use the Right Read Mode for Queries

Select the right read mode. Three query read modes in

BW determine the amount of data to be fetched from a
database:

1. Read all data (all data is read from a database and stored in user
memory space)

2. Read data during navigation (data is read from a database only

on demand during navigation)

3. Read data during navigation and when expanding the hierarchy

Reading data during navigation minimizes the impact on

the application server resources because only data that
the user requires will be retrieved.
Source: Catherine Roze, MyITgroup
18
Tip 6: Query read mode for large hierarchies NEW

For queries involving large hierarchies with many nodes, it

would be wise to select Read data during navigation and when
expanding the hierarchy option to avoid reading data for the
hierarchy nodes that are not expanded.

Reserve the Read all data mode for special queriesfor

instance, when a majority of the users need a given query to
slice and dice against all dimensions, or when the data is
needed for data mining. This mode places heavy demand on
database & memory resources and might impact other SAP BW Source: Catherine Roze, MyITgroup

processes and tasks.

A query read mode can be defined either on an individual query

basis or as a default for new queries using the query monitor
(transaction RSRT).
Run Program ANALYZE_RSZ_TABLES to check and fix
inconsistencies within query definition, please see note:
https://service.sap.com/sap/support/notes/792779 19
Tip 7: Condition & Exceptions

Minimize conditions-and-exceptions reporting. Conditions

&
exceptions are usually processed by the SAP application
server. This generates additional data transfer between
database and application servers.

If conditions and exceptions have to be used, the amount of data to

be processed should be minimized with filters. When multiple drill-
downs are required, separate the drill-down steps by using free
characteristics rather than rows and columns.

This strategy results in a smaller initial result set, and therefore

faster query processing and data transport as compared to a query
where all characteristics are in rows.
This strategy does not reduce the query result set. It just
separates the drill-down steps. In addition to accelerating query
processing, it provides the user more manageable portions of
20
data. Source: Catherine Roze, MyITgroup
 Some Performance settings for Query Execution
This decides how many records are read
during navigation .
New in 7.0 BI:
OLAP Engine can
read deltas into the
Examine the cache. Does not
request status invalidate existing
when reading query cache.
the InfoProvider
Turn off/on
parallel
processing
When will the
query program be
regenerated
based on
database
Displays the level of statistics
statistics collected.
21
Tip 8: Filters

Leverage filters as much as possible. Using filters contributes to

reducing the number of database reads and the size of the result
set, thereby significantly improving query runtimes.

Filters are especially valuable when associated with big

dimensions where there is a large number of characteristics such
as customers and document numbers.

IfIf large
large reports
reports have
have toto be
be produced,
produced, leverage
leverage the
the
BEx
BEx Broadcaster
Broadcaster toto generate
generate batch
batch reports
reports and
and pre-
pre-
deliver
deliver them
them each
each morning
morning toto their
their email,
email, PDF
PDF oror
printer.
printer. 22
Tip 8: Use RSRT Transaction to examine slow queries

P1 of 3

23
Look for patterns and see the performance details

P2 of 3

In this real case, aggregates

was needed for those cubes
flagged

24
Real Example: This system has issues with the Oracle DB

P3 of 3

Work with the basis

team to research the
settings and the Oracle
issues. Focus on SAP
notes and the index
issue.

The RSRT and RSRV

codes are a gold mine
for debugging and
analyzing slow queries. 25
Look at the query details for each slow query

Notice
Notice the
the yellow
yellow flag
flag for
for the
the 66 base
base
cubes
cubes inin the
the MultiProvider
MultiProvider and and the
the
yellow
yellow flag
flag for
for the
the 14
14 free
free chars.
chars.
(Note:
(Note:no
nohints
hintswere
wereused
usedininthis
this
MultiProvider,
MultiProvider,which
whichled
ledtotovery
verypoor
poor
performance).
performance).

You can also trace the front-end data

transfers and OLAP performance by using
RSTT in SAP 7.0 BI (RSRTRACE in BW 3.5)
26
 Tip 9: Use the BEx Broadcaster to Pre-Fill the Cache

Distribution Types

You
You can
can increase
increase query
query speed
speed by by
broadcasting
broadcasting the
the query
query result
result ofof commonly
commonly
used
used queries
queries toto the
the cache.
cache.
Users
Users do
do not
not need
need toto execute
execute the
the query
query from
from
the
the database.
database. Instead
Instead thethe result
result isis already
already inin
the
the system
system memory
memory (much
(much faster).
faster).
27
 Tip 10: Debugging Queries - RSRT

Here you can execute the query

and see each breakpoint,
thereby debugging the query
and see where the execution is
slow.

Worth a try: Try running slow

queries in debug mode with
parallel processing deactivated
to see if they run faster.. 28
 Tip 11: Upgrade to AS-Java service pack 14 asap. NEW
1. In service pack 14 we find several performance improvements
including:
Better Java execution and performance
Increased OLAP cache abilities (Enhanced Cluster table -BLOB)

2. In 7.0 BI at all service packs upto number 14, it is also

impossible to populate the OLAP cache by broadcasting query
views. If you use earlier service packs, you may be forced to
create many different queries to provide this performance.
3. Use quicksizer and OSS note 927530 for BI JAVA sizing.

The implementation of service pack 14 is highly

recommended by SAP for these performance reasons.
When implemented the Java execution will also
29
improve .
A Real Example

This company saw a

39% decrease in Query
execution time after
implementing SP-14.

They had 38 cockpits

and 82 queries that
improved substantially
without any further
changes..

30
 NEW
Tip 12: Restrictive Key Figures & Line Item Dimensions
1. When Restrictive Key Figures (RKF) are included in a query, conditioning is done
for each of them during query execution. This is very time consuming and a high
number of RKFs can seriously hurt query performance

Recommendation: Reduce RKFs in the query to as few as possible. Also, define

calculated & RKFs on the Infoprovider level instead of locally within the query.
Why?:
Good: Formulas within an Infoprovider are returned at runtime and held in
cache.
Bad: Local formulas and selections are calculated with each navigation step.

2. Line item dimensions are basically fields that are transaction oriented and
therefore, once flagged as a line item dimension, is actually stored in the fact
table. This results in faster query access (no table join).

Explore the use line item dimensions for fields

that are frequently conditioned in queries. 31
 Tip 13: Reducing the Query processing time NEW

Problem: Calculated Key Figures (CKF) are computed

during run-time, and a many CKFs can slow down the
query performance.

Solution: Many of the CKF can be done during data loads & physically
stored in the InfoProvider. This reduces the number of computations and
the query can use simple table reads instead. Do not use total rows
when not required (this require additional processing on the OLAP side).

Problem: Sorting the data in reports with large result sets can be time
consuming.
Solution: Reducing the number of sorts in the default view can improve
the report execution & provide the users with data faster.

PS! Reducing the text in query will

also speed up the processing some. 32
 Tip 14: Make your web templates Smaller & Faster NEW

Web templates in SAP BI can become

really large. Since they contain both
scripts and Cascading Stylesheets (CSS),
the code can become large..

To reduce the CSS, you can try several

compression tools that may help you limit
the overall size of your web templates.

There are no lack of free tools available,

and the quality varies. Therefore you must
remember to test, test and test.

PS! Web Applications run significantly faster

compared to BEx Analyzer reports in WAN
environments

Compression tools for CSS and Java scripts can reduce the overall web
template size. If you have thousands of users, this can be a life saver 33
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs EarlyWatch Reports

Wrap-up

34
 Tip 15: Is the Memory Cache Is Set Too Low?

Cache has a system default of 100 MB for local and 200 MB for global
cache. This may be too low for a system that can be optimized via
broadcaster .
Review the settings with the
Basis team and look at the
available hardware.

Use the transaction code

RSCUSTV14 in SAP NetWeaver
BI to increase the cache. Focus
particularly on the global cache.

The Cache is not used when a query contains a virtual key

figure or virtual characteristics, or when the query is
accessing a transactional DSO, or a virtual InfoProvider
35
 Tip 15: Monitor and adjust Cache Size

To monitor the usage of the cache, use transaction code RSRCACHE

and also periodically review the analysis of load distribution using
ST03N Expert Mode

The size of OLAP Cache is physically limited by the amount

of memory set in system parameter rsdb/esm/buffersize_kb.
The settings are available in RSPFPAR and RZ11.
Source: V. Rudnytskiy, 2008

36
 Tip 16: The Right OLAP Cache Persistence Settings

CACHE OLAP Persistence settings

Note When What t-code

Change the logical file

BW_OLAP_CACHE when
installing the system (not
Default Flatfile valid name) FILE

RSR_CACHE_DBS_IX
Optional Cluster table Medium and small result sets RSR_CACHE_DB_IX

Binary Large Objects RSR_CACHE_DBS_BL

Optional (blob) Best for large result sets RSR_CACHE_DB_BL

No central cache directory or

Blob/Cluster lock concept (enqueue). The Set
Enhanced (new in mode is not available by RSR_CACHE_ACTIVATE_NEW
SP 14 SAP 7.0 BI) default. RSADMIN VALUE=x
Source: SAP AG 2008.

37
Monitor Memory Usage Do you need more?

Roll memory was never maxed out in

the period 12/23/07 through 1/27/08

Paging memory was never maxed out

in the period 12/23/07 through 1/27/08

Extended memory was never maxed out in

the period 12/23/07 through 1/27/08

Only 3GB of 9 GB of Heap memory was ever

used in the period 12/23/07 through 1/27/08

38
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs

E arlyWatch Reports
Wrap-up

39
 Tip 17: Avoid Outdated Indexes and Database statistics

Database statistics are used by the optimizer to route queries.

Outdated statistics leads to performance degradation. Outdated
indexes can lead to very poor search performance in all queries where
conditioning is used (i.e. mandatory prompts).

e
l e xampl
Rea

For high volume Infocubes, or cubes that have a high number of users, the
percentage used to build the DB stats can be increased from the default
10% to 20%. This may yield more accurate query routing and better query
performance (consider this especially for cubes with old data partitioned)
 Tip 18: Avoid replicating the transaction system in SAP
BI
It is tempting to load cross-reference tables and do lookups inside SAP BI instead
of extending extractors. This creates DSOs that cannot be queried efficiently
without many table joins. In this example, of all DSOs contains less than 9
fields, & six have less than 4.

exa mple Programs that can help you

Real
monitor the system design:

1.SAP_ANALYZE_ALL_INFOCUBES
2.ANALYZE_RSZ_TABLES
3.SAP_INFOCUBE_DESIGNS

As much logic as possible should be moved to the extraction,

and needed data fields should be denormalized and stored in
logically organized ODSs and Infocubes.
 InfoCube Design & Indexes
When you flag a dimension as high cardinality SAP BI will
use a b-tree index instead of a bit-map index.

This can be substantially slower if the high cardinality does

not exist in the data in general (star-joins cannot be used with
b-trees).

exa mple
Real

Validate the high-cardinality of the data and reset the flag if

needed this will give a better index type and performance
42
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs

EarlyWatch Reports
Wrap-up

43
 TIP 19: Use BI Accelerator ASAP
SAP Any
BW tool

The SAP BI Accelerator

makes query response
time 50-10,000 faster.

You use process chains to

maintain the HPA engine
after each data load

HP, Sun and IBM have standard solutions ranging

from $32K to $250K+ that can be installed and
tested in as little as 2-4 weeks (+ SAP license fees)
Breaking news: 32 Gb Blades are
now certified by SAP (July 2008) 44
 How does SAP BIA Work?

Currently, the BIA performs aggregation and data selection

for the query, all other processing is done by the OLAP
analytical engine. (this means that 99% of the previous
recommendations in this session still holds true)

SAP BIA is not used when the result set exceeds 3 million records
(max. default). When the result set is less, the data is sent
as one large data package to the application server (need
fast network).

In the next SAP NetWeaver release the BIA will handle more of the
analytics processing such as top-5 products sales which
is currently done in the OLAP analytical engine.

You get BIA sizing estimates by running the

SAP program available in SAP Note: 917803
45
Performance Benchmarks for BIA
BIAs strength resides in
its near-linear scalability.

Performance is measured
in terms of:
1.BIA index creation time
2.Multi-user throughput per hr.
3.Average report response time
4.Average number of records
touched by each report.

BIA Currently reads data from

InfoCubes. DSOs & InfoObjects
are still read from
base/physical tables (even
when the InfoObject is indexed
as part of master data). 46
A Real BI-A Client Example from 2008

The BIA should be sized for critical applications. Most companies use
BIA only for Production, while others have a complete landscape 47
 BIA is becoming mainstream
Some of SAP reference clients

BIA is no longer
something exotic.

Many of the large BI

systems have already
implemented BIA and
many more projects are
under way in Europe
and in the Americas.

Once you exceed a few hundred critical users and/or 3-4 Tb of

data you should seriously consider SAP BIA
48
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs
EarlyWatch Reports

Wrap-up

49
 Tip 20: SAP Solutions Manager - EarlyWatch Reports Are Great!

EarlyWatch reports provide a

simple way to confirm how your
system is running and to catch
problems
A goldmine for system
recommendations
Run them periodically & read the
details
This is a real EarlyWatch report
from a mid-sized company that
has been running SAP BW for
the last four years
On
Onaalarge
largeglobal
globalproject,
project,system
system
issues
issues can be hard to pin-downwithout
can be hard to pin-down without
access to EarlyWatch reports.
access to EarlyWatch reports. TheThe
monitoring
monitoringreports
reportsallows
allowsyou
youtototune
tune
the
thesystem
systembefore
beforethe
theuser
usercommunity
community
gets
getsaccess
accessand andcomplaints
complaintsarise.
arise. 50
EarlyWatch Performance Info

InInaa24-hour
24-houroperational
operationalsystems
systems
due
due to time-zones, you willhave
to time-zones, you will have
less
lesstime
timetotoreact
reactand
andfix
fixissues.
issues.
Therefore,
Therefore,early
earlydetection
detectionofof
system
systemissues
issuesare
arecritical
criticaltotothe
the
success of a global project.
success of a global project.

51
 EarlyWatch Reports Finds Oracle fixes
In this real example, we can the EarlyWatch report identified that the
system was several Oracle notes are behind that needed to be applied
to optimize DB performance.

Before this was done, this system took 24 to 26 minutes to execute

some queries.

e
l e xampl
Rea

52
EarlyWatch Reports Finds Backup Problems

In this real
example, the
EarlyWatch report
identified that there
were no valid
backups for almost
one month.
e
l exampl
Rea
53
 What Well Cover

Introduction
Performance Issues & Tips
MultiProviders and Partitioning
Aggregates
Query Design & Caching
Hardware & Servers
Designing for Performance
InfoCubes and DSOs
BI- Accelerator
Sizing and Implementation
Management and Costs
EarlyWatch Reports

Wrap-up

54
 7 Key Points to Take Home
Use best practices for query design before you start massive hardware
performance tuning efforts.

Plan for growth what is the plan when you have 200,500, 1000+ users?

Start with aggregates (poor mans BIA), thereafter go with caching.

Monitor the system usage- do you need more app servers, memory, HW?

Check database statistics and indexes and keep them up to date.

If you are building an Enterprise Data Warehouse, plan and budget for a
BIA installation.

EarlyWatch reports are a tool to live (and die) by. Use the report before
you have performance issues.

55
 Resources

Dr. Bjarne Berg's web page -- 75+ presentations, tutorials & articles http://csc-
studentweb.lrc.edu/swp/Berg/BB_index_main.htm

SAP SDN Community web page for Business Intelligence Performance Tuning
https://www.sdn.sap.com/irj/sdn/bi-performance-tuning

ASUG407 - SAP BW Query Performance Tuning with Aggregates by Ron Silberstein

(requires SDN or Marketplace log-on). 54 min movie.
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/media/uuid/d9fd84ad-0701-
0010-d9a5-ba726caa585d

Large scale testing of SAP BI Accelerator on a NetWeaver Platform

https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/b00e7bb5-3add-
2a10-3890-e8582df5c70f
56
Your Turn!

How to contact me:

Dr. Bjarne Berg
bberg@myitgroup.com
57