Oracle Demantra Worksheet

Performance
An Oracle White Paper
October 2012
 Oracle Demantra Worksheet Performance

Introduction ....................................................................................................... 3
Worksheet Design Considerations ................................................................. 3
Worksheet Caching ........................................................................................... 4
Appserver Properties Parameters.................................................................... 8
Client Server Attributes .................................................................................... 9
Database Parameters ....................................................................................... 10
Database Health .............................................................................................. 11

Oracle Demantra Worksheet Performance Page 2

 Oracle Demantra Worksheet Performance

Introduction
Worksheets are the primary user interface for the Oracle Demantra tools. Users
use worksheets to view and edit all relevant data for planning purposes, generate
reports and conduct ad-hoc forecasting. Worksheets can be custom designed for
each implementation and individual users based on business requirements and
needs. This flexibility in the design of the worksheet allows the users to create
worksheets that may have a performance impact on the worksheets or system
overall. This white paper lists out some key points that need to be considered
before designing the worksheets to help address potential performance issues. In
addition this white paper lists some of the parameters that can be modified to
improve the performance of the worksheet

Worksheet Design Considerations

 Number of levels determines the number of combinations to be displayed

in a worksheet. Limiting this number greatly reduces the time to retrieve
these combinations from temporary or cached tables.
 In some cases levels are used more as informative – attributes of other
levels than real members (color, size…), such levels can be displayed as
series and not as real levels in the system, for more information see Levels
as series presentation (MOS Doc Id 1201977.1).
 Displaying levels in the page item section of the worksheet (i.e. as a
dropdown or members browser) reduces the time to paint the worksheet,
allowing for display of the first combination while the other combinations
are being downloaded from the tables. As levels are shifted to the display
window more information is retrieved and the time to initial display is
increased.
 Worksheet data is stored in a temporary table for non-cached worksheets
and within cached tables (CW %_{ Query_Id}_{User_Id}_%) for cached
worksheets. When using worksheets for reporting purposes with a large
number of combinations, caching should be considered.

Oracle Demantra Worksheet Performance Page 3

  When selecting levels in the worksheet wizard (either for aggregation or
for filters) the user should select in an ordered way. The order of Levels
chosen in Aggregation Level selection determines the order in which data
is aggregated (grouped in sql) and displayed in the worksheet. Make your
selections so you work your way down the hierarchy top-to-bottom. And
always make Promotion (or other General Levels) the last one selected.
Top to bottom is a typical way to view data so this selection also follows
the natural business logic.
This has no impact on how you view the different members in the UI, as
this is defined in the Layout tab.
 Although there is no seeded index on mdp_matrix with the hierarchy
levels, we recommend adding a custom index per worksheet in the
order of the selected levels (order is sensitive) - these indexes can be re-
used if you follow the same standard.
You can use (from 7.3.1) the “Index advisor” to help define the additional
indexes.
 Utilizing "Open With" functionality, instead of opening and then filtering
worksheets from the worksheet designer (or viewing the entire content of
the worksheet) is an advised way to keep the data that the worksheet
opens controlled; this also allows the user to see the relevant data for his
work without waiting for a bigger worksheet to fully load.
 If customer is using General Level worksheets that don’t include sales_data
series, set ApprovalProcessScope from 0 to 1. This will have a positive
performance impact because it eliminates the join with sales_data.

 Use “Program Permissions” to prevent novice users from creating

worksheets that don’t adhere to these recommendations; assign a select
number of users as ‘super users’ to define and maintain worksheets.
 Utilize User security to restrict ad hock the amount of data users can
review in the UI.
Create generic purpose worksheets that are not filtered in the actual
worksheet wizard, but are filtered based on the user security (and open
with functionality).
 Disabling summary lines for specific / all levels in the worksheet layout
designer can have a positive performance impact.

Worksheet Caching

 Caching enables the system to aggregate and store the subset of data that
is part of the worksheet in a separate table and hence the retrieval is much
faster compared to regular worksheets.

Oracle Demantra Worksheet Performance Page 4

 There are two types of caches – Automatic and Manual.
o Automatic caches are updated every time you display the
worksheet. This option works better for smaller worksheets, as
the time to check for updates could decrease performance for
larger worksheets.
o Manually cached worksheets should be used for large worksheets
or when using worksheets as reports, where underlying data does
not change frequently
Please refer to the Oracle Demantra User Guide and Implementation
Guide for more information on Caching.
Some guidelines on how to manage caching jobs:
Often caching jobs are scheduled as the final step in a weekend or
overnight workflow process that also includes upload of new sales data.
Since the caching workflows compete for the same resources and access
the same database tables (SALES_DATA, PROMOTION_DATA,
MDP_MATRIX) as other processes (EP_LOAD, BLE, etc.), it is
important to schedule caching workflows to start automatically upon
completion of these other tasks. Further, depending on the number of
worksheets to be cached, you may want to control the number of active
caching jobs at any one time.

One way to accomplish this is by building an ‘umbrella’ workflow process

that uses a series of workflow steps that call out to caching workflows,
choosing whether or not the individual caching steps run synchronously
or asynchronously.

In Demantra 7.3 and onwards, this can be accomplished with the standard
workflow step to call another workflow, with stored procedure step(s) to
manage parallelism. In versions before 7.3, it may be necessary to leverage
the custom step:

Oracle Demantra Worksheet Performance Page 5

where
Parameter Description
schema_id Caching workflow identifier: SCHEMA_ID from
WF_SCHEMAS
user_id User identifier: USER_ID from USER_ID
sync true – run synchronously
false – run asynchronously

An example workflow process is displayed below:

Between caching steps, there is a stored procedure step to manage the

waiting period. This procedure is used to implement logic such as the
following:
 Check if two or more caching workflows are running

Oracle Demantra Worksheet Performance Page 6

 If so, sleep for a random period of time between 3 and 4 minutes,
and then loop.

 If not, exit and proceed to next step.

A sample procedure is included below:

CREATE OR REPLACE PROCEDURE CACHE_WF_WAIT AS

/*
This procedure is called by the Wait steps of the worksheet
caching workflows. If MAX_CACHE_WFS or more instances of
workflows in the Worksheet Caching workflow group are running,
then this procedure will sleep for a random number of seconds
near WAIT_BETWEEN_RETRIES and then loop. It will exit when less
than MAX_CACHE_WFS (or no) instances are running.
*/

-- maximum number of active caching workflows

MAX_CACHE_WFS number := 2;

-- workflow group that contains caching workflows

CACHE_WF_GROUP_NAME varchar2(200) := 'Worksheet Caching';

-- number of seconds to wait before checking that a caching

workflow has completed
WAIT_BETWEEN_RETRIES number := 180;

v_num_running number;

BEGIN

v_num_running := MAX_CACHE_WFS;
WHILE (v_num_running >= MAX_CACHE_WFS) LOOP
-- get count of wfs in Worksheet Caching group that are
running
select count(*)
into v_num_running
from wf_process_log
where status not in(0,-1,-2)
and schema_id in
(select schema_id from wf_groups_schemas gs,
wf_schema_groups sg
where gs.group_id = sg.group_id
and sg.name = CACHE_WF_GROUP_NAME);

-- if MAX_CACHE_WFS or more workflows are running, then

sleep for 3-4 minutes
IF (v_num_running >= MAX_CACHE_WFS) THEN
--dbms_output.put_line('sleeping...');

dbms_lock.sleep(dbms_random.value(WAIT_BETWEEN_RETRIES,WAIT_BET
WEEN_RETRIES+60));
END IF;
END LOOP;

EXCEPTION
WHEN OTHERS THEN
null; -- TBD: log exception

END CACHE_WF_WAIT;
/
show errors procedure CACHE_WF_WAIT
exit;

Oracle Demantra Worksheet Performance Page 7

Worksheet Hints

 Some worksheets access a large amount of data which can cause

them render slowly. A can be implemented to improve performance
for such worksheets.
 Worksheet hints needs to be manually implemented via the
Worksheet_hints table.
 There are 3 drill down levels for creating worksheet hints:
o Worksheet level (all users for relevant worksheet)
o Worksheet – User level (in the case of there is a variation in
the application data security rules between users)
o Worksheet – User – Context level (in the case of open with).
 For each level one can define hints both on Meta data
(Population_Hint column) as well as on the Data itself (Data_Hint
column)
 When configuring worksheet hints, you need to take into
consideration the number of CPUs on the DB side, the system
statistics and the concurrency of relevant users on the system, this is
to avoid an overload of the Database with too many parallel
requests.

Appserver Properties Parameters

In versions prior to 7.3, the Appserver.properties file has many parameters that can
be leveraged by the administrator to control the behavior of the Demantra system.
In versions after 7.3, these parameters are maintained in the APS_PARAMS table.
The following list of parameters has an impact on the performance of worksheets.
Appserver properties can be located in the appserver machine where the
application is deployed in the following folder (.../ {application name}/ WEB-
INF\classes\com\demantra\applicationServer\services\AppServer.properties)
 threadpool.query_run.per_user: This parameter determines the number
of concurrent threads to use for querying data from tables for each user.
Number of concurrent users determines this setting. Experiment, by
increasing it in chunks of 4 to determine an optimal value. Caution should
be exercised in increasing this number; too big a number could result in a
lot of threads waiting for the database to respond if the database doesn’t
have a lot of open connections.
 threadpool.query_run.size: This is the total number of threads allocated
for the query run. This should be adjusted based on the number of
concurrent users and the threadpool.query_run.per_user. (Product of
number of concurrent users and threadpool.query_run.per_user). For
example if the threadpool.query_run.per_user is set to 4 and expected
number of concurrent users is 10, this parameter should be set to 40.

Oracle Demantra Worksheet Performance Page 8

 MaxDBConnections parameter should also be modified accordingly to
account for the increase in number of threads for this parameter.
 worksheet.data.comb.block_size: This parameter controls the block
size for worksheet data retrieval. It defaults to 0, downloading all
combinations in a single select. This setting is appropriate for crosstab
worksheets (that is, worksheets in which the levels are displayed within
the table vs. as dropdowns or within the members browser), to display all
the combinations together.
 client.worksheet.calcSummaryExpressions: This parameter defaults to
1. If true, summary line client expressions are supported. Complex
summary expressions can have a performance impact. If this is set to 0,
simple summary expressions (Sum, Max, etc.) are only supported.
 MaxDBConnections: It is important to make sure that the application
server will have all the needed threads to execute all parallel requests
(query executions, workflow runs, updates and so), a rule of thumb will be
the number of concurrent users * threadpool.query_run.per_user *10
(if threadpool.query_run.per_user > 4) .

Client Server Attributes

The following client server attributes have an impact on the worksheet
performance
 It has been observed that use of newer versions of JRE improves
performance. (1.6.x or 1.5.x demonstrates improved performance over
1.4.x)
 It is suggested that the maximum JRE heap size parameter be set greater
than 64k (default) by setting the -Xmx parameter. Setting this parameter
prevents the Java out of memory error and could increase performance.
A good starting point will be –Xmx = 256M.
All of the Client Machine’s JRE heap size parameter can be set to a
default common value by following the steps outlined below
o Set the parameter CLIENT.JREMAXMEMORY to the desired
setting in the appserver.properties file
o Once this is set up, each of the client machines JRE heap size
parameter can be adjusted to the above setting by visiting this
page from the client machine one time.
http://<web_root_url>/portal/configureEnv.jsp (web_root_url
is the root url for a given deployment)

Oracle Demantra Worksheet Performance Page 9

Database Parameters
Some Oracle database parameters have an impact on the worksheet performance.
Oracle Database parameters can be edited using the Oracle Enterprise Console
tool and/or directly in sqlplus using the ALTER SYSTEM command.
For more comprehensive and detailed documentation of DB parameters and set
up please refer to the Oracle_Demantra_DB_Best_Practices document (MOS Doc
Id 1499638.1).
This new document is more comprehensive and more up to date with the latest
DB tuning and maintenance options.
 The database must be maintained at least weekly by analyzing the schema
or rebuilding the tables and indexes. Demantra system provides the
ANALYZE_SCHEMA stored procedure for doing this in an optimal
way.
 The block size in the schema should be set to 16k – if not possible, it is
recommended that the table spaces for Demantra main tables (data tables
and mdp_matrix) will be created with a 16k block size and not the entire
DB instance, also a sufficient 16K buffer cache should be set.
 Demantra system is very sensitive to Oracle database table scans. Table
scans can be controlled by modifying the Oracle database parameters
optimizer_index_cost_adj and db_file_multiblock_read_count.
Optimizer_index_cost_adj should be set between 25 and 50; this reduces
the cost to access the indexes and should reduce the database response
time.
 If you are not using the Oracle SYSTEM_STATS, then the
db_file_multiblock_read_count (MBRC) should be set to a small number
(no larger than 8). This forces the system to use index scan. If you are
using the Oracle SYSTEM_STATS, then this initialization parameter
should be set based on the formula: (Number of Physical disk blocks *
SAN read-ahead) / Oracle block size. For example if the physical disk
blocks are 4k, the SAN does 32 blocks of read-ahead, and Oracle block
size is 16k, then you’d want to set this parameter to (4k*32 / 16k) which is
8. In addition to this initialization parameter, the MBRC parameter in the
SYSTEM_STATS should be set to a small number (4) so as to enable the
cost based optimizer (CBO) to use the index scan.
 cursor_sharing - The default value for cursor_sharing is EXACT; it
should be changed to ‘Force’ for better performance. If the DB has both
Demantra and ASCP on the same instance, it is recommended to keep
cursor_sharing = ‘EXCAT’.
 Demantra system runs better when it has more memory in the Oracle
Server Buffer Cache. This change should decrease the number of physical
I/O’s that are done and hence results in better performance. Demantra

Oracle Demantra Worksheet Performance Page 10

 recommends that at least 1 GB be dedicated to the buffer cache. This
value is, of course, dependent on the overall data volume.

Database Health
It is important to maintain the database health so that performance will not
deteriorate.
For more comprehensive and detailed documentation of DB parameters and set
up please refer to the Oracle_Demantra_DB_Best_Practices document (MOS Doc
Id 1499638.1).
This new document is more comprehensive and more up to date with the latest
DB tuning and maintenance options.
 Gather DB statistics on the main data tables (such as mdp_matrix,
sales_data, promotion_data) and indexes; after sufficient testing with good
results, statistics can be locked on the tables for the DB optimizer to use.
 Rebuild tables by PK: Larger tables in the Demantra schema (like
sales_data, promotion_data, mdp_matrix) should be rebuilt by copying
data to a new table in the order of the column sequence in the Primary
Key. Rebuild should occur when either of these conditions is true:
 Out of Order Sequence ratio is > 20%
 Chained row count > 10%
 Chained Row count: SQL below can be used to determine the %
fragmentation of tables in the Demantra schema. Please note: you should
have run analyze schema before running this sql to get an accurate result.
select table_name, chain_cnt, num_rows,
(chain_cnt/num_rows)*100 percent
from user_tables
where chain_cnt >0 and num_rows > 0
order by chain_cnt desc;

 Out of Order Sequence ratio: Performance can be compromised if

rows are not in consecutive order according to the natural primary key
order. SQL below can be used to get this ratio for a given table and key
columns in the sequence of the Primary Key columns.

select (round(((select count(*) as cnt

from (select <key columns>
,relative_fno , block_number, row_number, data_row
,(lag(data_row) over(partition by relative_fno, block_number
order by row_number)) as prev_data_row
from (select <key columns>
,relative_fno, block_number, row_number
,(dense_rank() over(partition by relative_fno,
block_number order by <key columns>)) as data_row
from (select <key columns>

Oracle Demantra Worksheet Performance Page 11

 ,dbms_rowid.rowid_relative_fno(rowid) relative_fno
,dbms_rowid.rowid_block_number(rowid) as
block_number
,dbms_rowid.rowid_row_number(rowid) as
row_number
from <table>
)c
)b
)a
where data_row != prev_data_row
and data_row != prev_data_row + 1) / (select count(*) from
<table>)),3)*100) as "out of order ratio %"
from dual;

 Chained Row count and Out of Order Sequence ratio should be run every
3 months and after any major data load that would increase data volume
by 20%. Make sure you have sufficient disk space to copy data from the
original data tables to the new copy of the tables. If your table is
partitioned you could break the copy by partitions to save on disk space.
It is also recommended (at least once after the data model is finalized and
all the series are created) to re-org the columns on the data tables by
moving empty columns to the end of the table (to minimize table
chaining).
 To determine potential savings from a reorder, do the following:
o Determine the avg_row_len, num_rows per table, for example:
 select table_name,avg_row_len,num_rows from
sys.dba_tables where table_name ='SALES_DATA' and
owner='&your_owner'
o Use the result of ‘num_rows’ above to compute the maximum
upper limit of the savings.
o Calculate the avg_savings_per_row. Let’s say, for example, the
num_rows was 1072963:
 select (sum(num_nulls) / 1072963) as
avg_saving_per_row
from dba_tab_columns
where table_name = 'SALES_DATA'
and owner = '&your_owner';
o Now compare the avg_row_len with the avg_saving_per_row to
see your maximum savings. Your real savings will be less than
this but you will be able to get a reading on the possibile
performance improvement.
 To rebuild a table create a procedure similar to this:
o Create a new table NOLOGGING.

Oracle Demantra Worksheet Performance Page 12

 o Copy the data to the new table.
o Rename the old table.
o Rename the new table.
o Recreate the indexes on the new table.
o Turn on logging.
To create the table:

CREATE TABLE SD_NEW

NOLOGGING
PARALLEL 3 -- should be the same as the old existing table
PCTFREE 30 -- should be the same as the old existing table
AS
SELECT
-- fill in this part with the results of the SQL query
-- SELECT column_name||','
-- FROM user_tab_cols
-- WHERE table_name = 'SALES_DATA'
-- ORDER BY num_nulls;
-- Edit with notepad to remove the extra ',' at the end
FROM SALES_DATA WHERE ROWNUM < 1;

To copy the data:

CREATE OR REPLACE PROCEDURE MYLOAD
AS
V_COUNT PLS_INTEGER := 0;
v_log_count pls_integer := 0;
BEGIN
FOR C1_REC IN (
select /*+ PARALLEL(sales_data, 3) */

-- fill in this part with the results of the SQL query

-- SELECT column_name||','
-- FROM user_tab_cols
-- WHERE table_name = 'SALES_DATA'
-- ORDER BY num_nulls;
-- Edit with notepad to remove the extra ',' at the end from
sales_data
)
LOOP
insert /*+ APPEND PARALLEL(sd_new, 3) */ INTO SD_NEW
VALUES (

Oracle Demantra Worksheet Performance Page 13

 -- fill in this part with the results of the SQL query
-- SELECT 'C1_REC.'||column_name||','
-- FROM user_tab_cols
-- WHERE table_name = 'SALES_DATA'
-- ORDER BY num_nulls;
-- Edit with notepad to remove the extra ',' at the end
);

-- Commit every 10,000 rows and log your progress

V_COUNT := V_COUNT + 1;
IF (V_COUNT = 10000) THEN
v_log_count := v_log_count + 1;
insert into my_sales_log values(v_log_count, SYSDATE);
COMMIT;
V_COUNT := 0;
END IF;

END LOOP;

COMMIT;

END;

 Partitioning:
o If the data table (sales_data) is big (over 50M rows) it is
recommended to consider partitioning of the table.
o The most intuitive partitioning option is by date range. Based on
the business flows and requirements, other partitioning strategies
such as a combination of range and hash sub partitioning can
also be evaluated.
o It is also important to distribute the data on the storage system
(either in different physical partitions or via ASM).

Oracle Demantra Worksheet Performance Page 14

Oracle Demantra Worksheet Performance
May 2010
Authors: Meyyappan Meyappan, Arup Chatterjee, Margie Bell, Harish Wadhwa, Amit Ben Zvi

Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.

Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200
oracle.com

Copyright © 2010, Oracle. All rights reserved.

This document is provided for information purposes only and the
contents hereof are subject to change without notice.
This document is not warranted to be error-free, nor subject to any
other warranties or conditions, whether expressed orally or implied
in law, including implied warranties and conditions of merchantability
or fitness for a particular purpose. We specifically disclaim any
liability with respect to this document and no contractual obligations
are formed either directly or indirectly by this document. This document
may not be reproduced or transmitted in any form or by any means,
electronic or mechanical, for any purpose, without our prior written permission.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates.
Other names may be trademarks of their respective owners.