How to Turn New Recruits Into

Oracle EPM Infrastructure Gurus

Session ID#: 13888

Prepared by:
Nicholas King

REMINDER
Check in on the
COLLABORATE mobile app
Topics

Level 1 Beginner
What is Oracle EPM?
Whats so Tough about EPM?
EPM Modules
Level 2 - Intermediate
Monitoring
Patching
SSL
Level 3 Guru
EPM Technology Stack
Troubleshooting
Level 1 - Beginner
------------------------------------------------------------------

-What is Oracle EPM Infrastructure?

-Whats so Tough about EPM?
Why is Infrastructure Support Critical?

Financial close is so critical that companies will spend

millions of dollars to streamline the process

Software issues during close can be catastrophic

Oracle EPM is a best-of-breed product representing some

of the most advanced tools in Finance
How to Become Best-of-Breed

Key to Understanding EPM Architecture

Acquisitions:
1998 - Hyperion Software merges with Arbor and the combined
company is renamed Hyperion Solutions (Essbase)
1999 - Hyperion acquires Sapling Corporation (Strategic
Finance)
2003 - BI Consolidation continues as Hyperion acquiers Brio
(Reporting)
2006 - Hyperion acquires UpStream (FDM)
2007 - Oracle buys Hyperion
How to Become Best-of-Breed
Implications of Acquisitions

How do you integrate millions of lines of code from many

different products?
When will work on Linux?
What else?
Whats so Tough?
Whats so Tough?
Oracle EPM is:
- a conglomeration of many custom modules
- proprietary and learned only from experience

Helpful Background Absolutely Required

Coding Java, .NET, Time
Scripting Patience
OS Administration Dedication to Learn
SSL Proactively
Web / Application Server
High Availability
Troubleshooting
Beginners Homework

Become familiar with EPM modules

Execute basic validation procedures
Install and configure EPM test environments
Internalize the Oracle documentation Read up!
http://docs.oracle.com/cd/E17236_01/index.htm
EPM Module Admin Guides
EPM Module User Guides
Infrastructure:
Support Matrix
Installation and Configuration
High Availability and Disaster Recovery
Security and SSL
Level 2 - Intermediate
------------------------------------------------------------------

-Monitoring
-Patching
-SSL
Monitoring and Alerting

Keeping track of the modules

Alert when modules are down
Good monitoring identifies problems without having to
perform intensive manual effort
Dashboard Known good state
Types of Monitoring

Passive Monitoring
Servers up and running
EPM modules up and running
Tip: Use OHS access_log to find specific URLs
https://server/hfm/GlobalWorkspaceNav/bpm/conf/HfmConfig.xml
https://server:8543/calcmgr/conf/CalcMgrConfig.xml
https://server/hyperion-bpma-server/Sessions.asmx
https://server/awb/conf/AwbConfig.xml
https://server:28443/interop/index.jsp
https://server:10083/easconsole/console.html

Active Monitoring
Load Runner simulate user activity
Patching Why Patch?

Patches are NOT included with the installation media

Business needs and stability
Patches come out frequently
Proactively Patch
A typical install has about 20 separate patches to apply after
installation
Patching Process
Search Dependencies Plan

Search for new patches for each module

What other patches are required dependencies?

Detailed Implementation plan repeatable, concise

Searching for Patches
Search Dependencies Plan

http://support.oracle.com
Myriad of HFM Patches
Search Dependencies Plan

PSU 300

PSU 301

PSU 302
Patch Plan Review Readme
Search Dependencies Plan

Cumulative or need
specific base
version, like .300
first?
Prerequisite patches
for other Modules
Patch Plan Review Readme
Search Dependencies Plan

For Each Patch Identified

Full Opatch command
Post-install steps
When to redeploy webapp? If patch has EAR/WAR
What nodes should the patch be applied
Are there any client patches or adapters? Fr studio.. FDM..
If redeploying reset java heap sizes
 Sample Plan
Search Dependencies Plan
--- Requirement: ADF Patch (hosts: Web1, Web2, Plan1, Plan2):
opatch.bat apply D:\Oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\16763038 -oh
D:\Oracle\Middleware\oracle_common -jre D:\Oracle\Middleware\jdk160_29

--- Requirement: CalcMgr Patch .303 (hosts: Web1, Web2)

requires - Oracle Hyperion Workspace 11.1.2.2.300 (EPM_PSU13936087_IE9) (ok)
opatch.bat apply D:\oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\16384766 -oh
D:\oracle\Middleware\EPMSystem11R1 -jdk D:\oracle\Middleware\jdk160_29

delete <MIDDLEWARE_HOME>/user_projects/domains/<DOMAIN_NAME>/servers/<CalcMgr0>/tmp/
Redeploy CalcMgr

--- Planning (hosts: Web1, Web2, Plan1, Plan2)

opatch.bat apply D:\oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\17413112 -oh
D:\oracle\Middleware\EPMSystem11R1 -jdk D:\oracle\Middleware\jdk160_29

del <MIDDLEWARE_HOME>/user_projects/domains/EPMSystem/servers/<server>
Web Hosts Only:
copy /Y \oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\17413112\files\common\ADM\Planning\11.1.2.0\lib\HspJS.jar
\oracle\Middleware\EPMSystem11R1\common\ADM\Planning\11.1.2.0\lib
copy /Y \oracle\Middleware\EPMSystem11R1\OPatch\2013Oct\17413112\files\common\ADM\Planning\11.1.2.0\lib\HspJS.jar
\oracle\Middleware\EPMSystem11R1\common\ADM-64\Planning\11.1.2.0\lib

Redeploy Planning

Reset Heap sizes for CalcMgr and Planning

SSL Keystore / Wallet

You must know when your SSL certificates expire and how to rotate them.
Understanding SSL is critical if it is enabled in the environment.

Identity/Private
Keystore / Wallet

What cert the server will serve up when URL is requested

- https://mylbname.com/workspace

Trusted
What certificates from other servers are accepted
- mydomain.com
- serv1.mydomain.com
- serv2.mydomain.com
SSL Wallet Example
Wallet
Each Layer Uses a Different SSL
Management Tool
OHS
OHS

Oracle Wallet
Manager
.sso
WebLogic IIS

Admin Server MMC Certificates

Trusted SSL
Console Inetmgr Site
.jks Bindings
Java .p12

Keytool
.jks
SSL Implementation

Installing and Configuring Oracle Hyperion Enterprise

Performance Management System 11.1.2 with SSL Enabled
on All Layers
http://www.oracle.com/technetwork/middleware/performance-
management/tutorials/index-087654.html
Review each technology stacks SSL documentation
IIS
WebLogic
Java Keytool
OHS
Tip: WLST can be used to script WebLogic SSL components
Intermediate Homework

Become familiar with operational tasks

Patching
Perform SSL certificate rotation
Jobs - LCM exports, FDM loads, etc.
Backup and Restore procedure
Password changes admin and database accounts
Oracle Service Requests (SR) - follow tickets from beginning to
resolution
Level 3 EPM Support Guru
------------------------------------------------------------------

-EPM Technology Stack

-EPM Troubleshooting
- Approach
- Examples
Understanding The Technology Stack

OHS
Web Proxy

Application
WebLogic IIS
Server

Core Code Java .Net VB/C++

EPM Modules and the Technology Stack

WebLogic
WebLogic IIS
Web Workspace DRM
Financial Reporting HFM Web*
Application Planning EPMA Web*
Server Most other Modules FDM Web

Java
Java .NET VB/C++
.Net VB/C++
Core Code Essbase
Framework Services HFM & FDM App

* Some components also WebLogic

Understanding WebLogic

WebLogic
WebLogic
Web Workspace
Financial Reporting
Application Planning
Server Most other Modules

A WebLogic Application is
Framework for Running Java Web Applications
Java Web Applications are bundled into packages and deployed
as part of the WebLogic configuration
Applications are exposed via web pages to end users

 First Build the Application
1. Core Code (Java)
2. Java Libraries

4. EAR File

3. WAR File
 Next Deploy App in WebLogic
WebLogic Admin Server: http://wlserver:7001/console

Main Menu

Deployment Details
WebLogic in Oracle EPM
Deploy to Application Server

Deployed using WLST WebLogic Scripting Language

Each product has an EAR file on the file system
EPMSystem11R1\products\Planning\AppServer\
InstallableApps\Common\HyperionPlanning.ear
Patching modifies the ear files and requires redeploying the
application server
Distributed Env WebLogic Domain

WebLogic Domain Server Pushes Deployment Files

WebLogic to Each Node
Domain HyperionPlanning.ear
server1 HFMWeb1.ear

Planning1
server2

Planning2
server3
On WebLogic Domain node:
- must install all web products
HFMWeb - must be patched along with all products
server4
 What is OHS?

OHS is a proxy and load balancer to all EPM modules

Essentially an Oracle branded Apache Web Server

Three main configuration files
httpd.conf proxy to IIS
ssl.conf
mod_wl_ohs.conf proxy to WebLogic modules
Files are auto-generated in Configure Web Server task in
the EPM Configuration tool.
Pictorial OHS & Load Balancing
 Textual OHS & Load Balancing

httpd.conf - IIS
ProxyPass /hfm balancer://hfmCluster stickysession=ORA_EPM_hfm_ROUTE_ID nofailover=Off
timeout=7200

<Proxy balancer://hfmCluster>
Header add Set-Cookie "ORA_EPM_hfm_ROUTE_ID=hfm.%{BALANCER_WORKER_ROUTE}e;
path=/hfm;" env=BALANCER_ROUTE_CHANGED
BalancerMember https://hfm1.com:443/hfm loadfactor=1 route=0
BalancerMember https://hfm2.com:443/hfm loadfactor=1 route=1
ProxySet lbmethod=bytraffic
</Proxy>
mod_wl_ohs.conf - WebLogic
<LocationMatch ^/HyperionPlanning>
SetHandler weblogic-handler
WeblogicCluster plan2.com:8343,plan1.com:8343
WLIOTimeoutSecs 14400
Idempotent OFF
WLSocketTimeoutSecs 600
Troubleshooting Basics

Check monitoring dashboard everything up?

Reproduce the problem
Narrow down the products involved
Look through the logs
Do the messages make sense?
External Resources
Google
Oracle Knowledgebase
Oracle Forums
Oracle Support
Sifting Through Logs

100+ log files in EPM deployment

Types of logs
WebLogic Application Logs
user_projects\domains\EPMSystem\servers
EPM Application Logs
Oracle ODL logging framework
user_projects\epmsystem1\diagnostics\logs
Windows Event Viewer
HFM Error Log Viewer
FDM Logs in FDMData Folder
Troubleshooting EPM - Example

End user messages

are too generic and
misleading

Notice the
framework module
is having issues

Show Details
Troubleshooting EPM - Example

Check WebLogic Logs for Framework Module

Primary log
user_projects\domains\EPMSystem\servers\RaFramework0\logs\
RaFramework0.log
Other logs in this dir, sort by date
RaFramework_CommonClient.log
Framework.log
RaFramework_stdout_console_servlets.log
GsmNotFound
com.brio.one.services.globalservicemanager.GSMException:
GsmNotFound
Troubleshooting EPM - Example

Check Application Tier

Quick Port Test
telnet localhost 6800
Connecting To localhost...Could not open connection to the host, on port
6800: Connect failed
Primary EPM Application Log
user_projects\epmsystem1\diagnostics\logs\ReportingAnalysis\
stdout_console_RAF_AGENT_MODULE.log
[oracle.EPMRAF.stdout] Invoking Graceful shutdown...
agent.log
Caused by: com.oracle.cmc.agent.RTAgentException: Error While Loading LSM Info

Caused by: com.sqribe.transformer.DBConnectionException: Database Server error:

DBConnectionPool::get: ORA-28000: the account is locked

Guru Homework

Install and configure a new SSL, load balanced environment

Take responsibility for infrastructure issues
Troubleshoot EPM
Quickly understand interactions between products and evaluate
error messages from logs to quickly resolve issues
Become Part of the Oracle Community
Blog
Forums
Begin to train others in the art of EPM infrastructure
Thank you Gurus!
------------------------------------------------------------------
Questions?

kingn@google.com

Hyperion Error of the Day

epm-errors.blogspot.com
Please complete the session
evaluation
We appreciate your feedback and insight

You may complete the session evaluation either

on paper or online via the mobile app