Informatica Productivity Pack

Save Time and Money while Increasing the Quality of Your PowerCenter Deployment

Louis Hausle

Are these issues you have run into?

Best Practices Violations even though you have them documented!

Abuse and misuse of SQL overrides, log verbosity or commit/cache/dtm buffer size?
Zombie Workflows?

Where is Time Spent in a PowerCenter Environment? (source Informatica World 2012 Poll)
Development accounts for less than 50% of time Development 38%

Code Review and Best Practices Enforcement 10%

Monitoring of PowerCenter 21%

More Time and Money Spent on Testing and Monitoring

Data Validation and Testing 29%

You can shrink the TCO of PowerCenter by automated Testing and Monitoring

What Exactly is the Productivity Pack?

( it is a Sales Bundle of Three Products)

PowerCenter Productivity Pack

Proactive Proactive Data Monitoring for Monitoring for Validation PowerCenter PowerCenter Option - Governance - Operations

DEV

TEST

PROD

Agenda Development Proactive Monitoring for PowerCenter Governance

Testing The Informatica Data Validation Option DVO Production Proactive Monitoring for PowerCenter - Operations

Every PowerCenter Customer Has Best Practices, Nearly Every One Struggles to Enforce Them!

Challenges in Data Integration & Governance

Best Practice Enforcement
Most companies have no way to enforce best practices

Production Impact
Unchecked Development Practices and Choices can lead to Fatal Mistakes in Production

Measure and Quantify Development Work

How can you track many developers in different locations?
7

1. Enforcement of Best Practices

Customer Challenges
How do you monitor many developers in many locations, all the time? How do make your Best Practices Actionable and enforceable as they happen?

Mappings are created and saved without comments

Next Developer has no idea of what logic is doing

Time is wasted and Mistakes can be made

2. Production Impact of Development Options

Customer Challenges
There are hundreds of development flags and options on Mappings, Workflows, Sessions and Transformations

Customers can have dozens, hundreds and thousands of Workflows

A developer uses the truncate target table option for testing in development

He forgets to change the flag, and the workflow is migrated into production

Data is lost, and recovery steps can take time and introduce new data problems

Agenda Development Proactive Monitoring for PowerCenter Governance

Testing The Informatica Data Validation Option DVO Production Proactive Monitoring for PowerCenter - Operations

11

The Data Validation and Testing Dilemma

Informatica has over 4,000 PowerCenter Customers who believe using a metadata driven, graphical tool to move data is better than writing SQL and hand coding.

15 years later, 99% of those 4,000 Customers still use SQL and Excel to test the data they have moved in data integration products. There has to be a better way
12

Current Approach: Like a Photo Hunt

13

Data Validation Simple Illustration

Source A
Sales Orders 30 Sales Amount $150,000

Target (DW or Operational System Data Integration Or Movement

Source B
Sales Orders 23 Sales Amount $150,000

Sales Orders 52

Sales Amount $290,000

DVO - tells you 1 Order and $10,000 is Missing In a few clicks it does set, aggregation and value tests.
14

DVO: Test Creation

Leverage your PowerCenter Metadata!
GUI with Pre-Built Test Operators with NO SQL

15

DVO: Test Results

Summary of Test Results

16

Agenda Development Proactive Monitoring for PowerCenter Governance

Testing The Informatica Data Validation Option DVO Production Proactive Monitoring for PowerCenter - Operations

17

Informatica built these solutions to address three common customer concerns

1. Customers asked for a Complete, PowerCenter Centric Monitoring Solution

2. Customers want to become Proactive in seeing issues before they turn into problems. Send email alerts instead of logging in to a monitor.

3. Scalable way to identify anomalies such as Workflows running longer/shorter than normal or returning more/less rows than normal.

18

What is Proactive PowerCenter Monitoring?

Prebuilt Rules to Monitor
Informatica PowerCenter

Delivers Alerts to people,

dashboards and systems

Complete Operational and Best Practices Extensible build unlimited number of new rules

19

Solution Overview
Operational Monitoring via Proactive PowerCenter Monitoring Continuous Correlation (Sample Rules)
SQL Sources to Expose Repository Features and Allow Easy Rule and Report Building Environmental Monitoring Custom:
Environmental
(CPU, Memory..)

1 2 3 4

Workflow/session was successful but zero (0) records loaded Rejected Records in Session Disproportionate elapsed time between Workflow and Session Session/Workflow Failures Correlate session changes to workflow failures SLA Violations: Workflow/Folder/Repository Level Significant increase in Workflow/Session Elapsed Times

Notification Framework for Alerts

Prebuilt EMAIL Responses

5
6 7 9

Prebuilt Dashboard Alerts

Runtime Statistics JMS Receiver via Web Web Services Services Hub

DB (SQL)

Alert Enrichment

Significant decrease in applied row counts 8 DBs, Web Services and more Workflows missing schedules Alert Editing, Creation and Management Too many concurrent workflows

Ping PowerCenter Services

PowerCenter Workflow Response to Execute Commands

Operational Design Time

10
11 12

Automatic Restart of Zombie Workflows

Ping Domain/Integration Service/Repository Service

Start/Stop Workflows Create Help Desk Ticket Update Control Tables

20

Proactive Monitoring for PowerCenter Illustration of Components that are Monitored

PowerCenter

1. The PowerCenter Domain, Integration Services and Repository Services are Monitored 2. The PowerCenter Repository Database is Monitored for Up/Down and Tablespace Usage 3. The PowerCenter Server OS is Monitored for Memory and CPU Usage

Domain

Database
Administrator

Repository Service

Application Server

Integration Service

Server

4. Running Workflows and Sessions are Monitored 5. Over 500 Attributes can be used in rules

21

DEMONSTRATION of PROACTIVE MONITORING FOR GOVERNANCE

Check for Commit size violations Check On Rollback Transactions on Errors Naming Conventions and Check for Comments

22