Informatica PowerMart /

PowerCenter 7 Basics
Education Services
PC6B-20030512

Informatica Corporation, 2003. All rights reserved.

Course Objectives

At the end of this course you will:

 Understand how to use all major PowerCenter 7
components
 Be able to perform basic Repository administration tasks
 Be able to build basic ETL Mappings and Mapplets
 Be able to create, run and monitor Workflows
 Understand available options for loading target data
 Be able to troubleshoot most problems

2
Extract, Transform, and Load
Operational Systems Decision Support
Data
RDBMS Mainframe Other Warehouse

• Transaction level data Aggregate Data

Cleanse Data • Aggregated data
• Optimized for Transaction Consolidate Data • Historical
Response Time Apply Business Rules
• Current De-normalize
• Normalized or De-
Normalized data
Transform

ETL Load
Extract

3
PowerCenter Architecture

4
 PowerCenter 7 Architecture
Server
native native
Sources Targets

TCP/IP
Repository
Heterogeneous Server Heterogeneous
Targets Targets

TCP/IP Repository
Agent

native
Repository Designer Workflow Workflow Rep Server Repository
Manager Manager Monitor Administrative

Console
Not Shown: Client ODBC Connections for Source and Target metadata 5
PowerCenter 7 Components

 PowerCenter Repository
 PowerCenter Repository Server
 PowerCenter Client
• Designer
• Repository Manager
• Repository Server Administration Console
• Workflow Manager
• Workflow Monitor
 PowerCenter Server
 External Components
• Sources
• Targets

6
Repository Topics

By the end of this section you will be familiar with:

 The purpose of the Repository Server and Agent
 The Repository Server Administration Console GUI
interface
 The Repository Manager GUI interface
 Repository maintenance operations
 Security and privileges
 Object sharing, searching and locking
 Metadata Extensions
7
Repository Server
 Each Repository has an independent architecture for the
management of the physical Repository tables
 Components: one Repository Server, and a Repository
Agent for each Repository
Server

Repository
Server

Repository
Agent

Repository Repository Server

Administration Console
Manager Repository
Client overhead for Repository management is greatly
reduced by the Repository Server
8
Repository Server Features

 Manages connections to the Repository from client

applications
 Can manage multiple Repositories on different
machines on a network
 Uses one Repository Agent process to insert, update
and fetch objects from the Repository database tables,
for each Repository it manages
 Maintains object consistency by controlling object
locking

The Repository Server runs on the same system running the Repository Agent 9
 Repository Server Administration Console

Use Repository Administration console to Administer Repository Servers and

Repositories through Repository Server. Following tasks can be performed:

 Add, Edit and Remove Repository Configurations

 Export and Import Repository Configurations
 Create a Repository
 *Promote a local Repository to a Global Repository
 Copy a Repository
 Delete a Repository from the Database
 Backup and Restore a Repository
 Start, Stop, enable and Disable a Repositories
 View Repository connections and locks
 Close Repository connections.
 Upgrade a Repository

10
 Repository Server Administration Console

Information HTML View

Nodes

Console Tree
Hypertext Links to
Repository
Maintenance Tasks

11
 Repository Management
 Perform all Repository
maintenance tasks through
Repository Server from the
Repository Server Administration
Console
 Create the Repository
Configuration
 Select Repository Configuration
and perform maintenance tasks:

• Create • Notify
• Delete Users
• Backup • Propagate
• Copy from • Register
• Disable • Restore
• Export Connection • Un-Register
• Make Global • Upgrade
12
 Repository Manager

Use Repository manager to navigate through multiple folders and

repositories. Perform following tasks:

 Manage the Repository

• Launch Repository Server Administration Console for this purpose
 Implement Repository Security
• Managing Users and Users Groups
 Perform folder functions
• Create, Edit, Copy and Delete folders
 View Metadata
• Analyze Source, Target, Mappings and Shortcut dependencies.

13
Repository Manager Interface

Navigator
Window

Main Window

Dependency Window

Output Window

14
Users, Groups and Repository Privileges
Steps:

 Create groups
 Create users
 Assign users to
groups
 Assign privileges to
groups
 Assign additional
privileges to users
(optional)

15
Managing Privileges

Check box assignment of privileges

16
Folder Permissions

 Assign one user as the

folder owner for first tier
permissions

 Select one of the owner’s

groups for second tier
permissions

 All users and groups in

the Repository will be
assigned the third tier
permissions
17
Object Locking
 Object Locks preserve Repository integrity
 Use the Edit menu for Viewing Locks and Unlocking
Objects

18
Object Searching
(Menu- Analyze – Search)

 Keyword search
• Limited to keywords
previously defined in
the Repository
(via Warehouse
Designer)

 Search all
• Filter and search
objects

19
Object Sharing
 Reuse existing objects
 Enforces consistency
 Decreases development time
 Share objects by using copies and shortcuts
COPY SHORTCUT
Copy object to another folder Link to an object in another folder
Changes to original object not captured Dynamically reflects changes to original
object
Duplicates space Preserves space
Copy from shared or unshared folder Created from a shared folder

Required security settings for sharing objects:

• Repository Privilege: Use Designer
• Originating Folder Permission: Read
• Destination Folder Permissions: Read/Write
20
Adding Metadata Extensions

 Allows developers and partners to extend the metadata

stored in the Repository
 Accommodates the following metadata types:
• Vendor-defined - Third-party application vendor-created
metadata lists
• For example, Applications such as Ariba or PowerConnect for
Siebel can add information such as contacts, version, etc.
• User-defined - PowerCenter/PowerMart users can define
and create their own metadata
 Must have Administrator Repository or Super User
Repository privileges

21
 Sample Metadata Extensions

Sample User Defined

Metadata, e.g. - contact
information, business user

Reusable Metadata Extensions can also be created in the Repository Manager

22
Design Process

1. Create Source definition(s)

2. Create Target definition(s)
3. Create a Mapping
4. Create a Session Task
5. Create a Workflow from Task components
6. Run the Workflow
7. Monitor the Workflow and verify the results

23
Source Object Definitions

By the end of this section you will:

 Be familiar with the Designer GUI interface
 Be familiar with Source Types
 Be able to create Source Definitions
 Understand Source Definition properties
 Be able to use the Data Preview option

24
 Source Analyzer

Designer Tools

Analyzer Window

Navigation
Window

25
Methods of Analyzing Sources

Repository
 Import from Database
 Import from File
 Import from Cobol File
 Import from XML file
 Create manually Source
Analyzer

Relational XML file Flat file COBOL file

26
 Analyzing Relational Sources
Source Analyzer Relational Source
ODBC Table
View
Synonym
DEF

Repository
Server

TCP/IP
Repository
Agent
native

Repository
DEF
27
Analyzing Relational Sources
Editing Source Definition Properties

28
Analyzing Flat File Sources
Source Analyzer
Mapped Drive Flat File
NFS Mount
Local Directory DEF
Fixed Width or
Delimited

Repository
Server

TCP/IP
Repository
Agent
native

Repository
DEF
29
 Flat File Wizard
 Three-step
wizard
 Columns can
be renamed
within wizard
 Text, Numeric
and Datetime
datatypes are
supported
 Wizard
‘guesses’
datatype
30
XML Source Analysis
Source Analyzer Mapped Drive
.DTD File
NFS Mounting
Local Directory DEF

DATA
Repository
Server

TCP/IP
Repository
Agent In addition to the DTD file, an
XML Schema or XML file
native can be used as a Source
Definition
Repository
DEF
31
Analyzing VSAM Sources
Source Analyzer .CBL File
Mapped Drive
NFS Mounting
DEF
Local Directory

Repository DATA
Server

TCP/IP
Repository
Agent Supported Numeric Storage Options:
COMP, COMP-3, COMP-6
native

Repository
DEF
32
VSAM Source Properties

33
Target Object Definitions

By the end of this section you will:

 Be familiar with Target Definition types
 Know the supported methods of creating Target
Definitions
 Understand individual Target Definition properties

34
Creating Target Definitions

Methods of creating Target Definitions

 Import from Database
 Import from an XML file
 Manual Creation
 Automatic Creation

35
 Automatic Target Creation

Drag-and-
drop a
Source
Definition
into
the
Warehouse
Designer
Workspace

36
Import Definition from Database
Can “Reverse engineer” existing object definitions
from a database system catalog or data dictionary

Warehouse
Database
Designer
ODBC

Table
Repository View
Server DEF Synonym

TCP/IP Repository
Agent

native
Repository DEF
37
 Manual Target Creation
1. Create empty definition 2. Add desired columns

3. Finished target definition

ALT-F can also be used to create a new column

38
Target Definition Properties

39
Target Definition Properties

40
Creating Physical Tables

DEF

DEF

DEF Execute SQL

via
Designer

LOGICAL PHYSICAL
Repository target table Target database
definitions tables

41
Creating Physical Tables
Create tables that do not already exist in target database
 Connect - connect to the target database
 Generate SQL file - create DDL in a script file
 Edit SQL file - modify DDL script as needed
 Execute SQL file - create physical tables in target database

Use Preview Data to verify

the results (right mouse
click on object)
42
Transformation Concepts

By the end of this section you will be familiar with:

 Transformation types and views
 Transformation calculation error treatment
 Null data treatment
 Informatica data types
 Expression transformation
 Expression Editor
 Informatica Functions
 Expression validation
43
Transformation Types
Informatica PowerCenter 7 provides 23 objects for data
transformation

 Aggregator: performs aggregate calculations

 Application Source Qualifier: reads Application object sources as ERP
 Custom: Calls a procedure in shared library or DLL
 Expression: performs row-level calculations
 External Procedure (TX): calls compiled code for each row
 Filter: drops rows conditionally
 Joiner: joins heterogeneous sources
 Lookup: looks up values and passes them to other objects
 Normalizer: reorganizes records from VSAM, Relational and Flat File
 Rank: limits records to the top or bottom of a range
 Input: Defines mapplet input rows. Available in Mapplet designer
 Output: Defines mapplet output rows. Available in Mapplet designer

44
Transformation Types

Router: splits rows conditionally


Sequence Generator: generates unique ID values

Sorter: sorts data

Source Qualifier: reads data from Flat File and Relational Sources

Stored Procedure: calls a database stored procedure

Transaction Control: Defines Commit and Rollback transactions

Union: Merges data from different databases

Update Strategy: tags rows for insert, update, delete, reject

XML Generator: Reads data from one or more Input ports and outputs XML through single output port

XML Parser: Reads XML from one or more Input ports and outputs data through single output port

XML Source Qualifier: reads XML data


45
 Transformation Views

A transformation has
three views:
 Iconized - shows the
transformation in
relation to the rest of
the mapping
 Normal - shows the
flow of data through
the transformation
 Edit - shows
transformation ports
and properties; allows
editing
46
 Edit Mode
Allows users with folder “write” permissions to change
or create transformation ports and properties
Define transformation
Define port level handling
level properties

Enter comments
Make reusable

Switch
between
transformations

47
Expression Transformation

Perform calculations using non-aggregate functions

(row level)

Passive Transformation
Connected

Ports
• Mixed
• Variables allowed
Click here to invoke the
Expression Editor
Create expression in an
output or variable port

Usage
• Perform majority of
data manipulation
48
 Expression Editor
 An expression formula is a calculation or conditional statement
 Used in Expression, Aggregator, Rank, Filter, Router, Update Strategy
 Performs calculation based on ports, functions, operators, variables,
literals, constants and return values from other transformations

49
Informatica Functions - Samples

ASCII
CHR Character Functions
CHRCODE
CONCAT
 Used to manipulate character data
INITCAP
INSTR  CHRCODE returns the numeric value
LENGTH (ASCII or Unicode) of the first character
LOWER of the string passed to this function
LPAD
LTRIM
RPAD
RTRIM
SUBSTR For backwards compatibility only - use || instead
UPPER
REPLACESTR
REPLACECHR
50
Informatica Functions

TO_CHAR (numeric) Conversion Functions

TO_DATE
TO_DECIMAL  Used to convert datatypes
TO_FLOAT
TO_INTEGER
TO_NUMBER

ADD_TO_DATE
Date Functions
DATE_COMPARE  Used to round, truncate, or compare
DATE_DIFF
GET_DATE_PART
dates; extract one part of a date; or
LAST_DAY
perform arithmetic on a date
ROUND (date)  To pass a string to a date function,
SET_DATE_PART
first use the TO_DATE function to
TO_CHAR (date)
TRUNC (date)
convert it to an date/time datatype

51
Informatica Functions
Numerical Functions
ABS
CEIL  Used to perform mathematical
CUME operations on numeric data
EXP
FLOOR
LN Scientific Functions COS
LOG COSH
MOD  Used to calculate SIN
MOVINGAVG SINH
geometric values
MOVINGSUM TAN
POWER of numeric data TANH
ROUND
SIGN
SQRT
TRUNC

52
 Informatica Functions
Special Functions
ERROR
ABORT Used to handle specific conditions within a session;
DECODE search for certain values; test conditional
statements
IIF
IIF(Condition,True,False)

ISNULL Test Functions

IS_DATE
IS_NUMBER  Used to test if a lookup result is null
IS_SPACES  Used to validate data

SOUNDEX
Encoding Functions

METAPHONE  Used to encode string values

53
Expression Validation

The Validate or ‘OK’ button in the Expression Editor will:

 Parse the current expression
• Remote port searching (resolves references to ports in
other transformations)
 Parse transformation attributes
• e.g. - filter condition, lookup condition, SQL Query
 Parse default values
 Check spelling, correct number of arguments in
functions, other syntactical errors

54
Variable Ports
 Use to simplify complex expressions
• e.g. - create and store a depreciation formula to be
referenced more than once
 Use in another variable port or an output port expression
 Local to the transformation (a variable port cannot also be an input or
output port)
 Available in the Expression, Aggregator and Rank transformations

55
Informatica Data Types
NATIVE DATATYPES
Specific to the source and target
database types
Display in source and target tables
within Mapping Designer
TRANSFORMATION DATATYPES
PowerMart / PowerCenter internal
datatypes based on ANSI SQL-92
Display in transformations within
Mapping Designer

Native Transformation Native

 Transformation datatypes allow mix and match of source and target

database types
 When connecting ports, native and transformation datatypes must be
compatible (or must be explicitly converted) 56
Datatype Conversions
Integer Decimal Double Char Date Raw
Integer X X X X
Decimal X X X X
Double X X X X
Char X X X X X
Date X X
Raw X

 All numeric data can be converted to all other numeric datatypes,

e.g. - integer, double, and decimal
 All numeric data can be converted to string, and vice versa
 Date can be converted only to date and string, and vice versa
 Raw (binary) can only be linked to raw
 Other conversions not listed above are not supported
 These conversions are implicit; no function is necessary
57
Mappings

By the end of this section you will be familiar with:

 Mapping components
 Source Qualifier transformation
 Mapping validation
 Data flow rules
 System Variables
 Mapping Parameters and Variables

58
Mapping Designer

Transformation Toolbar

Mapping List

Iconized Mapping

59
Pre-SQL and Post-SQL Rules

 Can use any command that is valid for the database

type; no nested comments
 Can use Mapping Parameters and Variables in SQL
executed against the source
 Use a semi-colon (;) to separate multiple statements
 Informatica Server ignores semi-colons within single
quotes, double quotes or within /* ...*/
 To use a semi-colon outside of quotes or comments,
‘escape’ it with a back slash (\)
 Workflow Manager does not validate the SQL

60
Data Flow Rules
 Each Source Qualifier starts a single data stream
(a dataflow)
 Transformations can send rows to more than one
transformation (split one data flow into multiple pipelines)
 Two or more data flows can meet together -- if (and only if)
they originate from a common active transformation
 Cannot add an active transformation into the mix

ALLOWED DISALLOWED

Passive Active

T T T T

Example holds true with Normalizer in lieu of Source Qualifier. Exceptions are:
Mapplet Input and Joiner transformations 61
Connection Validation

Examples of invalid connections in a Mapping:

 Connecting ports with incompatible datatypes
 Connecting output ports to a Source
 Connecting a Source to anything but a Source
Qualifier or Normalizer transformation
 Connecting an output port to an output port or
an input port to another input port
 Connecting more than one active
transformation to another transformation
(invalid dataflow)

62
 Mapping Validation
 Mappings must:
• Be valid for a Session to run
• Be end-to-end complete and contain valid expressions
• Pass all data flow rules
 Mappings are always validated when saved; can be validated
without being saved
 Output Window will always display reason for invalidity

63
Workflows
By the end of this section, you will be familiar with:
 The Workflow Manager GUI interface
 Workflow Schedules
 Setting up Server Connections
 Relational, FTP and External Loader

 Creating and configuring Workflows

 Workflow properties
 Workflow components
 Workflow Tasks
64
 Workflow Manager Interface

Task
Tool Bar

Workflow
Designer
Tools

Workspace
Navigator
Window

Output Window

Status Bar
65
Workflow Manager Tools

 Workflow Designer
• Maps the execution order and dependencies of Sessions,
Tasks and Worklets, for the Informatica Server

 Task Developer
• Create Session, Shell Command and Email tasks
• Tasks created in the Task Developer are reusable

 Worklet Designer
• Creates objects that represent a set of tasks
• Worklet objects are reusable

66
Workflow Structure
 A Workflow is set of instructions for the Informatica Server
to perform data transformation and load
 Combines the logic of Session Tasks, other types of Tasks
and Worklets
 The simplest Workflow is composed of a Start Task, a Link
and one other Task
Link

Start Session
Task Task

67
Workflow Scheduler Objects

 Setup reusable schedules to

associate with multiple
Workflows
− Used in Workflows and
Session Tasks

68
Server Connections
 Configure Server data access connections
− Used in Session Tasks

Configure:
1. Relational
2. MQ Series
3. FTP
4. Custom
5. External Loader

69
Relational Connections (Native )
 Create a relational (database) connection
− Instructions to the Server to locate relational tables
− Used in Session Tasks

70
Relational Connection Properties
 Define native
relational (database)
connection

User Name/Password

Database connectivity
information

Rollback Segment
assignment (optional)

Optional Environment SQL

(executed with each use of
database connection)

71
FTP Connection
 Create an FTP connection
− Instructions to the Server to ftp flat files
− Used in Session Tasks

72
External Loader Connection
 Create an External Loader connection
− Instructions to the Server to invoke database bulk loaders
− Used in Session Tasks

73
Task Developer
 Create basic Reusable “building blocks” – to use in any Workflow
 Reusable Tasks
• Session Set of instructions to execute Mapping logic
• Command Specify OS shell / script command(s) to run
during the Workflow
• Email Send email at any point in the Workflow
Session
Command
Email

74
 Session Task
 Server instructions to runs the logic of ONE specific Mapping
• e.g. - source and target data location specifications,
memory allocation, optional Mapping overrides,
scheduling, processing and load instructions

 Becomes a
component of a
Workflow (or
Worklet)
 If configured in
the Task
Developer,
the Session Task
is reusable
(optional)

75
 Command Task
 Specify one (or more) Unix shell or DOS (NT, Win2000) commands to
run at a specific point in the Workflow
 Becomes a component of a Workflow (or Worklet)
 If configured in the Task Developer, the Command Task is reusable
(optional)

Commands can also be referenced in a Session through the Session

“Components” tab as Pre- or Post-Session commands
76
Command Task

77
Additional Workflow Components

 Two additional components are Worklets and Links

 Worklets are objects that contain a series of Tasks

 Links are required to connect objects in a Workflow

78
Developing Workflows
Create a new Workflow in the Workflow Designer

Customize
Workflow name

Select a
Server

79
Workflow Properties

Customize Workflow
Properties

Workflow log displays

Select a Workflow
Schedule (optional)

May be reusable or
non-reusable

80
Workflows Properties

Create a User-defined Event

which can later be used
with the Raise Event Task

Define Workflow Variables that can

be used in later Task objects
(example: Decision Task)

81
Building Workflow Components
 Add Sessions and other Tasks to the Workflow
 Connect all Workflow components with Links
 Save the Workflow
 Start the Workflow Save
Start Workflow

Sessions in a Workflow can be independently executed 82

Workflow Designer - Links
 Required to connect Workflow Tasks
 Can be used to create branches in a Workflow
 All links are executed -- unless a link condition is used
which makes a link false
Link 1 Link 3

Link 2
83
Session Tasks

After this section, you will be familiar with:

 How to create and configure Session Tasks
 Session Task properties
 Transformation property overrides
 Reusable vs. non-reusable Sessions
 Session partitions

84
Session Task

 Created to execute the logic of a mapping (one mapping only)

 Session Tasks can be created in the Task Developer
(reusable) or Workflow Developer (Workflow-specific)

 Steps to create a Session Task

• Select the Session button from the Task Toolbar or
• Select menu Tasks | Create

Session Task Bar Icon

85
Session Task - General

86
Session Task - Properties

87
Session Task – Config Object

88
Session Task - Sources

89
Session Task - Targets

90
 Session Task - Transformations

Allows overrides of
some transformation
properties
Does not change the
properties in the
Mapping

91
Session Task - Partitions

92
Monitor Workflows

By the end of this section you will be familiar with:

 The Workflow Monitor GUI interface
 Monitoring views
 Server monitoring modes
 Filtering displayed items
 Actions initiated from the Workflow Monitor
 Truncating Monitor Logs

93
Monitor Workflows
 The Workflow Monitor is the tool for monitoring
Workflows and Tasks
 Review details about a Workflow or Task in two views
• Gantt Chart view
• Task view

Task view
Gantt Chart view 94
Monitoring Workflows
 Perform operations in the Workflow Monitor
• Restart -- restart a Task, Workflow or Worklet
• Stop -- stop a Task, Workflow, or Worklet
• Abort -- abort a Task, Workflow, or Worklet
• Resume -- resume a suspended Workflow after a
failed Task is corrected

 View Session and Workflow logs

 Abort has a 60 second timeout
• If the Server has not completed processing and
committing data during the timeout period, the
threads and processes associated with the Session
are killed

Stopping a Session Task means the Server stops reading data 95

 Monitoring Workflows
Start
Task View Completion
Task Workflow Worklet Time Time

Start, Stop, Abort, Resume

Status Bar Tasks,Workflows and Worklets
96
Monitor Window Filtering
Task View provides filtering

Monitoring filters
can be set using
drop down menus

Minimizes items
displayed in
Task View

Right-click on Session to retrieve the

Session Log (from the Server to the
local PC Client)
97
Debugger

By the end of this section you will be familiar with:

 Creating a Debug Session
 Debugger windows & indicators
 Debugger functionality and options
 Viewing data with the Debugger
 Setting and using Breakpoints
 Tips for using the Debugger

98
Debugger Features

 Debugger is a Wizard driven tool

• View source / target data
• View transformation data
• Set break points and evaluate expressions
• Initialize variables
• Manually change variable values
 Debugger is
• Session Driven
• Data can be loaded or discarded
• Debug environment can be saved for later use

99
 Debugger Interface
Debugger windows & indicators Debugger Mode
indicator

Solid yellow
arrow Current
Transformation
indicator
Flashing
yellow
SQL
indicator

Transformation
Debugger Instance
Log tab Data window

Session Log tab Target Data window

100
 Filter Transformation

Drops rows conditionally

Active Transformation
Connected

Ports
• All input / output

Specify a Filter condition

Usage
• Filter rows from
flat file sources
• Single pass source(s)
into multiple targets
101
Aggregator Transformation

Performs aggregate calculations

Active Transformation
Connected

Ports
• Mixed
• Variables allowed
• Group By allowed

Create expressions in
output or variable ports

Usage
• Standard aggregations
102
Informatica Functions

Aggregate Functions
AVG  Return summary values for non-null data
COUNT in selected ports
FIRST
LAST
 Use only in Aggregator transformations
MAX
 Use in output ports only
MEDIAN
MIN  Calculate a single value (and row) for all
PERCENTILE
records in a group
STDDEV
SUM  Only one aggregate function can be
VARIANCE nested within an aggregate function
 Conditional statements can be used with
these functions
103
Aggregate Expressions

Aggregate
functions are
supported
only
in the
Aggregator
Transformation

Conditional
Aggregate
expressions
are supported Conditional SUM format: SUM(value, condition)
104
Aggregator Properties

Sorted Input Property

Instructs the
Aggregator to
expect the data
to be sorted

Set Aggregator
cache sizes (on
Informatica Server
machine)

105
Sorted Data

 The Aggregator can handle sorted or unsorted data

• Sorted data can be aggregated more efficiently, decreasing
total processing time

 The Server will cache data from each group and

release the cached data -- upon reaching the first
record of the next group
 Data must be sorted according to the order of the
Aggregator “Group By” ports
 Performance gain will depend upon varying factors

106
Incremental Aggregation
MTD
Trigger in calculation
Session Properties,
Performance
Tab

 Cache is saved into $PMCacheDir: aggregatorname.DAT

aggregatorname.IDX
 Upon next run, files are overwritten with new cache information

Example: When triggered, PowerCenter Server will save

new MTD totals. Upon next run (new totals), Server will
subtract old totals; difference will be passed forward

Best Practice is to copy these files in case a rerun of data is ever required.
Reinitialize when no longer needed, e.g. – at the beginning new month processing 107
Joiner Transformation

By the end of this section you will be familiar with:

 When to use a Joiner Transformation
 Homogeneous Joins
 Heterogeneous Joins
 Joiner properties
 Joiner Conditions
 Nested joins

108
Homogeneous Joins
Joins that can be performed with a SQL SELECT statement:
 Source Qualifier contains a SQL join

 Tables on same database server (or are synonyms)

 Database server does the join “work”
 Multiple homogenous tables can be joined

109
Heterogeneous Joins

Joins that cannot be done with a SQL statement:

 An Oracle table and a Sybase table

 Two Informix tables on different database servers

 Two flat files
 A flat file
and a
database
table

110
Joiner Transformation

Performs heterogeneous joins on records from

different databases or flat file sources

Active Transformation
Connected
Ports
• All input or input / output
• “M” denotes port comes
from master source
Specify the Join condition
Usage
• Join two flat files
• Join two tables from
different databases
• Join a flat file with a
relational table 111
Joiner Conditions

Multiple
join
conditions
are supported

112
 Joiner Properties

Join types:
• “Normal”
(inner)
• Master outer
• Detail outer
• Full outer

Set
Joiner Cache

Joiner can accept sorted data (configure the join condition to

use the sort origin ports) 113
Mid-Mapping Join

The Joiner does not accept input in the following situations:

 Both input pipelines begin with the same Source Qualifier
 Both input pipelines begin with the same Normalizer
 Both input pipelines begin with the same Joiner
 Either input pipeline contains an Update Strategy

114
Sorter Transformation

 Can sort data from relational tables or flat files

 Sort takes place on the Informatica Server machine
 Multiple sort keys are supported
 The Sorter transformation is often more efficient than
a sort performed on a database with an ORDER BY
clause

115
Lookup Transformation

By the end of this section you will be familiar with:

 Lookup principles
 Lookup properties
 Lookup conditions
 Lookup techniques
 Caching considerations

116
How a Lookup Transformation Works
 For each Mapping row, one or more port values are looked
up in a database table
 If a match is found, one or more table values are returned
to the Mapping. If no match is found, NULL is returned

Lookup Lookup transformation

value(s)

Return value(s)

117
Lookup Transformation
Looks up values in a database table and provides
data to other components in a Mapping

Passive Transformation
Connected / Unconnected
Ports
• Mixed
• “L” denotes Lookup port
• “R” denotes port used as a
return value (unconnected
Lookup only)
Specify the Lookup Condition
Usage
• Get related values
• Verify if records exists or
if data has changed 118
Lookup Properties

Override
Lookup SQL
option

Toggle
caching

Native
Database
Connection
Object name

119
Additional Lookup Properties

Set cache
directory

Make cache
persistent

Set
Lookup
cache sizes

120
Lookup Conditions
Multiple conditions are supported

121
To Cache or not to Cache?
Caching can significantly impact performance
 Cached
• Lookup table data is cached locally on the Server
• Mapping rows are looked up against the cache
• Only one SQL SELECT is needed

 Uncached
• Each Mapping row needs one SQL SELECT
 Rule Of Thumb: Cache if the number (and size) of
records in the Lookup table is small relative to the
number of mapping rows requiring lookup
122
Target Options

By the end of this section you will be familiar with:

 Row type indicators
 Row operations at load time
 Constraint-based loading considerations
 Rejected row handling options

123
 Target Properties

Session Task

Select target
instance

Row loading
operations

Error handling

Properties Tab
124
Constraint-based Loading
Maintains referential integrity in the Targets
pk1
Example 1
fk1, pk2 With only One Active source, rows
for Targets 1-3 will be loaded
properly and maintain referential
fk2 integrity

pk1 Example 2
With Two Active sources, it is not
fk1, pk2 possible to control whether rows for
Target 3 will be loaded before or
fk2 after those for Target 2

The following transformations are ‘Active sources’: Advanced External Procedure,

Source Qualifier, Normalizer, Aggregator, Sorter, Joiner, Rank, Mapplet (containing
any of the previous transformations)
125
Update Strategy Transformation

By the end of this section you will be familiar with:

 Update Strategy functionality
 Update Strategy expressions
 Refresh strategies
 Smart aggregation

126
 Update Strategy Transformation
Used to specify how each individual row will be used to
update target tables (insert, update, delete, reject)

Active Transformation
Connected

Ports
• All input / output

Specify the Update

Strategy Expression

Usage
• Updating Slowly
Changing Dimensions
• IIF or DECODE logic
determines how to
handle the record
127
Target Refresh Strategies

 Single snapshot: Target truncated, new records

inserted
 Sequential snapshot: new records inserted

 Incremental: Only new records are inserted.

Records already present in the target are ignored
 Incremental with Update: Only new records are
inserted. Records already present in the target are
updated

128
Router Transformation

Rows sent to multiple filter conditions

Active Transformation
Connected
Ports
• All input/output
• Specify filter conditions
for each Group
Usage
• Link source data in one
pass to multiple filter
conditions

129
Router Transformation in a Mapping

130
Parameters and Variables

By the end of this section you will understand:

 System Variables
 Creating Parameters and Variables
 Features and advantages
 Establishing values for Parameters and Variables

131
 System Variables
SYSDATE  Provides current datetime on the
Informatica Server machine
• Not a static value

$$$SessStartTime  Returns the system date value as a

string. Uses system clock on machine
hosting Informatica Server
• format of the string is database type
dependent
• Used in SQL override
• Has a constant value

SESSSTARTTIME  Returns the system date value on the

Informatica Server
• Used with any function that accepts
transformation date/time data types
• Not to be used in a SQL override
• Has a constant value 132
Mapping Parameters and Variables

 Apply to all transformations within one Mapping

 Represent declared values
 Variables can change in value during run-time
 Parameters remain constant during run-time
 Provide increased development flexibility
 Defined in Mapping menu
 Format is $$VariableName or $$ParameterName

133
 Mapping Parameters and Variables
Sample declarations

Set the
User- appropriate
defined aggregation
names type

Set optional
Initial Value

Declare Variables and Parameters in the Designer Mappings menu

134
Functions to Set Mapping Variables

 SetCountVariable -- Counts the number of

evaluated rows and increments or decrements a
mapping variable for each row
 SetMaxVariable -- Evaluates the value of a mapping
variable to the higher of two values
 SetMinVariable -- Evaluates the value of a mapping
variable to the lower of two values
 SetVariable -- Sets the value of a mapping variable
to a specified value

135
 Unconnected Lookup
 Will be physically “unconnected” from other transformations
• There can be NO data flow arrows leading to or from an
unconnected Lookup
Lookup function can be set within any
transformation that supports expressions

Lookup data is
called from the
point in the
Mapping that
needs it

Function in the
Aggregator calls the
unconnected Lookup
136
Conditional Lookup Technique
Two requirements:
 Must be Unconnected (or “function mode”) Lookup
 Lookup function used within a conditional statement
Row keys
Condition (passed to Lookup)

IIF ( ISNULL(customer_id),:lkp.MYLOOKUP(order_no))

Lookup function

 Conditional statement is evaluated for each row

 Lookup function is called only under the pre-defined
condition 137
Conditional Lookup Advantage
 Data lookup is performed only for those rows which
require it. Substantial performance can be gained

EXAMPLE: A Mapping will process 500,000 rows. For two

percent of those rows (10,000) the item_id value is NULL.
Item_ID can be derived from the SKU_NUMB.

IIF ( ISNULL(item_id), :lkp.MYLOOKUP (sku_numb))

Condition Lookup
(true for 2 percent of all (called only when condition is
rows) true)
Net savings = 490,000 lookups
138
Connected vs. Unconnected Lookups
CONNECTED LOOKUP
Part of the mapping data flow
Returns multiple values (by linking
output ports to another
transformation)
Executed for every record passing
through the transformation
More visible, shows where the
lookup values are used
Default values are used
UNCONNECTED LOOKUP
Separate from the mapping data
flow
Returns one value (by checking the
Return (R) port option for the output
port that provides the return value)
Only executed when the lookup
function is called
Less visible, as the lookup is called
from an expression within another
transformation
Default values are ignored
139
Heterogeneous Targets

By the end of this section you will be familiar with:

 Heterogeneous target types
 Heterogeneous target limitations
 Target conversions

140
Definition: Heterogeneous Targets

Supported target definition types:

 Relational database
 Flat file
 XML
 ERP (SAP BW, PeopleSoft, etc.)

A heterogeneous target is where the target types are

different or the target database connections are different
within a single Session Task

141
Step One: Identify Different Target Types

Oracle table

Oracle table
Tables are EITHER in two
different databases, or
require different (schema-
specific) connect strings
Flat file
One target is a flatfile load

142
Step Two: Different Database Connections

The two database

connections WILL
differ

Flatfile requires
separate location
information

143
Target Type Override (Conversion)

Example: Mapping has SQL Server target definitions.

Session Task can be set to load Oracle tables instead,
using an Oracle database connection.

Only the following overrides are supported:

 Relational target to flat file target
 Relational target to any other relational database type
 SAP BW target to a flat file target

CAUTION: If target definition datatypes are not compatible with datatypes in newly
selected database type, modify the target definition 144
Mapplet Designer

Mapplet Designer Tool Mapplet

Transformation
Icons

Mapplet Output
Transformation

145
Mapplet Advantages

 Useful for repetitive tasks / logic

 Represents a set of transformations
 Mapplets are reusable
 Use an ‘instance’ of a Mapplet in a Mapping
 Changes to a Mapplet are inherited by all instances
 Server expands the Mapplet at runtime

146
Active and Passive Mapplets

 Passive Mapplets contain only passive transformations

 Active Mapplets contain one or more active
transformations

CAUTION: changing a passive Mapplet into an active

Mapplet may invalidate Mappings which use that
Mapplet
• Do an impact analysis in Repository Manager first

147
Using Active and Passive Mapplets

Multiple Passive
Mapplets can populate
Passive the same target
instance

Multiple Active Mapplets

or Active and Passive
Mapplets cannot
Active populate the same
target instance

148
Reusable Transformations

By the end of this section you will be familiar with:

 Reusable transformation advantages
 Reusable transformation rules
 Promoting transformations to reusable
 Copying reusable transformations

149
Reusable Transformations
 Define once - reuse many times
 Reusable Transformations
• Can be a copy or a shortcut
• Edit Ports only in Transformation Developer
• Can edit Properties in the mapping
• Instances dynamically inherit changes
• Be careful: It is possible to invalidate mappings by
changing reusable transformations
 Transformations that cannot be made reusable
• Source Qualifier
• ERP Source Qualifier
• Normalizer used to read a Cobol data source

150
Promoting a Transformation to Reusable

Place a
check in the
“Make
reusable” box

This action
is not
reversible

151
Sequence Generator Transformation

Generates unique keys for any port on a row

Passive Transformation
Connected

Ports
• Two predefined
output ports,
NEXTVAL and
CURRVAL
• No input ports allowed

Usage
• Generate sequence numbers
• Shareable across mappings
152
Sequence Generator Properties

Number
of
Cached
Values

153
Dynamic Lookup

By the end of this section you will be familiar with:

 Dynamic lookup theory
 Dynamic lookup advantages
 Dynamic lookup rules

154
 Additional Lookup Cache Options

Make cache
persistent

Cache File Name Prefix

• Reuse cache by
name for another
similar business
purpose

Recache from Database

• Overrides other
Dynamic Lookup Cache
settings and Lookup
data is refreshed • Allows a row to know about the
handling of a previous row
155
Persistent Caches

 By default, Lookup caches are not persistent

 When Session completes, cache is erased
 Cache can be made persistent with the Lookup
properties
 When Session completes, the persistent cache is
stored on server hard disk files
 The next time Session runs, cached data is loaded
fully or partially into RAM and reused
 Can improve performance, but “stale” data may pose
a problem

156
Dynamic Lookup Cache Advantages

 When the target table is also the Lookup table,

cache is changed dynamically as the target load
rows are processed in the mapping
 New rows to be inserted into the target or for
update to the target will affect the dynamic Lookup
cache as they are processed
 Subsequent rows will know the handling of
previous rows
 Dynamic Lookup cache and target load rows
remain synchronized throughout the Session run

157
Update Dynamic Lookup Cache
 NewLookupRow port values
• 0 – static lookup, cache is not changed
• 1 – insert row to Lookup cache
• 2 – update row in Lookup cache
 Does NOT change row type
 Use the Update Strategy transformation before or after
Lookup, to flag rows for insert or update to the target
 Ignore NULL Property
• Per port
• Ignore NULL values from input row and update the cache
using only with non-NULL values from input

158
Example: Dynamic Lookup Configuration

Router Group Filter Condition should be:

NewLookupRow = 1

This allows isolation of insert rows from update rows

159
Concurrent and Sequential Workflows

By the end of this section you will be familiar with:

 Concurrent Workflows
 Sequential Workflows
 Scheduling Workflows
 Stopping, aborting, and suspending Tasks and
Workflows

160
Multi-Task Workflows - Sequential

Tasks can be run sequentially:

Tasks shows are all Sessions, but they can also be

other Tasks, such as Commands, Timer or Email Tasks

161
Multi-Task Workflows - Concurrent

Tasks can be run concurrently:

Tasks shows are all Sessions, but they can also be

other Tasks such as Commands, Timer or Email Tasks.

162
Multi-Task Workflows - Combined

 Tasks can be run in a combination concurrent and

sequential pattern within one Workflow:

 Tasks shows are all Sessions, but they can also be

other Tasks such as Commands, Timer or Email Tasks
163
Additional Transformations

By the end of this section you will be familiar with:

 The Rank transformation
 The Normalizer transformation
 The Stored Procedure transformation
 The External Procedure transformation
 The Advanced External Procedure transformation

164
Rank Transformation
Filters the top or bottom range of records

Active Transformation
Connected
Ports
• Mixed
• One pre-defined
output port
RANKINDEX
• Variables allowed
• Group By allowed

Usage
• Select top/bottom
• Number of records
165
Normalizer Transformation

Normalizes records from relational or VSAM sources

Active Transformation
Connected

Ports
• Input / output or output

Usage
• Required for VSAM
Source definitions
• Normalize flat file or
relational source
definitions
• Generate multiple
records from one record
166
Normalizer Transformation
Turn one row
YEAR,ACCOUNT,MONTH1,MONTH2,MONTH3, … MONTH12
1997,Salaries,21000,21000,22000,19000,23000,26000,29000,29000,34000,34000,40000,4500
0
1997,Benefits,4200,4200,4400,3800,4600,5200,5800,5800,6800,6800,8000,9000
1997,Expenses,10500,4000,5000,6500,3000,7000,9000,4500,7500,8000,8500,8250

Into multiple rows

167
Stored Procedure Transformation

Calls a database stored procedure

Passive Transformation
Connected/Unconnected

Ports
• Mixed
• “R” denotes port will
return a value from the
stored function to the
next transformation

Usage
• Perform transformation
logic outside PowerMart /
PowerCenter
168
External Procedure Transformation (TX)
Calls a passive procedure defined in a dynamic linked
library (DLL) or shared library

Passive Transformation
Connected/Unconnected

Ports
• Mixed
• “R” designates return
value port of an
unconnected
transformation
Usage
• Perform transformation
logic outside PowerMart /
PowerCenter
Option to allow partitioning 169
Advanced TX Transformation
Calls an active procedure defined in a dynamic linked
library (DLL) or shared library

Active Transformation
Connected Mode only

Ports
• Mixed

Usage
• Perform
transformation logic
outside PowerMart /
PowerCenter
• Sorting, Aggregation

Option to allow partitioning 170

Transaction Control Transformation
Allows custom commit types (source- or target-
based) and user-defined conditional commits

Passive Transformation
Connected Mode Only

Ports
• Input and Output

Properties
• Continue
• Commit Before
• Commit After
• Rollback Before
• Rollback After

171
Transaction Control Functionality
 Commit Types
• Target Based Commit -
Commit Based on “approximate” number of records
written to target
• Source Based Commit –
Ensures that a source record is committed in all
targets
• User Defined Commit –
Uses Transaction Control Transform to specify
commits and rollbacks in the mapping based on
conditions
 Set the Commit Type (and other specifications) in the
Transaction Control Condition
172
 Versioning

 View Object Version Properties

 Track Changes to an Object
 Check objects “in” and “out”
 Delete or Purge Object version
 Apply Labels and Run queries
 Deployment Groups

173
Informatica Business Analytics Suite

Modular
Plug-&-Play
Approach

Packaged
Analytic Solutions Custom Built
Analytic Solutions
174
Informatica Warehouses / Marts

Informatica Warehouse™
Customer Finance Human Supply Chain
Relationship Resources
Sales G/L Compensation Planning
Marketing Receivables Scorecard Sourcing
Service Payables Inventory
Web Profitability Quality

Common Dimensions
Customer Product Supplier Geography
Organization Time Employee

175
Inside the Informatica Warehouse
Business Intelligence  Business Adapters™ (Extract)
• Data Source Connectivity with Minimal
Load
Informatica Warehouse™ • Structural/Functional Knowledge of
Sources
Analytic Advanced
 Analytic Bus™ (Transform)
Data Model Calculation
• Transaction consolidation and
Engine standardization
• Source independent interface
Load

 Warehouse Loader (Load)

Warehouse Loader™ • Type I, II slowly changing dimensions
• History and changed record tracking
Transform

Analytic Bus™  Analytic Data Model

• Industry Best Practice Metrics
• Process-centric model & conformed
Extract

dimensions
Business Adapters™
 Advanced Calculation Engine
• Pre-aggregations for rapid query response
• Complex calculation metrics (e.g.
SAP ORCL i2 SEBL PSFT Custom statistical)
176
PowerConnect Products

Family of enterprise software products that allow

companies to directly source and integrate ERP, CRM,
real-time message queue, mainframe, AS/400, remote
data and metadata with other enterprise data
 PowerConnect for MQSeries (real time)
 PowerConnect for TIBCO (real time)
 PowerConnect for PeopleSoft
 PowerConnect for SAP R/3
 PowerConnect for SAP BW
 PowerConnect for Siebel
 PowerConnect for Mainframe
 PowerConnect for AS/400
 PowerConnect for Remote Data
 PowerConnect SDK
177
178