Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
December 1, 2010
COURSE AGENDA
Day 1 Module 1 : Review of CDI Architecture Module 2 : Overview of Application Integration Module 3 : Initial Data Loads into UCM Module 4 : The UCM Batch Workflow and SDH tables Day2 Module 5 : UCM Messaging and CRMML Module 6 : UCM Server Party Package Workflows Module 7 : The UCM Batch Management Workflows Module 8 : Publish and Subscribe Module 9: Wrap-up
CDI
UCM
SDQ
UAN
BL
.
Field BC BO
DL
SDH Column
SDH table
S_CIF_EXT_SYST
S_UCM_ORG_CHLD
S_UCM_CON_CHLD
10
CDI Configuration
External System Registration Data Quality Configuration Survivorship Rules
11
12
13
14
Cross-Referencing
Cross-reference entries are created automatically with the creation of every new account, contact, or household record External systems send requests to insert a record If there is an existing cross-reference entry, the entry is updated If there is no existing cross-reference entry, a new record is inserted The system ID is the value assigned when the system was registered and system name is the associated name There are multiple possible external Ids to account for situations where multiple records in an operational application map to a single record in UCM Only the first is stored in the ID tag in a message request, by default
16
17
19
Survivorship Rules
Customer data originates in multiple systems Conflicts arise when different sources have different data for the same attribute For eg: two systems have different addresses for customer The role of UCM is to resolve these conflicts, which ensures that the best version record survives Survivorship rules are an automated mechanism for consolidating customer data to produce a best-version record Evaluates updates from each registered system Applies rules to determine which is the best value for each attribute Can be constructed by a system integrator or a CDI data steward Three comparison methods Recent
21
High Score - 90 Records with match scores in the middle will be brought to the attention of the data steward for manual review and resolution Low Score - 70 Records with low match scores will be left as separate records
22
23
25
26
Unmerge
For records that have been merged Surviving record--- record that remains after the merge Victim record ----- record that is merged and ceases to exist If merged records are unmerged after changes have been made to be the merged (surviving) record All changes and updates since the merge will be applied to the surviving record The victim record will be resurrected from the Source Data history table Merge Unmerge
A A2
A1 B
A2 B
29
30
31
32
33
34
Outbound Data
35
36
37
Outbound Data
Inbound/Outbound Processing
UCM has automated much of the inbound processing --- a user must: Prepare input(CRMML) file Run (appropriate) Server Party Package workflow to place records of SDH tables Run UCM Batch workflow to activate data quality and place data on Siebel base tables The UCM application supports an outbound publish and subscribe facility Insert and Update records can be published to external system subscribers
Individually or as a group In batch or real time mode
39
40
Interface table
41
Staging Tables
The UCM application uses two sets of staging tables before making data available to the Business Logic layer through Siebel Base tables The EIM staging tables: Data is staged to be transformed into a Siebel standard format The Source ,Data and history tables: Data is staged for cleansing and matching
42
43
S_BU
S_ORG_TXT
S_ADDR_ORG
44
User Keys
Based on multiple columns, user keys are used to uniquely identify a row for EIM Primary and foreign keys based on ROW_ID are used for systemwide referential integrity
PK ROW_ID User Key FK BU_ID
Name
LOC
S_ORG_EXT
S_UCM_BU
45
46
Interface Tables
Staging area for data to be imported, updated, or merged into Source Data and History (SDH) Staging area for data imported by EIM EIM tables do not have ROW_Ids assigned
External Database
S_UCM_ORG_EXT
EIM_UCM_ORG
S_UCM_CONTACT
Non-Siebel process
48
49
50
Referential Integrity
User key column mappings in interface tables are used to resolve ROW_Ids for SDH tables, maintaining referential integrity
User key PK(ROW_ID)
Acme
1-8D
FK(ROW_ID)
EIM_UCM_ORG
S_UCM_CONTACT
External Data
52
53
54
55
56
57
SDH table
Base Table
SDH table
Base column
58
59
60
Interface (EIM)
SDH
61
Base Column
Interface (EIM)
62
SDH (UCM)
63
64
65
66
Query by Id
Error?
Process NoMatch
CDM Cleanse
CDM Match
Match id?
Error?
Converter
PubSub
End
67
68
Query By ID
Invokes the Query method of the EAI Siebel Adapter business service Reads an object to be inserted from a SDH table Outputs a Siebel message
Workflow Process Props SDH int Obj Name OutputintObjectName PrimaryRowId Object Id SiebeMessage Siebel Message
Query By Id
69
CDM Cleanse
Invokes the Cleanse method of the UCM Data Quality Manager business service Reads a Siebel Message containing object to be inserted from SDH table Outputs a cleansed XML representation of the original Siebel Workflow Process Message Props
Base int Obj Name IntObjectName
CDM Cleanse
70
CDM Match
Invokes the Match method of the UCM Data Quality Manager business service Reads an XML representation of a Siebel Message containing an object to be inserted from SDH table Output indicates if record is a duplicate and the auto-merge Workflow level Process Props
Cleansed Prop XML Hierarchy
Number of AutoMatch
NumAutoMatch
CDM Match
71
72
Process NoMatch
Is activated in the event that the incoming record is not scored as a match Record is inserted Invokes the HandleMatchCase method of the UCM Batch Manager business service
Workflow Process Props Matching Id Object Type Turn On Survivorship Cleansed Prop Processed Prop XML Hierarchy XML Hierarchy CaseType ObjectType TurnOnSE
Process NoMatch
73
Process AutoMatch
Is activated in the event that the incoming record is scored as a match Record evaluated by survivorship rule --- is either merged or inserted Invokes the HandleMatchCase method of the UCM Batch Workflow Process Manager business service Props CaseType
Matching Id Object Type Turn On Survivorship Cleansed Prop Processed Prop MatchingRowId ObjectType TurnOnSE XML Hierarchy XML Hierarchy
Process AutoMatch
74
Process Review
Is activated in the event that the incoming record is scored as a match but not resolved by survivorship rule Record is posted in Duplicate (Accounts) view Invokes the HandleMatchCase method of the UCM Batch Manager business service
Workflow Process Props CaseType Object Type Cleansed Prop ObjectType XML Hierarchy
Process Review
Processed Prop
XML Hierarchy
75
76
77
Data Layer
SDH Column
SDH table
78
Business Component
S_UCM_CONTACT
S_UCM_CON_CHILD Table
79
S_UCM_ORG_EXT
S_UCM_CONTACT
S_CIF_EXT_SYST
S_UCM_ORG_CHLD
S_UCM_CON_CHLD
81
82
83
84
85
87
88
CRMML
An XML based markup language for data exchange and remote business process invocation Used for communication with UCM Inbound requests (insert, query, update, delete) Responses Publishing messages
Request
Back-Office Application
UCM
Response Publication These exchanges use CRMML messages
89
CRMML Header
Contains Information identifying sending and receiving systems Used by Universal Customer Master for security verification Message descriptor identifying the message <CRMHeader> <FromSystem> <SystemInfo> <SystemId>System123</SystemId> <SystemName>External App</SystemName> </SystemInfo> </FromSystem> <ToSystem> </ToSystem> <MessageDescriptor> </ MessageDescriptor>
92
93
94
<CRMMLHeader>
<ToSystem>
<SystemInfo>
<CRMML>
<MessageDescriptor>
<Body>
95
96
CRMML Body
Contains one or more message elements representing: Inbound requests from an external applications to insert, query, update, or delete a database object Outbound response to a requested transaction Outbound publication messages to be sent to an external subscribing application Each message element includes: Software package must be CIFPartyPackage Method --- the requested action Argument the argument to the action
<SoftwarePackage> <Method> <Argument> <CIFPartyPackage> <InsertContact> <Contact>
97
98
Continued
The mapping rules are defined in user properties of a mapping integration object The default mapping object is CIFDispMap
Envelope
Package
Method
CRMML/CIFPartyPackage/InsertContact -> CRMML/CIFPartyPackage/InsertContact;a;b;CIFContactInsert;CIFContactInsertRs; IXML_Operation_ADD Integration Object for inbound requests Integration Object for outbound responses
99
<CRMHeader>
<ToSystem>
<SystemInfo>
<CRMML>
<MessageDescriptor>
<CIFPartyPackage>
<InsertContact>
<Contact>
Package
Method
Argument
100
Method <InsertContactAddress>
Argument <Contact>
Method <InsertContactAddress>
<Contact_ INSPersonalAddress>
<Contact_ INSPersonalAddress>
101
103
104
105
PubSub
Transaction Manager
Realtime Pub/Sub
End
Write To File
Convert to Hierarchy
106
XML
<value>
107
XML Hierarchy
Dispatch message
108
Dispatch Message
Invokes the DispatchMessage method of the UCM Dispatcher business service Verifies structure of CRMML message Acts information to message components that is used by downstream business services Convert to XML
WorkFlow Process Props Hierarchy
XML Hierarchy
Security Check
109
Dispatch Message
Continued
Uses two integration objects, specified as user properties of the business service, to represent the message
Integration object used to process body Integration object used to process header
Value CIFDispMap
110
111
112
Dispatch Message
Sample message before Dispatcher additions
Continued
<CRMML> <CRMHeader> . </CRMHeader> <CIFPartyPackage> <InsertContactProfile> <Contact> . </Contact> </InsertContactProfile> </CIFPartyPackage> </CRMML
113
Dispatch Message
Sample message after Dispatcher additions
Continued
SiebelFINSRespIntObjName=CIFContactProfileInsertRs Dispatcher SiebelFINSMapOut=b SiebelFINSMapin=a> adds these attribute <Contact> values which come from the triggered .. mapping rule </Contact> </InsertContactProfile> </CIFPartyPackage> <CRMML>
114
Security Check
Invokes the SecurityCheck method of the UCM Security Manager business service Verify the SystemId in the message header against system registrations Return an error if the system is not recognized For each method in the request, verify that the system has the privilege required to execute the operation Failed operations are removed from the request and Dispatch put into WorkFlow Process the Fail Security Req Prop property set
Props Message
Disp Req Prop Fail Security Req Prop Security Req Prop
XML Hierarchy FailSecurityHierarchy XML Hierarchy Security Check: UCM Security Manager Security Check
Security Check
115
Security Check
Operation Name IXMLOperation_ADD IXMLOperation_DELETE IXMLOperation_QUERY IXMLOperation_UPDATE Privilege Insert Delete Query Update
Continued
User properties map operation (inserted by the Dispatcher) to privileges (assigned to systems in system registration)
116
Convert Request In
Invokes the XMLPropSetToPropSet method of the UCM Converter business service Each method is converted into the format required by the Transaction Manager Requesting System information is extracted from the header to make it available to downstream business services Security WorkFlow Process Uses integration object, CIF Envelope Check
Props Security Req Prop CIF System prop Tran Req Prop XML Hierarchy CIFSystemContainer XML Hierarchy Convert Request In: UCM Converter XMLPropSetToPropSet
Transaction Manager
117
Convert Request In
Sample converted method
Before
<CIFPartyPackage.> <InsertContactProfile.> <Contact> <FirstName>Test</FirstName> <LastName>Contact</LastName> <Status>Active</Status> .. </Contact> </InsertContactProfile> </CIFPartyPackage>
Continued
After
<CIFPartyPackage.> <ListOfCIFContactProfileInsert.> <Contact> First_spcName=Test Last_spcName=Contact Status=Active .. /> </ListOfCIFContactProfileInsert> </CIFPartyPackage>
118
Transaction Manager
Invokes the Execute method of the UCM Transaction Manager business service Performs operations in message by invoking services configured in its user properties
WorkFlow Process Props Tran Req Prop Turn On Data Mgmt Turn On Survivorship Status Object Notification Req Prop Trans Resp Prop
XML Hierarchy XML Hierarchy TurnOnDM TurnOnSE StatusObject NotificationHierarchy
Convert Request In
119
Query Delete
121
Manager
Realtim e Pub/Su b
122
Considerations
Error Handling Configuring Inbound Server Communication
123
Error Handling
The UCM Server Party Package Workflow (MQSeries Server Transport Error Handling) demonstrates how error handling could be added to a workflow.
Convert to XML Hierarchy Dispatch Message Security Check Convert Request In Transaction Manager Error Handler: UCM Converter: GenerateErrorPropSet Error Handler: UCM Converter: GenerateErrorPropSet
End
124
Error Handling
Continued
The UCM Converter service includes a method to aid with error handling
WorkFlow Process Props Error Code Error Message Disp Req Prop
ErrorCode ErrorMessageText XML Hierarchy
XML Hierarchy
125
126
127
128
UCM Contact Batch Data Management Workflow Performs the following actions on a single input record
Cleansing Start Matching Real-Time publication Query by ID
CDM Cleanse Get Error Code
Error
CDM Match
match
Matching
Error?
Process NoMatch
Process AutoMatch
End
Converter
Publishing
PubSub
129
Data Cleansing
Performed by Cleanse method of the UCM Data Quality Manager business service Invokes third-party data cleansing software (if enabled) Output is cleansed record
Start Retrieve record data for processing
130
Auto Match ?
No Process Review
Error?
Process NoMatch
Process AutoMatch
131
Data Matching
Continued All three match cases are processed by the same business service method: UCM Batch Manager HandleMatchCase
Process NoMatch
Process AutoMatch
Process Review
132
Data Matching
Continued Thresshold settings determine how match is handled
Process NoMatch High Score - 90 Process AutoMatch Records with match scores in the middle will be brought to the attention of the data steward for manual review and resolution. Records with high match scores can be automerged,applying survivorship rules.
Low Score - 70 Process Review Records with low match scores will be left as seperate records.
133
134
Publishing
Data changes can be published to subscribing systems Result of auto merging of records Result of insert of record (no match condition) Publishing is addressed in the next module
Process NoMatch Process AutoMatch
True
Converter
PubSub
End
135
136
1. Create a Job
In the Administration Server Management > Jobs view. Create a new job to run the Workflow Process Batch Manager component/job
137
138
139
140
Subscriber 2
Update: Contact123
Subscriber 3
141
Publish/Subscribe Modes
Publish and Subscribe framework supports two operative modes: Real-time Transaction are published as they happen Each individual transaction is published to real-time subscribers Daily Batch Mode Transactions are published once per day Best version of record is published, individual transactions are not Systems can mix and match modes Contact updates received in real-time Account updates received from daily batch job
142
Publication Messages
Publication messages are CRMML messages FromSystem is UCM ToSystem is subscriber Message body content is constructed using integration objects Real-time publishing uses the response integration object associated with the particular UCM transaction Batch publishing uses the integration object type specified as input to the both Publish/Subscribe business service
143
NoMatch Process AutoMatch Realtim ePub/S ub Convert Request Out Converter Result of match is published
PubSub
145
Start
End
146
147
148
150
Subscribers
Registered systems can be configured as subscribers to receive publication messages in real-time or batch
URL where publication messages are sent
151
Module 9: Wrap-Up
152
Course Goal
To enable participants to integrate Siebel Universal Customer Master (UCM) with external applications Legacy systems External databases Was achieved by covering the following topics Initial data loads into UCM at the data layer Incremental data loads into UCM at the business logic layer Managing outbound data flows via publish and subscribe The CRMML messaging protocol for requesting UCM services
153
156
Some Guidelines
Initial data for UCM should be cleaned outside (Before loading into UCM) EIM may be more effective to load data into UCM from external systems for performance reasons Batch publish method is more effective when send data from UCM to external systems UCM is not complete Customer data store, UCM can not store complete customer information. UCM Contact table has limited columns only; this data is primarily for referencing customer data between multiple external systems UUID is not numeric and big in size (about 20+ characters string), so it may not be useful as general purpose contact Id
158
Thank You
159