Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
TalendOpenStudioForMDM:APracticalStarterGuide
TalendOpenStudioForMasterData
Management:APracticalStarterGuide
2ndEdition
AbouttheAuthor
:
DiethardSteinerisworkingasanindependentbusinessintelligenceconsultantinLondon,UK
andfocusesmainlyonopensourcebusinessintelligencesolutions.Heisalsoauthorofthe
DiethardSteineronBusinessIntelligence
blogwhichoffersregulartutorialsonvariousopen
sourcetoolsandtopics.Youcanfindmoreinfoon
LinkedIn
.Followhimon
Twitter
or
Google+
.
TableofContents:
Introduction
Installation
UninstallingtheMDMServer
AddtheserverdetailstoTOSMDMStudio
Preparation
Modeling
Howtocreateadatamodel
Howtocreatedatamodelentities
Howtodefineanautoincrementkey
Usingautoincrementinteger
UsingUUID
Setlabelsanddescriptions
Definedisplayformatfordateandnumbers
Howtodeployadatamodel
DataContainer
Howtocreateaview
TheUpdateReport
DataIntegration
GettingStarted
LoaddataintotheMDMhub
Standardload
CheckingthedatacontainerrecordsviatheMDMServerwebinterface
CheckingthedatacontainerrecordsviaTalendMDMStudio
Checkingthedatacontainerrecordsviayourdatabasesqueryclient
Bulkload
ExportdatafromtheMDMserver
Realtimedataintegration/propagation
1/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Processes
Triggers
TheExchangeDocument
CRU:Newrecordsandupdates
Creatingthedataintegrationjob
Creatingthetrigger
Testing
CRUD:RoutingintheDIjob
Dataintegration
Creatingthetrigger
Testing
TheIntegratedversionofCRUD
IntegratedVSnonintegratedwhatitisallabout
ExchangeDocumentStructure
CREATE
UPDATE
LOGIC_DELETE
PHYSICAL_DELETE
DataIntegration
GeneratingtheJobCallerTrigger
Checkinguniquenessofafunctionalkeybeforesaving
CreatingtheBeforeSavingprocess
DataIntegration
Creatingtheprocess
Testing
Checkinguniquenessofafunctionalkeybeforesaving(Integratedversion)
DataIntegration
BeforeSavingProcess
ProcessesAdvancedTopics
Howtocreateanonintegratedprocessesmanually
Howtotestprocesses
UnderstandingtheXMLdocumentwhichispassedonfromourprocesstothedata
integrationjob
Testingprocessviathewebinterface
ManagingthedeployedDIjobs,views,processesandtriggersontheMDMserver
Deletingtriggers,processes,jobfiles,viewsetc
Deactivatingatrigger
CautionwhenrenamingfilesintheMDMStudio
eXist
AccessingtheeXistDBclient
eXistAdministration:Browsingdatarecords
Backup,addingindexesandmore
Whereisthedatastored
2/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
H2embeddedrelationaldatabase
HowtoaccesstheH2databasefromaqueryclient
ObtainingtheJDBCDriver
JDBCconnectionstring
Howtosetuptheconnectioninyourqueryclient
Howtochangethedatatypeofanelementoncethedatamodelisdeployed
AdvancedMDMconcepts
Appendix
Howtoimportthetutorialprojectarchive
Versionused
:TalendOpenStudioforMDMv5.4.
Mission
:GetyouupandrunningquicklywithTalendOpenStudioforMDM!Iwillcoverthe
basicsandpointyoutofurtherresourcesforthemoreadvancedtopics.
Prerequisite
:YouarealreadyfamiliarwithTalendOpenStudioforDataIntegration,Linux
commandlineandhaveaSQLdatabaseinstalled.
Introduction
Masterdatamanagementhasgainedastrongmomentuminrecentyears.Sowhatisitactually
about?
DefinitionMasterDataManagement
Incomputing,masterdatamanagement(MDM)comprisesasetofprocessesandtoolsthat
consistentlydefinesandmanagesthemasterdata(i.e.nontransactionaldataentities)ofan
organization(whichmayincludereferencedata).MDMhastheobjectiveofproviding
processesforcollecting,aggregating,matching,consolidating,qualityassuring,persisting
anddistributingsuchdatathroughoutanorganizationtoensureconsistencyandcontrolinthe
ongoingmaintenanceandapplicationuseofthisinformation.(Source:
Wikipedia
)
Justtohighlightagain:TalendMDMServerissupposedtohold
nontransactional
datawhich
issharedacrossvariousbusinessdepartments.
TheTalendMDMServerstoresallthedatainanXMLdatabase(
eXist
).Allchangesonthe
MDMdataareversionedautomatically.
Forthistutorialwetakeontheroleofasecondhandcamerafranchisewhichhasbranches
acrossvariouscities.Eachstorekeepstheirownlocaldatabase.Theinitialaimistohavea
centrallistoflensmasterdata.
Installation
3/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Youcanfinddetailedinstructions
online
onTalendswebsite.HereIjustwanttoofferashort
summary.
Inanutshell(Linux/Ubuntu):
DownloadtheTOSMDMpackagefromthe
Talendwebsite
.
Thedownloadpackagewillbenamedlikethis:
TOS_MDMAllr
ReleaseNumber
V
VersionNumber
.zip.
Unzipthepackageandyouwillseethatitcontainstwofiles:
MDMStudio
:TOS_MDMServerr
ReleaseNumber
V
VersionNumber
.jar
MDMServer
:TOS_MDMStudior
ReleaseNumber
V
VersionNumber
.zip
Note
:AmandTOSMDMversionnumbersinallcommandslistedbelow.
Firstinstalltheserverfirstusingwizard:
cd~/Downloads
unzipTOS_MDMAllr84309V5.1.1.zipd./TOS_MDM
cd./TOS_MDM
javajarTOS_MDMServerr84309V5.1.1.jar
Followthewizard.
Pleasenotethatthecommunityeditiononlysupportstwodatabases:
eXist:XMLdatabase
H2:inmemoryRDBMS
Oncetheserversetupiscomplete,unziptheTOSMDMStudiozipfile:
unzipTOS_MDMStudior84309V5.1.1.zip
Thencopyittoaconvenientlocation.
Navigatetothe
TOSMDMStudio
rootdirectoryandrun:
chmod700*.sh
Nowyoucanstartthe
MDMStudio
likethis:
sh./TOS_MDMlinuxgtkx86.sh
Onstartupyouwillbeaskedwhichprojectyouwanttoaccess.Wewillcreateadedicated
projectforthistutorial:Clickonthe
Create
buttonandnametheproject
Tutorial
.
4/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Click
OK
.
Thenmark
Tutorial
andclickon
Open
.
TostarttheMDMserver,navigatetothe
MDMserver
rootdirectory:
cdjboss4.2.2.GA
chmod700*.sh
sh./run.sh
Openyourfavouriteinternetbrowserandcheckiftheserverisproperlyrunning:
WebGUI
Onsuccessfulinstallation,
http://localhost:8080/talendmdm
willshow:
Theopensourceversioncomeswithonlytwouseraccounts(itisrestrictedtothesetwoones):
standarduser
user:user
password:user
5/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
admin
user:administrator
password:administrator
UninstallingtheMDMServer
ShouldyouhavetouninstalltheMDMserveratsomepoint,youcanfindanuninstallutility
inside<jboss>/Uninstaller.Justrun:
sudojavajaruninstaller.jar
AddtheserverdetailstoTOSMDMStudio
Youshouldhave
TOSMDMStudio
alreadyrunning.Youwillrealizethatitcomeswith3
perspectives
(toprighthandcorner):
Makesurethe
MDMperspective
isselected.
Ifthe
ServerExplorer
viewisnotvisible,choosefromthemainmenu
Window>ShowView
andselect
ServerExplorer
.
Rightclickinthe
ServerExplorer
areaandchoose
AddServerLocation
:
ThenprovidetheMDMserverdetails:
6/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Toconnecttotheserveruse
:
server:
http://localhost:8080/talend/TalendPort
user:admin
password:talend
Theseaccountdetailsworkaswell,butyouwillnothaveadminrights:
user:administrator
password:administrator
Note:YoumighthavetoadjusttheportnumberanddomainintheserverURLdependingon
yoursetup!
Click
OK
.
Preparation
LetsfirstcreateasampledatabaseforourimaginarysecondhandcamerastoreinLondon.
Thisisjustrepresentationofoneofmanystores.Wewillusethisdatabaselateronto
synchronizethemasterdatawiththelocalstoredata.IwillbeworkingwithPostgreSQL.Incase
youareworkingwithanotherdatabase,pleaseamendtheSQLrespectively:
$psqlUpostgres
CREATEDATABASEstorelondon
\cstorelondon
CREATESCHEMAlenses
SETsearch_pathTO'lenses'
CREATETABLE
lens_metadata
(
lens_nameVARCHAR(100)
7/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
,vendorVARCHAR(30)
,release_yearINT4
,lens_categoryVARCHAR(20)
,min_focal_lengthINT4
,max_focal_lengthINT4
,min_apertureNUMERIC
,max_apertureNUMERIC
,min_focus_distance_mmINT4
,filter_size_mmINT2
,talend_mdm_keyINT4
)
INSERTINTO
lens_metadata
VALUES
('AsahiOpticalCo.SuperTakumar28mmf/3.522.0','Pentax',1962,'Wide',28,
28,3.5,22,40,58,null),
('CarlZeissJenaBiotar75mmf/1.5','CarlZeissJena',1951,'Short
Telephoto',75,75,1.5,1.5,80,58,null)
Pleasenotewethatwedecidedtoaddatalend_mdm_keycolumntoourrawdatasothatwe
canlateronstorethemdmkeyinhere.
Modeling
Beforewegetstarted,letsfirstgetacommonunderstandingofthemostimportantMDMterms:
Term
Description
(business)element
Alsoreferredtoasbusinessattribute.Theactualnameofthe
datapoint.
(business)entity
Describestheactualdata(theelements),itsnature,itsstructure
1
anditsrelationships.
Anentitycanhaveoneormorebusiness
elements.TheTalendMDMjargonforthisconceptis
data
modelentity
.
datamodeltype
Thisisanelementorcollectionofelementswhichisglobally
definedandcanbeusedacrossvariousentities.Thismakes
maintenanceofcommonelementseasier.
datamodel
Definestheattributes(elements),useraccessrightsand
relationshipsofentitiesmasteredbytheMDMHub.Thedata
modelisthecentralcomponentofTalendMDM.Adatamodel
mapstooneormore(business)entitiesthatcanbeexplicitly
8/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
1
defined.Anyconceptcanbedefinedbyadatamodel.
Adata
modelcanhavemultipleentities.
(business)domain
Acollectionofdatamodelsthatdefineaparticularconcept.For
instance,thecustomerdomainmaybedefinedbythe
organization,account,contactandopportunitydatamodels.A
productdomainmaybedefinedbyaproduct,productfamilyand
pricelist.
Ultimately,thedomainisthecollectionofalldatamodelsthat
relatetoaconcept.TalendMDMcanmodelanyandmany
domainswithinasinglehub.ItisagenericmultidomainMDM
1
solution.
datacontainer
Holdsdataofoneorseveralbusinessentities.Datacontainers
1
aretypicallyusedtoseparatemasterdatadomains.
Source:TalendMDMGuide
Youcanthinkoftheseonesasahierarchy:A
datamodel/domain
canhaveoneormany
businessentities
andabusinessentitycanhaveoneormany
elements
.
Howtocreateadatamodel
Rightclickon
DataModel
intherepositoryandchoose
New
:
Namethedatamodel
Lenses
andclick
OK
.Theoretically,lensescouldbepartofamuch
biggercontext,butletskeepitsimplehere.
Howtocreatedatamodelentities
Asalreadymentionedabove,datamodelentitiesaretheactualbusinessentities.Soinour
casewewillhavealensentity(Forthepurposeofthistutorialwekeepitverysimplehere).
Rightclickonthemaindatamodelareaandchoose
NewEntity
:
9/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Namethenewentity
Lens
andleaveeverythingelseempty:
Whatfollowsisashortdescriptionoftheavailableoptionsinthisdialogwindow:
Letshavealookattheavailable
types
:
Simpletype
:Usedforsingle,selfcontainedelementslikeemailaddresses.
Complextype
:Usedforstructureslikeaddresswhichconsistsofmultipleelements.A
complextypecanalsoinheritelementsfromanothercomplextype.
The
Complextype
hasfollowingoptions:Theyaregrayedout/disabledbydefault
All
:liststheentityinanysequence.Thisisthe
default
one.
Sequence
:liststheentitiesinthedefinedsequence
Choice
:tohaveachoiceontheentities
10/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Option:
Pleaseenternameofthecomplextype.Leaveblankforanonymous.
Ifyouspecifyanamehere,youcreateareusabletype.Thistypewillshowupunder
Data
ModelTypes
inthe
DataModel
designarea.Onceyoudefineaname,youwillalsobeableto
select
All
,
Sequence
or
Choice
.Ifyoudontdefineanamehere,
All
willbedefaultone.
Option:
Thiscomplextypeextendsanothercomplextype
Complextypescaninheritelementsfromanothercomplextype.Thisisveryuseful(and
advanced)feature.Example:YoucancreateanAddresstypewhichhasfirstnameand
surnameaselements,thenyouaddUSaddressandEUaddresstype,whichbothinheritthe
elementsfromtheAddresstypeandaddtheareaspecificelements.
Forourentityweleaveeverythingonthedefaultsettingsandclick
OK
.
Youwillseethatnowyouhaveanewentityunder
DataModelEntities
:
Howtodefineanautoincrementkey
Inourcasetheuniquekeywillbeanautoincrementedidentifier.Expandthe
Lens
entity
completelyandthenrightclick
subelement
,whichiseasytoidentifyasithasa
key
symbol
nexttoit.Choose
EditElement
:
Changethenameto
Id
andthenclick
OK
:
Usingautoincrementinteger
Rightclickonthekeyelementandchoose
ChangetoSimpleType
.Thenchoose
Custom
and
choose
AUTO_INCREMENT
as
Type
.
11/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Nextrightclickthe
AUTO_INCREMENT
elementandchoose
ChangeBaseType
:
Setthe
builtinType
to
Integer
:
UsingUUID
Thissectionisforyourinformationonly,hencedoonlyreadthis,donotfollowstepspractically:
Rightclickonthekeyelementandchoose
ChangetoSimpleType
.Thenchoose
Custom
and
choose
AUTO_INCREMENT
as
Type
.
12/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Nextrightclickthe
AUTO_INCREMENT
elementandchoose
ChangeBaseType
:
Thenchoose
Custom
andchoose
UUID
as
Type
.
Howtocreateelements
Wehavetodefineallthelensattributesnow,soletsstartwiththelensname.Rightclickon
Lens
andchoose
AddElement
:
Definethename
LensName
andset
MinimumOccurrence
to
1
.Thiswaythebusiness
13/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
elementhastobedefinedbytheenduser:
Thenclick
OK
.
Inthiscasethedefaulttype
String
isjustwhatweneed.
Theentityoverviewshouldlooklikethisnow:
Press
CTRL+S
tosavethemodel.
Note
:Analternativewaytoaddanewelementistorightclickonanexistingelement,liketheId
(ourkey)andchoose
AddElement(after)>AddstringElement
.Thisisabitmoreconvenient
asyoucanchoosethedatatypestraightaway[ThisfunctionisnotavailableinTalendStudio
v5.0]:
Nextcreateanewelementcalled
Vendor
whichwillbeoftypeStringaswell.Thiselementis
optional,sotheminimumoccurrencehastobe0.
Rightclickonthe
Lensentity
andchoose
AddElement
:nameit
ReleaseYear
.Thisonehasto
beoftype
Integer
,hencerightclickon
ReleaseYear
andchoose
ChangetoSimpleType
:
14/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Nowchoose
Integer
fromthebuiltintypes:
Click
OK
.
Sofaryoufamiliarwithaddingelementsinthefollowingways:
Fromtheentitynode,whichdoesntallowyoutospecifythedatatypestraightaway.You
learnthowtochangethedatatypelateron.
Fromanexistingelement,whichisthemostconvenientmethod,asyoucanchoosethe
datatypestraightaway.Irecommendusingthismethod.
Finallyaddalltheotherbusinesselements:
LensCategory:String
MinFocalLength:Integer
MaxFocalLength:Integer
MinAperture:Double
MaxAperture:Double
MinFocusDistanceMM:Integer
FilterSizeMM:Integer
Whencompleted,yourentityshouldlooklikethis:
15/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Save
yourmodel.Aswearequitecuriouspeople,wewouldliketounderstandwhatTalend
MDMStudioactuallydidunderthehoodforus.Clickon
SchemaSource
tabandtrytoread
throughtheschema:
RememberthatearlieronImentionedthatthedefaultcomplexTypeis
All
,ifyouleavethetype
setananonymous(ifyoudontcreateadedicatedtypewhichcanbeinherited).Asyoucansee
fromtheschema,thisiswhathappenedinourcase.
Youjustcreatedyourfirstbusinessentity!Wedontwanttomakeitmorecomplexatthispoint
intermsofmodelling.Justreflectonceagainwhatyouhavebeendoingsofarandthenyou
shouldbereadyforthenextstep.
Setlabelsanddescriptions
16/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Eachentityandelementshouldhavehumanreadable/prettyprintlabelsandaproper
description.ThegreatthingsaboutTalendMDMStudioisthatitincludesinternationalization,
meaningyoucanaddlabelsanddescriptionsinvariouslanguages/locals.
OurimaginarysecondhandcamerachainhappenstohaveastoreinAustriaaswell,hencewe
willnotonlyaddEnglishlabelsanddescriptions,butGermanonesaswell.
Rightclickonthe
LensName
businesselementandchoose
SetLabels
:
Choosethe
English
languageandtype
Lens
intotheinputfield,thenclickthepressthe
+
icon.
Nowchangethelanguageto
German
andwrite
Objektivname
,click
+
again:
Intheentityoverviewtheselabelswillnowshowupunderthe
annotations
element:
Nowletsaddadescription,sothatendusersknowexactlywhatinformationtheyhaveto
provide:Rightclick
LensName
againandchoose
Setthedescriptions
.Theapproachis
similartothelabels,soIwillnotgothroughitagain.Pleaseinsertfollowingtext:
English
:
Thefullnameofthelensasengravedonthelens.
German
:
17/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
DervollstaendigeNamewieamObjektivablesbar.
Save
yourmodelagain.
AllthisinternationalizationwillcomeintoplaywhentheenduserslogontotheTalendMDM
webinterface.IfinexampleanAustrianuserlogson,allthelabelsanddescriptionswillbe
displayedinGerman.
Definedisplayformatfordateandnumbers
Justsomeshortinfo:TalendOpenStudioforMDMmakesuseofthejava.util.Formatterclass.
Havealook
here
foradetaileddescription.Rightclickonaelementandchoose
Setdisplay
format
.
NotethatJavaautomaticallysubstitutesthelocalspecificdecimalseparator.
Howtodeployadatamodel
Onceyouhavefinalizedyourdatamodel,youcandeployittotheMDMserver.Rightclickthe
datamodelintherepositoryandandchoose
DeployTo
.
NOTE
:Ifyougetanerrormessagethatyourdatamodelcannotbedeployedbecauseitis
locked,makesurethatyoucloseyourdatamodelincaseitisopeninthedesignarea.
Choosetheserveryouwantthedatamodeltobedeployedto.Youwillgetasuccessmessage
ifdeploymentprocesscompleteswithoutproblems.Note,thattheiconnexttothedatamodelin
therepositorywillchangeslightlyanddisplayagreenplaysymbol.Also,nexttothedatamodel
nameyouwillfindtheservername.
DataContainer
Allthemasterdataisstoredina
DataContainer
.Adatacontainercanholdthedataofvarious
18/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
businessentities.Notethatabusinessentitystoredinonedatacontainerisnotvisiblefrom
anotherdatacontainer.
Tocreateadatacontainer,simplerightclickonthe
DataContainer
intherepositorytreeand
choose
New
:
Defineaname:
Important
:Sinceversion5.2the
datacontainerandthedatamodelmusthavethesame
name!
(seealso
here
)
Oncethecontainerisdefined,rightclickonitandchoose
Deployto
todeployyourdata
containertothe
MDMserver
.
Ifeverythinggoesfine,youwillgetasuccessmessageandseeaplayiconnexttothedata
model:
Note:
Itisworthkeepinganeyeontheserverlog.TheTalendMDMStudiooffersafunctionto
showtheserverlog:Inthe
ServerExplorer
tab,rightclickonaregisteredserverandchoose
ViewServerLog
:
19/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Anewtabcalled
Console
willbeactivated:
Ifyouseefollowingerror(orsimilar)inthelog:
Causedby:java.lang.UnsupportedOperationException:Datamodel
'SecondHandLenses'mustexistbeforecontainer'SecondHandLenses'canbe
created.
Thiserrorwasraisedbecauseyour
datacontainer
doesnothavethesamenameasthe
data
model
.Tofixthisrenameyourdatacontainertoexactlythesamenameasyourdatamodel
andthendeployit.
Pleasenotethatinv5.0this1:1relationshipdatacontainerdatamodelwasnotenforced.This
mighthavetodowiththerecentadditionofRDBMS.ThelasttimeItestedtheexampleproject
discussedinthisbookwaswithTalendMDMv5.3usingtheembeddedH2databaseandIgot
theexceptionshownabove.Someofthecontentinthisbook(especiallyscreenshots)arestill
showingadatacontainername
SecondHandLenses.
Ididnthavethetimetoupdatethose
screenshots,sopleasekeepthisinfoinmind.
Howtocreateaview
Aviewisbasicallywhatanendusercanseeviathewebinterface,whichincludestheformand
searchfunctionality.Therearevariousviewsthatyoucancreate.Wewillonlyhavealookatthe
mostsimpleviewhere,whichbasicallywillallowenduserstocreatethebusinessentityonline
andsearchforvalueswithincertainattributes.
Toautogenerateaview,openthe
businessmodel
andrightclickontheentityforwhichyou
wanttocreateaview.Choose
GeneratedefaultBrowserItemsViews
:
20/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Inthefollowingdialogclickon
Finish
:
Theautogeneratedviewwillshowupinthe
View
nodeoftherepositorybrowser:
Doubleclickontheviewelementtoinspectitinthedesignview.Thedesignsectionisdivided
into
viewable
and
searchable
businesselements.Theautogeneratedviewmightnotinclude
alltheelementsyourequire.YoucansimplyaddadditionalelementsbyspecifyingtheXPath.
TocreatetheXPath,youcanuseavisualnavigator,soevenusersnotfamiliarwithXPathcan
easilycreatetheviews.
Inourcasewecanseethatonly
Id
wasaddedautomatically.Tospeedthisprocessup,clickon
21/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
the
AddMultiple
button:
Marktheelementswhichyouwanttoaddtotheview(InourcaseeverythingexcepttheId)and
click
Add
:
Inthe
SearchableBusinessElements
sectiondeletethe
Id
elementandaddthe
LensName
:
Thiswillallowtheenduserstosearchforrecordsby
LensName
.
Oncedone,
save
theviewand
close
it(thedesignviewonly,notTalendOpenStudio).Then
rightclickontheviewintheMDM
repositorybrowser
andchoose
UpdateServer
.
22/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Note
:LaterversionsofTalendMDMsupportstoringthedatainrelationaldatabases(RDBMS).
PayspecialattentiontothestatusmessagethatisshownafterpublishingtheviewontheMDM
server:
Asyoucanunderstandfromtheinfotext,redeploythedatamodeltotheMDMserversothat
theappropriateindexesontheRDBMScanbecreatedautomaticallyforthesearchable
elements.Thisisquiteaconvenientfeature!
Inordertobetterunderstandwhatyouhavecreatedsofar,openyourfavouritewebbrowser
andnavigateto
http://localhost:8080/talendmdm
(AdjustURLaccordinglytoyoursetup).Logon
withuser
user
andpassword
user.
Followthesesteps:
1. Onthelefthandside,setthe
DataContainer
to
Lenses
and
DataModel
to
Lenses.
2. Ontherighthandside,doubleclickonon
DataBrowser
.
3. Inthemenu(inthetopcentreofthescreen)choose
Lens
fromtheentitiespulldown
menu.
23/97
DiethardSteiner
4.
5.
6.
7.
TalendOpenStudioForMDM:APracticalStarterGuide
Clickon
Create
toaddsomedata.
Fillouttheform.
Clickon
Save
.
Thenewrecordwillshowupinthedataarea.
Congratulations!YouhavejustcreatedyoufirstMasterDatamodelandprovidedenduserswith
awebinterfacetomanagethedataofthismodel.
TheUpdateReport
TheTalendMDMserverkeepsachangelogcalled
UpdateReport
.ThedataisstoredinXML
formataswell.
TheUpdateReport
Wheneverarecordiscreated/updated/deleted(CRUD),the
MDMServer
logsthisactivityintheUpdateReportthisissimilartoanauditreport.This
logholdsinformationaboutwhoadded/updated/deletedarecord,when,for
whichdatacontainer,datamodel,businessentityetcsoinanutshellsome
metadata
butitdoesnotcontainthedatarecorditself
.Incaseofupdates,
theoldandnewvaluesofabusinessattributearepartoftheUpdateReport.
ThisUpdateReportisthensenttothe
EventManager
,whichevaluatesall
Trigger
conditionsagainstthisreport.Ifaconditionismet,the
Event
Manager
willroutethereporttothespecifiedprocessorjob.
Youwillbepleasedtohearthatthereisadatamodelforthe
UpdateReport
aswellforyouto
inspect.Under
DataModel>System
doubleclickon
UpdateReport
:
Nowyoucananalysethestructureofthisdatamodelinthemaindesignarea:
24/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Letsimportthe
UpdateReport
datacontainerfromtheMDMserversothatwehavesomedata
toplaywith:
Clickthe
ImportServerObjectsfromMDMServer
icon:
Choose(1)therespectiveserverand(2)untickallelementsintheimportsectionandthentick
the
DataContainer>System>UpdateReport
element:
25/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Click
Finish
.
The
UpdateReport
willnowshowupinyourlocalrepositoryunder
DataContainer>System>
UpdateReport
.Doubleclickonit:
Specifythesearchcriteria(ifanyinourcaseyoucanjustleaveeverythingempty),runa
searchand
doubleclick
ontherecordforwhichyouwanttoretrievetheXMLfragment:
IntheXMLviewerclickthe
Source
tabtoseetheXMLfragment:
26/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Justtohighlightitoncemore,the
UpdateReport
doesnotcontaintheactualdata(thefull
record).Letslookatsomeexamples:
UpdateReport
foranewlycreatedrecord:
AsyoucanseetheXMLcontainsalltheMetadataontherecord(Who,Where,When,).The
Keypointstotheactualdatarecord.
UpdateReport
foranupdatedrecord:
27/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Incaseofanupdate,theUpdateReportcontainstheoldandnewvaluesoftheaffectedfields.
DataIntegration
Aftercreatingthemodelandtheview(s),thenexttaskistoloadexistingdataintotheMDMdata
containerandsetupprocesseswhichinexamplesynchronizenew,deletedandupdated
recordswithremotedatabases.Inourcasewewanttoloadalistofexistinglensmetadataand
thenmakesurethateachnewandupdatedrecordgetssynchronizedwithallourstores,which
eachholdanindividualdatabase.
ItisbestifyouchangetheTOSperspectiveto
Integration
now:
GettingStarted
Itsgoodpracticetostoretheconnectiondetailsinthe
TalendMetadata
:
Createaconnectionforyourdatabaseandretrievethe
lens_metadata
schema.Iassumeyou
alreadyknowhowtosetupadatabaseconnection,henceIwillnotdiscussthisintoomuch
detail.
Createthesourcedatabaseconnection:
1.
Rightclickon
Dbconnections
andchoose
Createnewconnection
.Nameit
DBConnectionLenses
.
2.
Onthenextscreenprovidetheconnectiondetails.Thedatabaseiscalled
storelondon
andthe
schema
lenses.Click
Check
tomakesurethattheconnectiondetailsare
correct.
3.
Rightclickon
DbConnections>DBConnectionLenses
andchoose
RetrieveSchema
:
28/97
DiethardSteiner
4.
TalendOpenStudioForMDM:APracticalStarterGuide
Click
Next
andonthesecondscreenexpandthenodesandthentick
lens_metadata
:
5.
Click
Next
.
Onthenextscreenyoucanseeallthecolumnsofthetable.Youcanmapthe
DB
columnnames
toTalendDataIntegration
columnnames
.Forthe
databaseschema
definition
setthe
columnnames
tothesamenamesasyoudefinedtheminthe
businessentity
.Doingitthiswaysavesuscreatinganadditionalmappingstepinthe
DIjobs.
Note
:Currently,whenyouchangethe
Column
name,alsothe
DbColumn
namegetsautomaticallysettothesamevalue.Mysuggestionisthatyoufirstcopythe
original
Db
Column
namebeforechangingthe
Column
name.Onceyouchangedthe
Column
name,simply
pastetheoldvaluebacktothe
DbColumn
.
5.1.
Alsotick
Key
forthe
talend_mdm_key
,sothatthisfieldisusedasakeyfor
updatesanddeletions.Notethatherethe
Column
namehastobe
Id
,thesame
aswedefinedinourdatamodel.Thereasonwhywesetthisfieldasakeyisthat
wewanttouseitlaterontoidentifyupdatesanddeletions.
5.2.
Movethe
Id
(talend_mdm_key)columntothetop(sothatitistheveryfirstrow)
byusingthearrowupbutton.Thiswillfacilitatethemappinglaterontothe
TalendMDMdataset.
5.3.
Makesuretosetthe
Type
correctlyforeachcolumn.
5.4.
Yourdatabaseschemadefinitionshouldlooklikethis:
29/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Click
Finish
.
NowletsfocusontheMDMconnection:
Rightclickon
TalendMDM
andchoose
CreateMDMconnection
.Specify
MDMConnectionLenses
asthenamefortheconnection,click
Next
.Thenfillintheconnection
detailsasshownbelow:
server
:localhost
port
:8080
username
:admin
password
:talend
Alternatively:
username
:administrator
password
:administrator
Adjustserverandportdetailsaccordingtoyoursetup.
Clickon
Check
tomakesurethatyouprovidedthecorrectconnectiondetails.Thenclick
Next
.
Onthefollowingdialogchoose
Lenses
forthedatamodeland
Lenses
forthedatacontainer:
Click
Finish
.
30/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Thenrightclick
MDMConnectionLenses
andchoose
RetrieveEntity
:
FirstwecreatetheentitydefinitionfortheinputtoTalendDI:
Click
Next
twice,thenthewizardallowsyoutodefineanXMLmapping:
1. Draganddrop
Lens
intothe
AbsoluteXPathexpression
section.
2. Markalltheotherfieldsanddraganddropthemintothe
Fieldstoextract
section.
Click
Next
.Nowyouhaveachancetochecktheschemaandamenditifnecessary.Onceyou
arehappywithit,click
Finish
.
Rightclick
onthe
LensIn
nodeintherepositorytreeandchoose
ReadEntity
:
31/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Click
Next
severaltimesuntilyouarriveonthe
schema
definitiondialog.Adjustthecolumn
typeswherenecessary:
Thenclickon
Finish
andconfirmtopropagatethechanges.(Note:Theveryfirsttimeyouset
thisup,youmightnotgettotheschemascreen.Simplyrightclickonthe
LensIn
nodeand
choose
EditEntity
then).
Nextwewillcreatethedefinitionforthe
MDMoutput
.Rightclickagainon
MDMConnectionLenses
andchoose
RetrieveEntity
.Thistimechoose
OutputMDM
andclick
Next
twice.
Thewizardwillthenallowyoutosetupthemapping.Pleasenotethatthemappingshownis
basedonwhatTalendMDMStudioknowsaboutthedatamodel(asyoucanseebothsides
lookidentical).The
SchemaList
representsthefieldsinourdataintegration
job
,the
Linker
Target
listrepresentsthefields/elementsinourdatamodel.
32/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Inourcasethemappingissimple,becausewewillmakeuseofallthefields.Ifyouhaveto
adjusttheschemaforsomereason,youcandosoviathe
SchemaManagement
button.
Clickon
Finish
.
Finally,wewillsetuptheMetadatafortheMDMreceiveoption:
1. Rightclickagainon
MDMConnectionLenses
andchoose
RetrieveEntity
.Thistime
choose
ReceiveMDM
andclick
Next
.
2. Set
XPathPrefix
to
/exchange/item
.Thendraganddropthe
Lens
elementfromthe
SourceSchema
onthefirstrowofthe
AbsoluteXPathloopexpression
.Nextmarkall
theremainingsourceschemafieldsanddraganddropthemon
Fieldstoextract
.
3. Clickon
Finish
.
33/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
4. Rightclickon
LensReceive
inthe
TalendDIMetadata
treeandchoose
EditEntity
:
5. Clickseveraltimes
Next
twice.Nowyouhavetoopportunitytocorrecttheautomatically
generated
Schema
.Inourcase,allisfineasitisnochangesrequired:
6. Click
Finish
andagreetopropagatethechanges.
LoaddataintotheMDMhub
AlltheexamplesassumethatyouarefamiliarwithTalendDI.Iwillnotdiscussindetailonhow
tocreatethejobs,butmainlyprovideabriefdescription.
NOTE:
AlldataloadedintotheMDMserverhastobeinXMLformat!
IMPORTANT:
TheattributenamesoftheMDM
datamodelentity
are
casesensitive
!Soid
andIdaretwodifferentattributes.Makesurethatyoupayattentiontothisasthiswillavoid
errors.
IMPORTANT:
Ifyourbusinessentityhasanautoincrementkey,thenthiskeywillbegenerated
automatically,hencedonotattempttoimportit.
Standardload
Createanewjobcalled
StandardLoad
.Thisverysimpleexamplewillillustrateloadingasmall
datasetfromaPostgreSQLdatabase:
34/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Thisexamplecanbecreatedextremelyquickly:
1. Justdraganddropthe
lens_metadata
Metadata
connectionsontothecanvas.When
promptedforthecomponenttype,chooseasuitableinputcomponent.
2. Adda
tMap
component
3. Fromthe
Metadata
repositorydraganddropthe
LensOut
entityontothecanvas.
Connectallthreecomponentsasshownonthescreenshotabove.Thecomponent
settingsareallprepopulatedalready,soitisallsetupforus!
4. Doubleclickonthe
tMap
componentandmapthefieldsasshownbelow.Wedonot
importtheId(firstofallitisnotpopulatedanyways[wewillupdatethisfieldlateronwith
theTalendMDMIds]andsecondofallTalendMDMwillcreatetheIdsforus):
35/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Executethejobnow.
ThenextsteponlyappliesifyoudidnotfollowtheMDMmetadataconnectionsetupinthis
tutorialorarenotusingaMDMmetadataconnection:
Inthe
tMDMOutput
componentsettingsprovidealltheconnectiondetailsfortheMDMserver,
thedatamodelandthedatacontainer.
ThiscomponenthasabuiltinfeaturetocreateXMLbasedontheinputrow.Toaccessthis
feature,doubleclickonthecomponentinthedesignarea.
CreatetheXMLstructure:
1. Doubleclickonthecomponentinthedesignarea.
2. TheXMLroottaghastohavetosamenameasthe
datamodelentity/businessentity
youwanttopopulate!Sorenameitaccordingly.Note:Thisis
casesensitive
!
3. DraganddropthesourcecolumnsontotheXMLroottag:
SpecifyhowyouwantthemtobeaddedtotheXMLstructure.Thesubelementshere
representthe
entityattributes
.Theyare
casesensitive
!
4. Defineoneelementasthe
loopelement
.Inourcase,itistheidelement.Rightclickon
theelementandchoose
Setloopelement
.
Click
OK
.
5. Nextwehavetochangetheschema.Basically,nowthatwehaveourXMLstructure
defined,weenduphavingonlyonecolumnlefttohandle.Inthe
Componentsettings
clickontheeditbutton(...)anddeleteallcolumnsapartfromthefirstone.Changethe
nameofthefirstcolumntosomethinglikexmlRecord.
6. Inthe
componentsettings
specify
xmlRecord
asthe
XMLField
.
CheckingthedatacontainerrecordsviatheMDMServerwebinterface
Nowletscheckifourjobworkedproperly.LogintotheMDMwebsiteandsearchforthelens
records:
36/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
AsyoucanseeourdatabaserecordsshowupontheMDMserver.Notethatyouridsmightbe
differentfromtheonesshownonthescreenshot.
CheckingthedatacontainerrecordsviaTalendMDMStudio
Youcanalsochecktherecordsfromwithin
TalendMDMStudio
.Todoso,
rightclick
on
Onthenextscreenclicktheellipsisbutton[...]andchooseyour
MDMServer
.Oncetheserver
repositoryisloaded,click
DeselectAll
andthenexpandthe
DataContainer
nodeandtick
Lenses
:
37/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Click
Finish
.
Nowyouhavethesamedatasetavailableinthe
TalendMDMStudio
asonthe
TalendMDM
Server
.
Nextdoubleclickon
Lenses
in
LocalRepository>DataContainer
.ConfirmtheServer
selection.Thenthe
DataContainerBrowser
willopen.Aswehaveonlyafewrecordsloaded,
hitthe
Search
buttonstraightawaywithoutspecifyinganysearchcriteria.
Entityrecordswillbelistedbelow.Doubleclickononerowtoretrievethedetails,whichwillbe
showninthe
XMLViewer
:
Checkingthedatacontainerrecordsviayourdatabasesqueryclient
Thethirdoptionofcourseistochecktherecordsdirectlyinthequeryclientthatissuppliedwith
yourdatabasepackage.
TheembeddedTalendMDMServerdatabaseeXistcomeswithawebinterfacewhichallows
youtoconvenientlyanalyzetherecordswithouthavingtosetupanadditionalqueryclient.
IncaseyouarestoringyourrecordsinastandaloneRDBMSyouarequitelikelyfamiliarwith
thededicatedqueryclient,soIwillnotcoverthishere.
Bulkload
Note
:
tMDMBulkLoad
hassomelimitationswhenbulkloadinganXMLdatabasesee
ComponentGuidefordetails.
Createanewjobcalled
BulkLoad
.Iwillonlyprovideashortdescriptionhere,astheprocessis
similartothestandardload.
Themaindifferencetothestandardloadisthatthe
tMDMBulkLoad
componentdoesnotcome
withabuiltinXMLtransformationfeature,hencewehavetouseadedicatedXMLcomponent
(
tWriteXMLField
)beforehand.
Youcandraganddropthe
lens_metadata
MDMdefinitionasadatabaseinputcomponentonto
thedesigncanvas.Addthe
tWriteXMLField
and
tMDMBulkLoad
componentsviathe
Palette
.
Linethemupandconnectthemasshowninthescreenshotbelow:
38/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Letstalkaboutsomedetails:
1. Inthe
tWriteXMLField
componenttabclickon
ConfigureXMLTree
:
2. InthenewdialogrenametheXML
rootTag
inthe
Linktarget
(
ontherighthandside
)
to
Lens
(ourbusinessentityname).
3. Markallattributesontheleftanddraganddropthemontothe
Lens
rowontheright
handside:
4. Tick
Createassubelementoftargetnode
andclick
OK
:
5. MarktheLensNamerowbyclickingontheveryleftside.Thenrightclickandchoose
Set
AsLoopElement
:
39/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
6. Themappingshouldnowlooklikethis:
7. Click
OK
toclosethedialog.Nextclickon
EditSchema
inthe
Component
tab:
8. AllourinputcolumnsgetbasicallyfoldedintoonecolumnwhichholdstheXML.Hence
40/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
wehavetochangetheschemaaccordingly.Wewillonlyhaveoneoutputcolumn,
thereforedeletealloutputcolumns(ifthereareany)andaddanewonecalled
xmlRecord
oftype
String
andchangethe
Length
toalargervalue(orjustleaveit
empty):
Oncefinishedclick
OK
.Youwillseethethe
OutputColumn
isnowsetto
xmlRecord
in
thecomponentsettings.
9. Click
OK
.Apopupwindowwillcomeupconfirmthatyouwanttopropagatethe
changes.
Instructionforthe
tMDMBulkLoad
component:
1. Click
Synccolumns
tosourcetheschemadefinitionfromtherow.
2. ProvidealltheMDMserverdetails.Currentlythiscomponentdoesnotsupportsourcing
theconnectiondetailsfromthe
TalendDI
Metadata
,henceyouhavetofilloutallthe
requiredconfigurationfields.
3. Makesurethatyoutick
Validate
sothatthedataisvalidatedagainsttheMDMdata
model.
4. Ifyourbusinessentityhasanautoincrementedkey(whichistrueforourexample),tick
GenerateID
.
Onceyouarefinished,runthejob:
41/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Allshouldexecutesuccessfully.Ifyoudoasearchviathewebinterfaceforall
Lens
records,
youshouldnowseethesenewrecords:
Forthepurposeofthistutorialweimportedthesamerecordstwice,whichshouldneverhappen
inanormalsetup.Also,weseeaclearproblemwithourbusinessentity:Thelensesshouldbe
unique.Normallywewouldsetauniquekeyonthelensname,butasthelensnameisalonger
stringandcancontainspecialcharacters,itisnotpossibletodothisinTalendMDM.
TheoreticallyyoucansetauniquekeyonitandnotuseadedicatedIdentifier,butyouwillsoon
releasethatTalendMDMisnothappywithenforcinguniquekeysonstringswithspecial
characters.
Thereisasolutionthough:YoucancreateadedicatedDIjobwhichcheckseachtimethe
uniquenessofthelensnamesbeforeanewrecordgetsinsertedusingTalendMDMtriggers
moreabouttriggerslateron.
Deletetheduplicatedrecordsviathewebinterfacefornow.
ExportdatafromtheMDMserver
Createanewjobcalled
Export
.Thepurposeofourexportjobistobasicallyupdatetheexisting
recordsintheexternaldatabasewiththekeysthattheTalendMDMhubcreatedforthem.In
ourcase,thisisaoneoffoperation.Afterthis,weonlyintendtochangerecordsviatheTalend
MDMwebinterfaceandhaveanotherDIjobupdatetheexternalDB.
1. Draganddropthe
LensIn
metadataschemaonthedesigncanvas.Thiswill
automaticallycreatea
tMDMInput
prefilledwithalltheTalendDIMetadatadetails:
42/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
2. Leave
Usemultipleconditions
ticked.Deletethedefaultentryifoneshowsupinthe
operationstable.The
Operations
tablecanremainempty!
3. Clickon
AdvancedSettings
.Notethatthe
LoopXPathquery
isalreadyspecified.This
willnormallybeyour
businessentity
name.Ifyoudonotusethe
TalendDI
Metadata
definition,makesureyoualwaysaddthepreceding
/
whenspecifyingtheXPath.Aswe
use
TalendDI
Metadata,wedonthavetoworryaboutthis.Youcanevenseebelow
thatattributeshavebeenalreadydefined(Note:Thisis
casesensitive
!).Wedonot
wanttoretrievetheXMLtags,henceweleave
GetNotes
unticked:
4. Fromthe
Repository>Metadata
draganddrop
lens_metadata
onthedesignareaand
chooseasuitableDBoutputcomponent.Createamainrowfrom
LensIn
tothis
component.
5. Clickonthe
DBoutput
componentandthenonthe
Component
tab.Set
Actionon
data
to
Update
.
6. Clickon
EditSchema
,thenchoose
Changetobuiltinproperty
.Forthe
Output
datasetchangethekeyto
LensName
:
43/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Asyouareaware,currentlywecanonlyupdatetherecordsintheexternalDBbasedon
thelensname.Afterweranthisjob,wewillbeabletousetheTalendMDMkey(Id)in
otherdataintegrationjobs.
Click
OK
.
7. Tick
Dieonerror
.
8. Run
thejob.
9. Inyourfavouritequeryclient,checkiftherecordsintheexternalDBshowtheTalend
MDMkeys:
Realtimedataintegration/propagation
TalendMDMcomeswithpowerfulprocessandtriggerfeatures,whichallowinexampleinstant
propagationofchangestoothersystems.Wewillcreateonejobwhichpropagatesupdatesand
newrecordsandanotherjobforthedeletedrecords.Thefinalsectionwilltrytocombinethese
jobsintoone.
Processes
Aprocessconsistsofvarioussteps(plugins)whichareexecutedinthespecifiedordersoit
behavesprettymuchlikeaworkflow.
Youcanquicklycreatesimpleprocessesandtriggersbyrightclickingonthedataintegration
jobsintheMDMperspectiveandchoosingtheautogeneratefeature.
BeforeSavingandBeforeDeletingProcessesarespecialtypeofprocesseswhichcanbeused
toperformanactionbeforethedatarecordsgetssavedordeleted.
Triggers
Youareprobablyfamiliarwithtriggersfromadatabasecontext.Theconceptoftriggersonthe
TalendMDMserver
isquitesimilar:Acertaineventcantriggeracertainprocess.
Followingeventsexist:
44/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
CREATE
UPDATE
LOGIC_DELETE
PHYSICAL_DELETE
YoucandefinevarioustriggerconditionswhichcanbebasedonvariousfieldsfromtheUpdate
Report(YouwilllearnmoreabouttheUpdateReportabitlateron):
TriggerXPathExpressionsOverview
XPath
Description
PossibleValues
Update/Source
whatinitiatedthisevent
genericUI,...
Update/OperationType
PHYSICAL_DELETE,
LOGIC_DELETE,UPDATE,
CREATE
Update/DataCluster
DataContainer
Update/DataModel
BusinessModel
Update/Concept
BusinessEntity
Update/Item/path
Businessattributewhichgot
changedincaseofanupdate
TheExchangeDocument
Inmostcases,youwillsendanXMLdocument(letscallitExchangeDocument)fromthe
triggerorprocesstoyourdataintegrationjob,whichconsistsofthethe
Update
Report
andthe
actualdatarecord
(inXMLformat).Therootelementofthisdocumentis
exchange
.
TriggerssendthisExchangeDocumentbydefault(noadditionalconfigurationrequired),
nonintegratedprocesseshowevernot.Fornonintegratedprocessesyouhavetocombine2
stepstoachievethesameoutcome(moreaboutthislateron).
SomesectionsoftheExchangeDocumentlookdifferentdependingontheaction(create,
update,delete).IfyouwanttolearnmoreabouttheXMLstructure,takealook
here
.
CRU:Newrecordsandupdates
WewillcreatethenexttwoDIjobstheoldway,sothatyoulearnthisapproachaswell.The
thirdexample(
CRUDIntegrated
)willbebasedonthenewwaythe
integrated
waybut
moreaboutthislater.
45/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Creatingthedataintegrationjob
Createanewjobcalled
RealTimeDI_CRU
.Wewillcreatethisextremelysimpleexample:
Letsgetstarted:
1. Fromthe
TalendDIMetadata
draganddroptheMDM
LensReceive
schemaonthe
designcanvas.Thiswillcreatea
tMDMReceive
componentwhichisdesignedtoreceive
MDMXMLrecordsfromTOSMDMtriggersandprocesses.[*Ifyouarenotusing
TalendDIMetadata
pleasefindsomeinstructionattheendofthissection].
2. Fromthe
TalendDIMetadata
draganddroptheDBconnection
lens_metadata
schemaonthedesigncanvasandchoosethe
databaseoutput
component.
3. Connectthetwocomponentsasshowninthescreenshotabove.
4. The
tMDMReceive
componentwillreceiveanXMLdocumentfromtheMDMserver.
ThisXMLdocumenttheExchangeDocumentconsistsoftheUpdateReportandthe
actualdatarecord.Aswedonotknowthenameofthisfilebeforehand,wewillcreatea
contextvariablesothatitcanbesuppliedatruntime.Createa
contextvariable
called
xmlInput
:
5. Clickonthe
Valuesastable
tabandinsertthefollowing
XMLfragment
intothe
Default
valuefield.ThisisactuallyaspeciallypreparedXMLdocumentwhichcontainsthe
UpdateReport
ANDtheactual
datarecord
.ItisthesameXMLfragmentwhichwillbe
outputbytheMDMtriggerorprocesswhichwewillcreatelateron:
<exchange
xmlns:mdm="java:com.amalto.core.plugin.base.xslt.MdmExtension"><report><Update
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"><UserName>administrator</
46/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
UserName><Source>genericUI</Source><TimeInMillis>1336311970956</TimeInMillis>
<OperationType>CREATE</OperationType>
<RevisionID>null</RevisionID><DataCluster>
SecondHandLenses</DataCluster><DataModel>Lenses</DataModel><Concept>Lens</Conce
pt><Key>39</Key></Update></report><item><Lens
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"><Id>39</Id><LensName>Asah
iSuperTakumar55
1.8</LensName><Vendor>Pentax</Vendor><ReleaseYear/><LensCategory/><MinFocalLeng
th/><MaxFocalLength/><MinAperture/><MaxAperture/><MinFocusDistanceMM/><FilterSi
zeMM/></Lens></item></exchange>
ThiswillallowustotestourDIjobindependently!
6. Inthe
LensReceiveComponent
settingsspecify
context.xmlInput
for
XMLRecord
.
Alltheothersettingsarealreadyprovidedbythe
TalendDIMetadata
whichwesetup
before:
7. Next,clickonthe
databaseoutputcomponent
,whichisalreadymostlyprepopulated.
Justset
Actionondata
to
Insertorupdate
andtick
Dieonerror
:
8. Save
thejobandrunit.
47/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
*Incaseyouhavenotbeenfollowingthistutorialallthewaythroughoryouarenotusingthe
TalendDIMetadata
,justdraganddropthe
tMDMReceive
componentfromthe
Palette
and
specifythefollowing:
Definethe
Schema
XMLRecord
:Setupajobcontextvariablewiththesamenameastheparametername
ofthe
MDMprocess
whichcallsthisjob.
XPathPrefix
:Choose/itemincasetheXMLrecordiscomingfromaprocessand
/exchange/itemincasetheXMLrecordiscomingfromatrigger.
LoopXPathquery
:Thebusinessentity.
IMPORTANT
:Thishastobeprefixedwitha
forwardslash!
Mapping
:Definehowtheattributesofthebusinessentitymatchtothe
schema
.
Creatingthetrigger
1. IntheMDMperspective,rightclickonthejobintherepositoryviewandchoose
GenerateTalendJobCallerTrigger
.
2. Inthe
EventManagement>Trigger
sectionyouwillfindtherecentlycreatedtrigger.
Doubleclickonittoopenit(ifitisnotalreadyopen).
3. Mostsettingsarealreadypreconfigured,weonlyhavetomakeafewchanges:Inthe
Service
sectionchangethenameoftheparameterintheXMLfragmentto
xmlInput
(whichhappenstobethesamenameasyouassignedtothecontextvariableinthedata
integrationjob):
4. RemovetheDeleteconditionfromthe
TriggerXPathExpressions
andthefollowing
conditions,sothatwelinkthetriggertotherightdatacontainer,datamodelandentity:
XPath
Operator
Value
Condition
Update/DataCluster
Matches
SecondHandLenses
C3
48/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Update/DataModel
Matches
Lenses
C4
Update/Concept
Matches
Lens
C5
5. Settheconditionsto:
(C1OrC2)ANDC3ANDC4ANDC5
6. Save
thetriggeranddeployittotheMDMserver.
Testing
ViatheTalendMDMwebinterfaceaddandupdateafewrecordsandcheckifthesechanges
arereflectedintheexternalDB.KeepaneyeontheJBosslog.
CRUD:RoutingintheDIjob
Normallyyouwoulduseprocessesand/ortriggerstorouteupdates,newrecords,deletionsto
certainjobs.
Butthereisalsoawaytoachievethesamefunctionalitywithinyourdataintegrationjob,andit
isindeedaveryelegantsolution!
Thisisveryusefulinexampleforsynchronizingthedataasyoucanimplementthewholelogic
withinonejob.
Thewaytoachievethisistosendthe
ExchangeDocument
toyourDIjob.Thisonechecks
thenwhatkindofaction(
update,creation,deletion)
wasperformedandtakesactions
accordingly.
Dataintegration
49/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Letsgetstarted:
1. Createanewjobcalled
RealTimeDI_CRUD
.
2. Addacontextvariablecalled
xmlInput
oftypeString:
Thiscontextvariablewillbeusedtostorethe
ExchangeDocument
whichoriginates
fromaprocessortrigger.
Youcanprovideavalueaswell.Clickontba
Valuesastable
andcopyoneofthe
ExchangeDocumentslistedin
TheIntegratedversionofCRUD>ExchangeDocument
inthere,whichmakesittheneasytotesttheDIjob.
3. Adda
tFixedFlowInput
componenttothecanvas.Clickon
EditSchema
inthe
Component
settings.Createanewcolumncalled
MDM_Message
oftypeString:
4. Inthe
Values
tableclickinthe
Value
fieldnextto
MDM_Message
:Press
CTRL+SPACE
andchoosethecontextvariablewedefinedearlieron:
context.xmlInput
.
5. Thenextstepsareexactlythesameasdescribedin
TheIntegratedversionofCRUD
(WearejustusingherethetFixedFlowInputinsteadofthetMDMInputcomponent).
50/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Creatingthetrigger
1. Rightclickon
RealTimeDI_CRUD
andchoose
GenerateTalendJobCallerTrigger
:
2. Inthenextdialogchoose
Throughacontextvariable(backwardcompatibility)
and
Embedded(alwayson,nolatency)
:
3. Under
EventManagement>Trigger
youwillnowfindatriggercalled
CallJob_RealTimeDI_CRUD
.Doubleclickonit.
4. Inthe
Service
sectionchangethenameoftheparameterintheXMLfragmentto
xmlInput
(whichhappenstobethesamenameasyouassignedtothecontextvariable
inthedataintegrationjob):
51/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
5. Addthefollowingtothe
TriggerXPathExpressions
,sothatwelinkthetriggertothe
rightdatacontainer,datamodelandentity:
XPath
Operator
Value
Condition
Update/DataCluster
Matches
SecondHandLenses
C4
Update/DataModel
Matches
Lenses
C5
Update/Concept
Matches
Lens
C6
6. Settheconditionsto:
(C1OrC2OrC3)ANDC4ANDC5ANDC6
7. Save
thetrigger.
8. Undeploythetriggerweuploadedinthepreviousexercise:Rightclickon
CallJob_RealTimeDI_CRU
andchoose
UndeployFrom...
9. Nowdeploy
CallJob_RealTimeDI_CRUD
ittotheMDMserver.
52/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Testing
ViatheTalendMDMwebinterfaceadd,updateanddeleteafewrecordsandcheckifthese
changesarereflectedintheexternalDB.KeepaneyeontheJBosslog.
tMDMTriggerInput
:
Onceexecuted,tMDMTriggerInputreadstheXMLmessage(Document
type)sentbyMDMandpassesthemtothecomponentthatfollows.Everytimewhenyousave
achangeinyourMDM,thecorrespondingchangerecordisgeneratedinXMLformat.At
runtime,thiscomponentreadsthisrecordandsendstherelativeinformationtothefollowing
component.Withthiscomponent,youdonotneedtoconfigureyourJobanymoreinorderto
communicatethedatachangesfromMDMtoyourJob.
**
Nomorecontextvariableisrequiredwiththe'Integrated'method.MDMpassestheexchange
messagetransparentlytothejob.ThisispassedasaDOMDocumentsoit'spossibletouse
tXMLMaptoextractelements.
***
tMDMTriggerOutput
:
tMDMTriggerOutputreceivesanXMLflow(Documenttype)fromits
precedingcomponent.
ThiscomponentreceivesanXMLflowtosettheMDMmessagesothat
MDMretrievesthismessageatruntime.Withthiscomponent,youdonotneedtoconfigureyour
JobanymoreinordertocommunicatethedatachangesfromMDMtoyourJob.
**
ThiscomponentworksalongsidethenewtriggerserviceandprocessplugininMDMversion
5.0andhigher.TheMDMJobs,triggersandprocessesdevelopedinpreviousMDMversions
remainsupported.However,werecommendusingthiscomponentwhendesigningnew
MDMJobs.
**
**Source:TalendOpenStudioComponentsReferenceGuide
***Source:TalendForum
ThesecomponentshavethewordTriggerintheirname,whichmakesitabitconfusingso
yourememberthatthereareTriggersaswellintheMDMStudio?Hm,nowwhatdoesthisreally
mean?Aretheythesame?No.
Asmentionedabove,theyarepartofthe
MDMTriggerService.
The
tMDMTriggerInput
componentfetchesthewhole
ExchangeDocument
XMLdocumentandisfullyawareabout
therelatedbasicXMLstructure(apartfromthedatarecord/entityelementsinsidethe<item>
53/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
node),sonofurtherconfigurationisrequired.Thiscomponentdoesntallowyoutosetany
triggersassuch(oncreate,onupdate,ondelete),soitisbettertothinkofthiscomponentasa
tMDMReceivecomponentonsteroids
thanatrigger.
YouwillstillneedasimpleintegratedtriggerwithyourDIjob.Basicallyeachtimethereisa
changetothedata,thetriggerwillrunandstartthejob.Justthistimeyoudonthavetodefinea
variabletoexchangetheXMLdocumentplustheinputcomponentisawareofthemain
structureofthisXMLdocument.
Earlieronwecreatethe
RealTimeDI_CRUD
dataintegrationjob.Letsseehowwecan
implementthesamelogicusingthe
tMDMTriggerInput
component.Theadvantageofthis
approachisthatwedonothavetodefinethevariableanymorewhichgetsexchangedbetween
atriggerorprocessandthedataintegrationjob.
LetshaveadetailedlookagainatthestructureoftheExchangeDocumentforeachaction:
<exchange>
<report>
<Update>
<UserName>user</UserName>
<Source>genericUI</Source>
<TimeInMillis>1383156369929</TimeInMillis>
<RevisionID>null</RevisionID>
<DataCluster>Lenses</DataCluster>
<DataModel>Lenses</DataModel>
<Concept>Lens</Concept>
<Key>1</Key>
<OperationType>CREATE</OperationType>
</Update>
</report>
<item>
<Lens>
<Id>1</Id>
<LensName>AsahiOpticalCo.SuperTakumar28mmf/3.522.0</LensName>
<Vendor>Pentax</Vendor>
<ReleaseYear>1962</ReleaseYear>
<LensCategory>Wide</LensCategory>
<MinFocalLength>28</MinFocalLength>
<MaxFocalLength>28</MaxFocalLength>
<MinAperture>3.5</MinAperture>
<MaxAperture>22.0</MaxAperture>
<MinFocusDistanceMM>40</MinFocusDistanceMM>
54/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
<FilterSizeMM>58</FilterSizeMM>
</Lens>
</item>
</exchange>
UPDATE
<exchange>
<report>
<Update>
<UserName>user</UserName>
<Source>genericUI</Source>
<TimeInMillis>1383157466851</TimeInMillis>
<RevisionID>null</RevisionID>
<DataCluster>Lenses</DataCluster>
<DataModel>Lenses</DataModel>
<Concept>Lens</Concept>
<Key>18</Key>
<Item>
<path>Vendor</path>
<oldValue>Pentax</oldValue>
<newValue>PentaxAsahi</newValue>
</Item>
<OperationType>UPDATE</OperationType>
</Update>
</report>
<item>
<Lens>
<Id>1</Id>
<LensName>AsahiOpticalCo.SuperTakumar28mmf/3.522.0</LensName>
<Vendor>PentaxAsahi</Vendor>
<ReleaseYear>1962</ReleaseYear>
<LensCategory>Wide</LensCategory>
<MinFocalLength>28</MinFocalLength>
<MaxFocalLength>28</MaxFocalLength>
<MinAperture>3.5</MinAperture>
<MaxAperture>22.0</MaxAperture>
<MinFocusDistanceMM>40</MinFocusDistanceMM>
<FilterSizeMM>58</FilterSizeMM>
</Lens>
</item>
</exchange>
55/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
LOGIC_DELETE
<exchange>
<report>
<Update>
<UserName>user</UserName>
<Source>genericUI</Source>
<TimeInMillis>1383167579885</TimeInMillis>
<OperationType>LOGIC_DELETE</OperationType>
<RevisionID>null</RevisionID>
<DataCluster>Lenses</DataCluster>
<DataModel>Lenses</DataModel>
<Concept>Lens</Concept>
<Key>1</Key>
</Update>
</report>
<item/>
</exchange>
PHYSICAL_DELETE
<exchange>
<report>
<Update>
<UserName>user</UserName>
<Source>genericUI</Source>
<TimeInMillis>1383156401776</TimeInMillis>
<OperationType>PHYSICAL_DELETE</OperationType>
<RevisionID>null</RevisionID>
<DataCluster>Lenses</DataCluster>
<DataModel>Lenses</DataModel>
<Concept>Lens</Concept>
<Key>1</Key>
</Update>
</report>
<item/>
</exchange>
IMPORTANT:
Thedatarecorddoesnotexistanymoreatthistime,henceitisnotincluded!
56/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Data Integration
Letsgetstarted:
1. Letscreateanewjobcalled
RealTimeDI_CRUD_Integrated
.
2. Adda
tMDMTriggerInput
component.Thereisnothingtoconfigurehere.Maybetakea
lookattheschemayouwillseethatthereisonlyonecolumncalled
MDM_Message
anditisoftype
Document
.
3. Nextadda
tExtractXMLField
component.Connectit.Setitupsoitlookslikeshownon
thescreenshotbelow.Clickon
Editschema
anddefinethefieldsthatwewanttohave
inouroutputdataset:
Sohereweonlywanttoextractthe
OperationType
(CREATE,UPDATE,
PHYSICAL_DELETE),the
Key
andthe
item
(theactualdatarecord).
4. Set
XMLfield
to
MDM_Message
,the
LoopXPathquery
to"/exchange",thereason
beingthatwewanttogetbothdetailsfromthe
UpdateReport
aswellasthe
actual
data
item.Nextprovidethe
XPathquery
valuesinthe
Mapping
table,soinexamplefor
OperationType"report/Update/OperationType",fortheKey"report/Update/Key"andfor
theactualdatarecord"item".For"item"tick
GetNodes
.Inthiscasewewanttoretrieve
theXMLfragmenttodissectitlateron:
57/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
5. Adda
tMap
component.Connectit.Thendoubleclickonthe
tMap
componentand
createtwooutputdatatsets:
createOrUpdate
and
deletion
.
6. Forthe
createOrUpdate
datasetcreateoneoutputfieldcalled
item
oftype
String.
7. Forthe
deletion
datasetcreateoneoutputfieldcalled
Key
oftype
Integer
.Makesure
youtick
Key
:
8. For
createOrUpdate
activatethe
Expressionfilter
anddefinefollowingexpression:
row2.OperationType.equals("CREATE")||
row2.OperationType.equals("UPDATE")
WeonlywantthisoutputdatasettobepopulatedifthereisaCREATEorUPDATE
operationtakingplace.
9. For
deletion
activatethe
Expressionfilter
anddefinefollowingexpression:
row2.OperationType.equals("PHYSICAL_DELETE")
10. Maptheinput
Key
fieldtotheoutput
Key
fieldforthe
deletion
dataset.Alterthe
expressionsothatitconvertstheStringvaluetoanInteger:
Integer.parseInt(row2.Key)
Nextmapthe
item
inputfieldtothe
item
outputfield(dataset
createOrUpdate
)
Nowyoursetupshouldlooklikethis:
11. Add
tExtractXMLField
componentandconnectit.Itshouldusethe
createOrUpdate
datasetasinput.
12. Clickon
Editschema
.Withthiscomponentwebasicallywanttoretrievetheactualdata
record.Sodefineallthefieldsasyouknowthemfromtheentity.Makesureyoutick
Key
forthe
Id
field:
58/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
13. Setthe
LoopXPathquery
to"/item/Lens".Nowwewanttoextractthefieldsfromthe
actualdatarecordpartoftheXMLfragment.Setthe
XPathquery
forthecolumnstothe
samenameasthecolumnsitself,justenclosethemindoublequotationmarks:
14. Fromtherepository,
Metadata>DBConnections
choose
lens_metadata
anddrag
anddropitontothecanvas.Whenprompted,choosetheoutputcomponent(inmycase
tPostgreSQLOutput
).
15. Forthisoutputcomponent,set
Action
to
InsertandUpdate
andtick
Dieonerror
.
16. Finallyadda
tLogRow
componentandconnectit.Thiswillallowustoseeinthelogthe
valuesofthedatasetpassedontotheDB.Sowearefinishedwiththebranchfor
59/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
digestingthenewandupdatedrecords,nextwewillfocusonthedeletions.
17. Fromtherepository,
Metadata>DBConnections
choose
lens_metadata
anddrag
anddropitontothecanvas.Whenprompted,choosetheoutputcomponent(inmycase
tPostgreSQLOutput
).Connectthe
tMap
componentwith
tPostgreSQLOutput
by
choosingthe
deletion
row/output.Makesuretheschemagetspropagated.
18. FortheDBoutputcomponent,changethe
Action
to
Delete
.Tick
Dieonerror
.
19. Adda
tLogRow
componentandconnectit.ThiswillallowustoseetheKeyinthelog
oncearecordgetsdeleted.
20. DeploythejobtotheMDMserver.
Generating the Job Caller Trigger
Wearenowfinishedwiththejobdesign:
1. Changetothe
MDM
perspective.
2. Rightclickonthejob
RealTimeDI_CRUD_Integrated
andchoose
DeployTo
.Accept
thedefaultsettings.
3. Rightclickonthesamejobagainandchoose
GenerateTalendJobCallerTrigger
:
4. Makesurethat
Integrated
and
Embedded
isselectedonthenextscreen.Click
Generate
:
5. Youcannowfindtheprocessunder
EventManagement>Trigger
:
Doubleclickonittoopenit.
6. Notethatthe
Entity
valuewasautomaticallysetto
Update
:
60/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
7. Addthefollowingtothe
TriggerXPathExpressions
,sothatwelinkthetriggertothe
rightdatacontainer,datamodelandentity:
XPath
Operator
Value
Condition
Update/DataCluster
Matches
Lenses
C4
Update/DataModel
Matches
Lenses
C5
Update/Concept
Matches
Lens
C6
8. Settheconditionsto:
(C1OrC2OrC3)AndC4AndC5AndC6
9. Undeploythetriggerweuploadedinthepreviousexercise:Rightclickon
CallJob_RealTimeDI_CRU
andchoose
UndeployFrom...
10. DeploythetriggertotheMDMserver.
11. NextcreateoraltersomerecordsviatheTalendMDMwebinterface.Also,watchthelog
youshouldseeeitherthedatarecordorthekeydisplayed.
Checkinguniquenessofafunctionalkeybeforesaving
CreatingtheBeforeSavingprocess
Thereisquiteanastyprobleminour
Lens
data,inthesensethatfornowtheonlywaytocheck
foruniquenessoftherecordsisbasedonthelensname.Thisisratherinconvenientasitisa
longstring.
Thereisasolutionthoughtothisproblem:
BeforeSavingandBeforeDeletingProcesses
Talendfeaturesa
BeforeSaving
processaswellas
BeforeDeleting
process.Theyarehandleddifferently:Theseprocessesdonotfollowthe
standardTriggerProcessmechanismtheyaredirectlycalledby
naming
convention.
Furthermore,theseprocessesreceiveadifferentXMLdocument
thanthestandardprocesses:
acombinationoftheUpdateReportandthe
recordtobeadded/deleted
(ExchangeDocument).Finally,theseprocesses
arealsoexpectedtoreturnavariablecalled
output_reportw
hichcontainsa
statusreport
oran
errormessage
whichthewebinterfacecandisplay.
IMPORTANTPOINTS
61/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
BeforeSaving
and
BeforeDeleting
processeswillautomaticallysendtheUpdate
Reportandthedatarecord(theExchangeDocument).Toachieveexactlythesamewith
astandardprocesswewouldhavetousetwoadditionalsteps/plugins.Soyoumustnot
usethesameapproachhere.Itisalreadytakencareof.Usuallytheonlypluginyouwill
needtouseisthe
calljob
plugin.
The
DataIntegrationjob
issupposedtoreturneitherasuccessorerrormessagebut
nodata!
SinceTOSMDMversion5thesemessagesdohavetolooklikethis:
<report><messagetype="info">...successmessage...</message></report>
<report><messagetype="error">...errormessage...</message></report>
PriortoTOSMDMversion5theylookedlikethis:
<errorcode="0">...successmessage...</error>
<errorcode="1">...errormessage...</error>
DataIntegration
CreateanewDIjobcalled
FunctionalKeyCheck
.Thescreenshotbelowgivesyouanidea
whatweareupto:
1. Clickonthe
Context
tabandthenonthe
+
buttontoaddanewcontextvariable.Assign
thename
xmlInput
andthetype
String
.
2. Clickonthe
Valuesastable
tabwithintheContexttabandassignfollowing
Default
value:
<exchange
62/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
xmlns:mdm="java:com.amalto.core.plugin.base.xslt.MdmExtension"><report><Update
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"><UserName>administrator</
UserName><Source>genericUI</Source><TimeInMillis>1336311970956</TimeInMillis>
<OperationType>CREATE</OperationType><RevisionID>null</RevisionID>
<DataCluster>SecondHandLenses</DataCluster><DataModel>Lenses</DataModel><Concep
t>Lens</Concept><Key>39</Key></Update></report><item><Lens
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"><Id>39</Id><LensName>Asah
iSuperTakumar55
1.8</LensName><Vendor>Pentax</Vendor><ReleaseYear/><LensCategory/><MinFocalLeng
th/><MaxFocalLength/><MinAperture/><MaxAperture/><MinFocusDistanceMM/><FilterSi
zeMM/></Lens></item></exchange>
ThiswillallowustotesttheDIjobindependently.
9. Fromthe
TalendDIMetadata
draganddroptheMDM
LensReceive
schemaonthe
designcanvas.Thiswillcreatea
tMDMReceive
componentwhichisdesignedtoreceive
MDMXMLrecordsfromTOSMDMtriggersandprocesses.
10. Inthe
LensReceiveComponent
settingsspecify
context.xmlInput
for
XMLRecord
.
Alltheothersettingsarealreadyprovidedbythe
TalendDIMetadata
whichwesetup
earlieron:
11. Fromthe
MDMMetadata
addthe
LensIn
schema.
12. Fromthe
Palette
adda
tMap
component.
13. Createa
Mainrow
fromthe
LensReceive
tothe
tMap
component.Namethisrow
new_row
.
14. Createa
Mainrow
from
LensIn
tothe
tMap
component.Namethisrow
existing_row
.
15. Doubleclickonthe
tMap
component.Wewillnowjointhenewrecordwiththeexisting
recordstofindoutifthenewrecordisaduplicate.Draganddropthe
LensName
from
the
new_row
tableonthe
LensName
ofthe
existing_row
tabletocreatea
Join
:
63/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
16. Clickonthe
tMapsettings
iconintheexisting_rowtable.Set
JoinModel
to
InnerJoin
:
17. Addanew
outputtable
andnameit
new_record
.Clickonthe
tMapsettings
iconand
set
Catchlookupinnerjoinreject
to
true
.Addanew
column
named
result
andinsert
following
Expression
:
"<report><messagetype=\"info\">Recordfor"+new_row.LensName+"
successfullysaved</message></report>"
64/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
18. Addanewoutputtableandnameit
error
.Thisonewillreturnarowincasethereisa
matchinthejoin.
Addanew
column
named
result
andinsertfollowing
Expression
:
"<report><messagetype=\"error\">Error:Recordfor"+new_row.LensName+"
isaduplicate</message></report>"
19. Your
tMap
configurationshouldnowlooklikethis:
Click
OK
.
20. Fromthe
Palette
addtwo
tBufferOuput
components.Createrowsfromthe
tMap
componenttothem,eachonerepresentingonestate(successVSerror).
21. Dothefollowingforeachofthe
tBufferOutput
components:Clickon
Synccolumns
:
22. AlthoughtheDIjobisfunctionalcompletenow,itisdifficulttotestitinisolation.To
overcomethisshortcoming,weaddto
tLogRow
componentsfromthe
Palette
.Foreach
ofthemcreatearowfromthe
tBufferOutput
component.Clickoneachofthemand
thenpress
Synccolumns
:
65/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
23. Save
theDIjoband
run
it.Observethelogtoseewhichmessagegetsreturned(Note:
MakesureyourMDMserverisrunning):
Makesureyoutestforbothscenarios:ErrorandInfo/Success.Simplychangethe
defaultvalueXMLfragmentofthe
xmlInput
contextvariable.Ifyouwanttotesttheerror
message,makesuretheXMLfragmentholdsalensnamewhichisalreadyavailableon
yourMDMserver.Ifyouwanttotesttheinfo/successmessage,justusealensname
whichisnotyetstoredonyourMDMserver.
Note
:Pleasenotethatthissetupworksfineforsmalldatavolumes.Ifyouarehandlinglarge
datavolumes,youdonotwanttoimportallthedatajusttocheckifthenewrecordisaduplicate
ornot.Toimprovethisjob,youcouldjustfirstsetthelensnameasaglobalvariableandthenin
asubjobfetchrecordsfromtheMDMhubwhichhavethesamelensname.
Creatingtheprocess
Letsgetstarted:
1. Rightclickon
Process
.Choose
New
.
2. TalendMDMv5.0only
:Choose
CreateaBeforeSavingProcess
.
YouwillseethatTalendautomaticallyaddsthe
beforeSaving_
prefixfortheprocess
name.Note:Donotchangeordeletethisprefex!Theprocessnamehastobeprefixed
likethisasthisistheonlywayforTalendtoidentifythisprocessasa
BeforeSaving
Process
.Nextjustclickonthe[
]ellipsisbuttonandchoosetheentityinourcaseitis
Lens
.
66/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
TalendMDMv5.3andlateronly
:Choose
CreateaBeforeSaving/DeletingCheck
.On
thenextscreenchoose
CreateaBeforeSavingCheck
.Thenchoosethe
entity
to
whichthisprocessshouldbelinkedto:
Lens
.Notethatbelowthesysteminternal
processnamewillbedisplayed(grayedout):
Click
Next
.Onthenextscreenprovideainfoanderrormessage.Choosethe
Message
Type
andthenclickontheellipse[...]buttontospecifythemessageinvarious
languages.
Click
Finish
.
3. Click
OK
.Yourprocessshouldbenowdisplayedlikethisintherepositorytree:
TalendMDMv5.0only
:
67/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
TalendMDMv5.3andlateronly
:
4.
5.
6.
7.
8.
Inthe
ProcessEditor
providefollowing
Description
:
CheckuniquenessofLensName.
TalendMDMv5.3andlateronly:
Delete
RetrieveMessage
stepwealready
implementedareturnmessageinourDIjob.
Nextwritethefollowinginthe
StepDescriptionfield
:
CallTOSjob
andclickthe
+
button.
Choose
callJob
as
Pluginname
.
Choose
_DEFAULT_
as
InputVariable
and
text
as
InputParameter
andthenclickthe
AddLink
button
.
9. Choose
result
as
OutputParameter
andset
output_report
as
Outputvariable
andthen
clickthe
AddLink
button
10.
11. ReplacetheXMLinthe
Parameters
areawiththisone:
<configuration>
<url>ltj://FunctionalKeyCheck/0.1</url>
<contextParam>
<name>xmlInput</name>
<value>{_DEFAULT_}</value>
</contextParam>
</configuration>
Thisbasicallyspecifiesthejobthatshouldbeexecuted(urltag)andthenameandthe
valueofthecontextparameterwhichispassedontothejob.
12. DeploytheprocesstotheMDMserver.
Testing
Inthe
MDMperspective
,rightclickontheDIjobandchoose
DeployTo
.Chooseyour
MDMServerandclick
OK
.Dothesameforthe
BeforeSaving
process(Ifyouhaventdoneso
already).
68/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Logontothewebinterfaceandtrytoinsertthesame
LensName
twice.Youshouldgetfollow
errormessagesimilartothisone:
Checkinguniquenessofafunctionalkeybeforesaving(Integratedversion)
Theaimofthisexerciseisexactlythesameaswiththepreviousone,justthistimewewilluse
theMDMTriggerservice(andaddafewotherimprovementsalongtheway).
DataIntegration
Hereisaquickpreviewofwhatweareupto:
Letsgetstarted:
1. Createanewjobcalled
FunctionalKeyCheck_Integrated
.
2. Adda
tMDMTriggerInput
tothecanvas.
3. Adda
tXMLMap
tothecanvasandconnectit.
4. DoubleclickonthetXMLMapcomponent.
69/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
5. Onthelefthandside,rightclickonthe
MDMMessage
nodeandchoose
ImportFrom
Repository
6. SelecttherespectiveMDMmetadataentity,inourcase
LensReceive
:
7. Onthenextscreenclick
Cancel
sothatthe
UpdateReport
isnotaddedtothetree.We
donotrequireanydatafromtheUpdateReportforthepurposeofthisexercise:
8. Thetreeshouldlooklikethisnow:
70/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
9. Createanoutputdatasetandaddacolumncalled
LensName
oftypeString.Mapitto
theLensNamefieldfromtheinputdataset:
10. Adda
tSetGlobalVar
componentandconnectit.Createanewglobalvariablecalled
LensName
andmapittotheLensNamefromtheincomingrow(Note:yourrowmight
haveadifferentname,soreplaceout1withyourrowname):
11. Draganddrop
LensIn
fromtherepositoryontothecanvasjustbelowthecurrentsubjob.
12. Rightclickon
tMDMTriggerInput
andchoose
Trigger>OnSubjobOK
andpointitto
LensIn
.
13. Inthe
LensIn
componentsettingsspecify
Master
as
Type
andaddanewcondition(as
showninthescreenshotbelow)whichwillonlyfetcharecordfromtheMDMhubwhich
hasLensNameequaltotheglobalLensNamevariable.Toreferencetheglobalvariable
usethefollowing:
((String)globalMap.get("LensName"))
71/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
14. Add
tSetGlobalVar
componentandconnectit.AddanewKeycalled
ExistingLensName
andspecifyasvalue
row2.LensName
(Note:Yourincomingrow
mightbenameddifferently,soreplacerow2respectively):
Theimportantpointheretokeepinmindisthatthisglobalvariablewillbeonlysetifa
recordisretrievedfromtheMDMhub.Wecanusethisconditionthentopreparethe
returnmessages.
15. Adda
tFixedFlowInput
component.Rightclickon
LensIn
andchoose
Trigger>Runif
andpointittothe
tFixedFlowInput
component.ClickontheTriggerrowname
andspecifyfollowingconditioninthe
Component
tab:
((String)globalMap.get("ExistingLensName"))!=null
16. Clickonthe
tFixedFlowInput
componentandinthe
Component
settingson
Edit
Schema
.Addtwocolumnscalled
message
and
type
,bothoftypeString.Inthe
Values
tableset
message
to"Arecordforthislensalreadyexists"and
type
to"error:
72/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
17. Adda
tWriteXMLField
componentandconnectit.Clickon
Configure
XMLtree
andset
itupasshowninthescreenshotbelow:
18. Clickon
EditSchema
andsettheoutputcolumnnameto
MDM_Message
andmake
surethatitisoftype
Document
:
19. Adda
tMDMTriggerOutput
componentandconnectit.
20. Finallyadda
tLogRow
componentsothatwecanseetheoutputintheconsoleaswell.
21. Addanother
tFixedFlowInput
componentjustbelowthecurrentsubjob.Rightclickon
LensIn
andchoose
Trigger>Runif
andpointittothe
tFixedFlowInput
component.
ClickontheTriggerrownameandspecifyfollowingconditioninthe
Component
tab:
((String)globalMap.get("ExistingLensName"))==null
22. Specifythesamecolumnsasfortheother
tFixedFlowInput
component.Thistimeset
thevaluesfor
message
to"Allgood...thisisanewrecord!"andfor
type
to"info:
23. Adda
tWriteXMLField
,
tMDMTriggerOutput
and
tLogRow
componentandsetthem
upthesamewayaswedidfortheerrormessage.
24. Savethejobanddeployit.
BeforeSavingProcess
1. BackintheMDMperspective,rightclickontheDIjobandchoose
CreateTalendJob
CallerProcess
:
73/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
2. Nextchoose
Embedded
and
Integrated
:
Click
Generate
.
3. Nowyoucanfindtheautogeneratedprocessin
EventManagement>Process>
Other
:
4. Rightclickontheprocessandchoose
Rename
.Replacethe
CallJob
prefixwith
beforeSaving
74/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
5. Youwillnowfindtheprocessin
EventManagement>Process>BeforeSaving
Check
:
6. Thereisonesteplefttodo:Wehavetolinktheprocesstotheentity.Theonlywaytodo
thisistomaketheentitynamepartoftheprocessname:Rightclickontheprocessand
choose
Rename
.Thenclicktheellipsis[...]buttonnexttoentityandchoosethe
Lens
entity.ClickOk.(Note:Ifyoufollowedthepreviousexamples,youhavealreadya
processofthesamename.Inthiscase,firstundeploythisprocess,thenrenameitto
somethingelse).
7. Doubleclickontheprocessname.Markthethe
Invokethejob
step.Inthe
Step
area,
set
OutputParameters
to
result
and
OutputVariables
to
output_report
,thenclickthe
Link
button:
8. Savetheprocessanddeployit.
9. Addafewrecordsnowviathewebinterfacetotesttowholesetup.
ProcessesAdvancedTopics
Howtocreateanonintegratedprocessesmanually
75/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
PreparinganXMLdocumentforadataintegrationjob
BasicallywehavetocreateaspecialXMLdocumentcalled
Exchange
Document
whichholdsalltheinformationanddatawhichweneedfor
thejob.Inthefollowingexample,thefirstpartofthedocumentwill
consistofthe
UpdateReporta
ndforthesecondpartwewillretrievethe
actualrecorddatarecord
usingthe
mdm:getItemProjectionfunction.
This
isthefirststepofourprocessworkflow.Inthenextstepwehavetodo
somedecodingandthethirdstepwillpassthe
ExchangeDocument
XMLdocumenttothedataintegrationjobandexecuteit.
Whatfollowsbelowisageneralwalkthrough(sonotbackedupbyanyexample)ofsettingup
annonintegratedprocessfromscratch:
1. Inthe
MDMRepository
expend
EventManagement
andthenrightclickon
Process
.
Choose
New
:
2.
3.
4.
5.
6.
Inthenextdialogchoose
Createanormalprocess
(v5.0)/
Createanotherprocess
(v.5.3)andgiveitaname.
Click
OK
.Yourprocessshouldbenowdisplayedlikethisintherepositorytree.
Inthe
ProcessEditor
providea
Description
.
IfyouareworkingwithTalendMDMStudioV5.3orlateryouwillseethatthe
Step
sectioninthemaindesignareaisalreadyprepopulatedwith3steps:UpdateReport,
EscapetheitemXML,Redirect.Deleteallofthembymarkingthemandclickingonthe
redXbuttononthelefthandsidewewilltrytocreateaprocessfromscratch(which
basicallyreplacesthesameapproach).
Nowwewillstartpreparingour
ExchangeDocumentXMLdocument
.Writethe
followinginthe
StepDescriptionfield
:
Retrievethecompleteitemfromtheupdate
report
andclickthe
+
button:
76/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
7. Inthe
StepSpecification
sectionchoose
xslt
as
Pluginname
.Clickthe
?
buttonnext
tothepluginnametogetahelpdescriptionoftheselectedplugin:
8. Nextwehavetospecifyhowwemaptheinputvariabletothestepsinputparameters:
Theinputforthisstepwillbethe
UpdateReport
.
Inthe
InputVariables
comboboxwrite
_DEFAULT_,
whichbasicallymeansthatitwill
receivethewholeUpdateReport.
9. For
InputParameters
choose
xml
fromthepulldownmenu.
10. Clickonthe
Addlink
button:
11. Choose
text
from
OutputParameters
toandinsert
item_xml
for
Outputvariables
.
Thenclickthe
Addlink
button.Thesetupshouldlooklikethisnow:
12. Inthe
Parameters
areaaddfollowinglinesafter<xsl:templatematch="/"priority="1"/>:
<exchange>
<report>
<xsl:copyofselect="Update"/>
</report>
<item>
<xsl:copyof
select="mdm:getItemProjection(Update/RevisionID,Update/DataCluster,Update
/Concept,Update/Key)"/>
</item>
</exchange>
77/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
TheXMLfragmentshouldlooklikethisnow:
Inanutshell
:
a. Viatheinputvariable
_DEFAULT_
the
UpdateReport
XMLdocumentissentto
theinputparameterofthe
xslt
plugin.
b. WedefinethestructureofanewXMLdocument,the
ExchangeDocument
.
c. Withinthe<exchange><report>westorethe
UpdateReport
byusingthe
xsl:copyoffunction.
d. Within<exchange><item>wewanttostoretherelateddatarecord.Talendoffers
aspecial
XSLTextension
called
mdm:getItemProjection
whichenablesusto
retrievethewholedatarecord
(inencodedXMLformat).Notethatthe
mdm:getItemProjection
usesdetailsfromtheUpdateReportliketheKey/IDto
retrievetherecordfromtheMDMdatacontainer.InthisprocesseveryXMLsign
isescapedtoavoidcharacterencodingconflicts.Theresultisthensentfromthe
outputparametertothe
item_xml
variable.
13. ThenextstepistodecodetheXML:Inthe
StepDescription
write:
Decodeitemxml
andpressthe
+
button.
14. Choose
codec
as
pluginname
.
15. Mapthe
inputvariable
item_xml
(whichistheoutputvariableofthepreviousxsltstep)
to
law_text
inputparameter
.
16. Mapthe
outputparameter
codec_text
tothe
output_variable
decode_xml.
Thesetup
shouldnowlooklikethis:
17. Inthe
Parameter
sectionreplacethequestionmarkbetweenthe
method
tagswith
DECODE
andthequestionmarkbetweenthe
algorithm
tagswith
XMLESCAPE:
78/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
18. NowthedecodedXMLdocumentisreadytobesenttothelaststep:Insertanew
Step
Description
Invokethejob
andpressthe
+
buttontoaddittotheprocessflow.
19. Choose
callJob
as
Pluginname
.
20. Choose
decode_xml
as
InputVariable
and
text
as
InputParameter
.
21. Choose
result
as
OutputParameter
andset
output
as
Outputvariable
.
22. AdjusttheXMLinthe
Parameters
areasothatthecontextParametername(here:
xmlInput)matchestheoneinyourDIjob:
<configuration>
<url>ltj://sampleJob/0.1</url>
<contextParam>
<name>
xmlInput
</name>
<value>{decode_xml}</value>
</contextParam>
</configuration>
Thisbasicallyspecifiesthejobthatshouldbeexecuted(seeurltag)andthenameand
thevalueofthecontextparameterwhichispassedontothejob.
Veryimportant
:Note
how
decode_xml
fromtheprocessismatchedtotheexpectedcontextvariableinour
dataintegrationjobcalled
xmlInput
.Rememberthatwedefined
xmlInput
ascontext
variableearlieronwhenwecreatedourdataintegrationjobs?Thisishoweverything
tiestogether!
Howtotestprocesses
Thereisaneatwaytotestprocesses:
1. IntheMDMrepositorynavigateto
DataContainer>System>UpdateReport
.Double
clickonit:
79/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
2. Specifythesearchcriteria(ifany),runasearchand
doubleclick
ontherecordfor
whichyouwanttoretrievetheXMLfragment
3. IntheXMLviewerclickthe
Source
taband
copy
theXMLfragment:
Nowopentheprocessandclickthe
Execute
button:
Letsprovidethevalueoftheinputvariableofourveryfirststepintheprocess
Stylesheet
:Set
_DEFAULT_
as
InputVariable
andthenchoose
text/xml
as
ContentType
,pastetheXML
fragmentofthe
UpdateReport
inthe
Value
textboxandclickthe
+
button:
80/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Click
OK
.
PleasenotethatmorerecentversionsofTalendMDMStudiomaketestingeasierasyoucan
retrievetheUpdateReportdirectlyfromtheExecutedialog.
ThenchoosetheMDMserveryouwanttorunyourteston.Inthenextwindowyoucanhavea
lookatthevariablevaluesofallsteps.Thisisquiteusefulasyouwillnowactuallyveryeasily
understandwhateachstepoftheprocessisdoing:
1. Letsanalysetheoutputofthefirststep(
Stylesheet
),sochoose
item_xml
:
Noticethatrecordinformationwithinthe<item>elementisencoded.
81/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
2. Nowchoosetheoutputvariableofthesecondstep(
EscapetheitemXML
):
decode_xml
Aswewouldexpect,therecordinformationwithinthe<item>elementisnowproperly
decoded.
TheresultyouseehereistheXMLdocumentwhichwillbepassedonto
thedataintegrationjob.
3. Finally,ifyouchoose
output
youwillseethevalueoftheoutputvariableofthe3rdstep
(
Invokethejob
):
UnderstandingtheXMLdocumentwhichispassedonfromour
processtothedataintegrationjob
BeforecontinuingitisessentialtohaveagoodunderstandingoftheXMLdocument(the
ExchangeDocument
)whichispassedontothedataintegrationjobinourprocess.Lets
quicklygothroughtheprocessagain:
1. WebasicallycreateaspecialXMLdocument:Thefirstpartconsistingofthe
Update
Report
andforthesecondpartweretrievetheactualdatarecorddatausingthe
82/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
mdm:getItemProjection
function.
2. WedecodetheXMLdocument
3. WecallajobwhichreceivestheXMLdocument
Belowyoucanfindan
ExchangeDocument
examplefora
newrecord
.Notethatthe
<exchange>elementincludestwochildelements:<report>and<item>:
<report>includesthe
UpdateReport
(metadataaboutthesubmittedrecord).Pay
attentiontothe<OperationType>elementwhichtellsyouiftheeventwasanupdate,
deletionornewrecord.Alsoofimportancearethe<DataCluster>,<DataModel>and
<Concept>elements.TomapthesenamestotheirTalendMDMsynonyms:datacluster
isthe
datacontainer
andtheconceptisthe
businessentity
.
<item>ontheotherhandincludesalltherecordvaluesthatweresubmitted.
<exchangexmlns:mdm="java:com.amalto.core.plugin.base.xslt.MdmExtension">
<report>
<Updatexmlns:xsi="
http://www.w3.org/2001/XMLSchemainstance
">
<UserName>administrator</UserName>
<Source>genericUI</Source>
<TimeInMillis>1336311970956</TimeInMillis>
<OperationType>CREATE</OperationType>
<RevisionID>null</RevisionID>
<DataCluster>SecondHandLenses</DataCluster>
<DataModel>Lenses</DataModel>
<Concept>Lens</Concept>
<Key>39</Key>
</Update>
</report>
<item>
<Lens
xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"><Id>39</Id><LensNam
e>AsahiSuperTakumar55
1.8</LensName><Vendor>Pentax</Vendor><ReleaseYear/><LensCategory/><MinFoc
alLength/><MaxFocalLength/><MinAperture/><MaxAperture/><MinFocusDistanceM
M/><FilterSizeMM/>
</Lens>
</item>
</exchange>
Belowyoucanfindan
ExchangeDocument
exampleforan
update
.Notethestructureisabit
different:Inthiscasethe<item>elementdoesnotexist,butasimilaronecalled<Item>[Note:
startswithcapitali],whichisachildelementofthe<Update>element.<Item>holdstheoldand
newvaluesofthechangedbusinessattribute.Intheexamplebelowonlyonebusinessattribute
valuewaschanged:
<exchangexmlns:mdm="java:com.amalto.core.plugin.base.xslt.MdmExtension">
83/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
<report>
<Updatexmlns:xsi="http://www.w3.org/2001/XMLSchemainstance">
<UserName>administrator</UserName>
<Source>genericUI</Source>
<TimeInMillis>1336311787613</TimeInMillis>
<OperationType>UPDATE</OperationType>
<RevisionID>null</RevisionID>
<DataCluster>SecondHandLenses</DataCluster>
<DataModel>Lenses</DataModel>
<Concept>Lens</Concept>
<Key>10</Key>
<Item>
<path>Vendor</path>
<oldValue>CarlZeissJena</oldValue>
<newValue>Pentacon</newValue>
</Item>
</Update>
</report>
</exchange>
Thenextexampleisforadeletion:
<exchangexmlns:mdm="java:com.amalto.core.plugin.base.xslt.MdmExtension">
<report>
<Updatexmlns:xsi="http://www.w3.org/2001/XMLSchemainstance">
<UserName>administrator</UserName>
<Source>genericUI</Source>
<TimeInMillis>1336302748679</TimeInMillis>
<OperationType>PHYSICAL_DELETE</OperationType>
<RevisionID>null</RevisionID>
<DataCluster>SecondHandLenses</DataCluster>
<DataModel>Lenses</DataModel>
<Concept>Lens</Concept>
<Key>29</Key>
</Update>
</report>
<item/>
</exchange>
Testingprocessviathewebinterface
Thisgivesyouthechancetotesteverything(process,trigger,job):
Oncetheprocess,triggerandDIjobaredeployed,justlogintheuserconsoleandaddanew
recordtothebusinessentity
Lens
sothatthetriggergetscalled.Observetheconsoleoutput!
Yourconditionshouldvalidatetotrue.
Ifyougetanerror,itlookssomethinglikethis:
84/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Ifeverythingrunssuccessfully,theterminaloutputwilljustbelikethis:
ManagingthedeployedDIjobs,views,processesand
triggersontheMDMserver
Wehavetokeeponeyeonwhatisalreadydeployedontheserversothatwedontendup
havingvariousjobsrunningwithconflictinginterests.Inourexample,wemighthavealreadythe
CRU
processandDIjobdeployed,butwewanttoreplacethisnowwiththe
CRUD
ones.Iwill
notcoverthisinverymuchdetailhereasitisquitestraightforward.
TOSMDM
v5.0[Note:Morerecentversionsdonothavethisview,seealternativemethods
below]hasadepreciatedserverviewwhichcanstillbeusedtomanageallthedeployedfileson
theserver.
Toaddthisview,simplyclickon
Window>ShowView
andchoose
Talend>MDMServer
:
Click
OK
.Youhavenowanew
MDMServer
tabavailable.Tomanagethefiles,simplyright
85/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
clickonthemandchoosetherequiredaction:
TalendMDMStudiov5.3doesnothavethisserverviewanymore:Pleasefindbelowsome
alternativemethods:
Deletingtriggers,processes,jobfiles,viewsetc
Sinceversion5.4thereisaconvenient
Undeploy
optionavailableinthecontextmenu:
Forearlierversion,youhavetousethisworkaround:
Onceyoudeletefilesinthe
TalendOpenStudioforMDM
,thefileendsupintherecyclebin:
Thenyoucanremovethefilefromthe
Recyclebin
.Ifyouforgettosynchronizethestudiowith
theserver,youwillgetawarningmessage:
Atthisstagemakesureyoupress
Cancel
!
86/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Sotherecommendedmethodis:
1. Deletetheobject(process,trigger,jobetc)
2. Clickthe
Update
Server
button:
3. Thenextpopupwindowwillallowyoutoselecttheobjectwhichshouldbesynced:
Click
OK
.ThisshouldsynchronizethechangesfromtheStudiototheMDMserver.Note
thatintherecyclebintheredcrossnexttotheobjectnamenowgone:
4. Youcannowsafelydeletetheobjectorrestoreit,ifyourintentionwastoonlydeleteit
fromtheMDMserverbuttokeepitintheStudio.
Deactivatingatrigger
Sometimes,inexamplewhenyouaretestingsomething,itmightsufficetojustdeactivatethe
trigger,whichcanbespecifieddirectlyinthetriggerconfiguration:
Savethetriggerandredeployitthen.
CautionwhenrenamingfilesintheMDMStudio
WhenyourenamefilesintheMDMStudiothiswillcurrentlynotbereflectedontheMDM
Server.Ifyouclick
UpdateServer
afilewiththenewnamewillbedeployed,butthefilewiththe
oldnameisstillthere.
eXist
87/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
AccessingtheeXistDBclient
eXistDBcomeswithaJavaqueryclient.Youcanaccessitvia
http://localhost:8080/exist
.Scroll
downthepageandonthelefthandsideyouwillfindan
Administration
section.Clickthe
Launch
button
ThisusuallyrequirestheOracleJRE.Soifyouhavenotinstalleditdosobyrunningthe
followingontheTerminal(thesecommandsareUbuntuspecific):
sudoaptgetupdate
sudoaptgetinstallsunjava6jdk
sudoupdatejavaalternativessjava6sun
IftheJavaWebStartisnotopeningautomatically,thenjustdownloadthefile.Rightclickonit
Open>SunJava6WebStart.Ifthisdoesntwork,thenjustopenitfromthecommandlinewith
thiscommand:
$JAVA_HOME/bin/javawsexist.jnlp
Thelogindetailsareasfollows:
username
:admin
password
:1bc29b36f623ba82aaf6724fd3b16718
Thepasswordcanbechangedinjboss<version>/bin/mdm.conf
Youcancreateafavouritesothatyoudonthavetospecifyallthedetailsnexttimeagain.
Provideatitleandthenclickthe
Save
button:
88/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Thenclick
OK
.Thequeryclientwillnowcomeup:
Nameslistedinthe
Resource
columnrepresenttheTalend
DataContainers
.Youwillfindour
SecondHandLenses
asthe
UpdateReport
here.
Doubleclickonthe
UpdateReport
recordandthenyougetaccesstoallthedatarecords.Ifyou
wanttoseethecontentofthedatarecord,simplydoubleclickonit:
89/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Closethedatarecorddetailswindowandpressthe
Up
button(highlightedinthescreenshot
above)tojumpuponelevel.
ThisisanexcellentwaytolearnmoreabouttheXMLstructure.
eXistAdministration:Browsingdatarecords
YoucanaccesstheeXistadminpagesusingfollowingURL:
http://localhost:8080/exist/admin
.
Thelogindetailsareexactlythesameasmentionedaboveforthequeryclient:
username
:admin
password
:1bc29b36f623ba82aaf6724fd3b16718
Inthe
eXist
world
DataContainers
arecalled
Collections
,hencechoose
BrowseCollections
fromthelefthandsidenavigation:
90/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Thenlookforour
SecondHandLenses
collectionandclickonthelens:
Finallyyougetanoverviewofalltherecordscreatedforthebusinessentity:
Clickingononeofthemwillshowyoutherecorddata:
91/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
The
UpdateReport
justhappenstobeanordinarycollectionsaswell,soyoucanaccessits
recordsexactlythesameway.ThisisafairlygoodwaytolearnabouttheXMLstructure.
Backup,addingindexesandmore
Afewothertopicsyoumightwanttohavealookatare
backingupeXist
,creatingindexes(see
here
and
here
)andifyouwanttoevenfindoutmoreabouteXist,visittheir
online
documentation
.
Whereisthedatastored
Youcanfindthedatain:
<jboss_dir>/server/default/deploy/exist1.4.0rev11706TalendPatch.war/WEBINF/
data/
H2embeddedrelationaldatabase
HowtoaccesstheH2databasefromaqueryclient
ObtainingtheJDBCDriver
IfyourequiretheJDBCdriver,downloadthewholeinstallationpackagefromtheh2website
andextractit.Withinthebinfolderyoufindah2<version>.jarfilewhichyoucanuseasthe
JDBCdriver.
JDBCconnectionstring
Toeasiestwaytofindoutabouttheconnectiondetails,istolookatthedatasources.xmlfileon
theMDMserver:
<jbossroot>/server/default/conf/datasources.xml
Youwillseesomethingsimilartothis:
<!H2DATASOURCE>
<datasourcename="H2Default">
<master>
<type>RDBMS</type>
<rdbmsconfiguration>
<dialect>H2</dialect>
<connectiondriverclass>org.h2.Driver</connectiondriverclass>
<connectionurl>
jdbc:h2:///opt/talend/MDM/TOS_MDMV5.3.1/TOS_MDMServerr104014
V5.3.1/jboss4.2.2.GA/server/default/data/H2Default/${container}
</connection
92/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
url>
<connectionusername>
sa
</connectionusername>
<connectionpassword>
sa
</connectionpassword>
<fulltextindexdirectory>/opt/talend/MDM/TOS_MDMV5.3.1/TOS_MDMServerr104014
V5.3.1/jboss4.2.2.GA/server/default/data/indexes/H2Default</fulltextindexd
irectory>
Howtosetuptheconnectioninyourqueryclient
UsefollowingconnectiondetailstoconnecttotheembeddedH2database:
JDBCURLexample:
jdbc:h2:///home/dsteiner/development/software/talend/TOS_MDMAllr111943V5.4.1
/jboss4.2.2.GA/server/default/data/H2Default/LensesAUTO_SERVER=TRUEMVCC=TRU
ELOCK_TIMEOUT=15000
WhereLensesistheDataContainer.Youcanfindtheexactdetailsin:
<jbossroot>/server/default/conf/datasources.xml
Inthesamefileyoucanfindtheusernameandpasswordtoconnectaswell:
InSQLWorkbench/JIsetupthisconnectionlikethis:
Inthiscase,fortheextendedpropertiesIjustdefinedAUTO_SERVER.
Alsonotethatinthiscasethelogindetailsdidnthaveapasswordspecified.
93/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Note:Aportnumberisnotrequired!
IMPORTANT
:ToaccesstheH2databasefromtheSQLclientyouhavetostoptheMDM
server,otherwiseyouwillgetfollowingerrormessage:
Databasemaybealreadyinuse:"Lockedbyanotherprocess".Possible
solutions:closeallotherconnection(s)usetheservermode[90020174]
SoconnectviathequeryclientonceyoustoppedtheMDMserverandyoucanexplorethe
tablesinexampleinSQLWorkbenchs
DatabaseExplorer
:
Andherelookingatthedata:
Igoeswithoutsayingthatyoushouldbeverycarefulhereonwhatyouaredoing!Onlyusethis
forSELECTstatements.Ifdosomethingelse,makesurethatyouknow100%whatyouareup
to!
Howtochangethedatatypeofanelementoncethedatamodelis
deployed
IfyourTalendjobthrowsanerrorlikethisone:
94/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
faultString:org.hibernate.exception.DataException:CouldnotexecuteJDBC
batchupdatenestedexceptionis:
javax.ejb.TransactionRolledbackLocalException:
org.hibernate.exception.DataException:CouldnotexecuteJDBCbatchupdate
itmightbethatthefieldmappingisnotcorrect.Inspecttheserverlog
(jboss/server/default/log/server.log)formoredetails.Youmightfindanerrormessagelikethis
one:
Causedby:org.h2.jdbc.JdbcBatchUpdateException:Dataconversionerror
converting"'Joe'(EMPLOYEE:X_FIRSTNAMEBOOLEAN)"SQLstatement:
Thiscolumnisdefinitelywronglydefinedinthedatamodel.Followthesestepstosolvethis
problem(asalsohighlighted
here
):
1. Inyourdatamodelchangetheelementtype
2. Exportallthedataoftherelationaltable
3. Droptherelationaltable
4. Redeploythedatamodel
5. Nowyourrelationaltableshouldhavethecorrectfieldtypes.
6. Importthedata
Orletstakeanotherexample:Youjustcreatedadatamodelandforoneelementyoudefined
thewrongdatatype.Yourealizedthisshortlyafterdeployingit.Thereisnodatayetstoredfor
thismodel.Soyoucanjustsimplychangethedbcolumntypebyrunningastatementsimilarto
thisone(dothisonlyifyouareawareofthedbdatatypesthatTalendMDMuses!):
ALTERTABLEEMPLOYEEALTERCOLUMNX_FIRSTNAMEVARCHAR(255)
Openyourdatamodelandcorrectthedatatypeforthisspecificelement.Redeploythemodel.
Allshouldbegoodnow.
AdvancedMDMconcepts
Theintentionofthisguidewastoonlyprovideyouwithaquickstartingpoint.Istronglysuggest
thatyoureadthededicated
TalendMDM
guide(whichyoucanfindontheTalendwebsite)for
advancedconceptslikeforeignkeys,foreignkeyfilters,datamodelinheritanceand
polymorphism,howtosetlookupfieldsandmuchmore.Also,inregardsto
TalendDI
itis
definitelyworthhavingalookattheTalendDIcomponentsguide(whichyoucanfindonthe
Talendwebsiteaswell)tounderstandthefullpotentialofeach
MDMcomponent
.
Appendix
95/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Howtoimportthetutorialprojectarchive
Whenyoustartup
TOSMDMStudio
youhavethepossibilitytoimportexistingprojects.For
yourconvenienceIexportedallmyprojectfilesandmadethemavailableonlinesothatyoucan
compareyourfilestomineincaseyougetstuckatsomepoint.
1. Downloadthe
projectarchive
from
here
.Inthemenubarclick
File>Download
.
2. Startup
TOSMDMStudio
.
3. Press
Import...
4. Assigna
projectname
(inexample
MDMTutorial
).Clickon
Selectarchivefile
and
Browse
forthearchivefileyoujustdownloaded:
Click
Finish
.
5. Thenewprojectnameshouldnowshowuponthestartupwindow.Mark
MDMProject
andclick
Open
:
96/97
DiethardSteiner
TalendOpenStudioForMDM:APracticalStarterGuide
Deploysolutionsinfollowingorder:
1. datamodel
2. datacontainer
3. view
4. DIjob
5. processes
6. triggers
97/97