How To Convert Relational Data To XML - Melbourne

Relational Data to XML:
A Users Unexpected Journey

Paul Murray
Westpac New Zealand Ltd
Session Code: D02
Wednesday, September 11 13:00 to 14:00 | Platform: Cross Platform
Click to edit Master title style
OBJECTIVES
Describe reasons why conversion of relational data to XML may
be considered.
Detail situations when conversion appears to be a candidate
for consideration.
Describe how to convert relational data to XML for
A single column on a table

For multiple columns on a table
For data on multiple tables whether single or multiple columns
How to handle data where nulls exists
Detail what has been discovered / achieved so far.
Acknowledgements
Phil Grainger of BMC Software
From his education seminar, DB2 9 for z/OS In Depth presented at
IDUG Australia 2011, came the idea about using XML.
And for his introduction of me to
Dr Matthias Nicola from IBM

His presentation entitled - Publishing Relational Data as XML, from
IDUG North America 2006. This gave me the base knowledge to
develop the solution used in the situation described in this
presentation. (The presentation is available in the IDUG Website)
Robert Mala an Independent Database contractor

For his presentation entitled - Getting to grips with Complex SQL, from
IDUG Australia 2008. Again assisted with developing the solution with
relation to using and coding Common Table expressions.
Disclaimer
The following presentation is the views and opinions of the
author / presenter. It in no way represents the views of
Westpac NZ Ltd or IBM.
The facts presented as part of this presentation were observed
in a specific environmental configuration that existed within
Westpac NZ Ltd and are factual only to that environment.
They should by no means constitute a guarantee of results
achievable in any other environment and as such are therefore
indicative only.
Other observations are factual events that the author /
presenter encountered.
Agenda
Why convert to XML.

When you may wish to convert.
How to convert to XML. (The Basics)
Dealing with NULLS.
Outcomes achieved in Westpac.
Additional Suggestions / Observations and XML
syntax.
Conclusion.
Questions.
WHY?
6
Curiosity at work
7
Reasons Why convert Relational Data to XML.

Understanding.
Pseudo Standard.
Flexibility.
Re-use.
Performance.
Why convert to XML - Understanding

This reason may come into play in two ways;
1. XML has been widely taught from several years now and as
such has numerous people who can read / write / develop it.
This is turn has lead to a desire from developers and
architects who have this understanding, to see it utilised
within the organisation they work for.
2. When XML is well formed it can be a self describing format.
E.G.
<AccountBalance>100.00</AccountBalance> is more
easily understood than having 100.00 sitting somewhere
within a record in a file.
E.G. Record 1ACCT 1234567890
100.00O
Why convert to XML Pseudo Standard

Due to widespread understanding of and potential easy of
understanding of the meaning of the data within XML, XML is
being accepted as a Pseudo Standard as a data transmission
format between applications / platforms and even
organisations.
Even to the stage whereby external organisations to yours may
dictate its usage as the format they must receive in cross
organisation data communications.
10
Why convert to XML - Flexibility

XML is a very flexible data format.
Need a new piece of data just add a new tag in.
If a data element is no longer needed then the tag can be removed by
not populating it or removing it from showing in the XML generated.
XML can also be versioned which means multiple versions of the same
XML document can be maintained to allow for changes in the elements
that the document contains.
From a database point of view changing the elements within an XML
Document, which is akin to changing columns in a table, can be done
without requiring an actual table change being made, if stored as XML
within an XML column on the database.
11
Why convert to XML Re-use.

By converting current relational data used in data selection
processes to transform data to XML first this can allow current
processes that create and maintain the data to still function as
is, thus avoiding mass changes to the current environment.
This of course can allow for a slower more incremental set of
changes to be made, reducing risk and allowing a more
targeted approach to be taken.
It also allows for the adoption of a hybrid methodology where
relational data and XML coexist in the same database(s) and
are utilised where they are most suited.
12
Why convert to XML - Performance

Transforming relational data to XML can provide performance
gains by
1.
2.
3.
Reducing Calls to the database.

Reducing CPU consumption.
Reducing Elapsed Time taken.
13
WHEN
14
Conversion to XML may be an option when

The data is of a hierarchical nature. E.G.
A Family Tree.
An Organisational Chart.
The nature or structure of the data looks like a

A squat pyramid.
A Star or Snow Flake schema.
15
HOW The Basics
16
The nature / structure of XML

For an XML Document
to be well formed it has
to relate back to a single
point or root element.
i.e. You are trying to
create Mount Everest
(from peak to base) not
the whole Himalayan
Mountain range.
17
18
How to convert to XML - (Base Data Definition)

EMP
EMPNO
FIRSTNME
LASTNAME
SEX
HIREDATE
CHAR(6)
VARCHAR(12)
VARCHAR(15)
CHAR(1)
DATE
000010
CHRISTINE
HAAS
1965-01-01
000020
MICHEAL
THOMPSON
1973-10-10
EMPPROJACT
EMPNO
PROJNO
EMSTDATE
EMENDATE
CHAR(6)
CHAR(6)
DATE with Nulls
DATE with Nulls
000010
MA2100
1982-01-01
1982-11-01
000010
MA2110
1982-01-01
1983-02-01
000010
AD3100
1982-01-01
1982-07-01
000020
PL2100
1982-01-01
1982-09-05
How to convert to XML

The basics functions
XMLELEMENT
Creates a single XML element within an XML Document.
Note that every XML document you create needs at least 1 of these.
XMLFOREST
Allows the creation of multiple XML elements within a XML Document
with one XML command.
XMLAGG (XML Aggregation)

Combines XML Elements or XML Documents together.
19
XMLELEMENT The syntax
20

(Single Column from a Single Table)
SELECT XMLELEMENT(NAME First, FIRSTNME)

FROM DSN8910.EMP
WHERE EMPNO = 00010
;
Result
<FIRST>CHRISTINE</FIRST>
21

(Single Column for multiple rows from 1 table)
SELECT XMLELEMENT(NAME First, FIRSTNME)
FROM DSN8910.EMP
;
Result
<FIRST>MICHAEL</FIRST>
22

(Multiple Columns for 1 row from 1 table Option 1)
SELECT XMLELEMENT(NAME Name,
XMLELEMENT(NAME First, FIRSTNME),
XMLELEMENT(NAME Last, LASTNAME))
FROM DSN8910.EMP
WHERE EMPNO = 000010
;
Result
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST></NAME>
23
XMLFOREST The syntax
24

(Multiple Columns for 1 row from 1 table Option 2)
XMLFOREST(FIRSTNME as First,
LASTNAME as Last))
FROM DSN8910.EMP
;
Result
25
XMLAGG The syntax
26
How to join XML together - XMLAGG

WITH PROJECT(EMPNO, PROJECTXML) as
(SELECT PROJ1.EMPNO,
XMLELEMENT(NAME ProjectDetails,
XMLFOREST(PROJNO as ProjectID,
EMSTDATE as StartDate,
EMENDATE as EndDate))
FROM DSN8910.EMPPROJACT PROJ1)
LASTNAME as Last),
(SELECT XMLAGG(PROJ.PROJECTXML)
FROM PROJECT PROJ
WHERE PROJ.EMPNO = EMP.EMPNO))
FROM DSN8910.EMP EMP
;
27
How to join XML together XMLAGG

The Result
Row 1
Row 2
<NAME>
<LAST>HAAS</LAST>
<PROJECTDETAILS>
<PROJECTID>MA2100</PROJECTID>
<STARTDATE>1982-01-01</STARTDATE>
<ENDDATE>1982-11-01</ENDDATE>
</PROJECTDETAILS>
<PROJECTDETAILS>
</PROJECTDETAILS>
<PROJECTDETAILS>
<PROJECTID>AD3100</PROJECTID>
</PROJECTDETAILS>
</NAME>
<NAME>
<LAST>THOMPSON</LAST>
<PROJECTDETAILS>
<PROJECTID>PL2100</PROJECTID>
</PROJECTDETAILS>
</NAME>
28
The power of XMLAGG

It was discovered by using an additional construct in the SQL
statement, used to generate the XML, that is was possible to
generate all the data in a single select statement.
This was achieved using Common Table Expressions and a final
outer query which utilised the XMLAGG command to string all
the previously created result set rows into one row.
29
The power of XMLAGG - Example

WITH PROJECT(EMPNO, PROJECTXML) AS
(SELECT PROJ1.EMPNO, XMLELEMENT(NAME ProjectDetails,
XMLFOREST(PROJNO AS ProjectID, EMSTDATE AS StartDate,
EMENDATE AS EndDate))
FROM DSN8910.EMPPROJACT PROJ1),
EMPLOYEEDETAIL(EMPLOYEEXML) AS
(SELECT XMLELEMENT(NAME Name,
XMLFOREST(FIRSTNME AS First, LASTNAME AS Last),
(SELECT XMLAGG(PROJ.PROJECTXML)
FROM PROJECT PROJ WHERE PROJ.EMPNO = EMP.EMPNO))
FROM DSN8910.EMP EMP)
SELECT XMLELEMENT(NAME EmployeeDetails,
(SELECT XMLAGG(EMPLOYEEXML) FROM EMPLOYEEDETAIL))
FROM SYSIBM.SYSDUMMY1
WITH UR
;
30
The power of XMLAGG - The Result

<EMPLOYEEDETAILS>
<NAME>
<LAST>HAAS</LAST>
<PROJECTDETAILS>
</PROJECTDETAILS>
<PROJECTDETAILS>
</PROJECTDETAILS>
<PROJECTDETAILS>
<PROJECTID>AD3100</PROJECTID>
</PROJECTDETAILS>
</NAME>
<NAME>
<LAST>THOMPSON</LAST>
<PROJECTDETAILS>
<PROJECTID>PL2100</PROJECTID>
</PROJECTDETAILS>
</NAME>
</EMPLOYEEDETAILS>
31
DEALING WITH NULLS
32
Options for Dealing with NULLS
1.
2.
3.
Three Choices
Do nothing specific and just create XML as you would.
Specifically ask to create an empty element.
Specifically dont create an element.
33
Create an Element (Treat as part of the data)

One option is to just create the XML as you would normally.
Therefore if we were to assume the EMP table had another
column called LEAVEDATE which allowed nulls and for Christine
the column contained a null, using the following SQL
LASTNAME as Last
LEAVEDATE as LeaveDate))
FROM DSN8910.EMP
;
Would result in
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST><LEAVEDATE/></NAME>
34
Specifically Create an Empty Element

Another option is to specify that an EMPTY element is created
as part of the XML.
LASTNAME as Last
LEAVEDATE as LeaveDate
OPTION EMPTY ON NULL))
FROM DSN8910.EMP
;
<NAME><FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST><LEAVEDATE/></NAME>
35
Specifically do not Create an Element

Another option is to specify that an element is not created as
part of the XML if the data in the element does not exist
LASTNAME as Last
LEAVEDATE as LeaveDate
OPTION NULL ON NULL))
FROM DSN8910.EMP
;
36
Dealing with NULLS - Recap
1.
2.
3.
3 Options
Create XML as is
Specifically create an EMPTY tag
Specifically do not create a tag.
Default behaviour if nothing specifically specified i.e. option 1
is followed, is to create an EMPTY tag.
Specification of an option is allowed in both the XMLELEMENT
and XMLFOREST syntax.
37
WHAT CAN BE ACHIEVED
38
What was achieved in Westpac

In a particular situation
Significant reduction in SQL calls to DB2
Significant reduction in average elapsed times to extract data
Significant reduction in CPU consumption to extract data.
39
40
What was achieved in Westpac - Context

Situation found was a COBOL program executing in the following
manner
Driver
Simpleton
Select
Cursor
Child
Cursor 10
Child
Cursor 9
Child
Cursor 8
Child
Cursor 1
Result set for

Driver Cursor
contained 64 rows.
Child
Cursor 7
Child
Cursor 4
Child
Cursor 6
Child
Cursor 5
Child
Cursor 2
Child
Cursor 3
Initial Redesign Option

Following suggested best practice it was decided to attempt to
rewrite all the cursors and the select statement into 1 cursor
using joins.
Result found that
Average CPU consumed increased 2000%
Average Elapsed Time increased 2000%
Calls to DB2 increased over 1000 %
41

Initial Results.
Within a Test Environment Current State Cobol Program
Consumed on average 0.1630 seconds of CPU in DB2 and 0.1693
overall.
Executed 4588 Calls to DB2.
This includes 1 commit statement.
Took on average 0.1808 seconds elapsed time in DB2.
After using XML Transformation

overall.
Executed 2 Calls to DB2.
This includes 1 commit statement.
Took on average 0.1660 seconds elapsed time in DB2.
42

Final Results.
After using an addition to the XML Transformation Technique it
was managed to achieve the following results.
overall.
Again executed 2 Calls to DB2.
This still included 1 commit statement.
Took 0.0034 seconds elapsed time on average in DB2.
43

Final Results - Explained.
The final solution was able to select the converted relational
data from a table as an already generated XML document.
This approach was taken due to the fact that the data being
read was static in its nature so therefore did not need to be
generated every time it was requested.
Therefore the initial XML transformation was inserted into
another table as XML in an XML Column with a relational data
identifier and the process was modified to extract this instead
of regenerating the data every time it was read.
44
Other Suggestions /
Observations /
XML Syntax
45
Other useful XML Syntax - XMLSERIALIZE

Allows the serialisation of XML into a string type.
SELECT XMLSERIALIZE(XMLELEMENT(NAME First, FIRSTNME) AS
CLOB(100))
FROM DSN8910.EMP
;
Results
Note be careful in z/OS as you cannot serialise to a VARCHAR, in version 9 at least.
46
Other useful XML Syntax - XMLATTRIBUTE

SELECT XMLELEMENT(NAME EmployeeDetails,
XMLATTRIBUTE(SELECT CASE WHEN SEX = F THEN FEMALE
WHEN SEX = M THEN MALE
ELSE OTHER
END AS SEX
FROM DSN8910.EMP
WHERE EMPNO = 000010),
XMLFOREST(FIRSTNME as FirstName,
LASTNAME as LastName))
FROM DSN8910.EMP
;
47
XMLATTRIBUTE The Result
<EMPLOYEEDETAILS SEX="FEMALE">
<FirstName>CHRISTINE</FirstName>
<LastName>HAAS</LastName>
</EMPLOYEEDETAILS>
48
How to convert to XML - XMLCONCAT

SELECT XMLCONCAT(XMLELEMENT(NAME First, FirstName),
XMLELEMENT(NAME Last, LastName))
FROM DSN8910.EMP
;
Result
NOT XML
<FIRST>CHRISTINE</FIRST><LAST>HAAS</LAST>
49
How to convert to XML XMLCONCAT (Proper usage)

XMLCONCAT(XMLELEMENT(NAME First, FirstName),
XMLELEMENT(NAME Last, LastName)))
FROM DSN8910.EMP
;
Result
50
Other Observations
Be careful of open Pandoras box.
People treating this a one size fits all solution or
A magic silver bullet.
System Configuration especially memory settings

In z/OS specifically zPARMs
XMLVALA The amount of memory a single agent is allowed to use while
processing XML
XMLVALS The amount of memory the sub system has allocated in total to
handle all XML processing at any point in time.
51
Conclusion What you colleagues might initially think

(Especially those of you who work with DB2 for z/OS)
52
Conclusion
Conversion from relational data to XML can provide benefits.
In certain circumstances these could be significant.
The benefits are not derived in all circumstances. Treat each
situation as its own and develop a solution accordingly. XML
conversion is not of a one size fits all approach so please be
sensible when applying.
The conversion process is not overly onerous and requires little
specialist knowledge of DB2 XML functionality. Just a good
understand of really only three SQL XML functions and
Common Table Expressions.
53
QUESTIONS
54
THANK YOU
55
Paul Murray
Westpac New Zealand Ltd
Paul_Murray@Westpac.co.nz
Session D02
From Relational Data to XML :
A Users unexpected journey.

How To Convert Relational Data To XML - Melbourne

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

How To Convert Relational Data To XML - Melbourne

Caricato da

Copyright:

Formati disponibili

Relational Data to XML:

A Users Unexpected Journey

Click to edit Master title style

A single column on a table

Detail what has been discovered / achieved so far.

Click to edit Master title style

Dr Matthias Nicola from IBM

Robert Mala an Independent Database contractor

Click to edit Master title style

Click to edit Master title style

Why convert to XML.

Click to edit Master title style

Click to edit Master title style

Click to edit Master title style

Reasons Why convert Relational Data to XML.

Click to edit Master title style

Why convert to XML - Understanding

Click to edit Master title style

Why convert to XML Pseudo Standard

Click to edit Master title style

Why convert to XML - Flexibility

Click to edit Master title style

Why convert to XML Re-use.

Click to edit Master title style

Why convert to XML - Performance

Reducing Calls to the database.

Click to edit Master title style

Click to edit Master title style

Conversion to XML may be an option when

The nature or structure of the data looks like a

Click to edit Master title style

HOW The Basics

Click to edit Master title style

The nature / structure of XML

Click to edit Master title style

How to convert to XML - (Base Data Definition)

DATE with Nulls

DATE with Nulls

Click to edit Master title style

How to convert to XML

XMLAGG (XML Aggregation)

Click to edit Master title style

XMLELEMENT The syntax

Click to edit Master title style

How to convert to XML

SELECT XMLELEMENT(NAME First, FIRSTNME)

Click to edit Master title style

How to convert to XML

Click to edit Master title style

How to convert to XML

Click to edit Master title style

XMLFOREST The syntax

Click to edit Master title style

How to convert to XML

Click to edit Master title style

XMLAGG The syntax

Click to edit Master title style

How to join XML together - XMLAGG

Click to edit Master title style

How to join XML together XMLAGG

Click to edit Master title style

The power of XMLAGG