DB2 Primer

DB2 for z/OS Performance Primer
Dave Beulke
Joe Burns
Dan Luksetich
Craig S. Mullins
Anne Stevens
Julian Stuhler
Brought to you by
DBAzine.com
DBAzine.com
DB2 for z/OS Primer

by Dave Beulke, Joe Burns, Dan Luksetvich, Craig S. Mullins, Anne Stevens, and Julian
Stuhler
Copyright 2004 by BMC Software and DBAzine. Used with permission.
Printed in the United States of America.
Printing History:
November, 2004 for First Edition
DB2, DB2 Connect, DB2 Universal Database, IBM, are trademarks or registered trademarks of International Business Machines Corporation
in the United States, other countries, or both.
Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other company, product, and service names may be trademarks or service marks of others.
Many of the designations used by computer vendors to distinguish their products are claimed as Trademarks. All names known to
DBAzine.com to be trademark names appear in this text as initial caps.
The information provided by the authors of this work is believed to be accurate and reliable, but because of the possibility of human error by
our authors and staff, BMC Software, DBAzine cannot guarantee the accuracy or completeness of any information included in this work and
is not responsible for any errors, omissions or inaccurate results obtained from the use of information or scripts in this work.
Links to external sites are subject to change; DBAZine.com and BMC Software do not control or endorse the content of these external web
sites, and are not responsible for their content.
ii
DBAzine.com
Table of Contents
A DB2 for z/OS Performance Roadmap..........................................................................................1
Tuning the System.......................................................................................................................1
No Magic Bullet.......................................................................................................................2
Tuning the Databases ..................................................................................................................3
Creating Indexes in the Dark.......................................................................................................5
Tuning the Applications ..............................................................................................................6
The Bottom Line .........................................................................................................................7
Can Your Left Outer Joins Perform Better? ....................................................................................9
Introduction .................................................................................................................................9
Left Outer Joins the Good and the Bad ....................................................................................9
A Simple Example of a Left Outer Join ....................................................................................10
Leveraging the ON Clause for Performance Benefits ...........................................................10
The ON Clause Predicate Does Not Affect the Result Set....................................................11
Benchmarking the Performance Improvement..........................................................................12
Design Considerations and Where to Use This Technique.......................................................15
Conclusion.................................................................................................................................15
Notes:.........................................................................................................................................16
Utilizing DB2 UDB for OS/390 and z/OS UNION Everywhere for Large Table Design ............17
Introduction ...............................................................................................................................17
Increasing Table Size and Concurrency with Union in View ...................................................17
Partitioning Tablespaces............................................................................................................18
The UNION Solution ................................................................................................................19
Predicate Distribution................................................................................................................20
Query Block Pruning.................................................................................................................21
Predicate Transitive Closure Applied to Joins and UNION in View........................................23
Limitations to Union Everywhere .............................................................................................24
Join Impact on UNION in View................................................................................................26
The Role of UNION in View in V8 ..........................................................................................29
Conclusion.................................................................................................................................29
REORG Redux: A Review of the DB2 REORG Utility................................................................31
Introduction ...............................................................................................................................31
What Is REORG For?................................................................................................................31
How Does REORG Work?........................................................................................................34
When Should You Run REORG? .............................................................................................35
Conclusion.................................................................................................................................37
DB2 Buffer Pool Essentials ...........................................................................................................39
Memory Basics..........................................................................................................................39
DBAzine.com
iii
DB2 Buffer Pools ......................................................................................................................40

Buffer Pool Sizing .....................................................................................................................42
Buffer Pool Tuning Knobs ........................................................................................................42
Buffer Pool Monitoring .............................................................................................................44
Some Additional Rules of Thumb.............................................................................................46
Summary ...................................................................................................................................46
The Key to Performance: Business Process Analysis....................................................................47
Introduction ...............................................................................................................................47
Bigger Is NOT Better ................................................................................................................47
Business Process Analysis = Big Savings.................................................................................47
Deep analysis........................................................................................................................48
Examining online transactions..............................................................................................49
Opportunities for improvement ............................................................................................49
How Much is Too Much?..........................................................................................................49
Conclusion.................................................................................................................................50
iv
DBAzine.com
About the Authors

David Beulke is an internationally recognized DB2 consultant, author and lecturer. He is
known for his extensive expertise in database performance, data warehouses and Internet
applications. He is member of IBM DB2 Gold Consultant program and is a certified DBA
for z/OS, Linux, Unix, and Windows platforms, a certified Business Intelligence Solutions
Expert, and the past president of IDUG.
Joe Burns is a senior database administrator with Highmark Inc. His primary job
responsibilities include database design and DB2 performance monitoring. He has worked
in the DB2 field for more than 10 years, both as a DBA and an application developer. Mr.
Burns has been an award winning speaker at the IDUG DB2 North America conference as
well as the IDUG Europe conference. Mr. Burns is an avid supporter of IDUG and the
DB2 community.
Daniel Luksetich is an internationally recognized consultant, educator, presenter, database
administrator, and application developer. His focus is primarily on the IBM DB2 database
product on all platforms. Daniel has 14 years DB2 experience, building and tuning high
performance, highly available DB2 systems on the OS/390, and AIX Platforms. Dan is an
IBM Certified Advanced Technical Solutions Expert in DB2 Database Administrations,
DB2 Application Programming, and Data Replication, and was recently named a top ten
speaker at the IDUG Europe Conference.
Craig S. Mullins is director of technology planning for BMC Software. He has 20 years of
experience dealing with data and database technologies. He is the author of the books
Database Administration: The Complete Guide to Practices and Procedures and the bestselling DB2 book, DB2 Developers Guide (currently in its fifth edition).
Anne Stevens is an independent consultant and freelance writer.
Julian Stuhler is a Principal Consultant with Triton Consulting. He has over 17 years
relational database experience working with a number of clients within the insurance,
telecommunications, banking, financial services and manufacturing sectors. Julian has
lectured widely on DB2 subjects, including presentations for the International DB2 Users
Group (IDUG), the UK GUIDE DB2 Working Group, and European GUIDE meetings.
Julian is an IBM Redbook author, and won the Best Overall Speaker award at
the European International DB2 User Group in 2000. He is an IBM DB2 Gold Consultant
and a member of the IDUG Board of Directors.
DBAzine.com
vi
DBAzine.com
Forward
One of the biggest problems faced by companies today is keeping applications running at
peak efficiency. It can be a never-ending task to diligently keep collecting, examining, and
reacting to performance-oriented statistics. This is certainly the case for DBAs charged with
maintaining optimal performance of DB2 databases and applications.
DB2 performance is a three-headed beast encompassing the program logic and SQL in your
applications, the database objects being accessed, and the database system itself all of which
must be tamed to assure optimal performance. Many experts agree that inefficient SQL is
probably the number one cause of poor DB2 performance. But that doesnt mean you can
safely neglect the state of your database objects or your DB2 subsystems.
Monitoring and managing these three aspects of performance can be a full-time job. Further
complicating this job is the constantly changing state of DB2. Just when you think you have
mastered one aspect of DB2, along comes that new release or PTF that changes everything.
Fortunately, you have this DB2 for z/OS Performance Primer in your hands to keep you upto-speed with DB2 performance issues. If you read the articles contained in this primer you
will be better armed to combat the three-headed DB2 performance beast.
Start off with the article I wrote titled A DB2 for z/OS Performance Roadmap. This article
offers up a high-level overview of DB2 performance issues and will start you on your way to
becoming a DB2 performance expert. Then move on to learn more about solving DB2
application performance issues by reading Joe Burns Can Your Left Outer Joins Perform
Better? Joe offers up a wealth of information about squeezing optimal performance out of
outer joins. We follow this with Dan Luksetichs Utilizing UNION Everywhere for Large
Table Design. In this informative article Dan teaches us how to bridge the gap between
application and database performance by targeting a newer SQL feature to simplify the design
of large DB2 tables.
Then we take full aim at your database objects with REORG Redux: A Review of the DB2
REORG Utility by Julian Stuhler. This article takes you through the paces of when, why, and
how to reorganize your DB2 databases to achieve optimal database performance. And we
dont neglect the system aspect of DB2 performance either. Anne Stevens discusses DB2
Buffer Pool Essentials for tuning your subsystem performance.
DBAzine.com
vii
Finally, Dave Beulke wraps it all up with his insightful article titled The Key to Performance:
Business Process Analysis. In it he discusses how to achieve business objectives using
business process analysis and better not bigger databases.
So, lets start our journey into the world of DB2 performance management
Craig S. Mullins
Director, Technology Planning
BMC Software
viii
DBAzine.com
A DB2 for z/OS Performance Roadmap

by Craig S. Mullins
* First published in eServer Magazine, Mainframe Edition.
Assuring optimal performance is one of a database administrators (DBAs) biggest, ongoing

challenges. The loudest complaints about poor performance come from the users who must
wait longer than they are used to waiting for their applications to respond. No one likes
waiting, especially one who is unaccustomed to waiting.
But what causes those once-fast applications to stall and deliver sub-par performance? If there
were an easy answer for that question, many DBAs would be out of work. Theres no short
answer to this question, either. But, as a starting point, lets examine the fundamentals of
performance management and optimization and come up with a roadmap for tuning DB2
databases and applications.
Every database application, at its core, requires three components in order to operate: the
system, the database, and the application. To deliver performance, the DBA must be able to
monitor and tune each of these components. This is easier said than done.
Tuning the System

The system consists of the system software and hardware required for the application to
provide service, A DB2 for z/OS Performance Roadmap. This includes the computer itself, its
disk subsystems, network connections, and all peripherals. From a software perspective, the
system includes the operating system, the file system, the DBMS itself, networking protocols,
and any related middleware such as transaction processors or message queues.
To deliver consistent system performance, the DBA must have the resources to monitor,
manage, and optimize the performance of these disparate pieces of hardware and software.
Some of the tasks required for system tuning include the proper allocation and management of
memory structures (e.g., buffer pools, program cache area), storage management, integration
of the DBMS with other system software, proper usage of database logs, and coordination of
the operating system resources used by the DBMS. Additionally, the DBA must control the
installation, configuration, and migration of the DBMS software. If the system isnt
performing properly, everything that uses the system will perform poorly. In other words, a
poorly performing system impacts every database application.
When managing DB2 system performance, the DBAs first job is to ensure that all of the
allied agent address spaces are tuned and connected properly. This includes, but is not limited
to, CICS, IMS/TM, TSO, WebSphere, and DB2 Connect. If not configured and tuned
appropriately, all of these system software components interact with, and can affect the
performance of, DB2 applications.
DBAzine.com
For example, when DB2 data is accessed using CICS, multiple threads can be active
simultaneously, giving multiple users concurrent access to a DB2 subsystem of a single CICS
region. A mechanism named the CICS Attach Facility connects CICS with DB2. Using the
CICS Attach Facility, you can connect each CICS region to only one DB2 subsystem at a
time. You can connect each DB2 subsystem, however, to multiple CICS regions
simultaneously. DB2 provides services to CICS via MVS TCBs. All of these TCBs reside in
the CICS address space and perform cross-memory instructions to execute the SQL code in
the DB2 database services address space (DSNDBM1).
Furthermore, the resource control table (RCT) must be configured for each DB2 program that
will run under CICS. The RCT applies only to CICS transactions that access DB2 data; it
defines the manner in which DB2 resources will be used by CICS transactions. In particular,
the RCT defines a plan for each transaction that can access DB2.
Additionally, it defines parameters detailing the number and type of threads available for
application plans and the DB2 command processor.
DB2 DBAs also must ensure that appropriate DB2 system parameters are set using DB2
commands and DSNZPARMs. One of the most important areas for tuning here is memory
usage. DB2 uses memory for buffer pools, the EDM pool, RID pool, and sort pools to cache
data and structures in memory. The better memory is allocated to these structures, the better
DB2 will perform.
When allocating DB2 buffer pools, keep these rules in mind:
Dont allocate everything to a single buffer pool (e.g., BP0); use a multiple buffer pool
strategy.
Explicitly specify a buffer pool for every tablespace and index.
Isolate the DB2 Catalog in BP0; put user and application DB2 objects into other buffer
pools.
Consider separating indexes from tablespaces with each in their own dedicated buffer
pools.
Consider isolating heavily hit data into its own buffer pool to better control performance.
Consider isolating sorts into a single buffer pool and tuning for mostly sequential access
(e.g., BP7).
Consider separating DB2 objects into separate buffer pools that have been configured for
sequential verses random access.
No Magic Bullet
Forget about trying to follow a cookie-cutter approach to buffer pool management. Every
shop must create and optimize a buffer pool strategy for its own data and application mix.
DBAzine.com
DB2 offers the following buffer pool tuning knobs that can be used to configure virtual
buffer pools to the type of processing they support:
DWQT This value is the deferred write threshold; it is expressed as a percentage of
the virtual buffer pool that might be occupied by unavailable pages. When this threshold
is reached, DB2 will start to schedule write I/Os to externalize data. The default is 50
percent, which is likely to be too high for most shops.
VDWQT This value is the vertical deferred write threshold; it is basically the same as
DWQT, but for individual data sets. The default is 10 percent, which once again is quite
likely to be too high for many shops.
VPSEQT This value is the sequential steal threshold; it is expressed as a percentage
of the virtual buffer pool that can be occupied by sequentially accessed pages. Tune
buffer pools for sequential access such as scans and sorting by modifying VPSEQT to a
larger value. The default is 80 percent.
VPPSEQT This value is the sequential steal threshold for parallel operations; the
default value is 50 percent.
VPXPSEQT This value is assisting parallel sequential threshold; it is basically the
VPPSEQT for operations from another DB2 subsystem in the data sharing group.
These parameters can be changed using the ALTER BUFFERPOOL command. Additionally,
hiperpools can be created to back up DB2 virtual buffer pools with additional memory. DB2
provides several tuning knobs for hiperpools, too, including HPSIZE to adjust the size of
hiperpools and HPSEQT to adjust the hiperpool sequential steal threshold.
The EDM pool is used for caching internal structures used by DB2 programs. This includes
DBDs, SKCTs, CTs, SKPTs, and PTs. It also includes the authorization cache for plans and
packages, as well as the cache for dynamic SQL mini-plans. As a general rule, you should
shoot for an 80 percent hit rate with the EDM pool; this means that in only one out every five
times should a structure need to be loaded from disk into the EDM pool.
Finally, remember that buffer and EDM pool tuning are in-depth subjects that cannot be
adequately covered in a high-level article such as this. Additionally, there is much more to
proper DB2 system performance tuning than allied agent and memory tuning. Other system
elements requiring attention include: locking, logging, and Parallel Sysplex configuration and
management for DB2 data-sharing shops.
Tuning the Databases

The second component of DB2 performance tuning is making sure the database is optimally
created and maintained. The database stores the data that is used by the application. When the
application needs to access data, it does so through DB2 to the database of choice. If the
DBAzine.com
database is not optimally organized or stored, the data it contains will be difficult or slow to
access. The performance of every application that requires this data will be negatively
impacted.
The first component of database optimization is assuring an optimal database design.
Database design is the process of transforming a logical data model into a physical database
design and then implementing the physical model as an actual database. Proper database
design requires up-front data modeling and normalization in other words, a logical data
model is required before you can even begin to design a physical database. Assuming a
logical model exists, it should be used as the basis for creating the physical database. The
physical database is created by transforming the logical data model into a physical
implementation based on an understanding of the DBMS to be used for deployment. But a
one-to-one mapping of logical entity to physical table is unlikely to result in an optimal
physical database design.
Successfully developing a physical database design requires a good working knowledge of
DB2s features. More precisely, the DBA must possess in-depth knowledge of physical
database objects supported by DB2 as well as the physical structures and files required to
support those objects. This must include knowledge of the manner in which DB2 supports
indexing, referential integrity, constraints, data types, and other features that augment the
functionality of database objects. Armed with the correct information, the DBA can create an
effective and efficient database from a logical data model.
The DBA creating the physical DB2 database implementation should keep these rules of
thumb in mind when building the databases:
Keep the physical database as normalized as possible. However, performance should win
out over aesthetics. In other words, dont let data modelers dictate physical design. If
you need to reorder columns in the table to optimize logging, do it.
In general, put each table in its own tablespace. Exceptions can be made for very small
tables such as code and reference tables.
In general, favor partitioned and segmented tablespaces over simple tablespaces. And
dont specify a DSSIZE greater than 4GB unless you really need a large tablespace
(doing so will waste space).
Dont create base table views.
Use NULL sparingly.
Use appropriate DB2 data types (e.g., use DATE for dates instead of CHAR or numeric
data types).
Consider using DB2 compression instead of using VARCHAR columns. With
compression, there is less overhead and no programmatic handling is required.
Favor using DB2 declarative RI instead of programmatic RI. It will usually perform
better and is easier to administer. However, its generally a bad idea to use RI for lookup
and reference tables.
DBAzine.com
Avoid the DDL defaults they are usually wrong. Explicitly code every single DDL
option when creating DB2 objects.
Calculate the appropriate amount of free space for both PCTFREE and FREEPAGE
based on the frequency of modification. Do not just default every tablespace to 10
percent free space. DB2 DBAs should always keep in mind that a proper indexing
strategy can be the primary factor to ensure optimal performance of DB2 applications.
Creating Indexes in the Dark

To create appropriate indexes, the DBA must have the SQL that is to be used against the DB2
objects to understand the access patterns. But many times, the DBA is called upon to create
indexes well in advance of the creation of any SQL. This can be a challenging dilemma: how
to create indexes to optimize SQL performance without any knowledge of the SQL? DBAs
can develop some basic rules of thumb, though. These steps can be used as a basic roadmap
for DB2 index creation:
First, take care of unique and primary key constraints by creating unique indexes on those
columns. Then consider creating indexes for each foreign key to optimize RI.
The next step is to examine the most heavily used queries. Look at the predicates and
build indexes to support these queries. Consider overloading indexes by adding columns
to encourage index only access.
Next, examine queries that invoke sorts. Look for ORDER BY, GROUP BY, UNION,
and SELECT DISTINCT. Consider building indexes to support these clauses so DB2 can
avoid initiating an expensive sort. Look at your indexes and make sure youve chosen the
first column wisely. In general, the first column of the index should have a high
cardinality.
In general, consider avoiding indexes on variable columns because DB2 will expand the
VARCHAR column to its full length in the index. After coming up with a first stab
indexing strategy consider the insert, update, and delete implications of those indexes. If
the columns of those indexes must be modified in any way, DB2 will incur additional
overhead keeping the indexes up-to-date.
Over time, as data is modified and updated, DB2 will have to move the data around within the
database. Such activity causes the data to become fragmented and inefficiently ordered. The
longer the database remains online and the more changes made to the data, the more
inefficient database access can become. To overcome disorganized and fragmented databases,
the DBA can run a reorganization utility to refresh the data and make the database efficient
once again. But the key to successful reorganization is to reorganize only when the database
requires it; instead, some companies over-reorganize by scheduling regular database
reorganization jobs to be run whether the database is fragmented, or not. This wastes valuable
CPU cycles.
But reorganization is only one of many database performance tasks performed by the DBA.
DBAzine.com
Others include data set placement, partitioning for parallel access, managing free space, and
assuring optimal compression.
Tuning the Applications

The third, and final, component of database performance is the application itself. Indeed, as
much as 80 percent of all database performance problems are caused by inefficient application
code. The application code consists of two parts: the SQL code and the host language code in
which the SQL is embedded.
SQL is simple to learn and easy to start using. But SQL tuning and optimization is an art that
takes years to master. Some general rules of thumb for creating efficient SQL statements
include:
Let SQL do the work instead of the application program. For example, code an SQL join
instead of two cursors and a programmatic join.
Simpler is generally better, but complex SQL can be very efficient.
Retrieve only the columns required, never more.
Retrieve the absolute minimum number of rows by specifying every WHERE clause that
is appropriate.
When joining tables, always provide join predicates. In other words, avoid Cartesian
products.
Favor using Stage 1 and Indexable predicates.
Avoid sorting if possible by creating indexes for ORDER BY, GROUP BY, and
DISTINCT operations.
Avoid black boxes - that is, avoid I/O routines that are called by programs instead of
using embedded SQL.
Avoid deadlocks by updating tables in the same sequence in every program.
Issue data modification statements (INSERT, UPDATE, DELETE) as close as possible to
the COMMIT statement as possible.
Be sure to build a COMMIT strategy into every batch program that changes data. Failing
to COMMIT can cause locking problems.
Every DBMS provides a method of inspecting the actual access paths that will be used to
satisfy SQL requests. The DBA must thoroughly understand the different types of access
paths and know which ones are best in a given situation. For DB2, the DBA must be able to
interpret the output of the access path explanation produced by EXPLAIN. This information
is encoded in the PLAN_TABLE and must be interpreted. To make matters more difficult, the
PLAN_TABLE doesnt contain 100 percent of the information required to determine if the
SQL will perform efficiently. The DBA (and, indeed, programmers too) must be able to read
the PLAN_TABLE in conjunction with the SQL code, host language code, and information
from the DB2 catalog to judge the efficiency and effectiveness of each SQL statement.
DBAzine.com
Host language code refers to the application programs written in C, COBOL, Java, Visual
Basic or the programming language du jour. SQL statements are usually embedded into host
language code and it is quite possible to have finely tuned SQL inside of inefficient host
language code. And, of course, that would cause a performance problem.
The Bottom Line

DBAs must understand all three aspects of database performance management. The system,
the databases, and the applications all must be monitored and tuned to assure efficient DB2
performance. Furthermore, the DBA must be able to identify performance problems as they
occur. Once identified, each problem must be analyzed to determine its cause. And only then
can a proper tuning strategy be deployed to rectify the problem.
The guidelines in this article can help, but remember each performance management
situation is different and general rules of thumb may not apply.
But remember that this article only touches the tip of the iceberg when it comes to DB2
performance tuning and management. Database performance management is a complex and
ever-changing discipline that requires rigorous attention and focus to master. And then, just
when you seem to understand things, that new version of DB2 will come out that changes
everything! But this is good. It might make life more difficult as you learn the latest DB2
nuances, but new releases invariably help make DB2 applications more efficient.
Good luck with DB2 for z/OS and happy performance tuning!
DBAzine.com
DBAzine.com
Can Your Left Outer Joins Perform Better?

by Joe Burns
Introduction
Have you ever heard the saying, The best performing SQL statement is the one that never
executes? This means that if you can avoid doing an unnecessary SQL call, then you have
saved all of the resources that would have been used by that call. But there should be a similar
guideline that says, The best performing Left Outer Join is the one that never does a join
except when it should.
Left Outer Joins have been part of DB2 for a while now, but starting in V6, the join
processing became much more sophisticated. Changes were made in how the ON predicates
are evaluated, and we can take advantage of those changes to improve the performance of our
Left Outer Joins. This article covers a technique that we are using in our shop to significantly
reduce the CPU costs of our most expensive Left Outer Joins.
Left Outer Joins the Good and the Bad

As DBAs, we face increasingly more complex systems and databases. Many of these designs
involve tables with optional relationships, and there is often a need to join those tables
together. Because the relationships are optional, we have seen a corresponding rise in the
number of Left Outer Joins used in our systems. There is nothing fundamentally wrong with
Left Outer Joins, but they can be slightly more expensive than regular inner joins from a
performance standpoint.
The benefit of a Left Outer Join, over a regular inner join, is that it returns the same result set
as a regular inner join, but also keeps rows from the parent table that have zero matching child
rows. Although this sounds like an ideal solution for joining tables that have an optional
relationship, there may be a hidden performance cost.
Keep in mind that the reason for using a Left Outer Join is that the relationship is optional.
This means that we believe our result set will have at least some (possibly many) parent rows
that will NOT have a matching child row. However, when DB2 is actually performing the
Left Outer Join, it doesnt know which parent rows will or will not have a matching child. The
only way DB2 can really determine if there is a child row is to actually look at the child table
to see if there is one. An index probe into the child table is often done to determine this, which
is where a potential performance issue may occur.
If the optional relationship is such that there are many parent rows without children, then the
Left Outer Join is doing numerous index probes into the child table, looking for potential
matching child rows. Because there are many parent rows without children, most of the time,
DBAzine.com
DB2 doesnt find a child row. This results in a lot of fruitless access to the child table looking
for rows that often are not going to be there. In a high-volume, high-performance application,
this can be a real problem. Spending resources looking for child rows that usually are not
there is far from an ideal design.
But are there actual alternatives to the issue? After all, how can DB2 determine whether a
child row does or doesnt exist unless it actually goes to the child table and looks? Believe it
or not, this can be done using careful database design and well-coded Left Outer Joins.
A Simple Example of a Left Outer Join

First, here is a simple example of a Left Outer Join. We start with a CUSTOMER table that
has an optional relationship to a table called COMPLAINTS. Not surprisingly, the
COMPLAINTS table keeps track of any complaints made by a particular customer.
Obviously, we hope that not every customer has a complaint and, in fact, if we are a good
business we expect that most of them dont. This creates an optional relationship in which
there is often not a child row. Nevertheless, when a customer calls our service center, it is
important that any pre-existing complaints be brought up for the service representative to see.
So, the COMPLAINTS table has to be accessed, and a simple Left Outer Join seems to be the
ideal solution.
SELECT CU.cols.., CP.cols..
FROM CUSTOMER
CU LEFT OUTER JOIN
COMPLAINTS CP ON CU.CUST_ID = CP.CUST_ID
WHERE CU.CUST_NAME LIKE %ABC%
This works, and it will return the desired rows and columns. However, keep in mind that most
of the time there is no matching COMPLAINT row. Despite this, the SQL is still constantly
probing the COMPLAINT table looking for one. If our customer service area is a highvolume application, then this SQL is spending a lot of valuable CPU time looking for
COMPLAINT rows that often are not there.
Fortunately, due to the way that Left Outer Joins are processed (V6 and above), there is an
alternative available to us. The ON clause can accept predicates that allow DB2 to eliminate
the unsuccessful accesses to the child table. The key concept to remember is that the ON
clause tells DB2 how to join the tables together, but it does not provide filtering. This sounds
a little cryptic, but if we understand it, we can leverage it to our benefit. The best way to see
this is by going back to our example.
Leveraging the ON Clause for Performance Benefits

We are going to change the table design slightly by adding an indicator column to the
CUSTOMER table. This indicator is maintained by the application and is switched on if a
CUSTOMER has at least one complaint. At this point, you may have some reservations about
such a design change since it would seem that the indicator could easily get out of synch.
More on those concerns a little later, but for now we will assume that we have added the
indicator. We can now change our SQL
10
DBAzine.com
to include the indicator in the ON clause. It is essential that this indicator is added to the
ON clause and not in the WHERE clause.
FROM CUSTOMER
CU LEFT OUTER JOIN
AND CU.COMPLAINT_IND = Y
WHERE CU.CUST_NAME LIKE %ABC%
This small change will not affect the actual result set in any way; however, it will dramatically
affect the way the Left Outer Join performs. Remember, the key concept was that the ON
clause tells DB2 how to join the tables, but does not provide filtering. Because it does not
filter the data in any way, the CU.COMPLAINT = Y predicate in the ON clause has no
effect on the final result set. In the example, DB2 will return a row whether or not the
COMPLAINT_IND = Y. The predicate in the ON clause is simply ignored when it
comes to determining the final result. This can seem very confusing, because as SQL coders,
it looks very much like a filtering predicate; consequently, we think that it is going to limit the
result set in some way. But it is not, and we must remember this important detail.
The ON Clause Predicate Does Not Affect the Result Set

The following is an example of the result set returned using the ON clause predicate. Note
that the result set includes rows that have a COMPLAINT_IND of Y or N. Remember
that the ON clause is NOT providing filtering.
If we, on the other hand, had actually added a true filtering predicate in the WHERE clause,
then we would have a serious problem with the result set.
FROM CUSTOMER
CU LEFT OUTER JOIN
WHERE CU.CUST_NAME = ABC Company
DBAzine.com
11
In this example, we have added a true filtering predicate in the WHERE clause. This is not
at all what we wanted, since this will only return rows for customers if they have complaints.
We really do not need a customer service application that only returns data for those
customers who complain all of the time. Instead, we prefer that it return a row whether or not
a customer had a complaint.
Benchmarking the Performance Improvement

So, once we have convinced ourselves that adding the predicate in the ON clause will not
affect the result set, then the question is How will it affect our performance? To answer
this, we have to go back to the key concept that the ON clause tells DB2 how to join the
tables together. Because we have added an indicator in the ON clause, DB2 can now use
this to make a determination about whether or not a join to the child table should even be
attempted.
If the complaint indicator on the CUSTOMER table is set to Y, then DB2 will probe the
child table in an attempt to make a join. And assuming our indicator is reliable, then it will
always find at least one child row. After all, the complaint indicator should only be set if there
is a complaint.
If, on the other hand, the complaint indicator is not set, then DB2 will not even attempt to
access the child table to do a join. The ON clause predicate is telling DB2 that there is no
need to look at the child table (the complaint indicator is not a Y, so there will be no child
rows). Instead, just use NULL values for any of the columns selected from the child table.
This will return exactly the same result set as if the additional ON predicate were not coded
at all, but it will do it in less time.
In essence, DB2 is only doing a join when it knows there is a child row (based on the
complaint indicator). This can be thought of as a Left Outer Join that never does a join
except when it should
This reduces both CPU and Getpages, while returning the identical result set. What more
could we ask for? The graphs below are benchmarks comparing a Left Outer Join with and
12
DBAzine.com
without the additional ON predicate. As you can see the Left Outer Join with the additional
ON predicate is clearly the better performer.
Figure 1: Number of rows on Child table.

Note in Figure 1, the CPU time used by the Left Outer Join with the ON predicate decreases
rapidly as the number of child rows decreases. This eventually levels off and stays consistent
as the number of child rows becomes negligible. On the other hand, the CPU time for the Left
Outer Join without the ON predicate tends to stay relatively high even when there are very
few rows on the child table.
DBAzine.com
13
Using the SQL with the ON predicate results in a very low number of Getpages against the
child tables index. This is because DB2 only accesses the child table if the indicator is set on
the parent. In fact, the Getpage activity literally drops off the chart as the number of child
rows decreases. On the other hand, the Getpages are consistently very high in the SQL
without the ON predicate. This is because DB2 has to check the child table every time to
determine if a child row exists. Note that the Getpage activity is still declining somewhat, but
this is only due to the reduction in the size and number of levels in the child index.
From the benchmarks, we can see that there are obvious performance savings from this
approach. However, another less elegant but somewhat simpler solution may jump to mind. If
we are putting an indicator on the CUSTOMER table that tells us whether a complaint exists
or not, then why bother doing a Left Outer Join at all? Why not just read the customer row
and, if the complaint indicator is a Y, go read the COMPLAINT table. Much like the Left
Outer Join, it would seem that we are now avoiding those fruitless lookups to the child table.
After all, we can check the complaint indicator just as easily as DB2 did.
And indeed, this is an option that should be considered and benchmarked since there is the
possibility that it is actually faster than letting DB2 do the check as part of a Left Outer Join.
Joins of any type are not free, and CPU cycles must be used to perform them.
In fact, in the extremely simple example covered in this article, it very well may execute
faster as two separate SQL calls. However, at our shop, we have found that our real-world
situations are not nearly as simple as the example we covered. As a result, more often than
not, the Left Outer Join outperforms the two separate SQL calls. The main reason is that there
will be overhead associated with each separate SQL call since both would need to be coded as
14
DBAzine.com
cursors. And, of course, each cursor would require a separate open, fetch, and close statement.
So, as the number of times the second SQL call needs to be executed goes up, it eats into any
potential performance savings. The Left Outer Join with the additional ON predicate does
not suffer from this degradation since it is all done in a single SQL call (one cursor).
Design Considerations and Where to Use This Technique

Lastly, where is the Left Outer Join technique with the additional ON predicate really
applicable? As mentioned before, there are some very real design concerns in adding such an
indicator to the CUSTOMER table. There is the possibility that the indicator gets out of synch
with the child table and that could cause an incorrect answer to be returned. That is a real
issue and should be carefully considered before designing a table this way. This type of
design should probably only be considered in cases where the application is processing very
high volumes and peak performance is crucial. Further, if this were designed into the
database, then methods should be considered to help insure the reliability of the indicator.
For example, perhaps a trigger could be used on the child table to fire back to the parent table
to either set or unset the indicator as the case may be. This would help to ensure that the
indicator and the child table stay in synch. This may or may not be practical depending on the
update activity on the table, as the triggers may slow the updates down.
Another area where this technique is applicable is when this arrangement occurs almost
naturally, so to speak. There are cases in which certain columns can be thought of as
indicators, even though they truly are not. A good example of this would be a cancel date
column, in combination with a child table that holds the textual description of the reason for
customer cancellation. In this situation, only customer rows that have a cancel date would
have a matching row in the child table with the textual description. The cancel date on the
customer row could be thought of as a type of indicator, which could be used in the ON
predicate. So this would tell the Left Outer Join to only do the join to the textual description
table if the cancel date on the parent customer table is populated.
Optional code tables are another good example of instances for which this may prove
appropriate. Again, if we use the idea of a cancel code on the customer table, then we can see
where this may prove useful. If the cancel code on the customer table is populated, then the
Left Outer Join should attempt to join to the actual cancel code table itself. If the cancel code
on the customer table is not populated, then do not bother going out to the code table. There is
not a matching row there.
Conclusion
We have seen that there are definite performance advantages to using this technique. We can
save CPU and reduce Getpages by adding some simple predicates in the ON clause. An
added benefit is that it often simplifies code since Left Outer Joins allow us to combine access
to tables in a single SQL. So, to take full advantage of the powerful functionality offered by
the Left Outer Join, consider using this interesting technique.
DBAzine.com
15
Notes:
1. The benchmarks used involved a Left Outer Join that was using a Nested Loop Join
method as the access path. If DB2 had selected a Merge Scan Join access path instead,
there would be no performance benefits to this technique. This is because the Merge Scan
never probes the child index like the Nested Loop join; instead, it sorts and scans the child
table.
2. In the key concepts, it was shown that the ON predicate does not affect the result set.
This is true because the ON predicate is referencing the table on the left (the parent
table). If the ON predicate were specified for the table on the right (the child table), then
it could have affected the result set. This is because DB2 will apply the child ON
predicate to the child table during the join, which may affect the final result set.
3. All benchmarks were performed using DB2 for z/OS version 7.
16
DBAzine.com
Utilizing DB2 UDB for OS/390 and z/OS

UNION Everywhere for Large Table Design
by Dan Luksetich
Introduction
Around the time of the introduction of DB2 for OS/390 and z/OS version 7, many people
began talking about the concept of utilizing the new UNION everywhere feature to
accommodate some of our large table designs. This new feature of DB2 allowed UNIONs to
be placed inside of nested table expressions, subqueries, and views. If we could put a UNION
clause into a view, then that view could represent a large table to our applications. This view
could exceed the current table size limits, as well as some of our current large table
concurrency issues.
The idea of storing the contents of a single DB2 table in many physical tables is nothing new.
There are situations in which the amount of data that can be stored in a single table exceeds
the limit (in V7, thats 16TB), or other situations in which data needs to be separated into
many tables for improved concurrency, data organization, or legal issues. In these situations,
the data is divided into several identical tables by applying some sort of business rule to
determine which data goes into which table. When the data is separated in this manner, it is
more difficult for application programs to access the data when the data required spans more
than one table. Clever finder tables have been invented to direct application queries to the
appropriate table where data is stored, or package switching techniques have been established
to direct SQL statements within an application to the appropriate DB2 table. These techniques
lead to additional database or programming complexity.
If we can now create a view against all of the underlying tables, that view could contain a
reference to each of the appropriate underlying tables, with the appropriate business rule
directing access to the table or tables of interest. This would eliminate any finder tables and
package switching algorithms from our applications, and simplify access to these underlying
tables. Programs and SQL can now access the view as if it were a normal table. Doing this is
very desirable; it simplifies managing these large objects, and the programming required to
access them. However, there are always trade-offs, and users must take care to avoid the
limits and pitfalls associated with UNION in view.
Increasing Table Size and Concurrency with Union in View

The amount of data being stored in DB2 is increasing in dramatic fashion. Availability is also
an increasingly important feature of our databases. So, as we build these giant tables in our
system, we must make sure that they are built in a way that makes them available 24 hours a
day, 7 days per week. But these high demands are pushing database administrators to the
edge: They have to build large tables that are easy to access, hold significant quantities of
data, are easy to manage, and available all of the time.
DBAzine.com
17
Partitioning Tablespaces
Traditionally, our larger database tables have been placed into partitioned tablespaces. A
partitioned tablespace allows us to physically distribute our table data across several physical
datasets. Partitioning is determined by key values of the partitioning index, which is the index
used to determine both the clustering and the partitioning of the data. The database engine to
distribute the data across the partitions uses distinct key value ranges. Partitioning helps with
database management because its easier to manage several small objects versus one very
large object. DB2 utilities can act on individual partitions, and the datasets representing each
of the partitions can be distributed across many DASD units. Tables in partitioned tablespaces
can be read using SQL statements much in the same way as tables in segmented or simple
tablespaces. Updates to tables in partitioned tablespaces can sometimes be more difficult if
the update affects the partitioning key.
There are still some limits to partitioning. For example, each partition is limited to a
maximum size of 64GB, a partitioning index is required (this changes in DB2 for z/OS V8),
and if alternate access paths to the data are desired, then non-partitioning indexes (NPIs) are
required. These NPIs are not partitioned, and exist as single, large database indices. Thus,
NPIs can present themselves as an obstacle to availability (i.e., a utility operation against a
single partition may make the entire NPI unavailable), and as impairment to database
management since it is more difficult to manage such large database objects.
Figure 1: Account history table.

A UNION in a view can be utilized as an alternative to table partitioning in support of very
large database tables. In this type of design, several database tables can be created to hold
different subsets of the data that would have otherwise be held in a single table. Key values,
similar to what may be used in partitioning, can be used to determine which data goes into
which of the various tables. For example, we can have an account history table, such as the
table in figure 1, which holds historical account payment information.
This table could be partitioned by the ACCOUNT_ID column into as many as 254 partitions.
However, since the partitioning key is also the clustering key, we would have to scan the
entire tablespace to determine which account history rows could be deleted or archived after
they have aged. If this table were extremely large, we may also have an issue with certain
partitions reaching the 64GB limit if the account history information stored in those partitions
is for very active accounts. In addition, if we want additional access paths by
PAYMENT_DATE or INVOICE_NUMBER, we are going to have to create NPIs to support
18
DBAzine.com
those access paths, and those NPIs could be extremely large and hard to manage. If we want
to reorganize a partition, or redistribute the data across the partitions, an outage will probably
be required to do these things. But in our highly available world, such outages may not be
tolerated.
The UNION Solution

A UNION in view solution for this tables design can solve a lot of the difficulties associated
with large table management. We can separate data into different tables using a simple
business rule say the ACCOUNT_ID. We can then define a view, using a UNION to bring
each tables data together, to create a logical account history table. For example, we can have
four account history tables as defined in figure 2. The union of those four tables could then
represent the logical account history table (the view). And we could accomplish this by using
the following DDL:
CREATE VIEW V_ACCOUNT_HISTORY
(ACCOUNT_ID, PAYMENT_DATE, PAYMENT_AMOUNT,
PAYMENT_TYPE, INVOICE_NUMBER)
AS
SELECT ACCOUNT_ID, PAYMENT_DATE, PAYMENT_AMOUNT,
PAYMENT_TYPE, INVOICE_NUMBER
FROM
ACCOUNT_HISTORY1
WHERE ACCOUNT_ID BETWEEN 1 AND 100000000
UNION ALL
FROM
ACCOUNT_HISTORY2
UNION ALL
FROM
ACCOUNT_HISTORY3
UNION ALL
FROM
ACCOUNT_HISTORY4
WHERE ACCOUNT_ID BETWEEN 300000001 AND 999999999;
DBAzine.com
19
Figure 2: Four account history tables.

By separating the data into different tables, and creating the view over the tables, we can
create a logical account history table with these distinct advantages over a single physical
table:
We can add or remove tables with very small outages, usually just the time it takes to
drop and recreate the view.
We can partition each of the underlying tables, creating still smaller physical database
objects.
NPIs on each of the underlying tables could be much smaller and easier to manage than
they would under a single table design.
Utility operations could execute against an individual underlying table, or just a partition
of that underlying table. This greatly shrinks utility times against these individual pieces,
improves concurrency, and truly gives us full partition independence.
The view can be referenced in any SELECT statement in exactly the same manner as
would be a physical table.
Each underlying table could be as large as 16TB, logically setting the size limit of the
table represented by the view at 64TB.
Each underlying table could be clustered differently, or could be a segmented or
partitioned tablespace.
Predicate Distribution
Each query against our account history view will have predicates distributed across all of the
underlying tables. DB2 creates a separate query block for each underlying table of the view,
and any predicates against the view are distributed to each query block. Note, for example, the
following query:
20
DBAzine.com
SELECT *
FROM
V_ACCOUNT_HISTORY
WHERE ACCOUNT_ID = 12000000;
The predicate will be distributed to all of the query blocks representing access to the
underlying tables. The query is, in essence, rewritten as the following query:
FROM
ACCOUNT_HISTORY1
AND
ACCOUNT_ID = 12000000
UNION ALL
FROM
ACCOUNT_HISTORY2
AND
UNION ALL
FROM
ACCOUNT_HISTORY3
AND
UNION ALL
FROM
ACCOUNT_HISTORY4
AND
ACCOUNT_ID = 12000000;
The same distribution of predicates can apply to predicates that utilize host variables or
parameter markers, as well as join predicates.
Query Block Pruning

While each SQL statement in the union results in an individual query block, and SQL
statements written against our view are distributed to each query block, DB2 does employ a
technique to prune query blocks for efficiency. DB2 can, depending upon the query, prune
(eliminate) query blocks at either statement compile time or during statement execution.
Consider the account history view defined earlier in this article, then consider again the
following query:
SELECT *
FROM
V_ACCOUNT_HISTORY
WHERE ACCOUNT_ID = 12000000
The predicate of this query contains the literal value 12,000,000, and this predicate is
distributed to all four of the query blocks generated. However, DB2 will compare the
DBAzine.com
21
distributed predicate against the predicates coded in the UNION inside our view, looking for
redundancies. In any situations in which the distributed predicate renders a particular query
block unnecessary, DB2 will prune (eliminate) that query block from the access path. So,
when our distributed predicates look like the following, DB2 will prune the query blocks
generated at statement compile time based on the literal value supplied in the predicate:
. . .
WHERE
AND
. . .
WHERE
AND
. . .
WHERE
AND
. . .
WHERE
AND
ACCOUNT_ID BETWEEN 1 AND 100000000

ACCOUNT_ID = 12000000;
Although four query blocks would be generated in the previous example, three of them will
be pruned when the statement is compiled. DB2 compares the literal predicate supplied in the
query against the view with the predicates in the view. Any unnecessary query blocks are
pruned. Since three of the four resulting combined predicates are impossible, DB2 eliminates
those query blocks. Only one underlying table will then be accessed.
Query block pruning can happen at statement compile (bind) time, or at run time if a host
variable or parameter marker is supplied for a redundant predicate. So, lets take the previous
query example, and replace the literal with a host variable:
SELECT *
FROM
V_ACCOUNT_HISTORY
WHERE ACCOUNT_ID = :H1
If this statement were embedded in a program and bound into a plan or package, four query
blocks would be generated. This is because DB2 does not know the value of the host variable
in advance, and distributes the predicate amongst all four generated query blocks. However, at
run time, DB2 will examine the supplied host variable value, and dynamically prune the query
blocks appropriately. So, if the value 12,000,000 was supplied for the host variable value,
then three of the four query blocks would be pruned at run time, and only one underlying
table would be accessed.
This dynamic runtime query block pruning gives the UNION in view design an advantage
over a single partitioned database design in that only the necessary data will be accessed at
run time without query re-optimization. If a partitioned table is accessed based on the
partitioning key, as supplied in a predicate via a host variable, then partition range scanning is
not a possible access path unless the REOPT(VARS) bind parameter is utilized. This may
result in increased CPU utilization because every statement will incur an incremental bind.
Thus, the UNION in view design provides a distinct advantage in that the query block pruning
is available at runtime, without any need for re-optimization.
22
DBAzine.com
Predicate Transitive Closure Applied to Joins and UNION in View

The concept of the distribution of predicates can become a little more complicated when we
start joining tables to our view containing a UNION, or when joining tables to a nested table
expression that contains a UNION. The same concepts apply to views or nested table
expression, and so for the following example, well stay with the previously created view.
Figure 3: Person account table.

To join the person account table described in figure 3 to the account history view, we could
code the following statement:
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
V_ACCOUNT_HISTORY B
ON
A.ACCOUNT_ID = B.ACCOUNT_ID
AND
A.PERSON_ID = 55;
The join, as well as the local predicates against the person account table, are distributed across
all of the tables in the UNION in the view. Once the join is distributed across all of the tables
in the UNION in view, predicate transitive closure can also be applied. This query is, in
essence, rewritten by the optimizer as the following query:
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY1 B
ON
AND
B.ACCOUNT_ID BETWEEN 1 AND 100000000
AND
A.ACCOUNT_ID BETWEEN 1 AND 100000000
AND
A.PERSON_ID = 55
UNION ALL
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY2 B
ON
WHERE B.ACCOUNT_ID BETWEEN 100000001 AND 200000000
AND
AND
A.PERSON_ID = 55
UNION ALL
SELECT *
DBAzine.com
23
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY3 B
ON
WHERE B.ACCOUNT_ID BETWEEN 200000001
AND
A.ACCOUNT_ID BETWEEN 200000001
AND
A.PERSON_ID = 55
UNION ALL
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY4 B
ON
WHERE B.ACCOUNT_ID BETWEEN 300000001
AND
A.ACCOUNT_ID BETWEEN 300000001
AND
A.PERSON_ID = 55;
AND 300000000
AND 300000000
AND 999999999
AND 999999999
Note that with the previously shown query, the optimizer has distributed the join, and the join
predicate (ON A.ACCOUNT_ID = B.ACCOUNT_ID) to all query blocks. The local predicate
against the person account table (AND A.PERSON_ID = 55) has also been distributed. And, the
range predicate within the view (e.g., WHERE B.ACCOUNT_ID BETWEEN 100000001 AND
200000000 for query block 2) has been applied to the person account table in each query
block via predicate transitive closure.
Limitations to Union Everywhere

Careful considerations need to be made when designing very large databases. When DB2
UDB for OS/390 and z/OS V7 was first announced, a lot of consultants (myself included)
touted the UNION everywhere feature, and UNION in view in particular, as a solution to
some of the very large table design and concurrency issues. We boasted that UNION in view
logically increased the size of the largest table to 3.6PB (16TB X 225 underlying tables). Of
course, with every great new feature of the database comes certain limitations that must be
considered when developing the database design.
The number of underlying tables unioned in our view is limited by the complexity of the
queries that are going to reference it. If we always reference only the view in every statement
we write, then maybe we could union together 225 tables. However, it is likely the statement
would fail to compile, receiving instead a SQLCODE of 101 (statement too large or
complex) when DB2 runs out of memory for its parse tree. The number of potential
underlying tables in the view would be further reduced if any joins were coded against the
view. As stated earlier, DB2 will attempt to distribute the join to all of the query blocks within
the view, dramatically increasing the complexity of the query, which again can lead to a
SQLCODE 101.
Also, if the number of tables resulting from the join distribution exceeds 225 for the entire
query, DB2 will stop distributing the join and revert to materializing the view within the
query. This would obviously result in the scanning of all of the underlying tables within the
view, and assuming our underlying tables are very large, performance would not be
24
DBAzine.com
acceptable. Our large database systems tend to be complex, resulting sometimes in large joins.
In one of our recent designs, we began with a large UNION in view design that unioned 51
tables. Complex statements involving joins against this view resulted in SQLCODE 101 or
materialization of the view, and we eventually reduced the number of underlying tables to
eight after several attempts at redesign.
Bind time or run time query block pruning is only available for predicates coded against the
view that utilizes a host variable or literal value. Predicates with literal values against the view
may result in bind time query block pruning, while predicates with host variables or parameter
markers may result in run time query block pruning. Join predicates are not used to prune
query blocks during statement execution.
Predicate transitive closure is applied after joins have been distributed to the view. But DB2
determines if bind time or runtime query block pruning is possible during the distribution of
joins and predicates to the view. So, any predicate derived via predicate transitive closure that
may have resulted in query block pruning does not result in query block pruning. Lets take a
look at our previous join example:
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
V_ACCOUNT_HISTORY B
ON
AND
A.PERSON_ID = 55;
If the local predicate against the person table is modified to filter on the ACCOUNT_ID
column:
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
V_ACCOUNT_HISTORY B
ON
AND
A.PERSON_ID = 55
AND
A.ACCOUNT_ID = 1204524;
All predicates will be distributed to all query blocks, and the predicate on the ACCOUNT_ID
column will be applied to all of the underlying tables of the view via predicate transitive
closure. However, since predicate transitive closure happens after the join is distributed, none
of the query blocks will be pruned at bind or runtime. All query blocks will be accessed
during the statement execution. So, it is important that when coding such queries, we do so by
applying the predicate that would result in query block pruning explicitly:
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
V_ACCOUNT_HISTORY B
ON
AND
A.PERSON_ID = 55
AND
B.ACCOUNT_ID = 1204524;
DBAzine.com
25
Now that the predicate against the ACCOUNT_ID references the column in the view, we can
apply query block pruning.
Because a view containing a UNION cannot be updated or inserted into, any program
processes that need to update or insert into our view will have to update or insert into the
appropriate underlying table. Well have to create application logic to branch to the
appropriate insert or update statement, or we can employ some other technique such as
package switching. To retain the powerful flexibility that UNION in view provides our
design, its prudent to implement a finder table so that applications that make changes to the
underlying tables can utilize flexible routines to do so. For the account history view, the finder
table (ACCOUNT_RANGE, figure 4) would contain the following data:
TABLE NAME
ACCOUNT_HISTORY1
ACCOUNT_HISTORY2
ACCOUNT_HISTORY3
ACCOUNT_HISTORY4
START RANGE
1
100000001
200000001
300000001
END RANGE
100000000
200000000
300000000
999999999
Figure 4: Account history finder table.

Applications could then read the account range table, and dynamically insert or update the
account history table that corresponds to the account they are processing.
Join Impact on UNION in View

In situations in which queries will join a UNION in view to other tables, we need to consider
the potential overhead associated with the multiple query blocks that will be generated.
Depending on the amount of query block pruning, and the joined table access sequence within
each query block, certain queries may introduce some inefficiency to query performance.
Lets go back to the following example of our join between the person account table and the
account history UNION in view:
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
V_ACCOUNT_HISTORY B
ON
AND
A.PERSON_ID = 55;
26
DBAzine.com
Considering that the high order column primary key on the person account table is the
PERSON_ID column, it is reasonable to assume that the optimizer will select the person
account table as the outer table of the join. So, after the distribution of the join, and the
predicates across all of the underlying tables of the view, the optimizers rewritten query may
look something like this:
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY1 B
ON
AND
B.ACCOUNT_ID BETWEEN 1 AND 100000000
AND
AND
A.PERSON_ID = 55
UNION ALL
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY2 B
ON
AND
AND
A.PERSON_ID = 55
UNION ALL
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY3 B
ON
AND
AND
A.PERSON_ID = 55
UNION ALL
SELECT *
FROM
PERSON_ACCOUNT A
INNER JOIN
ACCOUNT_HISTORY4 B
ON
AND
AND
A.PERSON_ID = 55;
The predicate on the PERSON_ID column has been distributed to every query block, the join
predicate has been distributed to every query block, and the range predicate from each query
block of the view has been applied to the person account table via predicate transitive closure.
However, since there is no literal value or host variable predicate against the range predicate
of the account history UNION in view, then there is no bind time or runtime query block
pruning. This means that every query block will be executed, and in this case, the person
account table will be accessed redundantly. Figure 5 visualizes the order of the table access
for each query block:
DBAzine.com
27
Figure 5: Order of the table access for each query block..

In each of the four query blocks, the person account table will be accessed, and the predicate
on the PERSON_ID column will be applied (AND A.PERSON_ID = 55), as well as the
predicate against the ACCOUNT_ID column generated by predicate transitive closure (e.g.,
ANDA.ACCOUNT_ID BETWEEN 300000001 AND 999999999 for query block 4). This is
assuming, of course, that the person account table is the first table accessed for each query
block.
This redundant access will increase the number of GETPAGEs issued and CPU utilized for
some queries that involve joins against views that employ UNION. In tests of a four-table
join, in which the fourth table was a UNION in view large table design containing eight
underlying tables, and in which query block pruning did not occur, there was a 100 percent
increase in CPU utilization when compared to an equivalent query in which the view was
replaced by a single table (both queries returned the same results). In this situation, predicate
transitive closure applied predicates from the union to the table joined to the view. However,
since that table was the third table accessed in the join sequence, the first two tables were
accessed redundantly in each query block. For this reason, most of the joins to the view will
be programmatic.
The performance impact of joins on UNION in view will vary greatly depending on the
number of tables involved in the join, the number of underlying tables in the view, the
potential query block pruning, the table join order for each query block, and how much
filtering can be applied to tables early in the join sequence. Careful planning will ensure a
happy marriage between the flexibility that UNION in view has to offer, and query
performance.
28
DBAzine.com
The Role of UNION in View in V8

Version 8 introduces some extremely useful enhancements to partitioned tablespaces. The
number of partitions has increased (up to 4096), and there is greater ability to dynamically
add or remove partitions. Version 8 also introduces data partitioned secondary indexes, which
improve utility concurrency and can potentially eliminate large non-partitioned indexes. Runtime partition elimination without re-optimization will bring partition elimination up to speed
with the run-time query block pruning we see today with UNION in view. Although the
maximum size of any tablespace remains 16TB, and the maximum size of any single partition
remains 64GB, union in view can be used to overcome these limitations. The ability to
dynamically create new tables without impacting current processing, and drop and recreate
the view in a matter of seconds with a new definition, is also a great new benefit. This means
that UNION in view is still a viable option for very large table design in version 8.
Conclusion
UNION in view for very large table design offers significant advantages for concurrency,
design flexibility, and very large virtual tables. As with any advanced database design, and
advanced database features, we must be very careful when weighing the advantages and
disadvantages of the design. And, we must utilize proper consideration and control when
accessing the view. I have deployed UNION in view in some of my very large
implementations, and will continue to do so when it is appropriate. I know that, with proper
considerations, these implementations will be a success.
DBAzine.com
29
30
DBAzine.com
REORG Redux: A Review of the DB2

REORG Utility
by Julian Stuhler
Introduction
Familiarity breeds contempt, or so the saying goes. Many DB2 professionals are certainly
familiar with the DB2 REORG utility, but how often do we really stop and think about
exactly what it is doing and why?
This article will outline the basic operation of the REORG TABLESPACE utility (hereafter
referred to simply as REORG), and provide some simple guidelines on when and how to run
it. This can be quite useful to DB2 rookies and may even be a handy reminder for some of
you battle-hardened veterans out there.
Throughout the article, I am going to concentrate on the basics of the REORG utility, as
supported by IBM and the other major tool vendors such as BMC Software. Although each of
these vendors supports additional advanced functionality within their REORG
implementations, we explore strictly the basics here.
What Is REORG For?

It is a sad fact of life that any ordered system tends to gradually deteriorate into chaos unless
steps are actively taken to prevent deterioration. As an avid movie collector, I experience this
almost daily, as my two daughters constantly attack my carefully indexed collection. Movies
taken from the rack are returned to the wrong place, and new movies are casually added in
empty slots at the end of the collection without regard for their rightful place.
DB2 tables are the same. Just as I carefully order my DVDs alphabetically to allow more
efficient movie selection, DB2 will attempt to keep all of the rows in the table physically
ordered according to the clustering index (the index created with the CLUSTER keyword, or
the first index created if CLUSTER is not specified for any index). When a new row is
inserted into the table, DB2 will try to place that row in the correct data page so that the
clustering order is maintained.
The PCTFREE and FREEPAGE parameters within the tablespace definition allow the
designer to tell DB2 how much empty space to leave within each page, and how often to leave
a completely empty page respectively. This free space will gradually be consumed as new
rows are inserted.
If there is not enough space on the target page for the row to be inserted, DB2 has no choice
but to select an alternative location for it. Likewise, if a variable length row increases in size
following an UPDATE operation and does not fit on the original page any more, DB2 will
DBAzine.com
31
have to move it to another location and leave behind a pointer to the new location. In either
case, just like my movie collection, the organisation of the table will gradually degrade over
time and the performance of operations that depend upon that clustering will do likewise.
This performance degradation can be especially noticeable in large batch programs and other
processes that tend to access lots of data sequentially. When the data is highly clustered
according to the processing sequence of the batch program (refer to Figure 1 below), the data
can be very efficiently accessed as DB2 is able to read all of the rows on a page with a single
GETPAGE operation and may use other tricks such as sequential or dynamic Prefetch to
make the process even more efficient.
Batch Process
SELECT .
FROM .
WHERE .
ORDER BY .
Data Pages
Data accessed sequentially
Figure 1: Data highly clustered according to batch process requirements.

Contrast this to a situation in which the data is very poorly clustered, as shown in Figure 2. In
this case, DB2 has to randomly access pages throughout the tablespace, which will probably
result in significantly higher GETPAGE counts and inhibit DB2s ability to efficiently
Prefetch pages into the buffer pool to reduce I/O and improve performance.
32
DBAzine.com
Batch Process
SELECT .
FROM .
WHERE .
ORDER BY .
Data Pages
Data accessed randomly

Figure 2: Data poorly clustered according to batch process requirements.
The REORG utility will re-establish the clustering of a table, ensuring that the rows are
physically ordered in the same sequence as the clustering index. At the same time, it will reestablish the free space defined by the PCTFREE and FREEPAGE parameters to allow
INSERTs and UPDATEs to take place on the correct target page. If you drop a table in a
multi-table tablespace, REORG will also reclaim the space used by the dropped table.
But the REORG utility is more than just a means of ordering your data and re-establishing
free space. Using the PREFORMAT option, you can instruct DB2 to pre-format all of the
space up to the highly used RBA of the dataset as part of the REORG operation. This will
speed up very high-volume INSERT operations, which will no longer need to wait while DB2
formats data pages.
REORG can also be a very useful way of removing old or unwanted data from your tables.
Coding the DISCARD statement as part of your REORG job will allow you to filter out
unwanted rows as part of the unload operation, which can sometimes save time and effort
associated with writing a separate data clean-up program.
Because most REORG operations also involve dropping and recreating the underlying VSAM
datasets for a tablespace, it is an excellent way of implementing dataset-level changes such as
altering the primary and secondary space allocations, removing extents or moving the datasets
to a different volume following a STOGROUP or SMS change. Be careful though
specifying the REUSE option (introduced in Version 6) tells DB2 to avoid dropping and
recreating the VSAM datasets, which means none of the above items will change.
There is another class of changes, implemented by the SQL ALTER statement that will take
effect when REORG is executed. These include switching compression on and off for a given
tablespace, and materialising the contents of new columns added via ALTER TABLEADD
DBAzine.com
33
COLUMN (important for performance reasons). With the new dynamic schema change
functionality added in Version 8 allowing DBAs to make many more changes via a simple
ALTER, running REORG is essential to ensure that good performance is maintained
following the change.
Finally, there is the data availability aspect. If you are one of those poor souls asked to
maintain a 24x7 production database, making time for data housekeeping operations such as
REORG can be painful, to say the least. There are two major features offered by REORG that
can help. First, if the tablespace is partitioned you are able to REORG multiple partitions in
parallel to reduce the overall elapsed time of the process, with very little contention1. An even
better option might be to use online REORG, by specifying SHLEVEL REFERENCE or
SHRLEVEL CHANGE on your REORG parms. This allows full-read access (and, or
SHRLEVEL CHANGE, write access) to the data while the REORG is running, with only a
small window at the end of the process in which data is unavailable (see section on How
Does REORG Work? below).
So REORG has many abilities, and can be a valuable aid to the hard-pressed DBA when it
comes to getting a performance boost or rapidly implementing those space allocation changes.
Next we will look at how it actually works.
How Does REORG Work?

As mentioned earlier, there are two different types or REORG: the traditional or offline
version (which works by unloading and then reloading the tablespace data) and the newer
online REORG (which works by unloading the data and reloading it to a separate shadow
copy of the tablespace to maintain full data access while the REORG takes place).
As we are only looking at the basics of REORG in this article, we will concentrate on the
simpler offline version. The following diagram shows the most basic operation of a REORG
SHRLEVEL NONE utility. Use of additional keywords such as SORTDATA and
SORTKEYS will complicate this process somewhat, but can have a very positive impact on
the overall performance of the utility.
As you can see from the diagram, if we ignore the usual UTILINIT and UTILTERM phases
used for utility initialisation and cleanup respectively, we are left with four major steps:
The UNLOAD phase will unload the table data into a sequential dataset (using the clustering
index unless the SORTDATA option is specified, in which case a tablespace scan is used and
the data is sorted afterwards).
Prior to Version 8, the existence of non-partitioning indexes (NPIs) on the partitioned table can cause some
contention during the index rebuild phase. Version 8 makes it possible to avoid this issue by introducing Data
Partitioning Secondary Indexes (DPSIs).
34
DBAzine.com
The RELOAD step will then reload the table data back into the tablespace, re-establishing any
FREESPACE and PCTFREE allocations as it goes. As the sequential dataset holds the rows
in clustering sequence, the table will be 100 percent clustered following the reload operation.
Next, the SORT step extracts the keys for any indexes on the table, and sorts them into key
sequence.
Finally the BUILD phase uses the sorted and extracted keys to rebuild the tables indexes.
Table
Index
Table
Data
UTILINIT
UNLOAD
Index
Keys
RELOAD
SORT
BUILD
UTILTERM
Phase
As previously mentioned, both the IBM and the third-party tool vendors implementations of the
REORG utility contain a large number of additional keywords and options to improve the
performance of this process, but these are beyond the scope of this article.
When Should You Run REORG?

So, now we know why we need a REORG utility and how it works. When do you need to
actually run it? As usual, the answer is, It depends!
The diagram below shows the three major factors that affect the organisation of a table:
Volatility: The number of INSERT/UPDATE/DELETE operations conducted against a table
during a given period of time is obviously a major factor. The more frequently these
operations are conducted, the quicker a table is likely to become disorganised.
Free Space: As previously discussed, the FREESPACE and PCTFREE tablespace parameters
allow the designer to specify the amount of space to be set aside to accommodate newly
inserted rows (or updated rows that have increased in length and will no longer fit in their
original position). The higher these values are set, the greater the tables capacity for coping
with INSERT/UPDATE operations without the clustering beginning to degrade.
DBAzine.com
35
REORG Frequency: The elapsed time between REORG operations is the final factor. The
more frequently it is run, the less time the data has to become disorganised with a given
amount of update activity.
Table
Volatility
?
Free
Space
Reorg
Frequency
Fix any two of these variables, and the third one can usually be calculated. As we usually
have very limited control over the tables volatility (this is usually a function of the business
processes that use it), it is most common for us to have to juggle the remaining two variables
in order to ensure that application performance remains within acceptable boundaries.
For example, operational issues may make it impossible to reorganise the table more
frequently than once per day. In this case, enough space must be allowed in PCTFREE and
FREEPAGE to accommodate a typical days worth of activity without the clustering
degrading to an unacceptable level.
So what constitutes an unacceptable level? The DB2 catalog contains a wealth of
information that can help, and every DBA has his or her favourite queries for extracting the
relevant information. You will find just a few of the more simple queries immediately
following.
Running this subsequent query will list the total number of rows in the tablespace or partition
(CARD), together with the number of varying-length rows located near to and far from their
original position (NEARINDREF and FARINDREF respectively).
36
DBAzine.com
SELECT CARD, NEARINDREF, FARINDREF

FROM SYSIBM.SYSTABLEPART
WHERE DBNAME = database_name
AND TSNAME = tablespace_name;
The higher that NEARINDREF and FARINDREF become as a percentage of the overall
table, the poorer your I/O performance will become as DB2 has to constantly follow row
pointers in order to retrieve the relocated rows.
If you use non-segmented multi-table tablespaces, the PERCDROP column in the same
catalog table will tell you what proportion of the tablespace pages are occupied by dropped
tables. Running REORG will reclaim this wasted space.
The degree to which the table is ordered according to the clustering index can be determined
in a number of ways. Looking at the CLUSTERRATIO in SYSIBM.SYSINDEXES will tell
you the percentage of rows in clustering order; but, if your tablespace is partitioned, this will
only be an average across all partitions. Running the following query against
SYSINDEXPART will give a clear indication on a partition-by-partition basis:
SELECT NEAROFFPOSF, FAROFFPOSF
FROM SYSIBM.SYSINDEXPART
WHERE IXCREATOR = index_creator_name
AND IXNAME = index_name;
This will return the number of rows in each partition, which are near and far from their
optimal positions, respectively. This is due to INSERT operations having insufficient free
space on the target page. Again, as these figures climb, sequential access performance will
degrade.
DB2 Version 6 introduced some REORG enhancements that allowed limits to be specified
for both NEARINDREF/FARINDREF and NEAROFFPOSF/FAROFFPOSF, so that the
REORG will only be executed if these limits are exceeded. This allows you to schedule the
REORG to run on a regular basis, but only have it actually executed when necessary.
Do not forget that the catalog is only as up-to-date as the last time you ran RUNSTATS
against the tablespace, so make sure your stats are current before making any decisions based
upon them. Also, if you are collecting statistics history (using the HISTORY keyword on your
RUNSTATS jobs), then all of these figures can be analysed over time in order to see how
they degrade this can be a big help in determining the sweet spot for REORG frequency
and/or free space allocation.
Conclusion
Reorganising your tablespace can be a critical factor in maintaining the desired performance
levels for your application. Hopefully, this article has prompted you to take a closer look at an
aspect of DB2 performance usually neglected. Now, if youll excuse me, Im off to re-order
my DVD collection!
DBAzine.com
37
38
DBAzine.com
DB2 Buffer Pool Essentials

by Anne Stevens
If you use DB2 for z/OS and OS/390 as your database management system, you know that
you need to keep a watchful eye on your memory. DB2 just loves memory. Some might say
that the more memory you throw at DB2, the better. But simply adding more and more
memory to your DB2 buffer pools is not always the best approach. The issue is more about
how to practically manage the memory at DB2s disposal.
Memory Basics
By caching data and internal structures in memory, instead of accessing them from disk every
time they are needed, DB2 optimizes I/O efficiency. A read from memory is much less
expensive than a read from disk.
Whenever we speak of memory, most of us jump to conclusions and think about buffer pools.
Well, buffer pools are just one resource for which DB2 uses memory to manage. There are
others.
One of the more important memory structures used by DB2 is the EDM pool. Proper EDM
Pool allocation is important because DB2 uses this memory to cache structures required by
DB2 application programs as they run.
Every DB2 program requires a plan or a package that contains the optimized access paths for
the SQL it contains. When a DB2 application program is executed, these access paths must be
loaded into the EDM Pool for DB2 to use. This takes the form of Cursor Tables (or CTs) for
plans, and Packages Tables (or PTs) for packages. DB2 also maintains SKCTs and SKPTs,
skeletons for creating additional PTs and CTs when the same program is being run
concurrently.
SKCT/SKPT pages represent cursor table and package table pages. If the plan was bound with
packages, then SKPT pages are allocated. If the plan was bound with DBRMs, then SKCT
pages are allocated. There is only one SKCT page allocated for the plan regardless of the
number of threads executing the plan; the CT/PT pages are allocated for each thread instance
of the plan.
Additionally, DB2 caches DBDs in the EDM Pool. DBD pages are control block structures
that represent databases and their dependent objects. Any time a table is accessed, the DBD
for the database in which that table was created must be loaded into the EDM Pool.
There are other structures cached into the EDM Pool as well, such as an authorization cache
and perhaps cached dynamic SQL mini-plans (if the dynamic statement cache is in use). Also,
DBAzine.com
39
as of DB2 V8, the EDM Pool is broken apart into separate pools for DBDs and CTs/PTs. This
helps so that loading a DBD does not overwrite CTs/PTs/SKCTs/SKPTs, and vice versa.
If the EDM Pool is too small, your system will experience increased I/O to the DB2 Directory
(tables SCT02, SPT01 and DBD01), worse response time as the system loads SKCTs, SKPTs,
and DBDs, and perhaps re-preparation on cached dynamic SQL statements.
DB2 also sets aside memory for the RID Pool. The RID pool is used for all record identifier
(RID) processing. For example, it is used for enforcing unique keys while updating multiple
rows and for sorting RIDs during the following operations:
List Prefetch
Multiple index access
Hybrid joins
If there is insufficient RID pool storage at execution time, it is possible that DB2 will revert
back to a tablespace scan and that can severely impact performance.
Another DB2-memory consumer is the Sort Pool. As might be expected, the Sort Pool is used
by DB2 when data must be sorted. The default Sort Pool size is 1MB, but keep in mind that a
Sort Pool is allocated for every concurrent sort user. So, there is not just one Sort Pool, but as
many as are needed based on your workload. The bigger the Sort Pool, the more efficient
DB2 sorting becomes because more data can be sorted in memory, instead of requiring it to be
stored in the sort work database, DSNDB07.
Which brings us to buffer pools.
DB2 Buffer Pools

DB2 provides 80 separate buffer pools for caching data in memory. There are 50 4K buffer
pools, ten 8K buffer pools, ten 16K buffer pools, and ten 32K buffer pools. The buffer pools
are named as follows:
4K: BP0 thru BP49

8K: BP8K0 thru BP8K9
16K: BP16K0 thru BP16K9
32K: BP32K, BP32K1 thru BP32K9
DB2 tablespaces and index spaces are assigned to a specific buffer pool when they are
created. If no buffer pool is specified on the CREATE statement, then the object will default
to a buffer pool as specified in the CREATE DATABASE statement. There is both a
tablespace buffer pool default and an index-space buffer pool default that can be specified for
each database. If no buffer pools are specified at the database level, then DB2 will use the
default assigned for buffer pools at install time.
40
DBAzine.com
As a general rule of thumb, though, it makes sense to explicitly specify the buffer pool to be
used for each tablespace and index space. Doing so provides better control and management
of buffering and performance.
But how do you decide which buffer pool to use for which object? This is a much trickier
task, but there are some general guidelines to follow. First of all, it is better to use multiple
buffer pools than to use a single, large buffer pool for everything. In the early days of DB2
(1980s), many shops put everything into BP0ki and let DB2 manage how best to buffer. As
IBM provided additional buffer pools to use, and organizations tested different approaches to
buffer pool management, the general recommendation has changed to use multiple buffer
pools tuned for different types of processing instead of a single big buffer pool.
So, as we build DB2 databases, we need to gather knowledge of how our data will be
accessed by our end users and applications. Therefore, a good, first approach to buffer pool
design is to break apart the objects into different types of usage.
First of all, separate system objects (DB2 Catalog and Directory) from application data
objects (tablespaces and indexes). The system objects will be assigned to BP0 so you can
track and tune access to the system catalog and directory separate from application data. Be
sure to assign only system objects into BP0.
Next, separate your tablespaces and indexes. For example, place the tablespaces in BP1 and
the indexes in BP2. Doing so should help to maintain your index data in memory longer. This
is so because a large sequential Prefetch against a tablespace would not overwrite index data
in the buffer pool since the index and tablespace reside in different buffer pools.
Once you have taken care of this rudimentary separation, you can get down to the hard job of
allocating objects to buffer pools based on workload. Try to put objects that are more
frequently accessed randomly in separate buffer pools from objects that are more often
accessed sequentially. Of course, it is rare for an object to be accessed only randomly or only
sequentially, so you will have to determine which type of access is most frequent. Some
objects may be accessed as a true mix of random and sequential, and you might even have a
separate buffer pool for those.
An additional consideration is tuning buffer usage for DB2 sorting. DSNDB07 is the DB2
database for the sort worktable spaces used by DB2 when sorting data. Consider assigning
these tablespaces to a separate buffer pool tuned for sorting. (BP7 works out well for this
purpose because the 7 at the end of the buffer pool name aligns nicely with the 7 at the
end of the database name.) Then tweak the buffer pool parameters for mostly sequential
access. We will talk about the buffer pool tuning parameters in the next section of this article.
DBAzine.com
41
You might also want to think about pinning some data into a buffer pool. This tactic can be
beneficial for tables or indexes that are accessed very frequently, but are not very large, as are
lookup tables or code tables.
Buffer Pool Sizing

DB2 very efficiently manages data in large buffer pools. To search for data in a large buffer
pool does not consume any more resources than searching in a smaller buffer pool. With this
in mind, do not skimp on allocating memory to your DB2 buffer pools.
So, just how big should you make each buffer pool? One rule of thumb is to make the buffer
pool large enough to hold five minutes worth of randomly read pages during peak processing
time. To clarify, the goal is that once a page has been read from disk into the buffer pool, the
goal is to maintain it in memory for at least five minutes. The idea is that a page, once read, is
likely to be needed again by another process during peak periods. Of course, your particular
environment may differ.
But it can be difficult to know how much data is read at peak processing time and tricky to
know when peak processing time will occur, before an application is coded. You can gather
estimates from the subject matter experts, end users, and application designers, but they will
just be estimates. Over time, you will have to examine buffer pool usage and perhaps re-size
your buffer pools, or move objects from one buffer pool to another to improve performance.
You will want to make sure that you have not allocated so much memory to DB2 buffer pools
that the system starts paging When there is not enough real storage to back the buffer pool
storage, paging activity will cause performance to degrade.
Paging occurs when the virtual storage requirements for a buffer pool exceeds the real storage
capacity for the z/OS image. When this happens, DB2 migrates the least recently used pages
in the buffer pool to auxiliary storage. If the data that was migrated is then accessed, those
pages must be brought back into real storage from auxiliary storage. When you detect that
DB2 is paging, you should either increase the amount of real storage, or decrease the size of
your buffer pools.
Only DBAs or systems programmers should be allowed to add or modify the size of DB2
buffer pools. And these qualified professionals should be able to analyze the system to know
the amount of memory available, as well as the amount being used by other system software
and applications. (Note: For a good overview of mainframe storage management, check out
the IBM presentation on the Web at http://www306.ibm.com/software/data/db2/os390/techdocs/DMDB13.pdf.)
Buffer Pool Tuning Knobs

DB2 provides some nice tuning parameters to customize each buffer pool for the type of data
and application processing to be using the buffer pool. Lets examine the parameters at our
42
DBAzine.com
disposal by learning about the buffer pool thresholds that are monitored and maintained by
DB2.
First of all, DB2 has five adjustable thresholds that can be modified using the ALTER
BUFFERPOOL command. These thresholds are as follows:
Sequential Steal Threshold, or VPSEQT The percentage of the buffer pool that can
be occupied by sequentially-accessed pages. For example, at the default value of 80,
when this threshold is reached, 80 percent of the buffer pool represents pages for
sequential processing. Of course, 80 percent is the default; you can modify this value to
any value (ranging from 0 to 100) that makes sense for your data/processing mix. When
this threshold is reached, DB2 will steal a sequential page first before stealing a page
from a randomly accessed page. So, for data that is accessed mostly sequentially (for
example, through scans and perfecting), consider increasing the value of this parameter;
and for data that is accessed most randomly, consider decreasing the value of this
parameter. A VPSEQT value of zero will prevent any sequential pages from consuming
space in the buffer pool and it will turn off sequential Prefetch. A VPSEQT value of 100
allows the entire buffer pool to be monopolized by sequential pages.
Parallel Sequential Threshold, or VPPSEQT This threshold indicates the amount
of the buffer pool that can be consumed by sequentially-accessed data for parallel
queries. When this threshold is reached, DB2 will cease to steal random pages to store
sequential pages accessed by parallel queries. The default value for VPPSEQT is 50
percent, indicating its size as 50 percent of the sequential steal threshold (VPSEQT). For
example, if the buffer pool is defined as 1000 pages and VPSEQT is set at 80 percent, a
query using I/O parallelism can consume up to 400 sequential pages (that is, 1000 x 80
percent = 800 for the sequential steal threshold and 800 x 50 percent = 400 for the
parallel sequential threshold).
Assisting Parallel Sequential Threshold, or VPXPSEQT This threshold indicates
the portion of the buffer pool that might be used to assist with parallel operations initiated
from another DB2 in the data-sharing group. It is measured as a percentage of VPPSEQT.
Setting VPXPSEQT to zero disallows that DB2 from assisting with Sysplex query
parallelism at run time for queries using this buffer pool.
The final two modifiable DB2 buffer pool thresholds are used to indicate when modified data
is to be written from the buffer pool to disk. Log data is externalized when a COMMIT is
taken, but the two deferred write thresholds control writing of the actual data.
Deferred Write Threshold, or DWQT When DWQT is reached, DB2 starts
scheduling write I/Os to externalize the data pages to disk. By default, the deferred write
threshold is reached when 50percent of the buffer pool is allocated to unavailable pages,
whether updated or in use. The default is probably too high for most larger-sized buffer
pools.
DBAzine.com
43
Vertical Deferred Write Threshold, or VDWQT Basically the same as DWQT, but
for a single page set. By default, VDWQT is reached when 10 percent of the buffer pool
is allocated to one data set. When reached, DB2 will start scheduling write I/Os to
externalize the data pages to disk. Once again, this default is most likely too high for
most shops.
Consider ratcheting the deferred write thresholds down to smaller percentages for most of
your buffer pools. Doing so enables trickle write from the DB2 buffer pools. This means
that the data is written asynchronously to disk regularly, over time, in smaller amounts,
instead of storing up a lot of modified data that must be written all at once when the threshold
percentage is reached. The needs of every shop will vary, but some organizations have had
success by reducing DWQT as low as one.
All of the above thresholds can be changed using the -ALTER BUFFERPOOL command. Of
course, there are additional buffer pool thresholds that must be monitored, but cannot be
changed. Lets look at those three thresholds now:
Prefetch Disabled Threshold Reached when 90 percent of the buffer pool pages are
unavailable. When reached, DB2 will no longer initiate Prefetch operations, and will
cancel Prefetch operations in progress.
Data Manager Critical Threshold Reached when 95 percent of the buffer pool pages
are unavailable. When this threshold is reached, DB2 will access the pages in the virtual
buffer pool once for each row that is retrieved or updated in that page. In other words,
retrieving or updating several rows in one page causes several page access operations.
Immediate Write Threshold Reached when 97.5 percent of the buffer pool pages are
unavailable. When this threshold is reached, DB2 will write updated pages as soon as the
update is complete. The write is performed synchronously as opposed to asynchronously.
This method will potentially cause a large increase in the number of write I/Os performed
by DB2, and should be avoided completely.
Buffer Pool Monitoring

After setting up your buffer pools, you will want to regularly monitor your configuration for
performance. The most rudimentary way of doing this is by using the -DISPLAY
BUFFERPOOL command. There are many options of the DISPLAY command that can be
used to show different characteristics of your buffer pool environment; the simplest is the
summary report, requested as follows:
-DISPLAY BUFFERPOOL(BP0) LIST(*) DBNAME(DSN8*)
And a truncated version of the results will look something like this:
DSNB401I - BUFFERPOOL NAME BP0, BUFFERPOOL ID 0, USE COUNT 20
DSNB402I - VIRTUAL BUFFERPOOL SIZE = 500 BUFFERS 736
44
DBAzine.com
ALLOCATED = 500 TO BE DELETED = 0

IN-USE/UPDATED = 0
DSNB404I - THRESHOLDS - 739
VP SEQUENTIAL
= 80
HP SEQUENTIAL = 75
DEFERRED WRITE
= 85
VERTICAL DEFERRED WRT = 80,0
PARALLEL SEQUENTIAL = 50
ASSISTING PARALLEL SEQT = 0
Of course, you can request much more information to be displayed using the DISPLAY
BUFFERPOOL command by using the DETAIL parameter. Additionally, you can request
that DB2 return either incremental statistics (since the last DISPLAY) or cumulative statistics
(since DB2 was started). The statistics in a detail report are grouped in the following
categories: GETPAGE information, Sequential Prefetch information, List Prefetch
information, Dynamic Prefetch information, Page Update statistics, Page Write statistics, and
Parallel Processing Activity details.
A lot of interesting and useful details can be gathered simply using the DISPLAY
BUFFERPOOL command. For example, you can review GETPAGE requests for random and
sequential activity, number of Prefetch requests (whether static or dynamic, or for sequential
or list Prefetch), number of times each of the thresholds were tripped, and much more. Also, if
you are using DB2 V7 or earlier, you will also get hiperpool statistics. Refer to the DB2
Command Reference Manual for a definition of each of the actual statistics returned by
DISPLAY BUFFERPOOL.
Many organizations also have a performance monitor that simplifies the gathering and display
of buffer pool statistics. Such tools are highly recommended for in-depth buffer pool
monitoring and tuning. More sophisticated tools also exist that offer guidance on how to tune
your buffer pools or that automatically adjust your buffer pool parameters according to
your workload. Most monitors also provide more in-depth statistics, such as buffer pool hit
ratio calculations.
The buffer pool hit ratio is an important tool for monitoring and tuning your DB2 buffer
pools. It is calculated as follows:
Hit ratio = GETPAGES - pages_read_from_disk / GETPAGEs
Pages read from disk is a calculated field that is the sum of all random and sequential reads.
The highest possible buffer pool hit ratio is 1.0. This value is achieved when each requested
page is always in the buffer pool. When requested pages are not in the buffer pool, the hit
ratio will be lower. You can have a negative hit ratio this simply means that Prefetch was
requested to bring pages into the buffer pool that were never actually referenced.
In general, the higher the hit ratio, the better, because it indicates that pages are being
referenced from memory in the buffer pool more often. Of course, a low hit ratio is not always
bad. The larger the amount of data that must be accessed by the application, the lower the hit
ratio will tend to be. Hit ratios should be monitored in the context of the applications assigned
DBAzine.com
45
to the buffer pool and should be compared against hit ratios from prior processing periods.
Fluctuation can indicate problems.
Some Additional Rules of Thumb

Do not create lots of small buffer pools. Pools of less than 500 pages are not generally useful
and should be avoided unless you are trying to peg an entire tablespace or index into one pool
and it is small enough to fit into such a small memory structure. In general, buffer pools of
1,000 pages or more are more efficient than smaller buffer pools.
Remember, though: We do not want to lump everything into one very large buffer pool.
Instead, we want to create several buffer pools for the different types of processing as we
outlined earlier in this paper.
One final consideration is to set up separate buffer pool environments that you switch back
and forth in between online and batch processing. For example, you might want to set up a
buffer pool favoring random processing during heavy online activity and switch to a set up
favoring sequential processing during heavy batch processing. This can be accomplished
using scheduled jobs that process -ALTER BUFFERPOOL commands.
Summary
DB2 loves memory. In general, the more memory you can assign to DB2 processing, the
better performance can be. But there are no silver bullets. Everyones DB2 buffer pool set-up
will be different, based on your shops particular processing needs.
Use the tools at your disposal. DB2 provides basic commands that can be used for buffer pool
monitoring and tuning, but a performance monitor can greatly simplify your buffer pool
tuning tasks.
You should align each buffer pool to the workload that DB2 will process using that buffer
pool. And , understand the difference between sequential and random processing and learn
which application processes rely most heavily on each type; tune your buffer pools
accordingly. Separate indexes from table spaces, separate system work from application work,
separate work files from other files, and isolate your code and lookup tables when possible.
Of course, we have only covered the basics of buffer pool management in this article. For
large environments, one could make an entire career out of monitoring and tuning buffer pool
usage to improve performance. Dont stop here keep reading and learning about DB2
buffer pools. Other good resources for learning about buffering techniques include the IBM
DB2 Administration Guide manual, IDUG and IBM Technical conference presentations, DB2
books, and vendor products and Web sites.
46
DBAzine.com
The Key to Performance: Business Process

Analysis
by Dave Beulke
Introduction
We used to think that bigger databases were better. Consequently, many of us have designed
and implemented very large systems that process billions of transactions and SQL statements
daily. The thought was that the big database could contain all of the companys data and serve
multiple purposes. As a consultant, I have seen many issues with these types of large
databases and helped numerous companies fix large data warehousing, Internet and
operational database performance problems I am convinced that bigger databases are not
necessarily better.
Bigger Is NOT Better

It used to be that a ten-terabyte database was big, but not anymore. With over a terabyte of
data stored on over six PCs in my own small office, for example , it is clear that gathering
data is no big accomplishment. Data proliferation within companies often led to the
development of huge data warehouses. But many data warehouses have failed because they
didnt answer the questions that users were asking. These systems created huge, multiterabyte pools of data that couldnt perform, often because users were unable to load or
retrieve the data they contained efficiently, or because the systems did not have the right data
for the right business analysts questions at the right time.
Bigger databases are not better because all of the data I/O does not really benefit from
Moores Law of CPU price performance improvement. Sure, cheap memory can cache large
data segments, but not a ten-terabyte database. I/O continues to be the biggest performance
problem, and I/O has not really gained speed over the years like CPU speeds.
Business Process Analysis = Big Savings

Database performance plays into the bottom line of every company because it can mean big
cost savings. On a recent consulting assignment, we saved a company eight million dollars in
CPU charges, and helped them avoid a CPU upgrade and save millions of I/Os daily by tuning
their systems, application, and business processes. All of these efforts resulted in reducing
their I/O requirements and overall database size. On this same assignment, we saved the
company almost $40 million dollars by avoiding the CPU upgrade and related software
upgrade costs. Database performance that translates into a $40 million savings means the
difference between a companys ability to budget for outsourcing, implementing new
software solutions, start new projects and, most importantly, offer bonuses to the IT
department or having no money in the budget for any of this.
DBAzine.com
47
How did business process analysis save this company so much? Partnering with this
companys employees, we worked through their toughest performance issues. First, we
looked at their overall business processes and discovered that the systems were generally I/O
bound. This meant that, regardless of how much CPU we threw at the problem, the processing
would not improve. We looked at the top CPU and I/O processes that existed in their overall
system. These processes were daily, long-running batch jobs that were processing large data
flows; a few large online transactions that were executed millions of times; and some utilities.
Since the utilities showed up in the list, we decided to look more closely at each of the daily,
weekly, and monthly utilities running against their system.
Deep analysis
Long-running batch programs are notoriously symptomatic of overall performance aspects of
the processing within a company. To discover the batch processes performance issues, we
did a deep dive into the programming logic and the business processes. First, we looked at the
input files, the key data elements for the various database tables, and indexing associated with
the processing.
This analysis revealed that the programs were using different keys for different processing,
and that good indexing themes were not available to quickly and uniquely retrieve the data.
For example, we found a database table designed using an IDENTITY data type as its major
processing key. This IDENTITY data type was great for quickly retrieving data throughout its
database indexes, but the key had to be manipulated to relate it to other database systems and
older systems. Because of this additional processing to translate the IDENTITY data type into
another index key along with not being able to join tables, the application had to do multiple
SQL statements instead of SQL Joins. It was also discovered that the program was processing
across and through all the transaction information over six years of data. This database
table was clustered in such a way that the historic data was intermingled with the current data,
causing more I/O to get at the recent transaction data.
Since the input files were not able to match the IDENTITY column key, the SQL retrievals
using other keys had worse cardinality, resulting in more database pages to be referenced and
causing more I/O within the system. Also, since a cursor had to be opened for each of the
batch programs, the system required large sort work areas for cursor result sets triggering
even more I/O issues.
We were able to add additional columns and indexes to the IDENTITY column table. This
provided other indexed columns and complementary keys used outside that particular
database system; these new index keys also allowed the application to quickly focus on its
data replacing the large cursor with a direct SQL statement that did not require sorting.
This business process analysis also exposed a large number of decisions that the program was
making before doing the SQL. Based on the results of the input fields, some of these IF ELSE
conditions were unnecessary and were moved to other sections to be processed only when
required.
48
DBAzine.com
Examining online transactions

Next, we analyzed the business processes done in the online transaction. When processing for
a customer order, the customers previous orders were also made available by calling a stored
procedure. We discovered that marketing had previously wanted customer service operators
to ask customers about their last order to make sure they were completely satisfied with the
companys service.
Although this practice had been stopped after customers were surveyed for a six-month
period, someone forgot to tell the programming staff, and the code to prepare to retrieve the
previous orders was still being executed. Even though the code was efficient, executing an
extra stored procedure involved making a call to another address space and then retrieving the
additional data from the system. Also, looking at the code, we discovered that all the customer
orders were being retrieved, not just their last order. Retrieving all the customers orders
could retrieve orders up to six years old (I dont know about you, but I cant remember last
years orders, let alone those from six years ago). We eliminated calling the stored procedure
and also documented that next time, they should only retrieve the customers last order
instead of all their orders. This eliminated a minimum of 35 million I/Os from the system if a
customer had only one old order (which was rare), so the I/O savings were actually higher.
Opportunities for improvement

Looking at the utilities scheduled, we discovered many performance improvement
opportunities. Since the customer orders database had six years of data and there was no good
strategy for making room for new data, the database had to be reorganized on a regular basis.
These reorgs made the database bigger by adding free space and free pages into the database
tablespace for handling the new order data. Talking through the performance requirements,
we realized that mixing the current data with the old order data was causing the current data to
be spread across too many database pages. For example, when an order was retrieved, its line
items were spread across several database pages, forcing the system to do multiple I/Os to
retrieve all the customer order detail data.
To consolidate the orders onto a few pages, we enhanced the clustering index to include
additional columns to bring the order details closer together. This improved data clustering,
helped reduce the number of I/Os necessary to get the data, and reduced the number of reorgs
that were required. Instead of reorganizing the data every month, it would only be necessary
every six to nine months. Since reorganizations require system downtime and are very CPUintensive, this saved an enormous amount of CPU and I/O and major monthly grief for the
DBAs that had to monitor the REORG schedule.
How Much is Too Much?

During our analysis of the reorganization schedule, we discussed whether six years of data
was really necessary in the active transaction database. We heard many stories about how the
old data was needed by customer service, so we performed more analysis on the transaction
age on all the SQL retrievals. Our analysis showed that 98.7 percent of all the queries were
related to data that was three-and-a-half years old and that the database was carrying two-anda half years of data that was not really being referenced. This extra data had to be backed up
DBAzine.com
49
every day, causing additional I/Os. Daily, weekly, and monthly batch processes were required
to go through the old data to get to the new transactions. Our analysis showed that cutting
down the database from six years to four years of data (by one-third) would significantly
reduce the number of I/O done by the batch processes. This significant reduction of I/O also
saved bottom-line CPU costs and precious time in the nightly processing window.
Analyzing the data retention requirements and how the data flowed through the business
processes showed that the company had overlapping data across their operational, reporting,
and business intelligence systems. By aligning all of these platforms, departments, and their
systems, we were able to reduce the data overlap and reduce their DASD requirements
significantly across several platforms. This DASD reduction saved processing, downtime
backup time, and CPU across all programs, utilities, and reporting tools.
Conclusion
The next time you are wondering whether your company is going to start that development
effort, buy that software package, or give out bonuses, let me know. You and I might be able
to help the company accomplish these with savings from business process analysis and with
better not bigger databases.
50
DBAzine.com

DB2 Primer

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

DB2 Primer

Caricato da

Copyright:

Formati disponibili

DB2 for z/OS Performance Primer

DB2 for z/OS Primer

DB2 Buffer Pools ......................................................................................................................40

About the Authors

A DB2 for z/OS Performance Roadmap

* First published in eServer Magazine, Mainframe Edition.

Assuring optimal performance is one of a database administrators (DBAs) biggest, ongoing

Tuning the System

Tuning the Databases

Creating Indexes in the Dark

Tuning the Applications

The Bottom Line

Can Your Left Outer Joins Perform Better?

Left Outer Joins the Good and the Bad

A Simple Example of a Left Outer Join

Leveraging the ON Clause for Performance Benefits

The ON Clause Predicate Does Not Affect the Result Set

Benchmarking the Performance Improvement

Figure 1: Number of rows on Child table.

Design Considerations and Where to Use This Technique

Utilizing DB2 UDB for OS/390 and z/OS

Increasing Table Size and Concurrency with Union in View

Figure 1: Account history table.

The UNION Solution

Figure 2: Four account history tables.

Query Block Pruning

ACCOUNT_ID BETWEEN 1 AND 100000000

Predicate Transitive Closure Applied to Joins and UNION in View

Figure 3: Person account table.

Limitations to Union Everywhere

Figure 4: Account history finder table.

Join Impact on UNION in View

Figure 5: Order of the table access for each query block..

The Role of UNION in View in V8

REORG Redux: A Review of the DB2

What Is REORG For?

Data accessed sequentially

Figure 1: Data highly clustered according to batch process requirements.

Data accessed randomly

How Does REORG Work?

When Should You Run REORG?

SELECT CARD, NEARINDREF, FARINDREF

DB2 Buffer Pool Essentials

DB2 Buffer Pools

4K: BP0 thru BP49

Buffer Pool Sizing

Buffer Pool Tuning Knobs

Buffer Pool Monitoring

ALLOCATED = 500 TO BE DELETED = 0

Some Additional Rules of Thumb

The Key to Performance: Business Process

Bigger Is NOT Better

Business Process Analysis = Big Savings

Examining online transactions

Opportunities for improvement

How Much is Too Much?

Potrebbero piacerti anche