File Organisation

The database is stored as a collection of les. Each le is a sequence of records. A record is a sequence of elds. First approach:

assume record size is xed each le has records of one particular type only different les are used for different relations

This case is the easiest to implement.

26
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Fixed-Length Records
Simple approach: Store record i starting from byte n ! (i 1), where n is the size of each record. Calculate the block based on that. Record access is simple. But records may cross blocks!

Modication: do not allow records to cross block boundaries

Deletion of record i: alternatives: move records i + 1, . . ., n to i, . . . , n 1 move record n to i do not move records, but mark deleted records and link all free records on a free list
27

Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Free Lists
Store the address of the rst deleted record in the le header. Use this rst record to store the address of the second deleted record, and so on One can think of these stored addresses as pointers since they point to the location of a record. More space efcient representation: reuse space for normal attributes of free records to store pointers. (No pointers stored in in-use records.)

28
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Variable-Length Records
Variable-length records arise in database systems in several ways: Storage of multiple record types in a le. Record types that allow variable lengths for one or more elds. E.g varchars

Store several records, with variable length, in blocks (slotted page) A record cannot be bigger than a slotted page This limits the size of records in a database, which is usually the (default) case There are special types for big records, that are treated differently (remember the clobs and blobs in Oracle?)

29
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Variable-Length Records: Slotted Page Structure

Slotted pages are usually the size of a block Header contains: number of record entries end of free space in the block location and size of each record

Records can be moved around within a page to keep them contiguous with no empty space between them; entry in the header must be updated. (Other) pointers should not point directly to record instead they should point to the entry for the record in header.
30

Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Organisation of Records in Files

Heap a record can be placed anywhere in the le where there is space Sequential store records in sequential order, based on the value of the search key of each record Hashing a hash function computed on some attribute of each record; the result species in which block of the le the record should be placed Records of each relation may be stored in a separate le. In a multitable clustering le organisation records of several different relations can be stored in the same le Motivation: store related records on the same block to minimise I/O

The choice of a proper organisation of records in a le is important for the efciency of real databases!
31

Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Sequential File Organisation

Suitable for applications that require sequential processing of the entire le The records in the le are ordered by a search-key

32
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Sequential File Organization (Cont.)

Deletion use pointer chains Insertion locate the position where the record is to be inserted if there is free space insert there if no free space, insert the record in an overow block In either case, pointer chain must be updated

Need to reorganise the le from time to time to restore sequential order

33
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Multitable Clustering File Organisation

Store several relations in one le using a multitable clustering le organisation.

34
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Multitable Clustering File Organization (cont.)

Multitable clustering organisation of customer and depositor:

good for queries involving depositor customer, and for queries involving one single customer and her accounts bad for queries involving only customer but one can add pointer chains to link records of a particular relation results in variable size records
35

Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

File System
In sequential le organisation, each relation is stored in a le One may rely in the le system of the underlying operating system Multitable clustering may have signicant gains in efciency But this may not be compatible with the le system of the operating system

Several large scale database management systems do not rely directly on the underlying operating system The relations are all stored in a single (multitable) le The database management system manages the le by itself This requires the implementation of an own le system inside the DBMS

36
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Data Dictionary Storage

Data dictionary (also called system catalogue) stores metadata; that is, data about data, such as

Information about relations names of relations names and types of attributes of each relation names and denitions of views

integrity constraints User and accounting information, including passwords Statistical and descriptive data number of tuples in each relation Physical le organisation information How relation is stored (sequential/hash/) Physical location of relation Information about indices (in the sequel)
37

Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Data Dictionary Storage (Cont.)

Catalog structure Relational representation on disk specialised data structures designed for efcient access, in memory

A possible catalog representation:

Relation_metadata = (relation_name, number_of_attributes, storage_organization, location) Attribute_metadata = (attribute_name, relation_name, domain_type, ! position, length) User_metadata = (user_name, encrypted_password, group) Index_metadata = (index_name, relation_name, index_type, ! index_attributes) View_metadata = (view_name, denition)

38
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

File Organisation in Oracle

Oracle has its own buffer management, with complex policies Oracle doesn!t rely on the underlying operating system!s le system A database in Oracle consists of tablespaces: System tablespace: contains catalog meta-data User data tablespaces The space in a tablespace is divided into segments: Data segment Index segment Temporary segment (for sort operations) Rollback segment (for processing transactions)

Segments are divided into extents, each extent being a set of contiguous database blocks. A database block need not be the same size of an operating system block, but is always a multiple

39
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

File Organisation in Oracle (cont.)

A standard table is organised in a heap (no sequence is imposed) Partitioning of tables is possible for optimisation Range partitioning (e.g by dates) Hash partitioning Composite partitioning Table data in Oracle can also be (multitable) clustered One may tune the clusters to signicantly improve the efciency of query to frequently used joins.

Hash le organisation (to be studied next week) is also possible for fetching the appropriate cluster A database can be tuned by an appropriate choice for the organisation of data: Choosing partitions Appropriate choice of clusters Hash or sequential

Tuning makes the difference in big (real) databases!

40

Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Chapter 12: Indexing and Hashing

Jos Alferes Sistemas de Bases de Dados 2012/13

41
Monday, March 4, 2013

Chapter 12: Indexing and Hashing

Basic Concepts Ordered Indices B+-Tree Index Files B-Tree Index Files Multiple-Key Access and Bitmap indices Index Denition in SQL Indexing in Oracle Hashing Static Hashing Dynamic Hashing Comparison of Ordered Indexing and Hashing

42
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Basic Concepts
Indexing mechanisms are used to speed up access to desired data. Search Key - attribute or set of attributes used to look up records in a le. An index le consists of records (called index entries) of the form search-key pointer

Index les are typically much smaller than the original le Two basic kinds of indices: Ordered indices: search keys are stored in sorted order Hash indices: search keys are distributed uniformly across buckets using a hash function.

43
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Index Evaluation Metrics

Access time Insertion time Deletion time Space overhead Access types supported efciently. E.g., records with a specied value in the attribute or records with an attribute value falling in a specied range of values. This strongly inuences the choice of index, and depends on usage.

44
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Ordered Indices
In an ordered index, index entries are stored sorted on the search key value. E.g. as an authors catalog in a library. Primary index: in a sequentially ordered le, the index whose search key species the sequential order of the le. Also called clustering index The search key of a primary index is usually but not necessarily the primary key.

Secondary index: an index whose search key species an order different from the sequential order of the le. Also called non-clustering index. Index-sequential le: ordered sequential le with a primary index.

45
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Dense Index Files

Dense index Index record appears for every search-key value in the le.

46
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Sparse Index Files

Sparse Index: contains index records for only some search-key values. Only applicable (in primary index) when records are sequentially ordered on search-key To locate a record with search-key value K we: Find index record with largest search-key value < K Search le sequentially starting at the record to which the index record points

47
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Multilevel Index
If the primary index does not t in memory, access becomes expensive. Solution: treat primary index kept on disk as a sequential le and construct a sparse index on it. outer index a sparse index of primary index inner index the primary index le

If the outer index is still too large to t in main memory, yet another level of index can be created, and so on. Indices at all levels must be updated on insertion or deletion from the le.

48
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Multilevel Index (Cont.)

49
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Index Update: Deletion

If deleted record was the only record in the le with its particular searchkey value, the search-key is deleted from the index also. Single-level index deletion: Dense indices deletion of search-key: similar to le record deletion. Sparse indices

if an entry for the search key exists in the index, it is deleted by replacing the entry in the index with the next search-key value in the le (in search-key order). If the next search-key value already has an index entry, the entry is deleted instead of being replaced.

50
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Index Update: Insertion

Single-level index insertion: Perform a lookup using the search-key value appearing in the record to be inserted. Dense indices if the search-key value does not appear in the index, insert it. Sparse indices if index stores an entry for each block of the le, no change needs to be made to the index unless a new block is created.

If a new block is created, the rst search-key value appearing in the new block is inserted into the index.

Multilevel insertion (as well as deletion) algorithms are simple extensions of the single-level algorithms the outer indices are sparse, whereas the inner one is dense

51
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Secondary Indices
Frequently, one wants to nd all the records whose values in a certain eld (which is not the search-key of the primary index) satisfy some condition. Example 1: In the account relation stored sequentially by account number, we may want to nd all accounts in a particular branch Example 2: as above, but where we want to nd all accounts with a specied balance or range of balances

We can have a secondary index with an index record for each search-key value

52
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Secondary Indices Example

Secondary index on balance eld of account

Index record points to a bucket that contains pointers to all the actual records with that particular search-key value. Secondary indices have to be dense, since the le is not sorted by the search key.
53

Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013

Primary and Secondary Indices

Indices offer substantial benets when searching for records, but updating indices imposes overhead on database modication - when a le is modied, every index on the le must be updated, Sequential scan using primary index is efcient, but a sequential scan using a secondary index is expensive Each record access may fetch a new block from disk Block fetch requires about 5 to 10 micro seconds, versus about 100 nanoseconds for memory access

54
Jos Alferes - Adaptado de Database System Concepts - 5th Edition

Monday, March 4, 2013