Oracle Data Pump

www.Fullinterview.com

Oracle Data Pump

Overview
Oracle Data Pump is a fully integrated new feature of Oracle Database that provides high-speed data/metadata loading (Import) and unloading (Export) process to and from the database. It is available only on Oracle Database 10g release 1 (10.1) and later. It automatically manages and schedules multiple parallel streams of loading or unloading for maximum throughput. Data Pump is a replacement for the venerable IMP and EXP applications used for creating logical backups of Oracle tables, schemas or databases. Data www.Fullinterview.com 1 of 10

Oracle Data Pump

www.Fullinterview.com

Pump is a server application. Data Pump Export and Import operations are processed in the database as a Data Pump job, which is much more efficient than the client-side execution of original Export and Import. Data Pump requires us to specify directory paths as directory objects. A directory object maps a name to a directory path on the file system. The database administrator must create a directory object and grant privileges to the user on that directory object. If a directory object is not specified, a default directory object called data_pump_dir is provided. The default data_pump_dir is available only to privileged users unless access is granted by the DBA. In Oracle Database 11g, Oracle Data Pump provides a server-side infrastructure and new high-speed, parallel Export and Import utilities for highly efficient bulk data and metadata movement between databases. Database administrators can now move data and metadata between databases faster and easier than ever before.

Architecture
Data Pump jobs use a master table, a master process, and worker processes to perform the work and keep track of the progress.

Master table:
A master table is used to track the progress within a job while the data and metadata are being transferred. The master table is implemented as a user table within the database. The specific function of the master table for export and import jobs is as follows:

For export jobs, the master table records the location of database objects within a dump file set. Export builds and maintains the master table for the duration of the job. At the end of an export job, the content of the master table is written to a file in the dump file set. For import jobs, the master table is loaded from the dump file set and is used to control the sequence of operations for locating objects that need to be imported into the target database.

The master table is created in the schema of the current user performing the export or import operation. Therefore, that user must have sufficient tablespace quota for its creation. The name of the master table is the same as the name of the job that created it. So, we cannot explicitly give a Data Pump job the same name as a pre-existing table or view. For all operations, the information in the master table is used to restart a job. www.Fullinterview.com 2 of 10

Oracle Data Pump

www.Fullinterview.com

The master table is either retained or dropped, depending on the circumstances, as follows:

Upon successful job completion, the master table is dropped. If a job is stopped using the STOP_JOB interactive command, the master table is retained for use in restarting the job. If a job is killed using the KILL_JOB interactive command, the master table is dropped and the job cannot be restarted. If a job terminates unexpectedly, the master table is retained. You can delete it if you do not intend to restart the job. If a job stops before it starts running (that is, it is in the Defining state), the master table is dropped

Master process:
For every Data Pump Export and Data Pump Import job, a master process is created. The master process controls the entire job, including communicating with the clients, creating and controlling a pool of worker processes and performing logging operations.

Worker processes:
The worker processes are the ones that actually unload and load metadata and table data in parallel. Worker processes are created as needed until the number of worker processes is equal to the value supplied for the PARALLEL command-line parameter. The number of active worker processes can be reset throughout the life of a job. When a worker process is assigned the task of loading or unloading a very large table or partition, it may choose to use the external tables access method to make maximum use of parallel execution. In such a case, the worker process becomes a parallel execution coordinator. The actual loading and unloading work is divided among some number of parallel I/O execution processes (sometimes called slaves) allocated from the instance wide pool of parallel I/O execution processes.

Data Pump Interfaces

You can interact with Data Pump by using a command line, a parameter file, or an interactive-command mode.

Command-Line Interface: Enables you to specify most of the Export/Import parameters directly on the command line. Parameter File Interface: Enables you to specify command-line parameters in a parameter file. The only exception is the PARFILE 3 of 10

www.Fullinterview.com

Oracle Data Pump

www.Fullinterview.com

parameter, because parameter files cannot be nested. The use of parameter files is recommended if you are using parameters whose values require quotation marks. Interactive-Command Interface: Stops logging to the terminal and displays the Export/Import prompt, from which you can enter various commands, some of which are specific to interactive-command mode. This mode is enabled by pressing Ctrl+C during an export/import operation started with the command-line interface or the parameter file interface. Interactive-command mode is also enabled when you attach to an executing or stopped job.

Data Pump Components

Oracle Data Pump is made up of three distinct parts:

The command-line clients, expdp and impdp The DBMS_DATAPUMP PL/SQL package (also known as the Data Pump API) The DBMS_METADATA PL/SQL package (also known as the Metadata API)

The Data Pump clients, expdp and impdp invoke the Data Pump Export utility and Data Pump Import utility, respectively. They provide a user interface that closely resembles the original export (exp) and import (imp) utilities. The expdp and impdp clients use the procedures provided in the DBMS_DATAPUMP PL/SQL package to execute export and import commands, using the parameters entered at the command-line. These parameters enable the exporting and importing of data and metadata for a complete database or subsets of a database. When data is moved, Data Pump automatically uses either direct path load (or unload) or the external tables mechanism, or a combination of both. When metadata is moved, Data Pump uses functionality provided by the DBMS_METADATA PL/SQL package. The DBMS_METADATA package provides a centralized facility for the extraction, manipulation, and resubmission of dictionary metadata. The DBMS_DATAPUMP and DBMS_METADATA PL/SQL packages can be used independently of the Data Pump clients.

Data Access Methods

Data Pump provides two access methods to load and unload data:

www.Fullinterview.com

4 of 10

Oracle Data Pump

www.Fullinterview.com

Direct Path
The Oracle database has provided direct path unload capability for export operations since Oracle release 7.3 and a direct path loader API for OCI since Oracle8i. Data Pump technology enhances direct path technology in the following ways:

Support of a direct path, proprietary format unloads. Improved performance through elimination of unnecessary conversions. This is possible because the direct path internal stream format is used as the format stored in the Data Pump dump files. Support of additional datatypes and transformations.

The default method that Data Pump uses for loading and unloading data is direct path, when the structure of a table allows it. Note that if the table has any columns of datatype LONG, then direct path must be used.

External Tables
The Oracle database has provided an external tables capability since Oracle9i that allows reading of data sources external to the database. As of Oracle Database 10g, the external tables feature also supports writing database data to destinations external to the database. Data Pump provides an external tables access driver (ORACLE_DATAPUMP) that reads and writes files. The format of the files is the same format used with the direct path method. This allows for high-speed loading and unloading of database tables. Data Pump uses external tables as the data access mechanism in the following situations: Loading and unloading very large tables and partitions in situations where parallel SQL can be used to advantage Loading tables with global or domain indexes defined on them, including partitioned object tables Loading tables with active triggers or clustered tables Loading and unloading tables with encrypted columns Loading tables with fine-grained access control enabled for inserts Loading tables that are partitioned differently at load time and unload time

New Features of Data Pump Export and Import

Parallel Threads:
The parallel parameter specifies the maximum number of threads of active execution operating on behalf of the export job. This enables you to adjust www.Fullinterview.com 5 of 10

Oracle Data Pump

www.Fullinterview.com

resource consumption versus elapsed time. This execution set consists of a combination of worker processes and parallel I/O server processes. Because each active worker processes or I/O server process works on one active file at a time, the DBA must specify a sufficient number of files. Therefore, the value the DBA specifies for this parameter should be less than or equal to the number of files in the dump file set. This important parameter helps the DBA to make a trade-off between resource consummation and the elapsed time. (PARALLEL is only available in the Enterprise Edition of Oracle Database.)
PARALLEL=integer

Restart Job:
The ability to restart a job is an extremely useful feature if the DBA is involved in moving large amounts of data. The Data Pump job can be restarted without any data loss or corruption after unexpected failure or if the DBA stopped the job with stop_job parameter.
Export> START_JOB

Ability to detach from and reattach to a job:

This allows DBAs and other operations personnel to monitor jobs from multiple locations. We can attach the Data Pump export and import utilities to one job at a time but we can have multiple clients attached to the same job.
ATTACH [=[schema_name.]job_name]

Support of export and import operations over the network:

The NETWORK_LINK parameter initiates an export and import using a database link. The system to which the expdp client is connected contacts the source database referenced by the source_database_link, retrieves data from it, and writes the data to a dump file set back on the connected system.
> expdp hr/hr DIRECTORY=dpump_dir1 NETWORK_LINK=source_database_link DUMPFILE=network_export.dmp LOGFILE=network_export.log

The impdp client initiates the import request, typically to the local database. That server contacts the remote source database referenced by source_database_link, retrieves the data, and writes it directly back to the target database. There are no dump files involved.
> impdp hr/hr TABLES=employees DIRECTORY=dpump_dir1 NETWORK_LINK=source_database_link EXCLUDE=CONSTRAINT

www.Fullinterview.com

6 of 10

Oracle Data Pump

www.Fullinterview.com

Change Source Datafile Name:

It allows you to change the name of the source datafile to the target datafile name in all SQL statements where the source datafile is referenced: CREATE TABLESPACE, CREATE LIBRARY, and CREATE DIRECTORY.
REMAP_DATAFILE=source_datafile:target_datafile

Remapping tablespaces during an import operation:

Remaps all objects selected for import with persistent data in the source tablespace to be created in the target tablespace
REMAP_TABLESPACE=source_tablespace:target_tablespace

Filtering Metadata:
Enables you to filter metadata that is exported and imported based on objects and object types using the EXCLUDE and INCLUDE options.
INCLUDE = object_type[:name_clause] [, ...] EXCLUDE=object_type[:name_clause] [, ...]

For example, suppose you have a parameter file named hr.par with the following content:
SCHEMAS=HR DUMPFILE=expinclude.dmp DIRECTORY=dpump_dir1 LOGFILE=expinclude.log INCLUDE=TABLE:"IN ('EMPLOYEES', 'DEPARTMENTS')" INCLUDE=PROCEDURE INCLUDE=INDEX:"LIKE 'EMP%'"

You could then use the hr.par file to start an export operation, without having to enter any other parameters on the command line:
> expdp hr/hr parfile=hr.par

Interactive-Command mode:
Interactive-command mode allows monitoring and interaction with ongoing jobs. In this mode the current job continues running, but logging to the terminal is suspended and the Export /Import prompt (Export>/Import>) is displayed www.Fullinterview.com 7 of 10

Oracle Data Pump

www.Fullinterview.com

To start interactive-command mode, do one of the following: From an attached client, press Ctrl+C. From a terminal other than the one on which the job is running, use the ATTACH parameter to attach to the job. This is a useful feature in situations in which you start a job at one location and need to check on it at a later time from a different location.

Estimate Only:
The ability to estimate how much space an export job would consume, without actually performing the export.
> expdp hr/hr ESTIMATE_ONLY=y NOLOGFILE=y

Version:
The ability to specify the version of database objects to be moved. In export jobs, VERSION applies to the version of the database objects to be exported. In import jobs, VERSION applies only to operations over the network. This means that VERSION applies to the version of database objects to be extracted from the source database.
VERSION={COMPATIBLE | LATEST | version_string}

Original Export/Import versus Data Pump Export/Import

Many of the concepts behind original Export and Import do not apply to Data Pump Export (expdp) and Data Pump Import (impdp).

Data Pump Export and Import operate on a group of files called a dump file set rather than on a single sequential dump file. Data Pump Export and Import access files on the server rather than on the client. This results in improved performance. It also means that directory objects are required when you specify file locations. The Data Pump Export and Import modes operate symmetrically, whereas original export and import did not always exhibit this behavior. For example, suppose you perform an export with FULL=Y, followed by an import using SCHEMAS=HR. This will produce the same results as if you performed an export with SCHEMAS=HR, followed by an import with FULL=Y.

www.Fullinterview.com

8 of 10

Oracle Data Pump

www.Fullinterview.com

Data Pump Export and Import use parallel execution rather than a single stream of execution, for improved performance. This means that the order of data within dump file sets and the information in the log files is more variable. Data Pump Export and Import represent metadata in the dump file set as XML documents rather than as DDL commands. This provides improved flexibility for transforming the metadata at import time. Data Pump Export and Import are self-tuning utilities. Tuning parameters that were used in original Export and Import, such as BUFFER and RECORDLENGTH, are neither required nor supported by Data Pump Export and Import. At import time there is no option to perform interim commits during the restoration of a partition. This was provided by the COMMIT parameter in original Import. There is no option to merge extents when you re-create tables. In original Import, this was provided by the COMPRESS parameter. Instead, extents are reallocated according to storage parameters for the target table. Sequential media, such as tapes and pipes, are not supported. The Data Pump method for moving data between different database versions is different than the method used by original Export/Import. With original Export, you had to run an older version of Export (exp) to produce a dump file that was compatible with an older database version. With Data Pump, you can use the current Export (expdp) version and simply use the VERSION parameter to specify the target database version. When you are importing data into an existing table using either APPEND or TRUNCATE, if any row violates an active constraint, the load is discontinued and no data is loaded. This is different from original Import, which logs any rows that are in violation and continues with the load. Data Pump Export and Import consume more undo tablespace than original Export and Import. This is due to additional metadata queries during export and some relatively long-running master table queries during import. As a result, for databases with large amounts of metadata, you may receive an ORA-01555: snapshot too old error. To avoid this, consider adding additional undo tablespace or increasing the value of the UNDO_RETENTION initialization parameter for the database. If a table has compression enabled, Data Pump Import attempts to compress the data being loaded. Whereas, the original Import utility loaded data in such a way that if a table had compression enabled, the data was not compressed upon import. Data Pump supports character set conversion for both direct path and external tables. Most of the restrictions that exist for character set 9 of 10

www.Fullinterview.com

Oracle Data Pump

www.Fullinterview.com

conversions in the original Import utility do not apply to Data Pump. The one case in which character set conversions are not supported under the Data Pump is when using transportable tablespaces.

Conclusion:
Oracle Data Pump is a newer, faster and more flexible feature of Oracle Database 10g that represents a major improvement on its predecessors, the export/import toolset. Data Pump provides a PL/SQL API and support for external tables. It provides flexibility, with the implementation of parameters such as INCLUDE, EXCLUDE, QUERY, and TRANSFORM that give the DBA more control of the data and objects being loaded and unloaded. Performance improvement is derived from the use of parameter parallelism. With all of these features, Data Pump is a welcome addition to DBA tools in a world that constantly redefines the size of the large database. In conclusion, based on increased functionality, relative ease of use and numerous advantages it has over its predecessors, if one is running Oracle 10g, Data Pump should be the utility of choice for data movement. If the exp/imp tools are still being used, you really are missing out!

www.Fullinterview.com

10 of 10