SQL*LOADER

Prepared By: Sidharth Date:27/04/2011

SQL*Loader (sqlldr) is the utility to use for high performance data loads. The data can be loaded from any text file and inserted into the database.
It is a high speed data loading utility supplied by Oracle that loads data from external files into table in an Oracle Database. We can transfer data from flat file to database i.e. from direct memory to conventional memory.

Note: Direct data are stored in OS memory where as conventional data are stored in Database In flat file data are separated by the help of some special symbol like comma, white space etc SQL Loader is an utility which accepts data in variety of formats, can perform transformation, filtering and can load into multiple tables from multiple files in same loading session

Files used in Sql *Loader

Control File - .ctl file Data File - .dat file Log File - .log file Bad File - .bad file Discard File - .dis file

Requirements- For Sql *Loader

Tables to be loaded must already exists in the Database. SQL *Loader never creates tables, it loads existing tables. Table may be empty or may already contain data. Privileges: INSERT privilege DELETE privilege, when using REPLACE or TRUNCATE insert option

Note: Control File - Whatever code is required we have to mention it in control file and its extension should be.ctl. It contains Source of Data, Destination of Data, Filtering of Data, Transformation of Data and also Data to be Load. Its compulsory. Whatever data you want to load into database we can mention it in either Data file or Control file. Bad File Contains the data which are not loaded due to some errors, it is not optional, if any one error occurred, SQL *Loader will create the bad file and write the offending input records into it. Discard File Contains the data that are discarded by when clause in control file, it is optional. Data File Contains the data to be loaded, it is optional Log File It is a record of SQL *Loaders activities during a load session It contains Control, Data, Bad, Discard file Name. Detailed breakdown of the fields & data types in data file that was loaded. Error Message for records that cause error. Message indicating when records have been discarded. Summary of the Load (no: of records read from file, no: of rows rejected because of errors, no: of rows discarded & elapsed time of load

Steps to Loading Of Data Step 1: Go to any drive and create a folder. Step 2:(Creation of Control File) Open Run Dialog box and type command. Provide the path up to the folder u have created against the command prompt. Open a notepad. Step 3: (Code for Data load written in Control file) Code: Load Data Infile d:\demo\mydata.csv Into table emp Fields terminated by , Optionally enclosed by (empno,ename,sal,deptno) Description: Infile: Here we have to give the complete path of the Data file Into table: Here we have to give the table in which u want to load data. Fields terminated by: separation symbol of data in flat file. Optionally enclosed by: It contains the special symbol that is used data to make it separate except that separation symbol. (): next part is the in between parenthesis in between the parenthesis we have to provide the column name in the order in which data are provided in Data file. Note: Now save the notepad as .ctl extension. Step 3: (Creation of Data File) Open a notepad in the path provided in Infile of control file. Write the data in Data file like below 1001,Sidharth,1000,40 1003,Sankar,1000,30 Step 4: (use the SQL*Loader utility) >sqlldr <username>/<pwd> control=<control file name>[ filename>] [bad=<badfilename>][discard=<discard file name>] log=<log

Data provided in Control File itself: LOAD DATA INFILE * REPLACE INTO TABLE EMP (DEPTNO POSITION(01:04) CHAR(4), DEPTNAME POSITION(07:26) CHAR(4)) BEGINDATA COSC Computer Science ENGL English Litreture MATH Mathematics Note: Use of Replace key word: If we r not using Replace key word then sqlldr dont able to load the data in the database table if there r some data present before. If we are using Replace keyword then sqlldr replace the data present before in the Database table. If we use the key word Append the sqlldr will append the new Data with the existing data in the Database Table. POSITION(01:04): If you dont want to give field terminated by <symbol> then we have to use position(sp:ep) with corresponding column name. Position(sp:ep): That means it directs the sqlldr to take the data from starting from sp position(01 to) to ep position(upto 04) e.g. Computer Science(Position(01:04)= Comp)

Infile *: If data are provided in control file itself then no need to give path in infile so in place of path we have to give *.

We can provide control file, username, password, Bad file name and Discard file name etc can be provided in a parameter file so that it is not required this information at the use of sqlldr e.g.

1) make text file save it with .par extension and inside the text file write like below

Userid=<username>/<pwd> control=<control file name> log=<log filename> bad=<badfilename> discard=<discard file name>

2) Open the command prompt and go to the path where the parameter file is created. Then sqlldr utility against t he prompt Sqlldr parfile=<parfilename> e.g. LOAD DATA INFILE * REPLACE INTO TABLE EMP FIELD ENCLOSED BY ; OPTIONALLY ENCLOSED BY (DEPTNO, DEPTNAME) BEGINDATA COSC; Computer Science ENGL; English Literature MATH; Mathematics

Adding more than one datafile into a table which contains some exising data LOAD DATA INFILE file1.dat INFILE file2.dat INFILE file3.dat REPLACE INTO TABLE EMP FIELD ENCLOSED BY , OPTIONALLY ENCLOSED BY (DEPTNO, DEPTNAME) File1.dat COSC, Computer Science ENGL, English Literature File2.dat MATH, Mathematics

Loading the Selected data LOAD DATA INFILE data.dat REPLACE INTO TABLE EMP when n2 = '222' ( n1 position(1:3), n2 position(5:7) ) 111 222 333 444 111 222 333 444 111 222 data.dat

e.g . of selective data load Look at this example, (01) is the first character, (30:37) are characters 30 to 37: LOAD DATA INFILE 'mydata.dat' BADFILE 'mydata.bad' DISCARDFILE 'mydata.dis' APPEND INTO TABLE my_selective_table WHEN (01) <> 'H' and (01) <> 'T' and (30:37) = '20031217' ( region CONSTANT '31', service_key POSITION(01:11) INTEGER EXTERNAL, call_b_no POSITION(12:29) CHAR ) NOTE: SQL*Loader does not allow the use of OR in the WHEN clause. You can only use AND as in the example above! To workaround this problem, code multiple "INTO TABLE ... WHEN" clauses. Here is an example: LOAD DATA INFILE 'mydata.dat' BADFILE 'mydata.bad' DISCARDFILE 'mydata.dis' APPEND INTO TABLE my_selective_table WHEN (01) <> 'H' and (01) <> 'T' ( region CONSTANT '31', service_key POSITION(01:11) INTEGER EXTERNAL, call_b_no POSITION(12:29) CHAR ) INTO TABLE my_selective_table WHEN (30:37) = '20031217' ( region CONSTANT '31', service_key POSITION(01:11) INTEGER EXTERNAL, call_b_no POSITION(12:29) CHAR )

Loading The Data from Data file as well as control file LOAD DATA INFILE * INFILE data.dat REPLACE INTO TABLE EMP

//IT MUST BE THE FIRST

when n2 = '222' ( n1 position(1:3), n2 position(5:7) ) BEGINDATA 333 444 111 222 Loading into Multiple Tables(CHECK THIS EXAMPLE) load data infile * REPLACE into table sql_ldr101 (n1 position(1:2), n2 position(4:5)) REPLACE into table sql_ldr102 (n1 position(1:2), n2 position(4:5)) begindata 11 33 22 44 33 55 Originally load data infile 'data10.dat' insert into table sql_ldr101 replace (n1 position(1:2), n2 position(4:5)) into table sql_ldr102 replace (n1 position(1:2), n2 position(4:5)) data10.dat 11 22 33 44 55 66

Note: Try to do in one table replace and in another append and try an example where from multiple datafile data load in multiple table Skipping records control2.ctl load data infile 'data2.dat' insert into table sql_ldr2 replace ( n1 position(1:3), n2 position(5:7), n3 position(9:11) ) Result data2.dat 111 222 666 333 444 666 555 666 666 777 888 666 999 000 666 N1 N2 N3 555 666 666 777 888 666 999 000 666

sqlldr control=control2.ctl userid=apps/secretone skip=2 Using LOAD option

sqlldr control=control14.ctl userid=apps/secretone LOAD = 2 Consider above example Result N1 N2 N3 111 222 666 333 444 666 Stream Record Format Control2.ctl load data infile data2.dat str | into table sql_ldr2 fields terminated by , ( a1 char, a2 char ) data2.dat aaa,bbb|ccc,dddd|eeee,ffff|

A1 aaa ccc eeee

A2 bbb dddd ffff

load data infile "test.dat" "str '|\n'" into test_table fields terminated by ';' TRAILING NULLCOLS ( desc, txt ) test.dat: one line;hello dear world;| two lines;Dear world, hello!;|

How does one load MS-Excel data into Oracle?

Open the MS-Excel spreadsheet and save it as a CSV (Comma Separated Values) file. This file can now be copied to the Oracle machine and loaded using the SQL*Loader utility. Possible problems and workarounds: The spreadsheet may contain cells with newline characters (ALT+ENTER). SQL*Loader expects the entire record to be on a single line. Run the following macro to remove newline characters (Tools -> Macro -> Visual Basic Editor): ' Removing tabs and carriage returns from worksheet cells Sub CleanUp() Dim TheCell As Range On Error Resume Next For Each TheCell In ActiveSheet.UsedRange With TheCell If .HasFormula = False Then .Value = Application.WorksheetFunction.Clean(.Value) End If End With Next TheCell End Sub

Is there a SQL*Unloader to download data to a flat file?

Oracle does not supply any data unload utilities. Here are some workarounds: Using SQL*Plus You can use SQL*Plus to select and format your data and then spool it to a file. This example spools out a CSV (comma separated values) file that can be imported into MS-Excel: set echo off newpage 0 space 0 pagesize 0 feed off head off trimspool on spool oradata.txt select col1 || ',' || col2 || ',' || col3 from tab1 where col2 = 'XYZ'; spool off

You can also use the "set colsep" command if you don't want to put the commas in by hand. This saves a lot of typing. Example:
set colsep ',' set echo off newpage 0 space 0 pagesize 0 feed off head off trimspool on spool oradata.txt select col1, col2, col3 from tab1 where col2 = 'XYZ'; spool off

Using PL/SQL
PL/SQL's UTL_FILE package can also be used to unload data. Example: declare fp utl_file.file_type; begin fp := utl_file.fopen('c:\oradata','tab1.txt','w'); utl_file.putf(fp, '%s, %sn', 'TextField', 55); utl_file.fclose(fp); end; /

Can one modify data as the database gets loaded?

Data can be modified as it loads into the Oracle Database. One can also populate columns with static or derived values. However, this only applies for the conventional load path (and not for direct path loads). Here are some examples: LOAD DATA INFILE * INTO TABLE modified_data ( rec_no "my_db_sequence.nextval", region CONSTANT '31', time_loaded "to_char(SYSDATE, 'HH24:MI')", data1 POSITION(1:5) ":data1/100", data2 POSITION(6:15) "upper(:data2)", data3 POSITION(16:22)"to_date(:data3, 'YYMMDD')" ) BEGINDATA 11111AAAAAAAAAA991201 22222BBBBBBBBBB990112 LOAD DATA

INFILE 'mail_orders.txt' BADFILE 'bad_orders.txt' APPEND INTO TABLE mailing_list FIELDS TERMINATED BY "," ( addr, city, state, zipcode, mailing_addr "decode(:mailing_addr, null, :addr, :mailing_addr)", mailing_city "decode(:mailing_city, null, :city, :mailing_city)", mailing_state, move_date "substr(:move_date, 3, 2) || substr(:move_date, 7, 2)" )

Can one skip certain columns while loading data?

One cannot use POSITION(x:y) with delimited data. Luckily, from Oracle 8i one can specify

FILLER columns. FILLER columns are used to skip columns/fields in the load file, ignoring fields
that one does not want. Look at this example: LOAD DATA TRUNCATE INTO TABLE T1 FIELDS TERMINATED BY ',' ( field1, field2 FILLER, field3 )

BOUNDFILLER (available with Oracle 9i and above) can be used if the skipped column's value
will be required later again. Here is an example: LOAD DATA INFILE * TRUNCATE INTO TABLE sometable FIELDS TERMINATED BY "," trailing nullcols ( c1, field2 BOUNDFILLER, field3 BOUNDFILLER, field4 BOUNDFILLER, field5 BOUNDFILLER, c2 ":field2 || :field3", c3 ":field4 + :field5"