A file is in fixed record format when all records in a data file are the same byte length.

Although this format is the least flexible, it results in better performance than variable or stream format. Fixed format is also simple to specify. For example:

INFILE datafile_name "fix n"

This example specifies that SQL*Loader should interpret the particular data file as being in fixed record format where every record is n bytes long.

Example 7-1 shows a control file that specifies a data file (example1.dat) to be interpreted in the fixed record format. The data file in the example contains five physical records; each record has fields that contain the number and name of an employee. Each of the five records is 11 bytes long, including spaces. For the purposes of explaining this example, periods are used to represent spaces in the records, but in the actual records there would be no periods. With that in mind, the first physical record is 396,...ty,. which is exactly eleven bytes (assuming a single-byte character set). The second record is 4922,beth, followed by the newline character (\n) which is the eleventh byte, and so on. (Newline characters are not required with the fixed record format; it is simply used here to illustrate that if used, it counts as a byte in the record length.)

Note that the length is always interpreted in bytes, even if character-length semantics are in effect for the file. This is necessary because the file could contain a mix of fields, some of which are processed with character-length semantics and others which are processed with byte-length semantics. See Character-Length Semantics.

load data
infile 'example1.dat'  "fix 11"
into table example
fields terminated by ',' optionally enclosed by '"'
(col1, col2)

396,...ty,.4922,beth,\n
68773,ben,.
1,.."dave",
5455,mike,.

A file is in variable record format when the length of each record in a character field is included at the beginning of each record in the data file.

This format provides some added flexibility over the fixed record format and a performance advantage over the stream record format. For example, you can specify a data file that is to be interpreted as being in variable record format as follows:

INFILE "datafile_name" "var n"

In this example, n specifies the number of bytes in the record length field. If n is not specified, then SQL*Loader assumes a length of 5 bytes. Specifying n larger than 40 results in an error.

Example 7-2 shows a control file specification that tells SQL*Loader to look for data in the data file example2.dat and to expect variable record format where the record's first three bytes indicate the length of the field. The example2.dat data file consists of three physical records. The first is specified to be 009 (9) bytes long, the second is 010 (10) bytes long (plus a 1-byte newline), and the third is 012 (12) bytes long (plus a 1-byte newline). Note that newline characters are not required with the variable record format. This example also assumes a single-byte character set for the data file. For the purposes of this example, periods in example2.dat represent spaces; the fields do not contain actual periods.

The lengths are always interpreted in bytes, even if character-length semantics are in effect for the file. This is necessary because the file could contain a mix of fields, some processed with character-length semantics and others processed with byte-length semantics. See Character-Length Semantics.

load data
infile 'example2.dat'  "var 3"
into table example
fields terminated by ',' optionally enclosed by '"'
(col1 char(5),
 col2 char(7))

example2.dat:
009.396,.ty,0104922,beth,
012..68773,ben,

A file is in stream record format when the records are not specified by size; instead SQL*Loader forms records by scanning for the record terminator.

Stream record format is the most flexible format, but there can be a negative effect on performance. The specification of a data file to be interpreted as being in stream record format looks similar to the following:

INFILE datafile_name ["str terminator_string"]

The str indicates the file is in stream record format. The terminator_string is specified as either 'char_string' or X'hex_string' where:

• 'char_string' is a string of characters enclosed in single or double quotation marks

• X'hex_string' is a byte string in hexadecimal format

When the terminator_string contains special (nonprintable) characters, it should be specified as an X'hex_string'. However, some nonprintable characters can be specified as ('char_string') by using a backslash. For example:

• \n indicates a line feed

• \t indicates a horizontal tab

• \f indicates a form feed

• \v indicates a vertical tab

• \r indicates a carriage return

If the character set specified with the NLS_LANG initialization parameter for your session is different from the character set of the data file, then character strings are converted to the character set of the data file. This is done before SQL*Loader checks for the default record terminator.

Hexadecimal strings are assumed to be in the character set of the data file, so no conversion is performed.

On UNIX-based platforms, if no terminator_string is specified, then SQL*Loader defaults to the line feed character, \n.

On Windows-based platforms, if no terminator_string is specified, then SQL*Loader uses either \n or \r\n as the record terminator, depending on which one it finds first in the data file. This means that if you know that one or more records in your data file has \n embedded in a field, but you want \r\n to be used as the record terminator, then you must specify it.

Example 7-3 illustrates loading data in stream record format where the terminator string is specified using a character string, '|\n'. The use of the backslash character allows the character string to specify the nonprintable line feed character.

load data
infile 'example3.dat'  "str '|\n'"
into table example
fields terminated by ',' optionally enclosed by '"'
(col1 char(5),
 col2 char(7))

example3.dat:
396,ty,|
4922,beth,|

SQL*Loader organizes the input data into physical records, according to the specified record format. By default, a physical record is a logical record.

For added flexibility, SQL*Loader can be instructed to combine several physical records into a logical record.

SQL*Loader can be instructed to follow one of the following logical record-forming strategies:

• Combine a fixed number of physical records to form each logical record.

• Combine physical records into logical records while a certain condition is true.

  See Also:

  • Assembling Logical Records from Physical Records

  • Case study 4, Loading Combined Physical Records (see SQL*Loader Case Studies for information on how to access case studies)

Once a logical record is formed, field setting on the logical record is done.

Field setting is a process in which SQL*Loader uses control-file field specifications to determine which parts of logical record data correspond to which control-file fields. It is possible for two or more field specifications to claim the same data. Also, it is possible for a logical record to contain data that is not claimed by any control-file field specification.

Most control-file field specifications claim a particular part of the logical record. This mapping takes the following forms:

• The byte position of the data field's beginning, end, or both, can be specified. This specification form is not the most flexible, but it provides high field-setting performance.

• The strings delimiting (enclosing, terminating, or both) a particular data field can be specified. A delimited data field is assumed to start where the last data field ended, unless the byte position of the start of the data field is specified.

• You can specify the byte offset, the length of the data field, or both. This way each field starts a specified number of bytes from where the last one ended and continues for a specified length.

• Length-value data types can be used. In this case, the first n number of bytes of the data field contain information about how long the rest of the data field is.

  See Also:

  • Specifying the Position of a Data Field

  • Specifying Delimiters

The bad file contains records that were rejected, either by SQL*Loader or by the Oracle database.

If you do not specify a bad file and there are rejected records, then SQL*Loader automatically creates one. It will have the same name as the data file, with a .bad extension. Some of the possible reasons for rejection are discussed in the next sections.

• Records Rejected by SQL*Loader
  Data file records are rejected by SQL*Loader when the input format is invalid.
• Records Rejected by Oracle Database During a SQL*Loader Operation
  After a data file record is accepted for processing by SQL*Loader, it is sent to the database for insertion into a table as a row.

As SQL*Loader executes, it may create a file called the discard file.

This file is created only when it is needed, and only if you have specified that a discard file should be enabled. The discard file contains records that were filtered out of the load because they did not match any record-selection criteria specified in the control file.

The discard file therefore contains records that were not inserted into any table in the database. You can specify the maximum number of such records that the discard file can accept. Data written to any database table is not written to the discard file.

During conventional path loads, the input records are parsed according to the field specifications, and each data field is copied to its corresponding bind array (an area in memory where SQL*Loader stores data to be loaded).

When the bind array is full (or no more data is left to read), an array insert operation is performed.

SQL*Loader stores LOB fields after a bind array insert is done. Thus, if there are any errors in processing the LOB field (for example, the LOBFILE could not be found), then the LOB field is left empty. Note also that because LOB data is loaded after the array insert has been performed, BEFORE and AFTER row triggers may not work as expected for LOB columns. This is because the triggers fire before SQL*Loader has a chance to load the LOB contents into the column. For instance, suppose you are loading a LOB column, C1, with data and you want a BEFORE row trigger to examine the contents of this LOB column and derive a value to be loaded for some other column, C2, based on its examination. This is not possible because the LOB contents will not have been loaded at the time the trigger fires.

A direct path load parses the input records according to the field specifications, converts the input field data to the column data type, and builds a column array.

The column array is passed to a block formatter, which creates data blocks in Oracle database block format. The newly formatted database blocks are written directly to the database, bypassing much of the data processing that normally takes place. Direct path load is much faster than conventional path load, but entails several restrictions.

• Parallel Direct Path
  A parallel direct path load allows multiple direct path load sessions to concurrently load the same data segments (allows intrasegment parallelism).

External tables are defined as tables that do not reside in the database, and can be in any format for which an access driver is provided.

Oracle Database provides two access drivers: ORACLE_LOADER and ORACLE_DATAPUMP. By providing the database with metadata describing an external table, the database is able to expose the data in the external table as if it were data residing in a regular database table.

An external table load creates an external table for data that is contained in an external data file. The load executes INSERT statements to insert the data from the data file into the target table.

The advantages of using external table loads over conventional path and direct path loads are as follows:

• If a data file is big enough, then an external table load attempts to load that file in parallel.

• An external table load allows modification of the data being loaded by using SQL functions and PL/SQL functions as part of the INSERT statement that is used to create the external table.

The record parsing of external tables and SQL*Loader is very similar, so normally there is not a major performance difference for the same record format. However, due to the different architecture of external tables and SQL*Loader, there are situations in which one method may be more appropriate than the other.

Use external tables for the best load performance in the following situations:

• You want to transform the data as it is being loaded into the database

• You want to use transparent parallel processing without having to split the external data first

Use SQL*Loader for the best load performance in the following situations:

• You want to load data remotely

• Transformations are not required on the data, and the data does not need to be loaded in parallel

• You want to load data, and additional indexing of the staging table is required

This section describes important differences between loading data with external tables, using the ORACLE_LOADER access driver, as opposed to loading data with SQL*Loader conventional and direct path loads.

This information does not apply to the ORACLE_DATAPUMP access driver.

• Multiple Primary Input Data Files
  If there are multiple primary input data files with SQL*Loader loads, then a bad file and a discard file are created for each input data file.
• Syntax and Data Types
  This section provides a description of unsupported syntax and data types with external table loads.
• Byte-Order Marks
  With SQL*Loader, if a primary data file uses a Unicode character set (UTF8 or UTF16) and it also contains a byte-order mark (BOM), then the byte-order mark is written at the beginning of the corresponding bad and discard files.
• Default Character Sets, Date Masks, and Decimal Separator
  For fields in a data file, the settings of NLS environment variables on the client determine the default character set, date mask, and decimal separator.
• Use of the Backslash Escape Character
  This section describes how to use the backslash escape character.

SQL*Loader supports loading of the column and row object types.

• column objects
  When a column of a table is of some object type, the objects in that column are referred to as column objects.
• row objects
  These objects are stored in tables, known as object tables, that have columns corresponding to the attributes of the object.

SQL*Loader supports loading of nested tables and VARRAY collection types.

• Nested Tables
  A nested table is a table that appears as a column in another table.
• VARRAYs
  A VARRAY is a variable sized arrays.

A LOB is a large object type.

This release of SQL*Loader supports loading of four LOB data types:

• BLOB: a LOB containing unstructured binary data

• CLOB: a LOB containing character data

• NCLOB: a LOB containing characters in a database national character set

• BFILE: a BLOB stored outside of the database tablespaces in a server-side operating system file

LOBs can be column data types, and except for NCLOB, they can be an object's attribute data types. LOBs can have an actual value, they can be null, or they can be "empty."

This section describes case study files.

This section describes running the case studies.

1. At the system prompt, type sqlplus and press Enter to start SQL*Plus. At the user-name prompt, enter scott. At the password prompt, enter tiger.

   The SQL prompt is displayed.

2. At the SQL prompt, execute the SQL script for the case study. For example, to execute the SQL script for case study 1, enter the following:

   SQL> @ulcase1
   

   This prepares and populates tables for the case study and then returns you to the system prompt.

3. At the system prompt, start SQL*Loader and run the case study, as follows:

   sqlldr USERID=scott CONTROL=ulcase1.ctl LOG=ulcase1.log
   

   Substitute the appropriate control file name and log file name for the CONTROL and LOG parameters and press Enter. When you are prompted for a password, type tiger and then press Enter.

Log files for the case studies are not provided in the $ORACLE_HOME/rdbms/demo directory.

This is because the log file for each case study is produced when you execute the case study, provided that you use the LOG parameter. If you do not want to produce a log file, then omit the LOG parameter from the command line.

To check the results of running a case study, start SQL*Plus and perform a select operation from the table that was loaded in the case study.

1. At the system prompt, type sqlplus and press Enter to start SQL*Plus. At the user-name prompt, enter scott. At the password prompt, enter tiger.

   The SQL prompt is displayed.

2. At the SQL prompt, use the SELECT statement to select all rows from the table that the case study loaded. For example, if the table emp was loaded, then enter:

   SQL> SELECT * FROM emp;
   

   The contents of each row in the emp table will be displayed.