This chapter includes the following sections:
The following properties set the log files and the type of logging.
Logging is controlled by the following properties.
Specifies the type of logging that is to be used. The default implementation for the Oracle GoldenGate Adapters is the JDK option. This is the built-in Java logging called java.util.logging (JUL). The other logging options are log4j or logback.
For example, to set the type of logging to log4j:
gg.log=log4j
The log file is created in the report subdirectory of the installation. The default log file name includes the group name of the associated Extract and the file extension is .log.
Specifies the overall log level for all modules. The syntax is:
gg.log.level={ERROR|WARN|INFO|DEBUG}
The log levels are defined as follows:
ERROR – Only write messages if errors occur
WARN – Write error and warning messages
INFO – Write error, warning and informational messages
DEBUG – Write all messages, including debug ones.
The default logging level is INFO. The messages in this case will be produced on startup, shutdown and periodically during operation. If the level is switched to DEBUG, large volumes of messages may occur which could impact performance. For example, the following sets the global logging level to INFO:
# global logging level gg.log.level=INFO
Specifies the path to the log file. The syntax is:
gg.log.file=path_to_file
Where the path_to_file is the fully defined location of the log file. This allows a change to the name of the log, but you must include the Extract name if you have more than one Extract to avoid one overwriting the log of the other.
The following properties apply to all writer type user exits and are not specific to the user exit.
Specifies the name of the writer. This is always jvm and should not be modified.
For example:
goldengate.userexit.writers=jvm
All other properties in the file should be prefixed by the writer name, jvm.
Specifies a string value for the prefix added to the checkpoint file name. For example:
goldengate.userexit.chkptprefix=javaue_
Disables or enables the user exit checkpoint file. The default is false, the checkpoint file is enabled. Set this property to true if transactions are supported and enabled on the target.
For exampe, if JMS is the target and JMS local transactions are enabled (the default), set goldengate.userexit.nochkpt=true to disable the user exit checkpoint file. If JMS transactions are disabled by setting localTx=false on the handler, the user exit checkpoint file should be enabled by setting goldengate.userexit.nochkpt=false.
goldengate.userexit.nochkpt={true|false}
The following options configure the Java Runtime Environment. In particular, this is where the maximum memory the JVM can use is specified; if you see Java out-of-memory errors, edit these settings.
Specifies the classpath and boot options that will be applied when the user exit starts up the JVM. The path needs colon (:) separators for UNIX/Linux and semicolons (;) for Windows. This is where to specify various options for the JVM, such as heap size and class path; for example:
-Xms: initial java heap size
-Xmx: maximum java heap size
-Djava.class.path: class path specifying location of at least the main application jar, ggjava.jar. Other jars, such as JMS provider jars, may also be specified here as well; alternatively, these may be specified in the Java application properties file.
-verbose:jni: run in verbose mode (for JNI)
For example (all on a single line):
jvm.bootoptions= -Djava.class.path=ggjava/ggjava.jar -Dlog4j.configuration=my-log4j.properties -Xmx512m
The log4j.configuration property can be a fully qualified URL to a log4j properties file; by default this file is searched for in the class path. You may use your own log4j configuration, or one of the preconfigured log4j settings: log4j.properties (default level of logging), debug-log4j.properties (debug logging) or trace-log4j.properties (very verbose logging).
The use of the user exit causes Extract to assume that the records handled by the exit are ignored. This causes the standard Oracle GoldenGate reporting to be incomplete. Oracle GoldenGate for Java adds its own reporting to handle this issue.
Statistics can be reported every t seconds or every n records - or if both are specified, whichever criteria is met first.
There are two sets of statistics recorded: those maintained by the User Exit shared library (on the C side) and those obtained from the Java libraries. The reports received from the Java side are formatted and returned by the individual handlers.
The User Exit statistics include the total number of operations, transactions and corresponding rates.
Controls the output of statistics to the Oracle GoldenGate report file and to the user exit log files.
The following example outputs these statistics.
jvm.stats.display=true
Controls the output of statistics from the Java side, in addition to the statistics from the C side.
Java side statistics are more detailed but also involve some additional overhead, so if statistics are reported often and a less detailed summary is adequate, it is recommended that stats.full property is set to false.
The following example will output Java statistics in addition to C.
jvm.stats.full=true
Specifies a time interval, in seconds or a number of records, after which statistics will be reported. The default is to report statistics every hour or every 10000 records (which ever occurs first).
For example, to report ever 10 minutes or every 1000 records, specify:
jvm.stats.time=600 jvm.stats.numrecs=1000
The Java application statistics are handler-dependent:
For the all handlers, there is at least the total elapsed time, processing time, number of operations, transactions;
For the JMS handler, there is additionally the total number of bytes received and sent.
The report can be customized using a template.
The following defines the properties which may be set in the Java application property file.
The following properties apply to all handlers.
The handler list is a list of active handlers separated by commas. These values are used in the rest of the property file to configure the individual handlers. For example:
gg.handlerlist=name1, name2 gg.handler.name1.propertyA=value1 gg.handler.name1.propertyB=value2 gg.handler.name1.propertyC=value3 gg.handler.name2.propertyA=value1 gg.handler.name2.propertyB=value2 gg.handler.name2.propertyC=value3
Using the handlerlist property, you may include completely configured handlers in the property file and just disable them by removing them from the handlerlist.
This type of handler. This is either a predefined value for built-in handlers, or a fully qualified Java class name. The syntax is:
gg.handler.name.type={jms|jms_map|aq|singlefile|rollingfile|custom_java_class}
Where:
All but the last are pre-defined handlers:
jms – Sends transactions, operations, and metadata as formatted messages to a JMS provider
aq – Sends transactions, operations, and metadata as formatted messages to Oracle Advanced Queuing (AQ)
jms_map – Sends JMS map messages
singlefile – Writes to a single file on disk, but does not roll the file
rollingfile – Writes transactions, operations, and metadata to a file on disk, rolling the file over after a certain size or after a certain amount of time
custom_java_class – Any class that extends the Oracle GoldenGate for Java AbstractHandler class and can handle transaction, operation, or metadata events
The following properties apply to all handlers capable of producing formatted output; this includes:
The jms_text handler (but not the jms_map handler)
The aq handler
The singlefile and rolling handlers, for writing formatted output to files
Specifies the format used to transform operations and transactions into messages sent to JMS, to the Big Data target or to a file. The format is specified uniquely for each handler. The value may be:
Velocity template
Java class name (fully qualified - the class specified must be a type of formatter)
csv for delimited values (such as comma separated values; the delimiter can be customized)
fixed for fixed-length fields
Built-in formatter, such as:
xml – demo XML format
xml2 – internal XML format
For example, to specify a custom Java class:
gg.handlerlist=abc gg.handler.abc.format=com.mycompany.MyFormat
Or, for a Velocity template:
gg.handlerlist=xyz gg.handler.xyz.format=path/to/sample.vm
If using templates, the file is found relative to some directory or JAR that is in the classpath. By default, the Oracle GoldenGate installation directory is in the classpath, so the preceding template could be placed in the dirprm directory of the Oracle GoldenGate installation location.
The default format is to use the built-in XML formatter.
Specifies a list of tables this handler will include. If the schema (or owner) of the table is specified, then only that schema matches the table name; otherwise, the table name matches any schema. A comma separated list of tables can be specified.
For example, to have the handler only process tables foo.customer and bar.orders:
gg.handler.myhandler.includeTables=foo.customer, bar.orders
Note:
In order to selectively process operations on a table by table basis, the handler must be processing in operation mode. If the handler is processing in transaction mode, then when a single transaction contains several operations spanning several tables, if any table matches the include list of tables, the transaction will be included.
Specifies a list of tables this handler will exclude. If the schema (or owner) of the table is specified, then only that schema matches the table name; otherwise, the table name matches any schema. A list of tables may be specified, comma-separated. For example, to have the handler process all operations on all tables except table date_modified in all schemas:
gg.handler.myhandler.excludeTables=date_modified
Specifies whether to output one operation per message (op) or one transaction per message (tx). The default is op. Use gg.handler.name.format.mode when you have a custom formatter.
Note:
This property must be set to one transaction per message (tx) if you are using group transaction properties. If it is set to one operation per message (op), gg.handler.name.minGroupSize and gg.handler.name.maxGroupSize will be ignored
If the handler is set to use either CSV or fixed format output, the following properties may also be set.
Specifies the delimiter to use between fields. Set this to no value to have no delimiter used. For example:
gg.handler.handler1.format.delim=,
Specifies the quote character to be used if column values are quoted. For example:
gg.handler.handler1.format.quote='
Specifies the metadata column values to appear at the beginning of the record, before any column data. Specify any of the following, in the order they should appear:
position – unique position indicator of records in a trail
opcode – I, U, or D for insert, update, or delete records (see: insertChar, updateChar, deleteChar)
txind – transaction indicator – such as 0=begin, 1=middle, 2=end, 3=whole tx (see beginTxChar, middleTxChar, endTxChar, wholeTxChar)
opcount – position of a record in a transaction, starting from 0
schema – schema/owner of the table for the record
tableonly – just table (no schema/owner)
table – full name of table, schema.table
timestamp – commit timestamp of record
For example:
gg.handler.handler1.format.metacols=opcode, table, txind, position
Specifies a special column prefix for a column value that was not captured from the source database transaction log. The column value is not in trail and it is unknown if it has a value or is NULL
The character used to represent the missing state of the column value can be customized. For example:
gg.handler.handler1.format.missingColumnChar=M
By default, the missing column value is set to an empty string and does not show.
Specifies a special column prefix for a column value that exists in the trail and is not NULL.
The character used to represent the state of the column can be customized. For example:
gg.handler.handler1.format.presentColumnChar=P
By default, the present column value is set to an empty string and does not show.
Specifies a special column prefix for a column value that exists in the trail and is set to NULL.
The character used to represent the state of the column can be customized. For example:
gg.handler.handler1.format.nullColumnChar=N
By default, the null column value is set to an empty string and does not show.
Specifies the header metadata character (see metacols) used to identify a record as the begin of a transaction. For example:
gg.handler.handler1.format.beginTxChar=B
Specifies the header metadata characters (see metacols) used to identify a record as the middle of a transaction. For example:
gg.handler.handler1.format.middleTxChar=M
Specifies the header metadata characters (see metacols) used to identify a record as the end of a transaction. For example:
gg.handler.handler1.format.endTxChar=E
Specifies the header metadata characters (see metacols) used to identify a record as a complete transaction; referred to as a whole transaction. For example:
gg.handler.handler1.format.wholeTxChar=W
Specifies the character to identify an insert operation. The default I.
For example, to use INS instead of I for insert operations:
gg.handler.handler1.format.insertChar=INS
Specifies the character to identify an update operation. The default is U.
For example, to use UPD instead of U for update operations:
gg.handler.handler1.format.updateChar=UPD
Specifies the character to identify a delete operation. The default is D.
For example, to use DEL instead of D for delete operations:
gg.handler.handler1.format.deleteChar=DEL
Specifies the end-of-line character as:
EOL - Native platform
CR - Neutral (UNIX-style \n)
CRLF - Windows (\r\n)
For example:
gg.handler.handler1.format.endOfLine=CR
Specifies the left or right justification of fixed fields. For example:
gg.handler.handler1.format.justify=left
The following properties only apply to handlers that write their output to files: the file handler and the singlefile handler.
Specifies the name of the output file for the given handler. If the handler is a rolling file, this name is used to derive the rolled file names. The default file name is output.xml.
Controls whether the file should be appended to (true) or overwritten upon restart (false).
If using the file handler, this specifies the size of the file before a rollover should be attempted. The file size will be at least this size, but will most likely be larger. Operations and transactions are not broken across files. The size is specified in bytes, but a suffix may be given to identify MB or KB. For example:
gg.handler.myfile.rolloverSize=5MB
The default rollover size is 10 MB.
The following properties apply to the JMS handlers. Some of these values may be defined in the Java application properties file using the name of the handler. Other properties may be placed into a separate JMS properties file, which is useful if using more than one JMS handler at a time. For example:
gg.handler.myjms.type=jms_text gg.handler.myjms.format=xml gg.handler.myjms.properties=weblogic.properties
Just as with Velocity templates and formatting property files, this additional JMS properties file is found in the classpath. The above properties file weblogic.properties would be found in {gg_install_dir}/dirprm/weblogic.properties, since the dirprm directory is included by default in the class path.
Settings that can be made in the Java application properties file will override the corresponding value set in the supplemental JMS properties file (weblogic.properties in the example above). In the following example, the destination property is specified in the Java application properties file. This allows the same default connection information for the two handlers myjms1 and myjms2, but customizes the target destination queue.
gg.handlerlist=myjms1,myjms2 gg.handler.myjms1.type=jms_text gg.handler.myjms1.destination=queue.sampleA gg.handler.myjms1.format=sample.vm gg.handler.myjms1.properties=tibco-default.properties gg.handler.myjms2.type=jms_map gg.handler.myjms2.destination=queue.sampleB gg.handler.myjms2.properties=tibco-default.properties
To set a property, specify the handler name as a prefix; for example:
gg.handlerlist=sample,sample2 gg.handler.sample.type=jms_text gg.handler.sample.format=my_template.vm gg.handler.sample.destination=gg.myqueue gg.handler.sample.queueortopic=queue gg.handler.sample.connectionUrl=tcp://host:61616?jms.useAsyncSend=true gg.handler.sample.useJndi=false gg.handler.sample.connectionFactory=ConnectionFactory gg.handler.sample.connectionFactoryClass=\
org.apache.activemq.ActiveMQConnectionFactory
gg.handler.sample.connectionUrl=\
tcp://localhost:61616?jms.useAsyncSend=true
gg.handler.sample.timeToLive=50000
The following outlines the JMS properties which may be set, and the accepted values. These apply for both JMS handler types: jms_text (TextMessage) and jms_map (MapMessage).
The queue or topic to which the message is sent. This must be correctly configured on the JMS server. Typical values may be: queue/A, queue.Test, example.MyTopic, etc.
gg.handler.name.destination=queue_or_topic
(Optional) User name required to send messages to the JMS server.
gg.handler.name.user=user_name
(Optional) Password required to send messages to the JMS server
gg.handler.name.password=password
Whether the handler is sending to a queue (a single receiver) or a topic (publish / subscribe). This must be correctly configured in the JMS provider. This property is an alias of gg.handler.name.destination. The syntax is:
gg.handler.name.queueOrTopic={queue|topic}
Where:
queue – a message is removed from the queue once it has been read. This is the default.
topic – messages are published and may be delivered to multiple subscribers.
If the delivery mode is set to persistent or not. If the messages are to be persistent, the JMS provider must be configured to log the message to stable storage as part of the client's send operation. The syntax is:
gg.handler.name.persistent={true|false}
JMS defines a 10 level priority value, with 0 as the lowest and 9 as the highest. Priority is set to 4 by default. The syntax is:
gg.handler.name.priority=integer
For example:
gg.handler.name.priority=5
The length of time in milliseconds from its dispatch time that a produced message should be retained by the message system. A value of zero specifies the time is unlimited. The default is zero. The syntax is:
gg.handler.name.timeToLive=milliseconds
For example:
gg.handler.name.timeToLive= 36000
Name of the connection factory to lookup via JNDI. ConnectionFactoryJNDIName is an alias. The syntax is:
gg.handler.name.connectionFactory=JNDI_name
If gg.handler.name.usejndi is false, then JNDI is not used to configure the JMS client. Instead, factories and connections are explicitly constructed. The syntax is:
gg.handler.name.useJndi={true|false}
Connection URL is used only when not using JNDI to explicitly create the connection. The syntax is:
gg.handler.name.connectionUrl=url
The Connection Factory Class is used to access a factory only when not using JNDI. The value of this property is the Java class name to instantiate; constructing a factory object explicitly.
gg.handler.name.connectionFactoryClass=java_class_name
Specifies whether or not local transactions are used. The default is true, local transactions are used. The syntax is:
gg.handler.name.localTX={true|false}
Disables the sending of JMS messages to allow testing of message generation. This is a global property used only for testing. The events are still generated and handled and the message is constructed. The default is false; do not disable message send. The syntax is:
gg.handlerlist.nop={true|false}
These properties set limits for grouping transactions.
Note:
When you use group transaction properties, you must:
Ensure that gg.handler.name.mode is set to one transaction per message (tx). Otherwise the group transaction properties will be ignored.
Ensure that the goldengate.userexit.nochkpt property is set to false.
Ignore the transaction indicator on the operations and not use it to determine transaction boundaries.
Use only one named handler per installation.
Specifies the minimum number of operations that must accumulate before the transaction will be sent.
The syntax is:
gg.handler.name.minGroupSize=number_ops
Where:
number_ops specifies the minimum number of operations that must be accumulated before the transaction is sent.
The maximum value allowed is integer.MAX.VALUE or 2147483647. The minimum value is one.
If you use both properties, the value set for gg.handler.name.minGroupSize should be less than or equal to the value set for gg.handler.name.maxGroupSize.
The following example will test for a minimum of 50 operations before a send.
gg.handler.name.minGroupSize=50
Specifies the maximum number of operations that will rigger the transaction send.
The syntax is:
gg.handler.name.maxGroupSize=number_ops
Where:
number_ops specifies the maximum number of operations that will be accumulated before the transaction is sent.
The maximum value allowed is integer.MAX.VALUE or 2147483647. The minimum value is one.
If you use both properties, the value set for gg.handler.name.minGroupSize should be less than or equal to the value set for gg.handler.name.maxGroupSize.
The following example will send when the maximum of 50 operations is reached.
gg.handler.name.maxGroupSize=50
These JNDI properties are required for connection to an Initial Context to look up the connection factory and initial destination.
java.naming.provider.url=url java.naming.factory.initial=java-class-name
If JNDI security is enabled, the following properties may be set:
java.naming.security.principal=user-name java.naming.security.credentials=password-or-other-authenticator
For example:
java.naming.provider.url= t3://localhost:7001 java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory java.naming.security.principal=jndiuser java.naming.security.credentials=jndipw
The following are general properties that are used for the user exit Java framework.
Specifies a comma delimited list of additional paths to directories or jars to add to the class path. Optionally, the list can be delimited by semicolons for Windows systems or by colons for UNIX. For example:
gg.classpath=C:\Program Files\MyProgram\bin;C:\Program Files\ProgramB\app\bin;
Specifies how often statistics are calculated and sent to Extract for the processing report. If Extract is configured to print a report, these statistics are included. The syntax is:
gg.report.time=report_interval{s|m|h}
Where:
report_interval is an integer
The valid time units are:
s - seconds
m - minutes
h - hours
If no value is entered, the default is to calculate and send every 24 hours.