12 Using the Kafka REST Proxy Handler

gg.handler.name.type

Required

kafkarestproxy

None

The configuration to select the Kafka REST Proxy Handler.

gg.handler.name.topicMappingTemplate

Required

A template string value to resolve the Kafka topic name at runtime.

None

See Using Templates to Resolve the Topic Name and Message Key.

gg.handler.name.keyMappingTemplate

Required

A template string value to resolve the Kafka message key at runtime.

None

See Using Templates to Resolve the Topic Name and Message Key.

gg.handler.name.postDataUrl

Required

The Listener address of the Rest Proxy.

None

Set to the URL of the Kafka REST proxy.

gg.handler.name.format

Required

avro | json

None

Set to the REST proxy payload data format

gg.handler.name.payloadsize

Optional

A value representing the payload size in mega bytes.

5MB

Set to the maximum size of the payload of the HTTP messages.

gg.handler.name.circularRedirectsAllowed

Optional

true | false

false

Set to allow or disallow circular redirects.

gg.handler.name.connectionRequestTimeout

Optional

A value representing milliseconds.

—1

Set the maximum time to wait for the connection manager to return a connection. The connection manager may not be able to return a connection if the max number of connections in the pool are used.

gg.handler.name.connectTimeout

Optional

true | false

-1

Set the timeout in milliseconds until a connection is established.

gg.handler.name.contentCompressionEnabled

Optional

true | false

true

Sets content compression to on or off.

gg.handler.name.maxRedirects

Optional

Integer value representing the redirect count.

50

Sets the maximum number of redirects.

gg.handler.name.proxy

Optional

host:port

None

Sets the proxy.

gg.handler.name.userName

Optional

Any string.

None

Sets the username for the proxy authentication.

gg.handler.name.password

Optional

Any string.

None

Sets the password for the proxy authentication.

gg.handler.name.redirectsEnabled

Optional

true | false

true

Set to check if redirects using relative naming is enabled.

gg.handler.name.relativeRedirectsAllowed

Optional

true | false

true

Set to check if redirects is enabled.

gg.handler.name.socketTimeout

Optional

A value representing milliseconds.

—1

Set the maximum time allowable between data packets on a read.

gg.handler.name.httpClientResetInterval

Optional

A value representing milliseconds.

0

Sets the wait interval between when the HTTP client is destroyed and when it is recreated.

gg.handler.name.apiVersion

Optional

v1 | v2

v2

Sets the API version to use.

gg.handler.name.mode

Optional

op | tx

op

Sets how operations are processed. In op mode, operations are processed as they are received. In tx mode, operations are cached and processed at the transaction commit.

${fullyQualifiedTableName}

Resolves to the fully qualified table name including the period (.) delimiter between the catalog, schema, and table names.

For example, test.dbo.table1.

No

${catalogName}

Resolves to the catalog name.

No

${schemaName}

Resolves to the schema name.

No

${tableName}

Resolves to the short table name.

No

${opType}

Resolves to the type of the operation: (INSERT, UPDATE, DELETE, or TRUNCATE)

No

${primaryKeys}

Resolves to the concatenated primary key values delimited by an underscore (_) character.

No

${position}

The sequence number of the source trail file followed by the offset (RBA).

Yes

${opTimestamp}

The operation timestamp from the source trail file.

Yes

${emptyString}

Resolves to “”.

Yes

${groupName}

Resolves to the name of the Replicat process. If using coordinated delivery, it resolves to the name of the Replicat process with the Replicate thread number appended.

Yes

${staticMap[]}

Resolves to a static value where the key is the fully-qualified table name. The keys and values are designated inside of the square brace in the following format:

${staticMap[dbo.table1=value1,dbo.table2=value2]}

No

${columnValue[]}

Resolves to a column value where the key is the fully-qualified table name and the value is the column name to be resolved. For example:

${staticMap[dbo.table1=col1,dbo.table2=col2]}

No

${currentTimestamp}

Or

${currentTimestamp[]}

Resolves to the current timestamp. You can control the format of the current timestamp using the Java based formatting as described in the SimpleDateFormat class, see https://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html.

Examples:

${currentDate}
${currentDate[yyyy-mm-dd hh:MM:ss.SSS]}

Yes

${null}

Resolves to a NULL string.

Yes

${custom[]}

It is possible to write a custom value resolver. If required, contact Oracle Support.

Implementation dependent

${groupName}_${fullyQualfiedTableName}

KAFKA001_dbo.table1

prefix_${schemaName}_${tableName}_suffix

prefix_dbo_table1_suffix

${currentDate[yyyy-mm-dd hh:MM:ss.SSS]}

2017-05-17 11:45:34.254

Optional

true | false

true

Set to true to create a field in the output messages called op_ts. The value is an indicator of the type of source database operation (for example, Ifor insert, Ufor update, Dfor delete).

Set to false to omit this field in the output.

Optional

true | false

true

Set to true to create a field in the output messages called op_type. The value is the operation timestamp (commit timestamp) from the source trail file.

Set to false to omit this field in the output.

Optional

true | false

true

Set to true to create a field in the output messages called current_ts. The value is the current timestamp of when the handler processes the operation.

Set to false to omit this field in the output.

Optional

true | false

true

Set to true to create a field in the output messages called pos. The value is the position (sequence number + offset) of the operation from the source trail file.

Set to false to omit this field in the output.

Optional

true | false

true

Set to true to create a field in the output messages called primary_keys. The value is an array of the column names of the primary key columns.

Set to false to omit this field in the output.

Optional

true | false

true

Set to true to include a map field in output messages. The key is tokens and the value is a map where the keys and values are the token keys and values from the Oracle GoldenGate source trail file.

Set to false to suppress this field.

Optional

Any string.

I

The value of the field op_type that indicates an insert operation.

Optional

Any string.

U

The value of the field op_type that indicates an update operation.

Optional

Any string.

D

The value of the field op_type that indicates an delete operation.

Optional

Any string.

T

The value of the field op_type that indicates an truncate operation.

Optional

true | false

false

Set to true treat all output fields as strings.

Set to false and the handler maps the corresponding field type from the source trail file to the best corresponding Kafka data type.

Optional

true | false

false

Set to true and these fields are mapped as strings to preserve precision. This property is specific to the Avro Formatter; it cannot be used with other formatters.

Optional

true | false

false

Set to true to output the current date in the ISO8601 format.

Optional

abend | update | delete-insert

abend

It is only applicable if you are modeling row messages with the .(gg.handler.name.format.messageFormatting=row property. It is not applicable if you are modeling operations messages as the before and after images are propagated to the message with an update.

Optional

${alltokens} | ${token} | ${env} | ${sys} | ${javaprop} | ${optype} | ${position} | ${timestamp} | ${catalog} | ${schema} | ${table} | ${objectname} | ${csn} | ${xid} | ${currenttimestamp} | ${opseqno} | ${timestampmicro} | ${currenttimestampmicro}

None

The current meta column information can be configured in a simple manner and removes the explicit need to use:

insertOpKey | updateOpKey | deleteOpKey | truncateOpKey | includeTableName | includeOpTimestamp | includeOpType | includePosition | includeCurrentTimestamp, useIso8601Format

It is a comma-delimited string consisting of one or more templated values that represent the template.