FAN provides immediate interrupt of clients following outages related to the database, nodes, and networks.

FAN is essential to break clients out of TCP/IP timeouts immediately following failures. FAN notifies clients immediately when resources become available and initiates draining of database sessions so clients experience no outages during planned maintenance. FAN also includes notifying configuration- and service-level information that includes changes in service status.

The Oracle client drivers and Oracle connection pools respond to FAN events and take immediate action. FAN UP and DOWN events apply to services, databases, instances, networks, and nodes.

Oracle connection pools, for example, use FAN to receive very fast notification of failures, to balance connections following failures, and to balance connections again after the failed components are repaired. So, when a service at an instance starts, the connection pool uses the FAN event to route work to that resource, immediately. When a service at an instance or node fails, the connection pool uses the FAN event to immediately interrupt applications to recover. FAN is essential to prevent applications from hanging on TCP/IP timeouts.

This section describes the information delivered in the FAN event to a callout program.

FAN event types are listed following the example, and Table 6-1 describes name-value pairs for the event parameters. The event type is always the first entry when you receive FAN information through a callout, as in the following example:

SERVICEMEMBER VERSION=1.0
   service=test.company.com database=ractest
   instance=ractest11 host=ractest1_host0343_1 status=up reason=FAILURE
   timestamp=2018-05-08 22:06:02 timezone=-07:00 db_domain=company.com

Note that the preceding example displays as one line.

FAN event types include:

• DATABASE
• INSTANCE
• NODE
• SERVICE
• SERVICEMEMBER
• SERVICEMETRICS

The DATABASE and INSTANCE types list the default database service as DB_UNIQUE_NAME.

All events except for NODE events include a db_domain field.

Events of SERVICEMETRICS type are load balancing advisory events.

See Also: Table 5-1 for more information about load balancing events

SERVICEMEMBER VERSION=1.0 service=swingbench
 database=orcl instance=orcl_2 host=rwsbj13 status=up
 reason=USER card=1 timestamp=2018-05-29 17:26:37
 timezone=-07:00 db_domain=

SERVICEMEMBER VERSION=1.0 service=swingbench.example.com
 database=orcl instance=orcl1 host=rwsbj09 status=up
 reason=USER card=2 timestamp=2018-05-03 17:29:28
 timezone=-07:00 db_domain=example.com

SERVICEMEMBER VERSION=1.0 service=swingbench.example.com
 database=orcl instance=orcl2 host=rwsbj10 status=up
 reason=USER card=1 timestamp=2018-07-03 17:29:18
 timezone=-07:00 db_domain=example.com

SERVICEMEMBER VERSION=1.0 service=swingbench.example.com
database=orcl instance=orcl_2 host=mjkbj09 status=up
reason=USER card=1 timestamp=2018-07-12 14:46:46  timezone=-07:00 db_domain=example.com

NODE VERSION=1.0 host=stru09 incarn=175615351 status=down
reason=member_leave timestamp=27-Jul-2018 14:49:32  timezone=-07:00

Some of the FAN event record parameters have values that correspond to values returned by the SYS_CONTEXT function using the default namespace USERENV, as shown in Table 6-2.

Oracle RAC uses FAN to notify applications about configuration changes and the current service level that is provided by each instance where the service is enabled. If you are using an OCI client or an ODP.NET client to receive FAN events, then you must enable the service used by that client to access the alert notification queue by using SRVCTL with the -notification parameter.

FAN callouts are server-side executables that Oracle RAC executes immediately when high availability events occur.

You can use FAN callouts to automate activities when events occur in a cluster configuration, such as:

• Opening fault tracking tickets

• Sending messages to pagers

• Sending e-mail

• Starting and stopping server-side applications

• Maintaining an uptime log by logging each event as it occurs

• Relocating low-priority services when high priority services come online

To use FAN callouts, place an executable in the Grid_home/racg/usrco directory on every node that runs Oracle Clusterware. The executable must be able to run standalone when called, with optional arguments, from another program. The following is an example of an executable shell script, named callout.sh, which is placed in the Grid_home/racg/usrco directory:

#! /bin/bash
FAN_LOGFILE= [your_path_name]/admin/log/'hostname'_uptime'.log
echo $* "reported="'date' >> $FAN_LOGFILE &

The previous example adds entries similar to the following in the log file, indicated by $FAN_LOGFILE in the shell script, each time a FAN event is generated:

NODE VERSION=1.0 host=sun880-2 incarn=23 status=nodedown reason=public_nw_down
timestamp=08-Oct-2012 04:02:14 timezone=-08:00 reported=Fri Oct 8 04:02:14 PDT 2012

The contents of a FAN event record matches the current session of the user logged on to the database, as shown in Table 6-2. The user environment (USERENV) information is also available using OCI connection handle and descriptor attributes (using OCIAttrGet()). Use this information to take actions on sessions that match the FAN event data.

In general, events are only posted to user callouts on the node from which the event originated. For example, if the database on node1 goes down, then the callout is posted to node1, only. The only exceptions to this are node down and VIP down events—these events are posted to all nodes, regardless of from where they originated.

Oracle recommends that you drain database sessions from the instance over a controlled time period from FAN-enabled Oracle or non-Oracle connection pools, or, beginning with Oracle Database 18c at the database, itself.

1. Use SRVCTL to relocate a singleton service or a service not running on all nodes. Use the -force flag with the previously listed SRVCTL commands. You must use the -force flag if you specify the -stopoption parameter on the command line when you run either srvctl relocate service or srvctl stop service. For example:

   $ srvctl relocate service –database mycdb01 –service myservice –drain_timeout 120
     –stopoption IMMEDIATE –oldinst mycdb01_01 -force

   The preceding command relocates the service named myservice01 from the instance named mycdb01_01 to any instance on which it is configured to run. Oracle Clusterware chooses this instance if you do not specify a target on the command line, and waits two minutes (in this example) for any active sessions to drain, after which any sessions remaining on mycdb01_01 are forcibly disconnected. The connection pool automatically releases a connection at a request boundary.

   Note:

   If the service you want to relocate is a uniform service that is currently running on all nodes, then the preceding command returns an error, unless the service is not up on all instances, in which case the preceding command example would succeed for a uniform service.
2. The FAN planned DOWN event clears idle sessions from the connection pool immediately and marks active sessions to be released at the next check-in. These FAN actions drain the sessions from the instance without disrupting the users.

   Existing connections on other instances remain usable, and new connections can be opened to these instances if needed.

3. Not all sessions, in all cases, will check their connections into the pool. Oracle recommends, as a best practice, to have a timeout period (by setting the -drain_timeout parameter), after which the instance is forcibly shut down or the service stopped, evicting any remaining client connections.
   After the drain interval expires, the -stopoption parameter is implemented, which you can define against a service or a database, as follows:

   • When stopping a service (svrctl stop service), you can specify one of the following stop options using the -stopoption parameter: TRANSACTIONAL or IMMEDIATE

   • When stopping a database (svrctl stop database), you can specify one of the following stop options using the -stopoption parameter: NORMAL, TRANSACTIONAL, IMMEDIATE, or ABORT

   The database stop options correlate to the service stop options, as follows:

   • NORMAL=NONE
   • TRANSACTIONAL/TRANSACTIONAL LOCAL=TRANSACTIONAL
   • IMMEDIATE/ABORT=IMMEDIATE

   For those services that are configured to use Application Continuity, an attempt is made to recover these remaining sessions, after they are terminated, masking the outage from users and applications.

   If you do not want Application Continuity to attempt to replay sessions, then specify the -noreplay flag on the SRVCTL command line.

4. Once maintenance is complete, restart the instance and the service on the original node.
5. The FAN UP event for the service informs the connection pool that a new instance is available for use, allowing sessions to be created on this instance at next request boundaries.

Many enterprises run a large number of services, whether it be many services offered by a single database or instance, or many databases offering a few services running on the same node.

• For example, if you want to stop all of the services running on a given node, then you could use the following command:

  $ srvctl stop service –node racnode01 –drain_timeout 60 –stopoption IMMEDIATE

  The command stops all services running on racnode01, allowing a drain interval of 60 seconds. After 60 seconds any remaining sessions are stopped immediately. The 60-second drain timeout interval overrides any attribute setting on any of the services.

  The command could also be qualified to stop the databases on a node, as in the following example:

  $ srvctl stop instance 	-node racnode01 -drain_timeout 60 –stopoption TRANSACTIONAL
    LOCAL -failover –force

  When you specify the -failover parameter:

  • All services are relocated, if possible, respecting the drain timeout interval and the stop option specified.

  • Any services that cannot be failed over are stopped, using the stop option specified.

  • Wait for the length of the drain timeout interval or until all sessions for targeted services are removed, whichever is sooner.

  • All instances stop according to the stop option specified.

  When you specify the –stopoption TRANSACTIONAL LOCAL parameter:

  • Remaining services stop according to the drain timeout interval and stop option specified.

  • Wait for the length of the drain timeout interval or until all sessions for targeted services are removed, whichever is sooner.

  • The instance stops using the TRANSACTIONAL LOCAL stop option.

Before planned maintenance, drain or fail over database sessions at the database instance so application work is not interrupted. Beginning with Oracle Database 18c, the database itself drains the sessions.

When you prepare for planned maintenance, you must stop or relocate the services that are using the server infrastructure. Relocating services is done over a period of time prior to the planned outage and is based on the nature of work associated with each service.

The procedure for rolling planned maintenance moves services in advance of maintenance to another database instance, and notifies the client-side drivers, connections pools, the database instance itself, and other subscribers that maintenance is pending, and what needs to be drained (either connections or sessions using this service). Once notified of draining, a Fast Application Notification (FAN) event is sent and the client pools behave as described elsewhere, in addition, the database begins to search for safe places to release connections and, if needed, to migrate the connections.

Moving or stopping a service triggers a FAN notification that is received by the subscribing Oracle drivers and Oracle connection pools. Starting with Oracle Database 18c, the FAN notification also triggers session draining at the server. Immediately, new work to that service is directed to another functioning instance of that service. Existing sessions are marked for release after their work completes. As work completes and the connections are returned to the connection pool, either the Oracle driver or the connection pool terminates these sessions.

SELECT 1 FROM DUAL;
SELECT COUNT(*) FROM DUAL;
SELECT 1;
BEGIN NULL;END;

isUsable

dbms_app_cont_admin.enable_connection_test(dbms_app_cont.sql_test,'select 1 from dual');

Select 1 from dual;

dbms_session.enable_connection_test(dbms_session.sql_test,'select 1 from dual');

dbms_app_cont.enable_connection_test(dbms_app_cont.sql_test,'select 1 from dual');

Select 1 from dual;

alias =(DESCRIPTION =
 (CONNECT_TIMEOUT=90)(RETRY_COUNT=20)(RETRY_DELAY=3)(TRANSPORT_CONNECT_TIMEOUT=3) 
   (ADDRESS_LIST =
   (LOAD_BALANCE=on)
   ( ADDRESS = (PROTOCOL = TCP)(HOST=primary-scan)(PORT=1521)))
   (ADDRESS_LIST =
   (LOAD_BALANCE=on)
   ( ADDRESS = (PROTOCOL = TCP)(HOST=secondary-scan)(PORT=1521)))
  (CONNECT_DATA=(SERVICE_NAME = gold-cloud)))

This section describes several terms and concepts that you must understand to use Application Continuity.

The following terms are used throughout this chapter:

A database request is a unit of work submitted to the database from the application, such as a transaction. A request typically corresponds to the SQL and PL/SQL, and other database calls, of a single web request on a single database connection, and it is generally demarcated by the calls made to check-out and check-in the database connection from a connection pool.

A recoverable error is an error that arises due to an external system failure, independent of the application session logic that is executing, such as a lost or invalid connection. Recoverable errors occur following planned and unplanned outages of foregrounds, networks, nodes, storage, and databases. The application receives an error code that can leave the application not knowing the status of the last operation submitted. Application Continuity reestablishes database sessions and resubmits the pending work for the class of recoverable errors.

Application Continuity does not resubmit work following call failures due to nonrecoverable errors. An example of a nonrecoverable error that would not be replayed is submission of invalid data values.

A transaction is committed by updating its entry in the transaction table. Oracle Database generates a redo-log record corresponding to this update and writes out this redo-log record. Once this redo-log record is written out to the redo log on disk, the transaction is considered committed at the database. From the client perspective, the transaction is considered committed when an Oracle message (called the commit outcome), generated after that redo is written, is received by the client. However, if a COMMIT has been issued, then the COMMIT failure message cannot be retrieved if it is not received by the client or the application.

Mutable functions are non-deterministic functions that can obtain a new value every time they are called, and thus their results can change frequently. Mutable functions cause a problem for replay because the results can change at replay. Consider sequence.NEXTVAL and SYSDATE, often used in key values. If a primary key is built with values from these function calls, and is used in later foreign keys or other binds, at replay the same function result must be returned.

Application Continuity provides mutable object value replacement at replay for granted Oracle function calls to provide opaque bind-variable consistency. If the call uses database functions that are mutable, including sequence.NEXTVAL, SYSDATE, SYSTIMESTAMP, and SYSGUID, the original values returned from the function execution are saved and are reapplied at replay.

After a COMMIT statement has executed, if state was changed in that transaction, it is not possible to replay the transaction to reestablish that state if the session is lost. When configuring Application Continuity, the applications are categorized depending on whether the session state after the initial setup is static or dynamic (or use AUTO so the decision is automatic), and thus whether it is correct to continue past a COMMIT operation within a request.

• A session has dynamic state if the session state changes are not fully encapsulated by the initialization, and cannot be fully captured by FAILOVER_RESTORE or in a callback at failover. After the first transaction completes, failover is internally disabled until the next request begins. Session state may change during the course of the request.

• A session has a static state if all session state changes (for example, NLS settings and PL/SQL package state) occur as part of initialization, and can be encapsulated by FAILOVER_RESTORE or in a callback at failover. Static applications are those that were able to use Transparent Application Failover (TAF) before Application Continuity. Session state does not change during the course of the request. (Choose setting session state consistency to AUTO over STATIC mode, when possible, because auto mode purges and cleans more efficiently than the pre-Application Continuity TAF mode.)

• With Transparent Application Continuity, the state is managed for you by setting session state consistency to AUTO (this is a mandatory setting for Transparent Application Continuity). These session states are tracked and verified at failover. You can add further states if outside the preset states.

Applications achieve continuous availability when planned maintenance and unplanned outages of the database are transparent.

Transparent Application Continuity is a functional mode of Application Continuity beginning with Oracle RAC 18c that transparently tracks and records session and transactional state so that a database session can be recovered following recoverable outages. This is done safely and with no need for a DBA to have any knowledge of the application or make application code changes. Transparency is achieved by using a state-tracking infrastructure that categorizes session state usage as an application issues user calls.

You can enable Transparent Application Continuity as default to protect applications during planned maintenance and when unplanned outages occur. For planned maintenance, database sessions that reach a safe place (such as a connection test or a known recoverable point) are drained at the database. For database sessions that do not drain, the database determines where to fail the database session over and invokes Application Continuity to do so. Application Continuity hides unplanned outages for Java-based applications, OCI and ODP.NET applications including SQL*Plus, all Oracle connection pools, Tuxedo, WebLogic Server, and third-party application servers using Universal Connection Pool.

When you use Transparent Application Continuity for Java (beginning with Oracle Database 18c) and OCI and ODP.NET Unmanaged Provider (beginning with Oracle Database 19c (19.3)), your applications will experience lower resource usage and faster recovery because statements that do not contribute to the state are not recorded, or they are purged when no longer needed, and request boundaries are advanced automatically.

For unplanned outages, Transparent Application Continuity is invoked for outages that result in recoverable errors, typically related to underlying software, foreground, hardware, communications, network, or storage layers, hiding most failures from applications and users.

Transparent Application Continuity is enabled when FAILOVER_TYPE=AUTO.

If a recoverable error occurs and if you enabled replay, then Application Continuity attempts recovery of the database session.

The following figure is a graphical representation of how Application Continuity works.

Figure 6-1 Application Continuity

Description of "Figure 6-1 Application Continuity"

1. The client application makes a request, which is passed to a middle tier (such as the Universal Connection Pool (UCP), ODP.NET, WebLogic Server, OCI session pool, Tuxedo, JDBC and OCI drivers, or third-party pool using UCP), or directly to the database using the JDBC replay driver or OCI driver.

2. The JDBC replay driver or OCI driver issues each call in the request.

3. A FAN unplanned or planned down interrupt or recoverable error occurs. FAN/FCF then aborts the dead physical session.

4. Application Continuity begins the replay and does the following:

   1. Replaces the dead physical session with a new clean session.

   2. Prepares for replay by using Transaction Guard to determine the outcome of the in-flight transaction, if one was open.

   3. If FAILOVER_RESTORE=LEVEL1 or FAILOVER_TYPE=AUTO, then Application Continuity restores the common initial session state. Application Continuity uses a label callback or initial callback if an application also sets session states that are not provided by FAILOVER_RESTORE in the callback

   4. Rebuilds the database session, recovering the transactional and non-transactional states, and validating at each step that the data and messages seen by the client driver are the same as those that the client may have seen and used to make a decision.

   5. Ends the replay and returns to runtime mode.

   6. Submits the last queued call.

      This is the last call made when the outage was discovered. During replay, only this call can execute a COMMIT. A COMMIT midway through rebuilding the session aborts replay (excluding autonomous transactions).

5. The response is returned to the application.

   If replay succeeded, then the application can continue with the problem masked. If not, then the application must handle the original error.

The behavior of Application Continuity after a communication failure depends on the Oracle products and technologies involved. For example:

• If you use Oracle RAC or an Oracle Active Data Guard farm, then, after the connection is reestablished on another running instance, Application Continuity attempts to rebuild the session and replay the last transaction if there is one in flight.

• If you use Oracle Active Data Guard and fail over to a standby site, then Application Continuity connects to the failover instance and attempts to rebuild the session and replay the last transaction there, if a transaction was in-flight. (Application Continuity does not replay if the Oracle Active Data Guard switchover or failover has lost data, and if this is not an Oracle Active Data Guard reader farm with approved lags.)

• If you are using Oracle RAC or Oracle RAC One Node and not using Oracle Active Data Guard, and if an outage causes a break in all public networks or causes the database or database session to shut down briefly, then Application Continuity attempts to rebuild the session and replay the last transaction (if a transaction was in flight) against the database after connectivity is restored.

Application Continuity masks outages with few or no application changes when you use the Oracle integrated stack.

Application Continuity is available for general use with the following Oracle technologies:

• ODP.NET, Unmanaged Driver

• OCI Session Pool

• Universal Connection Pool

• WebLogic Server

• JDBC Thin Oracle replay driver

• Oracle Tuxedo

• SQL*Plus

• Third-party JDBC application servers with Universal Connection Pool

• OCI-based applications with Transparent Application Continuity starting with Oracle Database 19c (19.3)

Application Continuity for Java is embedded in the Universal Connection Pool, WebLogic data sources, including non-XA and XA data sources, and is available with the thin JDBC replay driver, standalone (which is a JDBC replay driver without Oracle connection pools, such as Apache Tomcat or a custom Java connection pool). Application Continuity for OCI is embedded in SQL*Plus, OCI Session Pool. Oracle Tuxedo for non XA data sources, and ODP.NET, Unmanaged Provider. With Transparent Application Continuity, JDBC applications auto enable starting with Oracle Database 18c, and OCI applications starting with Oracle Database 19c (19.3).

Application Continuity is embedded in Oracle connection pools. When you use the Oracle connection pools, request boundaries are implicitly marked at check-out and check-in, delimiting the size of each replay. When using third-party connection pools, use UCP if Java, or you can add request boundaries, which are included on the JDK9 standard. When using Transparent Application Continuity for Java, only the first request boundary is required. Request boundaries are discovered using state tracking. With the OCI driver for Oracle Database 19c, support is now available for implicit boundaries, as it is with JDBC client drivers.

Support for Application Continuity is integrated into many Oracle applications, so the features in such applications are used automatically if you set the Application Continuity-related service attributes. For all applications, follow the steps described in this section.

The main actions for ensuring transparent replay for an application are the following:

1. Only if using Java, determine whether the application uses Oracle JDBC concrete classes. For Application Continuity to be used, the deprecated concrete classes must be replaced.

   Use the -acchk parameter with the ORAchk utility to verify whether an application has any concrete classes. Use a connection without Application Continuity if there is anything that should not be replayed. (Most applications will be replayable.)

   See Also:

   Oracle Autonomous Health Framework User's Guide for more information about ORAchk

2. Ensure that you have the necessary CPU and memory resources.

   • CPU: Application Continuity is managed on the client and server sides and requires minimal CPU overhead to operate.

     At the client, CPU is used to build proxy objects and for garbage collection (GC).

     At the server, CPU is used for validation. CPU overhead is reduced for platforms with current Intel and SPARC chips where validation is assisted in the hardware.

   • Memory: When using Application Continuity, the replay driver requires more memory than the base driver because the calls are retained until the end of a request. At the end of the request, the calls are released to the garbage collector. This action differs from the base driver that releases closed calls.

     The memory consumption of the replay driver depends on the number of calls per request. If this number is small, then the memory consumption of the replay driver is less, and comparable to the base driver.

     To obtain the best performance, you must set the same value for both the -Xmx and -Xms parameters on the client. For example, if there is sufficient memory, then allocate 4 to 8 GB (or more) of memory for the Virtual Machine (VM), for example, by setting -Xms4g for 4 GB. If the -Xms parameter has a lower value, then the VM also uses a lower value from the operating system, and performance might suffer and garbage collection operations increase.

3. Determine whether the application borrows and returns connections from the connection pool, for example WebLogic Server Pool, Universal Connection Pool, OCI Session Pool, Oracle Tuxedo request, or ODP.NET connection pool, for each request, or whether to add beginRequest and endRequest APIs to the application's own connection pool to identify request boundaries for Java, only.

   Important:

   Do not use the beginRequest and endRequest Java API calls anywhere other than at request boundaries (borrow and return connections from your connection pool). endRequest indicates that the request is complete, and that it is now stateless. Replay starts from the next beginRequest. If there is prior state, it must be reestablished using FAILOVER_RESTORE or callback.

4. Application Continuity replays all states in a request. If the application sets states before vending connections, FAILOVER_RESTORE or a callback is needed. When using Oracle WebLogic Server or the Universal Connection Pool, use FAILOVER_RESTORE, connection labeling, or triggers. When using OCI session pool, Oracle Tuxedo or ODP.NET with Oracle Database 18c, or later clients, use FAILOVER_RESTORE, and only add the TAF callback if it is needed. The labeling is used for both runtime and replay.

5. Determine whether the application requires, and therefore needs to configure keeping original values for, SYSDATE, SYSTIMESTAMP, and SYS_GUID and sequences during failover.

6. Assess the application style for the session_state_consistency value, and set the appropriate value on the service:

   • If session_state_consistency is set to AUTO, then Transparent Application Continuity monitors the session state and decides what to do. If you are unsure about state usage or know that states can change in the future, then use Transparent Application Continuity. See the list of preset session states because you may need to restore additional preset states.

   • If session_state_consistency is set to DYNAMIC, then the application changes the environment or settings during the request. Replay is disabled after the first COMMIT until the beginning of the next request. DYNAMIC is the default mode, appropriate for most applications.

   • If session_state_consistency is set to STATIC, then the application never changes the session state after initial setup. This mode is typical for database agnostic applications that do not use PL/SQL state and do not use ALTER part-way through transactions. Use this mode with caution, and only for "static" applications.

7. Determine if any requests in the application should not be replayed.

   For example, replay may need to be disabled for requests using external PL/SQL actions.

8. Follow these configuration guidelines:

   • Use Oracle Database 12c release 1 (12.1.0.1), or later, for Java. Use Oracle Database 12c release 2 (12.2), or later, for OCI-based applications.

   • For .NET applications, use ODP.NET, Unmanaged Driver 12.2, or later, connecting to an Oracle Database 12c Release 2 (12.2) or later. By default, Application Continuity is enabled for ODP.NET applications in this configuration.

   • For Java-based applications, use Universal Connection Pool 12.1 (or later) or WebLogic Server 12.1.2 (or later) configured with the JDBC Replay data source; or for third party applications, including third party JDBC pools, use JDBC replay driver. For IBM WebSphere, Apache Tomcat, RedHat Spring, and custom Java solutions, the most effective solution is to use UCP as the pooled data source.

     Custom Java pools and standalone Java applications can also use the JDBC Replay data source directly. When using custom Java pools and standalone applications, add the beginRequest and endRequest calls.

   • If the application does not borrow and return from the Oracle connection pools, explicitly mark request boundaries. For example, if using custom JDBC pools, or other pools, then call beginRequest at check-out and call endRequest at check-in. These APIs can also be used for standalone JDBC applications without a connection pool.

   • Enable FAN for fast interrupt on errors. This is essential to eliminate a TCP hang occurring before the failover can start. In 12.2 FAN is built into the JDBC and OCI drivers and is on by default for Java.

   • Use a database service to connect; never use a SID or an instance name, or the administration service that is the DB_NAME or DB_UNIQUE_NAME.

   • Use a connection string that sets retries for new incoming connections and a delay between these retries.

   • For the service, set FAILOVER_TYPE to TRANSACTION for the manual mode of Application Continuity or set FAILOVER_TYPE to AUTO for Transparent Application Continuity. Set COMMIT_OUTCOME to TRUE and, for OCI FAN, set NOTIFICATION to TRUE. Optionally to find the best connections to use, set GOAL to SERVICE_TIME and CLB_GOAL to SHORT.

   • Grant permission on the Transaction Guard package, DBMS_APP_CONT, to the database users that fail over using Application Continuity, as follows:

     GRANT EXECUTE ON DBMS_APP_CONT TO user_name;

   • Use the statistics for request boundaries and protection level to monitor the level of coverage. If you need more details, then use Application Continuity Check Coverage (with the ORAchk utility) to report the percentage of requests that are fully protected by Application Continuity, and the location of those requests that are not fully protected. Use this coverage check before deployment and after application changes. Developers and management will know how well protected an application release is from failures of the underlying infrastructure. If there is a problem, then it can be fixed before the application is released, or waived knowing the level of coverage.

If a connection pool or container does not use an Oracle connection pool, then many third-party Java applications fully support replacing the connection pool with the Universal Connection Pool. This includes IBM WebSphere and Apache Tomcat. Alternatively—for Java applications, only—an application can add its own request boundaries.

This section includes the following topics:

• Configuring Oracle Application Continuity

• Configuring Oracle Database for Application Continuity

• Establishing the Initial State Before Application Continuity Replays

• Delaying the Reconnection in Application Continuity

• Using Application Continuity for Planned Maintenance

• Running Without Application Continuity

• Disabling Replay in Application Continuity

• Terminating or Disconnecting a Session Without Replay

When a request is replayed, the default and desired treatment of mutable objects can vary.

A mutable function is a non-deterministic function that can obtain a new value every time it is called. An example of a mutable function is a call to the SYSTIMESTAMP function. Client applications using Application Continuity can determine whether to keep the original value for mutable functions if the request is replayed.

Support for keeping mutable function values is currently provided for SYSDATE, SYSTIMESTAMP, SYS_GUID, and sequence.NEXTVAL. If the original values are not kept and if different values for these mutable objects are returned to the client, then replay is rejected because the client observes different results. If the application can use original values, then configure mutable functions using the KEEP clause for owned sequences and GRANT KEEP for other users. (Most applications need sequence values to be kept at replay, for bind variable consistency.)

Table 6-4 shows examples of the treatment of mutable functions by products during replay. (Actual implementation depends on specific products and releases.)

To allow Application Continuity to keep and use original function results at replay:

• The database user running the application might have the KEEP DATE TIME and KEEP SYSGUID privileges granted, and the KEEP SEQUENCE object privilege on each sequence whose value is to be kept. For example:

  GRANT KEEP DATE TIME TO user2;
  GRANT KEEP SYSGUID TO user2;
  GRANT KEEP SEQUENCE ON sales.seq1 TO user2;

  Notes:

  • GRANT ALL ON object does not include (that is, does not grant the access provided by) the KEEP DATE TIME and KEEP SYSGUID privileges, and the KEEP SEQUENCE object privilege.

  • Grant privileges related to mutable function support only to application users, and to each application user grant only the necessary privileges.

  • Do not grant DBA privileges to database users running applications for which you want replay to be enabled.

• Sequences in the application can use the KEEP attribute, which keeps the original values of sequence.NEXTVAL for the sequence owner, so that the keys match during replay. Most applications need sequence values to be kept at replay. The following example sets the KEEP attribute for a sequence (in this case, one owned by the user executing the statement; for others, use GRANT KEEP SEQUENCE):

  SQL> CREATE SEQUENCE my_seq KEEP;
  SQL> -- Or, if the sequence already exists but without KEEP:
  SQL> ALTER SEQUENCE my_seq KEEP;

  Note:

  Specifying ALTER SEQUENCE ... KEEP/NOKEEP applies to the owner of the sequence. It does not affect other users (not the owner) that have the KEEP SEQUENCE object privileges. If you want NOKEEP for all users, then be sure not to grant the KEEP SEQUENCE object privilege to these users (or to revoke it from each user if the privilege has been granted).

• To keep function results (for named functions) at replay, the DBA must grant KEEP privileges to the user invoking the function. This security restriction ensures that it is valid for replay to save and restore function results for code that is not owned by that user.

Use the statistics for request boundaries and protection level to monitor the level of coverage.

Statistic                                        Total     per Second     per Trans
---------------------------------------- -------- ------------- ---------
cumulative requests                         177,406           49.2             5.0
cumulative user calls in request            493,329          136.8            13.8
cumulative user calls protected             493,329          136.8            13.8

To enable protection-level statistics, use (_request_boundaries = 3).

Session state consistency describes how non-transactional state is changed during a request.

Oracle recommends that you set session_state_consistency to AUTO available with Transparent Application Continuity, which tracks and manages session states. If you choose to use Transparent Application Continuity, then you do not have to do anything else to ensure session state consistency.

You can set session_state_consistency to DYNAMIC or STATIC for manual Application Continuity. Set session_state_consistency to DYNAMIC or STATIC if you fully understand the application, and the application is not expected to change from the value set.

Examples of session state are NLS settings, optimizer preferences, event settings, PL/SQL global variables, temporary tables, advanced queues, LOBs, and result cache. If non-transactional values change in committed transactions, then use the default value, DYNAMIC (session_state_consistency is a service level attribute, the default value of which is DYNAMIC).

Using DYNAMIC mode, after a COMMIT has executed, if the state was changed in that transaction, then it is not possible to replay the transaction to reestablish that state if the session is lost. Applications can be categorized depending on whether the session state after the initial setup is static or dynamic, and hence whether it is correct to continue past a COMMIT operation.

DYNAMIC mode is appropriate for almost all applications. If you are unsure, then use DYNAMIC mode. If your customers or users can modify your application, then you must use DYNAMIC mode.

This section includes the following topics:

• Auto Session State Consistency

• Dynamic Session State Consistency

• Static Session State Consistency

Before configuring services for Transaction Guard, use the following configuration checklist:

• Grant permission to the application user who will call GET_LTXID_OUTCOME, as follows:

  GRANT EXECUTE ON DBMS_APP_CONT to user_name;
  

  Note:

  Do not run this statement if you use Application Continuity.

• Locate and define the transaction history table for optimal performance.

  The transaction history table (LTXID_HIST) is created, by default, in the SYSAUX tablespace when you create or upgrade an Oracle Database. New partitions are added when you add instances, using the storage of the last partition. If the location of transaction history table is not optimal for performance, then you can move it to another tablespace and create partitions there. For example, the following statement moves the transaction history table to a tablespace named FastPace:

  ALTER TABLE LTXID_TRANS move partition LTXID_TRANS_1 tablespace FastPace
     storage ( initial 10G next 10G minextents 1 maxextents 121 );
  

• Set values for the -commit_outcome and -retention service parameters.

• If you are using Oracle RAC, Oracle Data Guard, or Oracle Active Data Guard, then Oracle recommends that you use FAN for fast notification of an outage.

To configure services to use Transaction Guard, set the following service parameters:

• -commit_outcome: Set the -commit_outcome service parameter to TRUE. This service parameter determines whether the transaction commit outcome is accessible after the COMMIT has executed and an outage has occurred. While Oracle Database has always made COMMIT durable, Transaction Guard makes the outcome of the COMMIT durable, and is used by applications to enforce the status of the last transaction executed before an outage.

• -retention: Use the -retention service parameter with -commit_outcome. This service parameter determines the amount of time, in seconds, that the COMMIT outcome is retained. Oracle recommends that most installations use the default value.

The following SRVCTL command configures a policy-managed service named sales for Transaction Guard:

$ srvctl add service -db crm -service sales -serverpool spool_1
  -commit_outcome TRUE -retention 86400 -notification TRUE

The following SRVCTL command configures an administrator-managed service named sales for Transaction Guard:

$ srvctl add service -db crm -service sales -preferred crm_1,crm_2
  -available crm_3,crm_4 -commit_outcome TRUE -retention 86400
  -notification TRUE

You can also modify an existing service to configure it for Transaction Guard by using the srvctl modify service command.