These are the guidelines for choosing a target standby database.

Use Cloud Control or DGMGRL to perform either a complete (recommended) or an immediate failover.

DGMGRL> FAILOVER TO database-name;

DGMGRL> FAILOVER TO database-name IMMEDIATE;

This list describes how the overall Oracle Data Guard protection mode is handled after a manual failover (complete or immediate).

To maintain a viable disaster-recovery solution in the event of another disaster, you may need to perform additional steps.

These are the actions the broker performs after you start a complete failover.

1. Verifies that the target standby database is enabled. If the database is not enabled, you will not be able to perform a failover to this database.

2. Waits for the target standby database to finish applying any unapplied redo data before stopping Redo Apply (if the target is a physical standby database) or SQL Apply (if the target is a logical standby database).

   If the target is a snapshot standby database, the broker first converts the database back to a physical standby and then starts Redo Apply to apply all the accumulated redo before completing the failover and opening the database as a primary database.

3. Transitions the target standby database into the primary database role, as follows:

   1. Changes the role of the database from standby to primary.

   2. Opens the new primary database in read/write mode.

   3. Determines whether or not any standby databases that did not participate in the failover operation have applied redo data beyond the new primary database, and thus need to be disabled.

      If a bystander standby database is not disabled by the broker during this failover, it will remain in the state it was in before the failover. For example, if a physical standby database was in the APPLY-OFF state, it will remain in the APPLY-OFF state.

      By default, the broker always determines whether bystander standby databases will be viable standby databases for the new primary when performing a complete failover. If you want the broker to skip this viability check of bystander standby databases during a complete failover, thus decreasing the overall failover time, set the BystandersFollowRoleChange configuration property to NONE.

      When this property is set to NONE, the broker will disable all bystander standby databases without checking whether they have applied more redo data than the new primary database. You will have to reinstate or re-create (see Reenabling Disabled Databases After a Role Change) the standby databases after failover has completed. The SHOW CONFIGURATION command will show you which databases can be reinstated and which databases must be re-created. Use the SHOW CONFIGURATION BystandersFollowRoleChange command to see the value of this property. The default value is ALL.

      This property also affects whether the broker skips viability checks of bystander standby databases when a fast-start failover occurs.

   4. Starts redo transport services to begin transmitting redo data to all bystander standby databases that were not disabled.

      Note:

      Bystander standby databases may be disabled by the broker during the failover, and they must be reinstated or re-created before they can serve as standby databases to the new primary database. Oracle recommends configuring Flashback Database on every database so that if failover occurs to a physical standby database, you can more easily reinstate any disabled standby databases. If failover occurs to a logical standby database, all physical and snapshot standby databases will be disabled by the broker. In this case, Flashback Database cannot be used to reinstate databases. They must be re-created from a copy of the new primary database. Logical standby databases that are disabled during failover can be reinstated.

4. If the failover target database is an Oracle RAC physical or snapshot standby database, the broker directs Oracle Clusterware to restart all instances that may have been shut down prior to the failover.

The broker allows the failover to proceed as long as there are no errors for the standby database that you selected to participate in the failover. Errors occurring for any other configuration members will not impede the switchover. If you initiated a complete failover and it fails, you might need to use immediate failover.

To start an immediate failover, use the DGMGRL FAILOVER TO database-name IMMEDIATE command.

Once an immediate failover is started, the broker:

1. Verifies that the target standby database is enabled. If the standby database is not enabled for management by the broker, then the failover cannot occur.

2. Stops Redo Apply or SQL Apply on the standby database immediately, without waiting until all available redo data has been applied. This may result in data loss.

3. Transitions the target standby database into the primary role, opens the new primary database in read/write mode, and starts redo transport services.

   After an immediate failover completes, all the standby databases in the configuration, regardless of their type, are disabled. They may be reinstated if Flashback Database is enabled on those databases. Otherwise, they must be re-created from a copy of the new primary database.

The broker allows a complete failover to proceed as long as there are no errors present on the standby database that you selected to participate in the failover.

The broker allows an immediate failover to proceed even if there are errors present on the standby database that you selected to participate in the failover.

You can use the broker's reinstate capability to make a failed primary database a viable standby database for the new primary.

This can be done regardless of whether the failover was done to a physical, logical, or snapshot standby database.

You can also reinstate bystander standby databases that were disabled during a failover operation.

Databases that can be reinstated will have the following status value:

ORA-16661: the standby database needs to be reinstated

For the REINSTATE command to succeed, Flashback Database must have been enabled on the database prior to the failover and there must be sufficient flashback logs on that database. In addition, the database to be reinstated and the new primary database must have network connectivity.

To reinstate a database:

1. Restart the database to the mounted state

2. Connect to the new primary database

3. Use Cloud Control or DGMGRL to reinstate the database

The broker reinstates a failed primary database as a standby database of the same type (physical or logical standby database) as the old standby database. The only exception to this is failovers to snapshot standby databases. In such cases, the failed primary database is reinstated as a physical standby database.

The broker reinstates bystander standby databases that were disabled during a failover as standby databases to the new primary database.

Reinstatement Using Cloud Control

On the Oracle Data Guard Overview page, click Database must be reinstated. This brings up the General Properties page that provides a Reinstate button. After you click the Reinstate button, Cloud Control begins reinstating the database.

When the process is complete, the database will be enabled as a standby database to the new primary database, and Cloud Control displays the Oracle Data Guard Overview page.

Reinstatement Using DGMGRL

Issue the following command while connected to any database in the broker configuration, except the database that is to be reinstated:

DGMGRL> REINSTATE DATABASE db_unique_name;

The newly reinstated standby database will begin serving as a standby database to the new primary database. If reinstatement of a database fails, its status changes to ORA-16795: the standby database needs to be re-created. You must then re-create it from a copy of the new primary database and reenable it as described in How to Re-create and Reenable a Disabled Database.

If you re-create the old primary database, it must be created as the standby type of the old standby database.

You must determine which available standby databases should be targets for failover.

Use the FastStartFailoverTarget configuration property on the current primary database to specify one or more fast-start failover targets.

If you cannot tolerate any loss of data, then ensure that the configuration protection mode is set to maximum availability or maximum protection. For zero data loss in maximum availability mode, the FastStartFailoverLagLimit property must be set to zero.

DGMGRL> EDIT DATABASE 'North_Sales' SET PROPERTY LogXptMode=SYNC;
DGMGRL> EDIT DATABASE 'South_Sales' SET PROPERTY LogXptMode=SYNC;
DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MaxAvailability;

DGMGRL> EDIT DATABASE 'North_Sales' SET PROPERTY LogXptMode=ASYNC;
DGMGRL> EDIT DATABASE 'South_Sales' SET PROPERTY LogXptMode=ASYNC;
DGMGRL> EDIT CONFIGURATION SET PROTECTION MODE AS MaxPerformance;
DGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverLagLimit=45;

Fast-start failover occurs if both the observer and the target standby database lose connection to the primary database for the period of time specified by the FastStartFailoverThreshold configuration property.

DGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold = 45;

This table describes the optional database properties that you can set.

By default, a fast-start failover is done when both the observer and the standby cannot reach the primary after the configured time threshold (FastStartFailoverThreshold) has passed.

Use the Cloud Control Fast-Start Failover wizard or the DGMGRL ENABLE FAST_START FAILOVER command to enable fast-start failover.

DGMGRL> ENABLE FAST_START FAILOVER;
Enabled.

The primary database must be running in order to start the observer.

DGMGRL> START OBSERVER;

To verify the readiness of the fast-start failover configuration, issue the DGMGRL SHOW CONFIGURATION VERBOSE command or the SHOW FAST_START FAILOVER command on the primary database.

DGMGRL> SHOW FAST_START FAILOVER;
 
Fast-Start Failover: Enabled in Zero Data Loss Mode

 Protection Mode: MaxAvailability
 Lag Limit: 0 seconds

 Threshold: 180 seconds
 Active Target: South_Sales
 Potential Targets: East_Sales, West_Sales
   East_Sales    valid
   West_Sales    valid
 Observer: observer.example.com
 Shutdown Primary: TRUE
 Auto-reinstate: TRUE
 Observer Reconnect: (none)
 Observer Override: FALSE
 
Configurable Failover Conditions
 Health Conditions:
  Corrupted Controlfile     YES
  Corrupted Dictionary      YES
  Inaccessible Logfile       NO
  Stuck Archiver             NO
  Datafile Write Errors     YES

Oracle Error Conditions:
(none)

After fast-start failover is enabled and up to three observers are started, one observer is nominated as the master observer that continuously monitors the environment to ensure the primary database is available.

This section lists the steps the master observer takes to determine if a fast-start failover is needed and then to perform one, if necessary.

1. Monitor the environment to ensure the primary database is available.

   The master observer waits the number of seconds specified by the FastStartFailoverThreshold configuration property before attempting a fast-start failover when the primary database has crashed or has lost connectivity with the observer, as in the following situations:

   • The primary database loses its connections with both the observer and target standby database

   • Instance failures

     If a single-instance primary database (either Oracle RAC or non-Oracle RAC), or if all instances of an Oracle RAC primary database fail, the observer attempts a fast-start failover.

   • Shutdown abort

     If a single-instance primary database (either Oracle RAC or non-Oracle RAC), or if all instances of an Oracle RAC primary database are shut down with the ABORT option, the observer attempts a fast-start failover. Fast-start failover will not be attempted for the other types of database shutdown (NORMAL, IMMEDIATE, TRANSACTIONAL).

   The master observer never waits for the threshold to expire to perform a fast-start failover in the following situations:

   • User-configurable condition

     If the master observer determines that any of the user-configurable conditions has been detected, then it attempts a fast-start failover.

   • Application calls to DBMS_DG.INITIATE_FS_FAILOVER

     If an application has called this function and it has received a status of SUCCESS, then the master observer attempts a fast-start failover.

2. Reconnect within the time specified by the FastStartFailoverThreshold property.

   If the master observer detects an availability problem with the primary database, then it typically attempts to reconnect to the primary database within the time specified by the FastStartFailoverThreshold configuration property. The FastStartFailoverThreshold time interval starts when the observer first detects there might be a failure with the primary database.

   The time interval specified by the FastStartFailoverThreshold property is ignored if the master observer detects that a user-configurable condition has occurred or if a fast-start failover has been requested by the DBMS_DG.INITIATE_FS_FAILOVER function.

   If the primary database is an Oracle Real Application Clusters (Oracle RAC) database, the master observer will attempt to connect to one of the remaining primary instances. Fast-start failover will not occur unless all instances comprising the Oracle RAC primary database are perceived to have failed. The master observer uses the value specified by either the DGConnectIdentifier or ObserverConnectIdentifier database properties to connect to the primary and fast-start failover target standby databases. The value specified for either of these properties should allow the master observer to connect to any instance of an Oracle RAC database.

3. Verify the target standby database is ready for failover.

   If fast-start failover is initiated, the master observer verifies the target standby database is ready to fail over to the primary database role.

   Fast-start failover cannot occur if:

   • Fast-start failover is no longer enabled

   • The master observer cannot connect to the target standby database

     See Also:

     What Happens if the Observer Fails? if the observer is not running

   • The master observer and the target standby database are inconsistent with regard to the current state of the broker configuration

   • The master observer is not running

   • If the protection mode is maximum availability or maximum protection and the target standby database was not synchronized with the primary database at the time the primary database failed

   • If the protection mode is maximum performance and the apply point of the target standby database lags the redo generation point of the primary database by more than the amount specified by the FastStartFailoverLagLimit configuration property at the time the primary database failed

   • The target standby database has contact with the primary database. However, failover is attempted if the ObserverOverride configuration property is set to TRUE.

   • The FS_FAILOVER_STATUS column in the V$DATABASE view for the target standby database displays a reason why fast-start failover cannot occur

   • A manual failover is already in progress. See Manual Failover for complete information about manual failovers.

   • The primary database was shut down without using the ABORT option

4. Initiate a fast-start failover.

   If the target standby database is ready for failover, then the master observer immediately directs the target standby database to fail over to the primary database role. If failover is not possible for some reason, then the master observer will continue checking whether the standby database is ready to fail over. But it will also continue trying to reconnect to the primary database indefinitely. If it reconnects to the primary database before the standby agrees to fail over, then the master observer will stop attempting to initiate a fast-start failover.

5. Reinstate the former primary database as a new standby database.

   After the fast-start failover completes successfully, the master observer will attempt to reinstate the former primary database as a new standby database when a connection to the former primary database is reestablished, and the FastStartFailoverAutoReinstate configuration property is set to TRUE. If the FastStartFailoverPmyShutdown configuration property is set to TRUE, then the former primary database will have been automatically shut down and must be manually restarted before the master observer can attempt to reinstate it.

   Note that these properties only affect whether primary shutdown and automatic reinstatement are performed if a fast-start failover occurs because the primary crashed or was isolated from the observer and target standby database.

   See Also:

   Reinstating the Former Primary Database in the Broker Configuration for more information about reinstatement

This list describes restrictions when fast-start failover is enabled.

• Change:

  • The configuration protection mode

  • The redo transport mode used to send redo to the target standby database or the database currently in the primary role

  • The FastStartFailoverTarget configuration property on the primary unless the new property value contains the current fast-start failover target.

  • The RedoRoutes property on the primary if the new value would result in the primary not being able to ship redo to the current fast-start failover target standby. (It is permissible to change the RedoRoutes property on all standby databases including target standby databases.)

  • The RedoRoutes property on a far sync instance if it is being used to receive redo from the primary database and ship redo to the target standby database

• Disable or delete:

  • The broker configuration

  • The standby database that is the target of fast-start failover

  • A far sync instance if it is being used to receive redo from the primary database and ship redo to the target standby database

• Perform a manual failover:

  • Unless the conditions listed in Performing Manual Role Changes When Fast-Start Failover Is Enabled have been met

  • To a standby database that is not configured as the fast-start failover target

    To determine if the configuration is ready for fast-start failover to occur, issue the DGMGRL SHOW DATABASE <target-standby-database> command, or query the V$DATABASE view on either the primary or target standby databases. The column value for V$DATABASE.FS_FAILOVER_STATUS will be SYNCHRONIZED in a configuration operating in maximum availability mode, and it will be TARGET UNDER LAG LIMIT in a configuration operating in maximum performance mode when ready to fast-start failover. The FS_FAILOVER_OBSERVER_PRESENT column displays YES for the target standby database.

• Perform a switchover to a standby database that is not configured as the fast-start failover target

• Perform a switchover to the target standby database in a configuration operating in maximum availability mode, unless the standby database is synchronized with the primary database

• Perform a switchover to the target standby database in a configuration operating in maximum performance mode, unless the standby database is within the lag limit of the primary database

• Attempt to open the primary database, or the following error may be returned:

  ORA-16649: possible failover to another database prevents this database from being opened
  

  This error may return if the fast-start failover validity check fails or does not complete in under two minutes.

• Use the SQL ALTER DATABASE MOVE DATAFILE command to rename or relocate an online data file on a physical standby that is a fast-start failover target if the standby is mounted, but not open.

Fast-start failover will not be triggered if the primary or standby database is shut down normally.

A normal shutdown uses SHUTDOWN NORMAL, SHUTDOWN IMMEDIATE, or SHUTDOWN TRANSACTIONAL. A normal shutdown prevents a fast-start failover until the primary database and standby database are connected and communicating again.

If fast-start failover is enabled you can still perform a switchover or a manual failover as long as certain conditions are met.

The conditions are as follows:

• The role change is directed to the same standby database that was specified for the FastStartFailoverTarget database property on the primary database.

• The target standby database is synchronized with the primary database if it is a configuration operating in maximum availability or maximum protection mode, or the target standby database is within the lag limit if it is a configuration operating in maximum performance mode.

• For manual failover, the observer is started and communicating with the target standby database. You must ensure that the primary database is shut down prior to performing a manual failover.

You can query the V$DATABASE view to verify that the observer is started and the configuration is ready for fast-start failover.

When querying the V$DATABASE view, pay special attention to the following:

• The FS_FAILOVER_STATUS column, which can contain the values described in Table 5-1. Note that if the V$DATABASE.FS_FAILOVER_STATUS column has a value of DISABLED, then any values returned for the remaining columns related to fast-start failover (V$DATABASE.FS_FAILOVER_*) become irrelevant.

• The FS_FAILOVER_OBSERVER_PRESENT column, which indicates whether the observer is running and actively pinging the database.

You can query the V$FS_FAILOVER_STATS view on the primary database to display statistics about fast-start failovers that have occurred on the system.

It can be useful to perform such queries because fast-start failovers are fully automated and can occur at any time. Some of the statistics that can be monitored are as follows:

• LAST_FAILOVER_TIME that shows the timestamp of last fast-start failover

• LAST_FAILOVER_REASON that shows the reason for the last fast-start failover

The following is an example of querying the V$FS_FAILOVER_STATS view:

SQL> SELECT LAST_FAILOVER_TIME, LAST_FAILOVER_REASON FROM V$FS_FAILOVER_STATS;
 
LAST_FAILOVER_TIME
--------------------
LAST_FAILOVER_REASON
------------------------------------------------------------------------------------------------------------------------------------
02/13/2007 16:53:10
Primary Disconnected

Observers should be installed and run on a computer system that is separate from the primary and standby systems.

Installing and starting an observer is an integral part of using fast-start failover and is described in detail in the following sections:

• Oracle Data Guard Installation explains that you can either install only the Oracle Client Administrator or you can install the complete Oracle Database Enterprise Edition or Personal Edition on the observer system.

• Enabling Fast-Start Failover describes how to start observers as a part of the step-by-step process to enable fast-start failover. Examples of starting observers using DGMGRL are included in Scenario 6: Enabling Fast-Start Failover and Starting the Observer.

To start an observer, you must be able to log in to DGMGRL with an account that has the SYSDG or SYSDBA privilege. An observer is an OCI client that connects to the primary and target standby databases using the same SYS credentials you used when you connected to the Oracle Data Guard configuration with DGMGRL.

DGMGRL> SET MASTEROBSERVER TO boston-obsever;
Succeeded.

DGMGRL> START OBSERVER IN BACKGROUND 
FILE IS  /net/sales/dat/oracle/broker/fsfo.dat 
LOGFILE IS /net/sales/dat/oracle/broker/observer.log 
CONNECT IDENTIFIER IS sales_p

Submitted command "START OBSERVER" using connect identifier "sales_p" 

DGMGRL>

% dgmgrl
DGMGRL> CONNECT /@primary1;
DGMGRL> START OBSERVER FILE IS '/u01/oracle/dbs/config1.dat' 
LOGFILE IS '/u01/oracle/log/config1.log'

% dgmgrl
DGMGRL> CONNECT /@primary2;
DGMGRL> START OBSERVER FILE IS '/u01/oracle/dbs/config2.dat' 
LOGFILE IS '/u01/oracle/log/config2.log'

You can find information about the master observer by querying the V$DATABASE view.

The behavior of the broker if the master observer fails depends on whether the broker configuration has one observer or multiple observers.

If there is only one observer, then it is considered to be the master observer. If there are multiple observers, then only one of them is the master observer.

ORA-16658: unobserved fast-start failover configuration
ORA-16820: fast-start failover observer is no longer observing this database

The ObserverOverride and ObserverReconnect properties allow you additional control over the connection to the primary.

When the observer loses its connection to the primary database for a period of time greater than that specified by the FastStartFailoverThreshold property, it attempts a failover to the standby database. However, if the standby has had contact from the primary within the period of time specified by the FastStartFailoverThreshold property, the standby prevents the failover attempt.

To override this behavior and allow a fast-start failover to occur if the observer is unable to contact the primary for more than FastStartFailoverThreshold seconds, set the ObserverOverride property to TRUE. For example:

DGMGRL> EDIT CONFIGURATION SET PROPERTY ObserverOverride=TRUE;

Ordinarily the observer connects once to the primary and does not attempt to reconnect unless the connection has failed. However, if you want the observer to reconnect to the primary database periodically as a means of testing the health of the network connection to the primary, then use the ObserverReconnect configuration property. This specifies how often the observer establishes a new connection to the primary database. In the following example, ObserverReconnect is set to 30 seconds. This results in the observer establishing a new connection to the primary database every 30 seconds.

DGMGRL> EDIT CONFIGURATION SET PROPERTY ObserverReconnect=30;

You can manually stop a specific observer or all observers.

DGMGRL> STOP OBSERVER observer_name;

An observer can be moved from one computer to another through a process of stopping it on one system and and re-starting it on another.

1. Stop the observer from any computer system in the broker configuration, as described in Stopping the Observer.
2. Start the observer on the new computer system, as described in Step 8 of Enabling Fast-Start Failover.

The observer persistently maintains information about the fast-start failover configuration in a binary file created in the working directory where you started the observer.

By default, the observer creates this file in the current working directory when it is started and names the file fsfo.dat. This file contains connect identifiers to both the primary and the target standby databases.

Ensure this file cannot be read by unauthorized users.

Once the observer is started, you cannot change the file's name and location. However, you can change the name or the location of the file if you start the observer using the DGMGRL START OBSERVER command and include the FILE IS qualifier. See the START OBSERVER command for more information.

To patch an environment where the Observer is running and fast-start failover is enabled, follow these steps prior to applying the patch.

1. Enable fast-start failover using the DGMGRL ENABLE FAST_START FAILOVER command. The primary and target standby must have connectivity for this command to complete successfully.
2. Start the observer using the DGMGRL START OBSERVER command. If the configuration has multiple observers, then be sure to specify the name of the observer whose environment has just been patched (START OBSERVER observer-name).

Reinstatement is supported only after failover in a broker configuration. It also requires Flashback Database to be enabled on both the primary and target standby databases.

Prerequisites for Enabling Fast-Start Failover provides complete information about all of the fast-start failover and reinstatement requirements.

This list describes conditions in which the broker cannot automatically reinstate the former primary database.

• A fast-start failover occurred because a user-configurable condition was detected or was requested by an application by calling the DBMS_DG.INITIATE_FS_FAILOVER function.

• FastStartFailoverAutoReinstate is set to FALSE

• Another failover or switchover occurred after the fast-start failover completed but before the former primary database restarted

• Fast-start failover was disabled

• The master observer cannot connect to the former primary database

• The former primary database cannot connect to the new primary database

• The former primary database and the new primary database are not configured in the same fast-start failover environment

• The former primary database was disabled because of a manual failover when fast-start failover was disabled

  Note:

  Standby databases that are disabled during switchover, manual failover, or fast-start failover will not be automatically reinstated.

If automatic reinstatement fails, the broker will log errors and the former primary database will remain in the mounted state. At this point, you can either:

• Disable fast-start failover (described in Disabling Fast-Start Failover) and attempt to open the former primary database

• Manually reinstate the former primary database, as described in Reenabling Disabled Databases After a Role Change

If a failure occurs once a reinstatement operation (automatic or manual) is underway, the broker logs the appropriate information in the broker configuration files and broker log files.

The former primary database is disabled. Most in-progress failures cannot be restarted (for example, archived redo log file corruption on the primary database). You must manually re-create the database as a standby database and then reenable it.

For Fast Connection Failover (FCF) to occur, a client must be able to locate the new primary database after a failover.

1. Configure the connect descriptor for connect-time failover. Add the primary database and each standby database to the address list. Set the CONNECT_TIMEOUT parameter to a small value to minimize the delay experienced if a network address is not available. Increase the value of this parameter if resource contention causes connection timeouts to occur during normal operation.

   For example:

   sales =
   	  (DESCRIPTION= 
   	    (FAILOVER=ON)
   	    (CONNECT_TIMEOUT=5)
   	    (ADDRESS_LIST=
   	      (ADDRESS=(HOST=boston-scan)(PORT=1521))
   	      (ADDRESS=(HOST=dallas-scan)(PORT=1521)))
   	    (CONNECT_DATA=(SERVICE_NAME=sales)))
   

2. Configure the connect descriptor with a single network name that is registered with a global naming service such as DNS or LDAP. Create a trigger based on the DB_ROLE_CHANGE system event that changes the network address associated with the network name to the network address of the new primary database after a failover.

   See Also:

   • Oracle Database PL/SQL Language Reference for more information about the DB_ROLE_CHANGE system event

Database services can be configured to be active in specific database roles on Oracle RAC databases and on single-instance databases managed by Oracle Restart.

The broker interacts with Oracle Clusterware or Oracle Restart to ensure that the appropriate database services are active and that the appropriate FAN events are published after a role change.

FAN events are always published through ONS. However, the event notifying a failover is only published for database services that have been configured to be active while the database is in the primary role on the new primary database.

Services that must be active in any given database role (primary, physical standby, logical standby, or snapshot standby) must be configured with the Server Control utility (SRVCTL) explicitly on each database where the service must be active.

In the following example commands, a service named PAYROLL is configured to be active in the PRIMARY role on the primary database NORTH. The service is then configured to be active in the PRIMARY role on the standby database SOUTH, so that it will be active on that database after a role transition. In these sample commands, the ellipse (...) signifies any other add service options you wish to supply.

On primary database NORTH, execute the following:

srvctl add service –db NORTH –service PAYROLL –role PRIMARY ...

On standby database SOUTH, execute the following:

srvctl add service –db SOUTH –service PAYROLL –role PRIMARY ...

Services that are to be active while the database is in the physical standby role must also be created and started on the current primary database regardless of whether the service will be started on that database or not. This is to ensure that the service definition gets propagated to the physical standby database via the redo stream and thus allows for the service to be started on the physical standby database. The service can be started on the physical standby only after the redo generated by starting the service has been applied. It is important that all SRVCTL add service options be identical on all the databases so that the services behave the same way before and after a role change.

If all the databases do not have the same values, SRVCTL attempts to override the values, which will fail on the physical standby database because it is open read-only. In the following example, a service named sales is configured to be active in the PHYSICAL_STANDBY role on the primary database NORTH. It is then started and stopped on the primary database. It could optionally also be removed from the primary database if there is no intention to ever run this service on the current primary database. It is then configured to be active in the PHYSICAL_STANDBY role on the physical standby database SOUTH.

Execute the following on primary database NORTH:

srvctl add service -db NORTH -service SALES -role PHYSICAL_STANDBY ...

srvctl start service –db NORTH –service SALES

srvctl stop service –db NORTH –service SALES

Execute the following on the physical standby database SOUTH:

srvctl add service -db SOUTH -service SALES -role PHYSICAL_STANDBY ...

If the broker now performs a switchover or failover, it automatically starts the SALES service on the correct database, based on the database's role.

The previous examples dealt with setting up only one service on a database. The following example shows you how to set up more than one service on a database and how using the broker ensures that the correct service starts on the correct database.

Suppose you have a primary database, BOSTON, and a standby database, CHICAGO. Issue the following SRVCTL commands so that both databases in the Data Guard configuration know about the two potential services for each database:

On BOSTON

srvctl add service -db BOSTON -service SALESRW  -role PRIMARY -y AUTOMATIC
srvctl add service -db BOSTON  -service SALESRO  -role PHYSICAL_STANDBY -policy AUTOMATIC

On CHICAGO:

srvctl add service -db CHICAGO  -s SALESRW  -role PRIMARY -y AUTOMATIC
srvctl add service -db CHICAGO  -s SALESRO  -role PHYSICAL_STANDBY -policy AUTOMATIC

To start things up initially, you must manually start the services on the right node. You must also start and stop the SALESRO service on the primary so that it can be started on the standby. Issue the following SRVCTL commands:

On BOSTON:

srvctl start service -db BOSTON -service SALESRW
srvctl start service -db BOSTON -service SALESRO
srvctl stop service -db BOSTON -service SALESRO

On CHICAGO:

srvctl start service -db CHICAGO -service SALESRO

Now the correct services are running on the correct databases.

If the broker performs a switchover or failover, then it starts the service SALESRW or SALESRO based on the current role of the database. So SALESRW will start on CHICAGO (which is now the primary) and SALESRO will start on BOSTON (which is now the physical standby). The same thing happens if a shutdown and startup of either database occurs - the service that is started is the one that matches the role of the database being started.

Be aware that if you issue the following manual commands on either of those databases, then both the SALESRO and SALESRW services would be started on the databases regardless of what you may have earlier specified with the SRVCTL -role qualifier. This is because the -role qualifier is taken into account only by Data Guard broker, and at database startup.

On BOSTON:

srvctl start service -db CHICAGO

On CHICAGO:

srvctl start service -db BOSTON

The SRVCTL utility does not automatically take the database role into account, so any time you start a service manually, you must specify the name(s) of the service you want started.

If client-side ONS configuration is used, the client-side ONS configuration file must specify the hostname and port of the ONS daemon(s) of the primary database and each standby database.

If the client uses remote ONS subscription, the client must specify the hostname and port of the ONS daemon(s) of the primary database and each standby database.

Application Continuity is an Oracle Database feature that enables rapid and nondisruptive replays of requests against the database after a recoverable error that made the database session unavailable.

Application Continuity is supported for Oracle Data Guard switchovers to physical standby databases. It is also supported for fast-start failover to physical standbys in maximum availability data protection mode. Note that primary and standby databases must be licensed for Oracle RAC or Oracle Active Data Guard in order to use Application Continuity.