Fleet Patching and Provisioning (FPP) is a software lifecycle management method for provisioning and maintaining Oracle homes. Fleet Patching and Provisioning enables mass deployment and maintenance of standard operating environments for databases, clusters, and user-defined software types.

Fleet Patching and Provisioning enables you to install clusters, and provision, patch, scale, and upgrade Oracle Grid Infrastructure, Oracle Restart, and Oracle Database homes. The supported versions are 11.2, 12.1, 12.2, 18c, and 19c. You can also provision applications and middleware using Fleet Patching and Provisioning.

Fleet Patching and Provisioning is a service in Oracle Grid Infrastructure that you can use in either of the following modes:

• Central Fleet Patching and Provisioning Server

  The Fleet Patching and Provisioning Server stores and manages standardized images, called gold images. Gold images can be deployed to any number of nodes across the data center. You can create new clusters and databases on the deployed homes and can use them to patch, upgrade, and scale existing installations.

  The Fleet Patching and Provisioning Server can manage the following types of installations:

  • Software homes on the cluster hosting the Fleet Patching and Provisioning Server itself.

  • Fleet Patching and Provisioning Clients running Oracle Grid Infrastructure 12c Release 2 (12.2), 18c, and 19c.

  • Installations running Oracle Grid Infrastructure 11g Release 2 (11.2) and 12c Release 1 (12.1).

  • Installations running without Oracle Grid Infrastructure.

  The Fleet Patching and Provisioning Server can provision new installations and can manage existing installations without requiring any changes to the existing installations. The Fleet Patching and Provisioning Server can automatically share gold images among peer servers to support enterprises with geographically distributed data centers.

• Fleet Patching and Provisioning Client

  The Fleet Patching and Provisioning Client can be managed from the Fleet Patching and Provisioning Server, or directly by executing commands on the client itself. The Fleet Patching and Provisioning Client is a service built into the Oracle Grid Infrastructure and is available in Oracle Grid Infrastructure 12c Release 2 (12.2) and later releases. The Fleet Patching and Provisioning Client can retrieve gold images from the Fleet Patching and Provisioning Server, upload new images based on the policy, and apply maintenance operations to itself.

Fleet Patching and Provisioning has several methods to provision and, optionally, configure Oracle Grid Infrastructure and Oracle Restart grid infrastructure homes.

• To inform Fleet Patching and Provisioning the nodes on which to install Oracle Grid Infrastructure, and to configure Oracle Grid Infrastructure, you provide directions in a response file, as in the following example:

  $ rhpctl add workingcopy -workingcopy GI_HOME_11204_WCPY -image GI_HOME_11204 –responsefile /u01/app/rhpinfo/GI_11204_install.txt
    {authentication_option}

  The preceding command provisions the GI_HOME_11204_WCPY working copy based on the GI_HOME_11204 gold image to a target specified in the GI_11204_install.txt response file. In addition to identifying the target nodes, the response file specifies information about the Oracle Grid Infrastructure configuration, such as Oracle ASM and GNS parameters.

  Note:

  The oracle.install.crs.rootconfig.executeRootScript=xxx response file parameter is overridden and always set to false for Fleet Patching and Provisioning, regardless of what you specify in the response file.
• To provision an Oracle Grid Infrastructure home to a node or cluster that is currently running Oracle Grid Infrastructure:

  $ rhpctl add workingcopy -workingcopy GI_HOME_12201_PATCHED_WCPY -image GI_HOME_12201_PSU1 –client CLUST_002 -softwareonly

  The preceding command provisions a new working copy based on the GI_HOME_12201_PSU1 gold image to the Fleet Patching and Provisioning Client (that is running Oracle Grid Infrastructure 12c release 2 (12.2)) named CLUST_002. When you provision to a target that is not running Oracle Grid Infrastructure 12c release 2 (12.2) (such as, a target running Oracle Grid Infrastructure 12c release 1 (12.1) or Oracle Grid Infrastructure 11g release 2 (11.2)), use the -targetnode parameter instead of -client.

• Specify a target node on which you want to provision an Oracle Restart grid infrastructure, as follows:

  $ rhpctl add workingcopy -workingcopy SIHA_GI -image goldimage -targetnode remote_node_name -resposnefile Oracle_Restart_resposne_file {authentication_option}

Fleet Patching and Provisioning provides three methods to patch Oracle Grid Infrastructure software homes: rolling, non-rolling, and in batches.

When you install Oracle Grid Infrastructure or when you upgrade an older version to this current version, the Fleet Patching and Provisioning Server is configured automatically in local mode.

$ srvctl enable rhpserver
$ srvctl start rhpserver

$ srvctl stop rhpserver
$ srvctl remove rhpserver

• The independent automaton for patching Oracle Grid Infrastructure performs all of the steps necessary to switch from one home to another. Because the automaton is not aware of gold images, moving the database requires two home paths, as follows:

  $ rhpctl move gihome –sourcehome Oracle_home_path -destinationhome Oracle_home_path

Fleet Patching and Provisioning has error prevention and automated recovery options to assist you during maintenance operations.

During maintenance operations, errors must be avoided whenever possible and, when they occur, you must have automated recovery paths to avoid service disruption.

If you are using Fleet Patching and Provisioning, then you can use a single command to upgrade an Oracle Grid Infrastructure home.

$ rhpctl upgrade gihome -sourcewc GIOH11204 -destwc GIOH12201

Use the rhpctl add workingcopy command to provision a working copy of a database home on a Fleet Patching and Provisioning Server, Client, or target.

• Run the rhpctl add workingcopy command on a Fleet Patching and Provisioning Server, similar to the following example:

  $ rhpctl add workingcopy -image db12c -path /u01/app/dbusr/product/12.2.0/db12201
    -client client_007 -oraclebase /u01/app/dbusr/ -workingcopy wc_db122_1

  The preceding command example creates a working copy named wc_db122_1 on all nodes of the Fleet Patching and Provisioning Client cluster named client_007. The gold image db12c is the source of the workingcopy. The directory path locations that you specify in the command must be empty.

Create an Oracle Database on a working copy.

• After you create a working copy of a gold image and provision that working copy to a target, you can create an Oracle Database on the working copy using the rhpctl add database command, similar to the following command example, which creates an Oracle Real Application Clusters (Oracle RAC) database called db12201 on a working copy called wc_db122_1:

  $ rhpctl add database –workingcopy wc_db122_1 –dbname db12201 -node client_007_node1,client_007_node2 -dbtype RAC -datafileDestination DATA007_DG

To patch an Oracle database, you move the database home to a new home, which includes the patches you want to implement.

1. Create a working copy of the Oracle Database that you want to patch, in this case DB122.
2. Apply the patch to the working copy you created.
3. Test and validate the patched working copy.
4. Use the rhpctl add image command to create a gold image (for example, DB122_PATCH) from the patched working copy.

   Note:

   The working copy you specify in the preceding command must be hosted on the Fleet Patching and Provisioning Server in Fleet Patching and Provisioning-managed storage.
5. Delete the patched working copy with the patched Oracle Database using the rhpctl delete workingcopy command.

   Note:

   Do not remove directly using the rm command or some other method, because this does not update the Fleet Patching and Provisioning inventory information.
6. Create a working copy from the patched gold image, (DB122_PATCH).
7. Move myorcldb to the working copy you just created.
8. When you are confident that you will not need to roll back to the working copy on which the database was running at the beginning of the procedure, delete that working copy using the rhpctl delete workingcopy command.

The independent local-mode automaton updates Oracle Database homes, including Oracle Database single-instance databases in a cluster or standalone (with no Oracle Grid Infrastructure), an Oracle RAC database, or an Oracle RAC One Node database.

• The independent automaton for Oracle Database patching performs all of the steps necessary to switch from one home to another. Because the automaton is not aware of gold images, moving the database requires two home paths, as follows:

  $ rhpctl move database –sourcehome Oracle_home_path -destinationhome Oracle_home_path

In addition to Oracle Grid Infrastructure and Oracle Database homes, Fleet Patching and Provisioning supports patching the Oracle Exadata components: database nodes, storage cells, and InfiniBand switches.

1. Import an image of the EXAPATCHSOFTWARE image type, using the rhpctl import image command, similar to the following:

   $ rhpctl import image -image EXA1 -imagetype EXAPATCHSOFTWARE -path /tmp/ExadataPatchBundle
     -version 12.1.2.2.3.160720

   Note:

   You must rename the patch zip files to include the words storage, database and, iso, so that Fleet Patching and Provisioning can distinguish among them.
2. If this is the first time you are using Fleet Patching and Provisioning to patch Oracle Exadata, then use the rhpctl add workingcopy command, as follows:

   $ rhpctl add workingcopy -image image_name -root {[-dbnodes dbnode_list]
     [-cells cell_list] [-ibswitches ibswitch_list]} [-fromnode node_name]
     [-unkey] [-smtpfrom "address"] [-smtpto "addresses"] [-precheckonly]
     [-modifyatprereq] [-resetforce] [-force]

   The preceding command stores the list of nodes (database nodes, cells, and InfiniBand switches) and the version of each node in Fleet Patching and Provisioning on the working copy, in addition to the type of node and the type of image.

3. For subsequent patching operations, after you create a gold image with updated database, storage cell, and switch files, run the rhpctl update workingcopy command to patch one, two, or all three Oracle Exadata components, as follows:

   $ rhpctl update workingcopy -image image_name -root {[-dbnodes dbnode_list] [-cells cell_list]
     [-ibswitches ibswitch_list]} [-fromnode node_name] [-unkey] [-smtpfrom "address"]
     [-smtpto "addresses"] [-precheckonly] [-modifyatprereq] [-resetforce] [-force]

   Note:

   The name of the working copy remains the same throughout the life cycle of patching the given Oracle Exadata target.

   You can choose to patch only the database nodes, cells, or InfiniBand switches or any combination of the three. Patching occurs in the following order: InfiniBand switches, cells, database nodes.

4. Display the list of nodes and their images, as follows:

   $ rhpctl query workingcopy -workingcopy working_copy_name

5. Delete the working copy, as follows:

   rhpctl delete workingcopy -workingcopy working_copy_name

Fleet Patching and Provisioning provides two options for upgrading Oracle Database. Both options are performed with a single command.

Run a zero-downtime upgrade using Oracle GoldenGate for replication.

• You can run the upgrade command on the Fleet Patching and Provisioning Server to upgrade a database hosted on the server, an Oracle Database 12c release 1 (12.1.0.2) target cluster, or a database hosted on a Fleet Patching and Provisioning Client 12c release 2 (12.2.0.1) or 18c. You can also run the command a Fleet Patching and Provisioning Client 18c to upgrade a database hosted on the client, itself.
  Use the upgrade command similar to the following:

  $ rhpctl zdtupgrade database -dbname sierra -destwc DB_Wcopy_121 -ggsrcwc
    GG_Wcopy_11g -ggdstwc GG_Wcopy_12c -targetnode 12102_cluster_node -root

  In the preceding command, 12102_cluster_node refers to the Oracle Grid Infrastructure 12c release 1 (12.1.0.2) cluster hosting the database you want to upgrade.

Run a zero-downtime upgrade using Oracle Data Guard for replication.

• The following example of a zero-downtime upgrade using Oracle Data Guard upgrades an Oracle Database 11g release 2 (11.2.0.4), sierra, running on the target cluster, which includes a node, targetclust003, to an Oracle Database 12c release 1 (12.1.0.2) (the destination working copy, which was provisioned from a Gold Image stored on the Fleet Patching and Provisioning Server named rhps.example.com):

  $ rhpctl zdtupgrade database -dbname sierra -destwc WC121DB4344 -clonedatadg DBDATA -targetnode node90743 -root
  
  Enter user "root" password:
  node90753.example.com: starting zero downtime upgrade operation ...
  node90753.example.com: verifying patches applied to Oracle homes ...
  node90753.example.com: verifying if database "sierra" can be upgraded with zero downtime ...
  node90743: 15:09:10.459: Verifying whether database "sierra" can be cloned ...
  node90743: 15:09:10.462: Verifying that database "sierra" is a primary database ...
  node90743: 15:09:14.672: Verifying that connections can be created to database "sierra" ...
  < ... >
  node90743: 15:14:58.015: Starting redo apply ...
  node90743: 15:15:07.133: Configuring primary database "sierra" ...
  ####################################################################
  node90753.example.com: retrieving information about database "xmvotkvd" ...
  node90753.example.com: creating services for snapshot database "xmvotkvd" ...
  ####################################################################
  node90743: 15:15:33.640: Macro step 1: Getting information and validating setup ...
  < ... >
  node90743: 15:16:02.844: Macro step 2: Backing up user environment in case upgrade is aborted  ...
  node90743: 15:16:02.848: Stopping media recovery for database "xmvotkvd" ...
  node90743: 15:16:05.858: Creating initial restore point "NZDRU_0000_0001" ...
  < ... >
  node90743: 15:16:17.611: Macro step 3: Creating transient logical standby from existing physical standby ...
  node90743: 15:16:18.719: Stopping instance "xmvotkvd2" of database "xmvotkvd" ...
  node90743: 15:16:43.187: Verifying that database "sierra" is a primary database ...
  < ... >
  node90743: 15:19:27.158: Macro step 4: Upgrading transient logical standby database ...
  node90743: 15:20:27.272: Disabling service "sierrasvc" of database "xmvotkvd" ...
  node90743: 16:36:54.684: Macro step 5: Validating upgraded transient logical standby database ...
  node90743: 16:37:09.576: Creating checkpoint "NZDRU_0301" for database "xmvotkvd" during stage "3" and task "1" ...
  node90743: 16:37:09.579: Stopping media recovery for database "xmvotkvd" ...
  node90743: 16:37:10.792: Creating restore point "NZDRU_0301" for database "xmvotkvd" ...
  node90743: 16:37:11.998: Macro step 6: Switching role of transient logical standby database ...
  node90743: 16:37:12.002: Verifying that database "sierra" is a primary database ...
  < ... >
  node90743: 16:39:07.425: Macro step 7: Flashback former primary database to pre-upgrade restore point and convert to physical standby ...
  node90743: 16:39:08.833: Stopping instance "sierra2" of database "sierra" ...
  < ... >
  node90743: 16:41:17.138: Macro step 8: Recovering former primary database ...
  node90743: 16:41:19.045: Verifying that database "sierra" is mounted ...
  < ... >
  node90743: 17:20:21.378: Macro step 9: Switching back ...
  < ... >
  ####################################################################
  node90753.example.com: deleting snapshot database "xmvotkvd" ...

You can customize zero-downtime upgrades using the user-action framework of Fleet Patching and Provisioning.

• To register a plugin to be run during a zero-downtime upgrade, run the following command:

  $ rhpctl add useraction -useraction user_action_name -actionscript script_name
    {-pre | -post} -optype {ZDTUPGRADE_DATABASE | ZDTUPGRADE_DATABASE_SNAPDB |
     ZDTUPGRADE_DATABASE_DBUA | ZDTUPGRADE_DATABASE_SWITCHBACK}

  You can specify run-time input to the plugins using the -useractiondata option of the rhpctl zdtupgrade database command.

Provision Oracle Grid Infrastructure software on two nodes that do not currently have a Grid home, and then configure Oracle Grid Infrastructure to form a multi-node Oracle Grid Infrastructure installation.

• From the Fleet Patching and Provisioning Server, run the command:

  $ rhpctl add workingcopy -workingcopy GI122 -image GI_HOME_12201 –responsefile /u01/app/rhpinfo/GI_12201_install.rsp {authentication_option}

  GI122 is the working copy based on the image GI_HOME_12201

  /u01/app/rhpinfo/GI_12201_install.rsp is the response file location.

  Cluster Verification Utility checks for preinstallation configuration as per requirements. Fleet Patching and Provisioning configures Oracle Grid Infrastructure.

This procedure provisions Oracle Database 12c release 2 (12.2) software and creates Oracle Database instances.

1. From the Fleet Patching and Provisioning Server, provision the Oracle Database home software:

   $ rhpctl add workingcopy -image db12201 -path /u01/app/dbusr/product/12.2.0/db12201
     -client client_001 -oraclebase /u01/app/dbusr/ -workingcopy db122 

   The command provisions the working copy db122 to the specified path on the cluster client_001, from the image db12201 .
2. Create the database instance:

   $ rhpctl add database -workingcopy db122 -dbname db -dbtype RAC

   The command creates an Oracle RAC database instance db . You can use the add database command repeatedly to create more instances on the working copy.

You can provision a pluggable database (PDB) on an existing container database (CDB) running in a provisioned database working copy.

• The following command example creates a PDB called pdb183 on a CDB called raccdb183, which is on a working copy called wc_db183:

  $ rhpctl addpdb database -workingcopy wc_db183 -cdbName raccdb183 -pdbName pdb183

• Use the rhpctl deletepdb database command to delete a PDB from an existing CDB on a working copy.
  The following command example deletes a PDB called pdb183 on a CDB called raccdb183, which is on a working copy called wc_db183:

  $ rhpctl deletepdb database -workingcopy wc_db183 -cdbName raccdb183 -pdbName pdb183

This procedure uses Fleet Patching and Provisioning to upgrade your Oracle Grid Infrastructure cluster from 11g release 2 (11.2.0.4) to 12c release 2 (12.2).

1. Provision a working copy of the Oracle Grid Infrastructure 12c release 2 (12.2.0.1) software:

   $ rhpctl add workingcopy -workingcopy GI122 -image GI_HOME_12201 {authentication_option}

   GI122 is the working copy based on the image GI_HOME_12201.

2. Upgrade your target cluster to the GI122 working copy:

   rhpctl upgrade gihome -sourcewc GI11204 -destwc GI122

   Rapid Home Provisioning identifies the cluster to upgrade based on the name of the source working copy, and upgrades to the working copy GI122.

This procedure explains how to patch Oracle Grid Infrastructure without changing the Grid home path.

• To move a non-OLFS-compliant Grid home to an OLFS Grid home, from the Fleet Patching and Provisioning Server, run two commands similar to the following:

  $ rhpctl add workingcopy -workingcopy GI_HOME_12201_PSU1 -aupath
    /u01/app/orabase/product/12.2.0.1/aupath -image image_name
    -client client_name -softwareonly

  $ rhpctl move gihome -srcwc GI_HOME_12201 -destwc GI_HOME_12201_PSU1 -agpath
    /u01/app/orabase/product/12.2.0.1/agpath -path
    /u01/app/orabase/product/12.2.0.1/grid

• To move an OLFS-compliant Grid home to a patched version, run two commands similar to the following:

  $ rhpctl add workingcopy -workingcopy gihome12201_psu1_lpm -aupath
    /u01/app/orabase/product/12.2.0.1/aupath -image image_name -client
    client_name -softwareonly

  $ rhpctl move gihome -sourcewc gihome12201_lpm -destwc gihome12201_psu1_lpm

This procedure patches Oracle Grid Infrastructure and Oracle Databases on the cluster to the latest patch level without cluster downtime.

1. Prepare target Oracle homes as follows:
   1. Provision software-only Grid home on the cluster to be patched:

      $ rhpctl add workingcopy -workingcopy GI_HOME_12201_PATCHED_WCPY 
        -image GI_HOME_12201_PSU1 –client CLUSTER_005 -softwareonly

   2. Provision each release Database home, without database instances, to be patched:

      $ rhpctl add workingcopy -workingcopy DB_HOME_12201_PATCHED_WCPY 
        -image DB_HOME_12201_PSU1
      $ rhpctl add workingcopy -workingcopy DB_HOME_12102_PATCHED_WCPY 
        -image DB_HOME_12102_PSU5

2. Patch Oracle Grid Infrastructure and all Oracle RAC Databases on node1 as follows:

   $ rhpctl move gihome -sourcewc GI_HOME_12201_WCPY -destwc GI_HOME_12201_PATCHED_WCPY -auto
     -dbhomes DB_HOME_12102_WCPY=DB_HOME_12102_PATCHED_WCPY,DB_HOME_12201_WCPY=DB_HOME_12201_PATCHED_WCPY  -targetnode node1 {authentication_option}

   When you run the command, you move your active Oracle Grid Infrastructure from working copy GI_HOME_12201_WCPY to GI_HOME_12201_PATCHED_WCPY, Oracle RAC Database db1 from DB_HOME_12201_WCPY to DB_HOME_12201_PATCHED_WCPY, and Oracle RAC Database db2 from DB_HOME_12102_WCPY to DB_HOME_12102_PATCHED_WCPY.

This procedure explains how to patch Oracle Database 12c release 1 (12.1.0.2) with the latest patching without bringing down the database.

1. To patch an Oracle Database home managed by Fleet Patching and Provisioning, and there exist working copies of the source and destination databases, run:

   rhpctl move database -sourcewc db12102 -patchedwc db12102_psu

   db12102 is the source working copy of the database being patched

   db12102_psu is the working copy of the Oracle Database software with patches applied, based on the image DB12102_PSU.

2. To patch an unmanaged Oracle Database home (source working copy does not exist because the Oracle home is not managed by Fleet Patching and Provisioning), run:

   rhpctl move database -sourcehome /u01/app/orabase/product/12.1.0.2/dbhome_1
    -patchedwc db12102_psu -targetnode node1

   targetnode specifies the node on which the database to be upgraded is running, because the source Oracle Database is on a 12.1.0.2 cluster.

   /u01/app/orabase/product/12.1.0.2/dbhome_1 is the path of the database being patched

   db12102_psu is the working copy of the Oracle Database software with patches applied, based on the image DB12102_PSU.

   Use the saved gold image for standardized patching of all your databases of release 12c release 1 to the same patch level.
3. If for some reason, you want to rollback the patches applied to a managed Oracle Database home, run:

   rhpctl move database -sourcewc db12102_psu 
   -patchedwc db12102 -ignorewcpatches

   db12102 is the working copy of the unpatched database to which you want to roll back.

   db12102_psu is the working copy of the Oracle Database software with patches applied, based on the image DB12102_PSU.

This procedure describes how to upgrade an Oracle database from Oracle Database 11g release 2 (11.2) to 12c release 2 with a single command, using Fleet Patching and Provisioning, both for managed and unmanaged Oracle homes.

1. From the Fleet Patching and Provisioning Server, run one of the following commands as per your source and destination database:
   1. To upgrade an Oracle home managed by Fleet Patching and Provisioning, and there exist working copies of the source and destination databases, run:

      $ rhpctl upgrade database -dbname test_database -sourcewc db112 -destwc db122
        {authentication_option}

      test_database is the name of the database being upgraded.

      db112 is the source working copy of the pre-upgrade database.

      db122 is the working copy of the upgraded Oracle Database software.

   2. To upgrade an unmanaged Oracle home (source working copy does not exist because the Oracle home is not managed by Fleet Patching and Provisioning), run:

      $ rhpctl move database -sourcehome /u01/app/orabase/product/11.2.0/dbhome_1
        -destwc db122 -targetnode node1 {authentication_option}

      /u01/app/orabase/product/11.2.0/dbhome_1 is the path of the database being upgraded.

      db122 is the working copy of the upgraded Oracle Database software.

      targetnode specifies the node on which the database to be upgraded is running, because the source Oracle Database is on a 11.2.0.4 cluster.

You can add a node to your two-node cluster by using Fleet Patching and Provisioning to add the node, and then extend an Oracle RAC database to the new node.

1. From the Fleet Patching and Provisioning Server, run the following command to add a node to the existing Oracle Grid Infrastructure working copy:

   rhpctl addnode gihome -workingcopy GI_HOME_12202_WCPY -newnodes n3:n3-vip {authentication_option}

   The command extends the cluster by adding node3.
2. Add instances to the administrator-managed Oracle RAC database on the new node:

   rhpctl addnode database -workingcopy DB_HOME_12202_WCPY -dbname db321 -node n3 {authentication_option}

   The command extends the database home on the node3 and creates database db321 on this node.

Create gold images of software home and store them on the Fleet Patching and Provisioning Server, to use later to provision Oracle homes.

1. Import an image from an installed Oracle home on the Fleet Patching and Provisioning Server:

   rhpctl import image -image db12201 -path /share/software/122/dbhome -imagetype ORACLEDBSOFTWARE 

   The gold image of imagetype Oracle Database 12c release 2 software is created and stored on the Fleet Patching and Provisioning Server.

   You can also create gold images of Oracle Grid Infrastructure or any other software by specifying -imagetype as ORACLEGISOFTWARE, ORACLEGGSOFTWARE, or SOFTWARE respectively.

2. Import an image from an installed Oracle home on a Fleet Patching and Provisioning Client by running the following command from the Fleet Patching and Provisioning Client:

   rhpctl import image -image db12201 -path /u01/app/dbusr/product/12.2.0/

   The command creates and adds the image db12201 based on the local Oracle home installed in the specified path.

You can use Fleet Patching and Provisioning user actions to perform many tasks, such as installing and configuring any type of software and running scripts.