2 Advanced Installation Oracle Grid Infrastructure for a Cluster Preinstallation Tasks

2.4.1 Determining If the Oracle Inventory and Oracle Inventory Group Exists

When you install Oracle software on the system for the first time, OUI creates the oraInst.loc file. This file identifies the name of the Oracle Inventory group (by default, oinstall), and the path of the Oracle Central Inventory directory. An oraInst.loc file has contents similar to the following:

inventory_loc=central_inventory_location
inst_group=group

In the preceding example, central_inventory_location is the location of the Oracle central inventory, and group is the name of the group that has permissions to write to the central inventory (the OINSTALL group privilege).

If you have an existing Oracle central inventory, then ensure that you use the same Oracle Inventory for all Oracle software installations, and ensure that all Oracle software users you intend to use for installation have permissions to write to this directory.

To determine if you have an Oracle Inventory on your system:

1. Enter the following command:

   # more /var/opt/oracle/oraInst.loc
   

   If the oraInst.loc file exists, then the output from this command is similar to the following:

   inventory_loc=/u01/app/oracle/oraInventory
   inst_group=oinstall
   

   In the previous output example:

   • The inventory_loc group shows the location of the Oracle Inventory

   • The inst_group parameter shows the name of the Oracle Inventory group (in this example, oinstall).

2. Ensure that Oracle Inventory group members are granted the HP-UX privileges RTPRIO, MLOCK, and RTSCHED. For example:

   # /usr/bin/getprivgrp oinstall
   oinstall: RTPRIO MLOCK RTSCHED
   

   If the group is not granted these privileges, then add these privileges as described in the next section.

2.4.2 Creating the Oracle Inventory Group If an Oracle Inventory Does Not Exist

If the oraInst.loc file does not exist, then complete the following tasks:

1. Create the Oracle Inventory group by entering a command similar to the following:

   # /usr/sbin/groupadd -g 1000 oinstall
   

   The preceding command creates the oraInventory group oinstall, with the group ID number 1000. Members of the oraInventory group are granted privileges to write to the Oracle central inventory (oraInventory).

   By default, if an oraInventory group does not exist, then the installer lists the primary group of the installation owner for the Oracle Grid Infrastructure for a Cluster software as the oraInventory group. Ensure that this group is available as a primary group for all planned Oracle software installation owners.

   Note:

   Group and user IDs must be identical on all nodes in the cluster. Check to make sure that the group and user IDs you want to use are available on each cluster member node, and confirm that the primary group for each Oracle Grid Infrastructure for a Cluster installation owner has the same name and group ID.

2. If it does not already exist, create the /etc/privgroup file. Add a line similar to the following to grant Oracle installation owners the RTPRIO, MLOCK, and RTSCHED privileges:

   oinstall RTPRIO MLOCK RTSCHED
   

   If /etc/privgroup exists, then add these privileges to the Oracle Inventory group. For example:

   # /usr/sbin/setprivgrp oinstall RTPRIO MLOCK RTSCHED
   

   Confirm the grant of privileges to the group. For example:

   # /usr/bin/getprivgrp oinstall
   oinstall: RTPRIO MLOCK RTSCHED
   

3. Repeat this procedure on all of the other nodes in the cluster.

2.4.3 Creating the Oracle Grid Infrastructure User

You must create a software owner for Oracle Clusterware in the following circumstances:

• If an Oracle software owner user does not exist; for example, if this is the first installation of Oracle software on the system

• If an Oracle software owner user exists, but you want to use a different operating system user, with different group membership, to separate Oracle Grid Infrastructure administrative privileges from Oracle Database administrative privileges.

  In Oracle documentation, a user created to own only Oracle Grid Infrastructure software installations is called the grid user. A user created to own either all Oracle installations, or only Oracle database installations, is called the oracle user.

# id oracle

uid=501(oracle) gid=501(oinstall) groups=502(dba),503(oper)

2.4.4 Creating the Oracle Base Directory Path

The Oracle base directory for the grid installation owner is the location where diagnostic and administrative logs, and other logs associated with Oracle ASM and Oracle Clusterware are stored.

If you have created a path for the Oracle Clusterware home that is compliant with Oracle Optimal Flexible Architecture (OFA) guidelines for Oracle software paths then you do not need to create an Oracle base directory. When OUI finds an OFA-compliant path, it creates the Oracle base directory in that path.

For OUI to recognize the path as an Oracle software path, it must be in the form u[00-99]/app, and it must be writable by any member of the oraInventory (oinstall) group. The OFA path for the Oracle base is /u01/app/user, where user is the name of the software installation owner.

Oracle recommends that you create an Oracle Grid Infrastructure Grid home and Oracle base homes manually, particularly if you have separate Oracle Grid Infrastructure for a cluster and Oracle Database software owners, so that you can separate log files.

For example:

# mkdir -p  /u01/app/11.2.0/grid
# mkdir -p /u01/app/grid
# mkdir -p /u01/app/oracle
# chown grid:oinstall /u01/app/11.2.0/grid
# chown grid:oinstall /u01/app/grid
# chown oracle:oinstall /u01/app/oracle
# chmod -R 775 /u01/
# chown -R grid:oinstall /u01

2.4.5 Creating Job Role Separation Operating System Privileges Groups and Users

A Job Role Separation privileges configuration of Oracle ASM is a configuration with groups and users that divide administrative access privileges to the Oracle ASM installation from other administrative privileges users and groups associated with other Oracle installations. Administrative privileges access is granted by membership in separate operating system groups, and installation privileges are granted by using different installation owners for each Oracle installation.

If you prefer, you can allocate operating system user privileges so that you can use one administrative user and one group for operating system authentication for all system privileges on the storage and database tiers.

For example, you can designate the oracle user to be the installation owner for all Oracle software, and designate oinstall to be the group whose members are granted all system privileges for Oracle Clusterware, Oracle ASM, and all Oracle Databases on the servers, and all privileges as installation owners. This group must also be the Oracle Inventory group.

Oracle recommends that you use at least two groups: A system privileges group whose members are granted administrative system privileges, and an installation owner group (the oraInventory group) to provide separate installation privileges (the OINSTALL privilege). To simplify using the defaults for Oracle tools such as Cluster Verification Utility, if you do choose to use a single operating system group to grant all system privileges and the right to write to the oraInventory, then that group name should be oinstall.

• Overview of Creating Job Role Separation Groups and Users

• Creating Database Groups and Users with Job Role Separation

# /usr/sbin/groupadd -g 1031 dba

# /usr/sbin/groupadd -g 1032 oper1

# /usr/sbin/groupadd -g 1020 asmadmin

# /usr/sbin/groupadd -g 1022 asmoper

# /usr/sbin/groupadd -g 1021 asmdba

# id oracle

uid=501(oracle) gid=501(oinstall) groups=502(dba),503(oper)

# /usr/sbin/usermod -g oinstall -G dba,asmdba oracle

2.4.6 Example of Creating Standard Groups, Users, and Paths

The following is an example of how to create the Oracle Inventory group (oinstall), and a single group (dba) as the OSDBA, OSASM and OSDBA for Oracle ASM groups. In addition, it shows how to create the Oracle Grid Infrastructure software owner (grid), and one Oracle Database owner (oracle) with correct group memberships. This example also shows how to configure an Oracle base path compliant with OFA structure with correct permissions:

# groupadd -g 1000 oinstall
# groupadd -g 1031 dba
# useradd -u 1100 -g oinstall -G dba grid
# useradd -u 1101 -g oinstall -G dba oracle
# mkdir -p  /u01/app/11.2.0/grid
# mkdir -p /u01/app/grid
# chown -R grid:oinstall /u01
# mkdir /u01/app/oracle
# chown oracle:oinstall /u01/app/oracle
# chmod -R 775 /u01/

After running these commands, you have the following groups and users:

• An Oracle central inventory group, or oraInventory group (oinstall). Members who have the central inventory group as their primary group, are granted the OINSTALL permission to write to the oraInventory directory.

• A single system privileges group that is used as the OSASM, OSDBA, OSDBA for ASM, and OSOPER for ASM group (dba), whose members are granted the SYSASM and SYSDBA privilege to administer Oracle Clusterware, Oracle ASM, and Oracle Database, and are granted SYSASM and OSOPER for ASM access to the Oracle ASM storage.

• An Oracle grid installation for a cluster owner (grid), with the oraInventory group as its primary group, and with the OSASM group as the secondary group, with its Oracle base directory /u01/app/grid.

• An Oracle Database owner (oracle) with the oraInventory group as its primary group, and the OSDBA group as its secondary group, with its Oracle base directory /u01/app/oracle.

• /u01/app owned by grid:oinstall with 775 permissions before installation, and by root after the root.sh script is run during installation. This ownership and permissions enables OUI to create the Oracle Inventory directory, in the path /u01/app/oraInventory.

• /u01 owned by grid:oinstall before installation, and by root after the root.sh script is run during installation.

• /u01/app/11.2.0/grid owned by grid:oinstall with 775 permissions. These permissions are required for installation, and are changed during the installation process.

• /u01/app/grid owned by grid:oinstall with 775 permissions before installation, and 755 permissions after installation.

• /u01/app/oracle owned by oracle:oinstall with 775 permissions.

2.4.7 Example of Creating Role-allocated Groups, Users, and Paths

The following is an example of how to create role-allocated groups and users that is compliant with an Optimal Flexible Architecture (OFA) deployment:

# groupadd -g 1000 oinstall
# groupadd -g 1020 asmadmin
# groupadd -g 1021 asmdba
# groupadd -g 1031 dba1
# groupadd -g 1041 dba2
# groupadd -g 1022 asmoper
# useradd -u 1100 -g oinstall -G asmadmin,asmdba grid
# useradd -u 1101 -g oinstall -G dba1,asmdba oracle1
# useradd -u 1102 -g oinstall -G dba2,asmdba oracle2
# mkdir -p  /u01/app/11.2.0/grid
# mkdir -p /u01/app/grid
# chown -R grid:oinstall /u01
# mkdir -p /u01/app/oracle1
# chown oracle1:oinstall /u01/app/oracle1
# mkdir -p /u01/app/oracle2
# chown oracle2:oinstall /u01/app/oracle2
# chmod -R 775 /u01

After running these commands, you have the following groups and users:

• An Oracle central inventory group, or oraInventory group (oinstall), whose members that have this group as their primary group are granted permissions to write to the oraInventory directory.

• A separate OSASM group (asmadmin), whose members are granted the SYSASM privilege to administer Oracle Clusterware and Oracle ASM.

• A separate OSDBA for ASM group (asmdba), whose members include grid, oracle1 and oracle2, and who are granted access to Oracle ASM.

• A separate OSOPER for ASM group (asmoper), whose members are granted limited Oracle ASM administrator privileges, including the permissions to start and stop the Oracle ASM instance.

• An Oracle grid installation for a cluster owner (grid), with the oraInventory group as its primary group, and with the OSASM (asmadmin), OSDBA for ASM (asmdba) group as a secondary group.

• Two separate OSDBA groups for two different databases (dba1 and dba2) to establish separate SYSDBA privileges for each database.

• Two Oracle Database software owners (oracle1 and oracle2), to divide ownership of the Oracle database binaries, with the OraInventory group as their primary group, and the OSDBA group for their database (dba1 or dba2) and the OSDBA for ASM group (asmdba) as their secondary groups.

• An OFA-compliant mount point /u01 owned by grid:oinstall before installation.

• An Oracle base for the grid installation owner /u01/app/grid owned by grid:oinstall with 775 permissions, and changed during the installation process to 755 permissions.

• An Oracle base /u01/app/oracle1 owned by oracle1:oinstall with 775 permissions.

• An Oracle base /u01/app/oracle 2 owned by oracle2:oinstall with 775 permissions.

• A Grid home /u01/app/11.2.0/grid owned by grid:oinstall with 775 (drwxdrwxr-x) permissions. These permissions are required for installation, and are changed during the installation process to root:oinstall with 755 permissions (drwxr-xr-x).

• /u01/app/oraInventory. This path remains owned by grid:oinstall, to enable other Oracle software owners to write to the central inventory.

2.4.8 Creating the External Jobs User Account for HP-UX

On the HP-UX platform, if you intend to install Oracle Database or Oracle Real Application Clusters on Oracle Grid Infrastructure, then use the following procedure to create an external jobs user account to provide a low-privilege user with which external jobs can be run:

1. Log in as root

2. Create the unprivileged user extjob. For example:

   # useradd extjob
   

2.6.1 Network Hardware Requirements

The following is a list of requirements for network configuration:

• Each node must have at least two network adapters or network interface cards (NICs): one for the public network interface, and one for the private network interface (the interconnect).

  With Redundant Interconnect Usage, you can identify multiple interfaces to use for the cluster private network, without the need of using bonding or other technologies. This functionality is available starting with Oracle Database 11g Release 2 (11.2.0.2).

  When you define multiple interfaces, Oracle Clusterware creates from one to four highly available IP (HAIP) addresses. Oracle RAC and Oracle ASM instances use these interface addresses to ensure highly available, load-balanced interface communication between nodes. The installer enables Redundant Interconnect Usage to provide a high availability private network.

  By default, Oracle Grid Infrastructure software uses all of the HAIP addresses for private network communication, providing load-balancing across the set of interfaces you identify for the private network. If a private interconnect interface fails or become non-communicative, then Oracle Clusterware transparently moves the corresponding HAIP address to a remaining functional interface.

  Note:

  If you define more than one interface for the private network interfaces, be aware that Oracle Clusterware activates only one interface at a time. However, if the active interface fails, then Oracle Clusterware transitions the HAIP addresses configured to the failed interface to one of the reserve interfaces in the defined set of private interfaces.

  When you upgrade a node to Oracle Grid Infrastructure 11g release 2 (11.2.0.2) and later, the upgraded system uses your existing network classifications.

  To configure multiple public interfaces, use a third-party technology for your platform to aggregate the multiple public interfaces before you start installation, and then select the single interface name for the combined interfaces as the public interface. Oracle recommends that you do not identify multiple public interface names during Oracle Grid Infrastructure installation. Note that if you configure two network interfaces as public network interfaces in the cluster without using an aggregation technology, the failure of one public interface on a node does not result in automatic VIP failover to the other public interface.

  Oracle recommends that you use the Redundant Interconnect Usage feature to make use of multiple interfaces for the private network. However, you can also use third-party technologies to provide redundancy for the private network.

• If you install Oracle Clusterware using OUI, then the public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes. This restriction does not apply if you use cloning, either to create a new cluster, or to add nodes to an existing cluster.

  For example: With a two-node cluster, you cannot configure network adapters on node1 with lan0 as the public interface, but on node2 have lan1 as the public interface. Public interface names must be the same, so you must configure lan0 as public on both nodes. You should configure the private interfaces on the same network adapters as well. If lan1 is the private interface for node1, then lan1 should be the private interface for node2.

  See Also:

  Oracle Clusterware Administration and Deployment Guide for information about how to add nodes using cloning

• For the public network, each network adapter must support TCP/IP.

• For the private network, the interface must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP (minimum requirement 1 Gigabit Ethernet).

  Note:

  UDP is the default interface protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware. You must use a switch for the interconnect. Oracle recommends that you use a dedicated switch.

  Oracle does not support token-rings or crossover cables for the interconnect.

• Each node's private interface for interconnects must be on the same subnet, and those subnets must connect to every node of the cluster. For example, if the private interfaces have a subnet mask of 255.255.255.0, then your private network is in the range 192.168.0.0--192.168.0.255, and your private addresses must be in the range of 192.168.0.[0-255]. If the private interfaces have a subnet mask of 255.255.0.0, then your private addresses can be in the range of 192.168.[0-255].[0-255].

• For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. There should be no node that is not connected to every private network interface. You can test if an interconnect interface is reachable using ping.

2.6.2 IP Address Requirements

Before starting the installation, you must have at least two interfaces configured on each node: One for the private IP address and one for the public IP address.

You can configure IP addresses with one of the following options:

• Dynamic IP address assignment using Oracle Grid Naming Service (GNS). If you select this option, then network administrators assign static IP address for the physical host name and dynamically allocated IPs for the Oracle Clusterware managed VIP addresses. In this case, IP addresses for the VIPs are assigned by a DHCP and resolved using a multicast domain name server configured as part of Oracle Clusterware within the cluster. If you plan to use GNS, then you must have the following:

  • A DHCP service running on the public network for the cluster

  • Enough addresses on the DHCP to provide 1 IP address for each node's virtual IP, and 3 IP addresses for the cluster used by the Single Client Access Name (SCAN) for the cluster

• Static IP address assignment. If you select this option, then network administrators assign a fixed IP address for each physical host name in the cluster and for IPs for the Oracle Clusterware managed VIPs. In addition, domain name server (DNS) based static name resolution is used for each node. Selecting this option requires that you request network administration updates when you modify the cluster.

root@node1]$ nslookup mycluster-scan
Server:         dns.example.com
Address:        192.0.2.001
 
Name:   mycluster-scan.example.com
Address: 192.0.2.201
Name:   mycluster-scan.example.com
Address: 192.0.2.202
Name:   mycluster-scan.example.com
Address: 192.0.2.203

2.6.3 Broadcast Requirements for Networks Used by Oracle Grid Infrastructure

Broadcast communications (ARP and UDP) must work properly across all the public and private interfaces configured for use by Oracle Grid Infrastructure release 2 patchset 1 (11.2.0.2) and later releases.

The broadcast must work across any configured VLANs as used by the public or private interfaces.

2.6.4 Multicast Requirements for Networks Used by Oracle Grid Infrastructure

With Oracle Grid Infrastructure release 2 (11.2), on each cluster member node, the Oracle mDNS daemon uses multicasting on all interfaces to communicate with other nodes in the cluster.

With Oracle Grid Infrastructure release 2 patchset 1 (11.2.0.2) and later releases, multicasting is required on the private interconnect. For this reason, at a minimum, you must enable multicasting for the cluster:

• Across the broadcast domain as defined for the private interconnect

• On the IP address subnet ranges 224.0.0.0/24 and 230.0.1.0/24

You do not need to enable multicast communications across routers.

2.6.5 DNS Configuration for Domain Delegation to Grid Naming Service

If you plan to use GNS, then before Oracle Grid Infrastructure installation, you must configure your domain name server (DNS) to send to GNS name resolution requests for the subdomain GNS serves, which are the cluster member nodes. The following is an overview of what needs to be done for domain delegation. Your actual procedure may be different from this example.

Configure the DNS to send GNS name resolution requests using delegation:

1. In the DNS, create an entry for the GNS virtual IP address, where the address uses the form gns-server.CLUSTERNAME.DOMAINNAME. For example, where the cluster name is mycluster, and the domain name is example.com, and the IP address is 192.0.2.1, create an entry similar to the following:

   mycluster-gns.example.com  A  192.0.2.1
   

   The address you provide must be routable.

2. Set up forwarding of the GNS subdomain to the GNS virtual IP address, so that GNS resolves addresses to the GNS subdomain. To do this, create a BIND configuration entry similar to the following for the delegated domain, where cluster01.example.com is the subdomain you want to delegate:

   cluster01.example.com  NS  mycluster-gns.example.com
   

3. When using GNS, you must configure resolve.conf on the nodes in the cluster (or the file on your system that provides resolution information) to contain name server entries that are resolvable to corporate DNS servers. The total timeout period configured—a combination of options attempts (retries) and options timeout (exponential backoff)—should be less than 30 seconds. For example, where xxx.xxx.xxx.42 and xxx.xxx.xxx.15 are valid name server addresses in your network, provide an entry similar to the following in /etc/resolv.conf:

   options attempts: 2
   options timeout: 1
   
   search cluster01.example.com example.com
   nameserver xxx.xxx.xxx.42
   nameserver xxx.xxx.xxx.15
   

   /etc/nsswitch.conf controls name service lookup order. In some system configurations, the Network Information System (NIS) can cause problems with Oracle SCAN address resolution. Oracle recommends that you place the nis entry at the end of the search list. For example:

   /etc/nsswitch.conf
        hosts:    files   dns   nis
   

   Note:

   Be aware that use of NIS is a frequent source of problems when doing cable pull tests, as host name and username resolution can fail.

2.6.6 Grid Naming Service Configuration Example

If you use GNS, then you must specify a static IP address for the GNS VIP address, and delegate a subdomain to be delegated to that static GNS IP address.

As nodes are added to the cluster, your organization's DHCP server can provide addresses for these nodes dynamically. These addresses are then registered automatically in GNS, and GNS provides resolution within the subdomain to cluster node addresses registered with GNS.

Because allocation and configuration of addresses is performed automatically with GNS, no further configuration is required. Oracle Clusterware provides dynamic network configuration as nodes are added to or removed from the cluster. The following example is provided only for information.

With a two node cluster where you have defined the GNS VIP, after installation you might have a configuration similar to the following for a two-node cluster, where the cluster name is mycluster, the GNS parent domain is example.com, the subdomain is grid.example.com, 192.0.2 in the IP addresses represent the cluster public IP address network, and 192.168.0 represents the private IP address subnet:

^Footnote 1 Node host names may resolve to multiple addresses, including VIP addresses currently running on that host.

2.6.7 Manual IP Address Configuration Example

If you choose not to use GNS, then before installation you must configure public, virtual, and private IP addresses. Also, check that the default gateway can be accessed by a ping command. To find the default gateway, use the route command, as described in your operating system's help utility.

For example, with a two node cluster where each node has one public and one private interface, and you have defined a SCAN domain address to resolve on your DNS to one of three IP addresses, you might have the configuration shown in the following table for your network interfaces:

^Footnote 1 Node host names may resolve to multiple addresses.

You do not need to provide a private name for the interconnect. If you want name resolution for the interconnect, then you can configure private IP names in the hosts file or the DNS. However, Oracle Clusterware assigns interconnect addresses on the interface defined during installation as the private interface (lan1, for example), and to the subnet used for the private subnet.

The addresses to which the SCAN resolves are assigned by Oracle Clusterware, so they are not fixed to a particular node. To enable VIP failover, the configuration shown in the preceding table defines the SCAN addresses and the public and VIP addresses of both nodes on the same subnet, 192.0.2.

2.6.8 Network Interface Configuration Options

The precise configuration you choose for your network depends on the size and use of the cluster you want to configure, and the level of availability you require.

If certified Network-attached Storage (NAS) is used for Oracle RAC and this storage is connected through Ethernet-based networks, then you must have a third network interface for NAS I/O. Failing to provide three separate interfaces in this case can cause performance and stability problems under load.

2.6.9 Checking the Run Level and Name Service Cache Daemon

To allow Oracle Clusterware to better tolerate network failures with NAS devices or NFS mounts, enable the Name Service Cache Daemon (nscd). The nscd provides a caching mechanism for the most common name service requests. It is automatically started when the system starts in a multiuser state. Oracle software requires that the server is started with multiuser run level (3), which is the default for HP-UX.

To check to see if the server is set to 3, enter the command who -r. For example:

# who -r 
.       run-level 3  Jan 4 14:04     3      0  S 

Refer to your operating system documentation if you must change the run level.

To check to see if the password and group caching daemon (pwgrd) is running, enter the following command:

ps -aef |grep pwgrd

2.7.1 Software Requirements List for HP-UX Itanium Platforms

PHCO_41479 (or later) 11.31 diskowner(1M) cumulative patch
PHKL_38038 vm cumulative patch
PHKL_38938 11.31 SCSI cumulative I/O patch 
PHKL_40941  Scheduler patch : post wait hang 
PHSS_36354 11.31 assembler patch
PHSS_37042  11.31 hppac (packed decimal)
PHSS_37959  Libcl patch for alternate stack issue fix (QXCR1000818011)
PHSS_39094 11.31 linker + fdp cumulative patch
PHSS_39100 11.31 Math Library Cumulative Patch
PHSS_39102 11.31 Integrity Unwind Library
PHSS_38141 11.31 aC++ Runtime

http://www.unixodbc.org

2.7.2 Software Requirements List for HP-UX PA-RISC (64-bit)

At the time of this release, the 11.2.0.3 release of Oracle Grid Infrastructure 11g Release 2 is available only for HP-UX Itanium. The following requirements are for the 11.2.0.2 release of Oracle Grid Infrastructure 11g Release 2 for HP-UX PA-RISC systems.

PHKL_39773 (11.31 VRTS 5.0 GARP6 VRTSvxfs Kernel Patch. 

PHCO_40381 (64 BIT DISKOWNER PATCH - lvmchk funtionality)
PHKL_38038 - VM patch - hot patching/Core file creation directory 
PHKL_38938 (11.31 SCSI cumulative I/O patch)
PHKL_39351 (11.31 scheduler cumulative patch 
PHSS_37959 (Libcl patch for alternate stack issue fix (QXCR1000818011)
PHSS_38141 (11.31 aC++ Runtime)
PHSS_39094 (11.31 linker + fdp cumulative patch)

2.11.1 Requirements for Enabling IPMI

You must have the following hardware and software configured to enable cluster nodes to be managed with IPMI:

• Each cluster member node requires a Baseboard Management Controller (BMC) running firmware compatible with IPMI version 1.5 or greater, which supports IPMI over LANs, and configured for remote control using LAN.

• The cluster requires a management network for IPMI. This can be a shared network, but Oracle recommends that you configure a dedicated network.

• Each cluster member node's port used by BMC must be connected to the IPMI management network.

• Each cluster member must be connected to the management network.

• Some server platforms put their network interfaces into a power saving mode when they are powered off. In this case, they may operate only at a lower link speed (for example, 100 MB, instead of 1 GB). For these platforms, the network switch port to which the BMC is connected must be able to auto-negotiate down to the lower speed, or IPMI will not function properly.

2.11.2 Configuring the IPMI Management Network

On HP-UX platforms, the BMC shares configuration information with the management processor (iLO). For Oracle Clusterware, you must configure the ilO/BMC for static IP addresses. Configuring the BMC with dynamic addresses (DHCP) is not supported on HP-UX.

2.11.3 Configuring the BMC

On each node, complete the following steps to configure the BMC to support IPMI-based node fencing:

• Configure a static IP address for the BMC.

• Set a password for the null (noname) account for the BMC.

On HP-UX, the BMC and the iLO share network configuration. They have the same IP address, the same hardware MAC address, and the same default gateway address.

If your iLO processor is already configured for network access using a static address, then you have already established the required BMC network configuration. If you have not set up a static address for the iLO, then you must set a static address, and note the address so that you can enter it into the Oracle Clusterware local registry after installation.

You must also set up a password for the null (noname) user account, as this is the single account that the BMC uses. Administration accounts in the iLO are unrelated to the BMC. For security reasons, Oracle requires that you set a password for the BMC account.

Refer to your HP-UX documentation for more information about how to configure the BMC. Ensure that the BMC is configured on each cluster member node.

2.13.1 Environment Requirements for Oracle Grid Infrastructure Software Owner

You must make the following changes to configure the Oracle Grid Infrastructure software owner environment:

• Set the installation software owner user (grid, oracle) default file mode creation mask (umask) to 022 in the shell startup file. Setting the mask to 022 ensures that the user performing the software installation creates files with 644 permissions.

• Set ulimit settings for file descriptors and processes for the installation software owner (grid, oracle)

• Set the software owner's environment variable DISPLAY environment variables in preparation for the Oracle Grid Infrastructure installation

2.13.2 Procedure for Configuring Oracle Software Owner Environments

To set the Oracle software owners' environments, follow these steps, for each software owner (grid, oracle):

1. Start a new terminal session; for example, start an X terminal (xterm).

2. Enter the following command to ensure that X Window applications can display on this system:

   $ xhost + hostname
   

   The hostname is the name of the local host.

3. If you are not already logged in to the system where you want to install the software, then log in to that system as the software owner user.

4. If you are not logged in as the user, then switch to the software owner user you are configuring. For example, with the grid user:

   $ su - grid
   

5. To determine the default shell for the user, enter the following command:

   $ echo $SHELL
   

6. Open the user's shell startup file in any text editor:

   • Bourne shell (sh) or Korn shell (ksh):

     $ vi .profile
     

   • C shell (csh or tcsh):

     % vi .login
     

7. If the ORACLE_SID, ORACLE_HOME, or ORACLE_BASE environment variable is set in the file, then remove the appropriate lines from the file.

8. Save the file, and exit from the text editor.

9. To run the shell startup script, enter one of the following commands:

   • Bourne or Korn shell:

     $ . ./.profile
     

   • C shell:

     % source ./.login
     

10. If you are not installing the software on the local system, then enter a command similar to the following to direct X applications to display on the local system:

    • Bourne or Korn shell:

      $ DISPLAY=local_host:0.0 ; export DISPLAY
      

    • C shell:

      % setenv DISPLAY local_host:0.0
      

    In this example, local_host is the host name or IP address of the system to use to display OUI (your workstation or PC).

11. If you determined that the /tmp directory has less than 7 GB of free disk space, then identify a file system with at least 7 GB of free space and set the TEMP and TMPDIR environment variables to specify a temporary directory on this file system:

    Note:

    You cannot use a shared file system as the location of the temporary file directory (typically /tmp) for Oracle RAC installation. If you place /tmp on a shared file system, then the installation fails.

    1. Use the bdf command to identify a suitable file system with sufficient free space.

    2. If necessary, enter commands similar to the following to create a temporary directory on the file system that you identified, and set the appropriate permissions on the directory:

       $ su - root
       # mkdir /mount_point/tmp
       # chmod a+wr /mount_point/tmp
       # exit
       

    3. Enter commands similar to the following to set the TEMP and TMPDIR environment variables:

       • Bourne or Korn shell:

         $ TEMP=/mount_point/tmp
         $ TMPDIR=/mount_point/tmp
         $ export TEMP TMPDIR
         

       • C shell:

         % setenv TEMP /mount_point/tmp
         % setenv TMPDIR /mount_point/tmp
         

12. Enter the following command to ensure that the ORACLE_HOME and TNS_ADMIN environment variables are not set:

    • Bourne or Korn shell:

      $ unset ORACLE_HOME
      $ unset TNS_ADMIN
      

    • C shell:

      % unsetenv ORACLE_HOME
      % unsetenv TNS_ADMIN
      

    Note:

    If the ORACLE_HOME environment variable is set, the Installer uses the value that it specifies as the default path for the Oracle home directory. However, if you set the ORACLE_BASE environment variable, Oracle recommends that you unset the ORACLE_HOME environment variable and choose the default path suggested by the Installer.

2.13.3 Configuring Kernel Parameters on HP-UX Systems

During installation, you can generate and run the Fixup script to check and set the kernel parameter values required for successful installation of the database. This script updates required kernel packages if necessary to minimum values.

If you cannot use the Fixup scripts, then verify that the kernel parameters are set to values greater than or equal to the minimum value shown in Appendix D, "Configuring Kernel Parameters Manually." The procedure following the table describes how to verify and set the values manually.

2.13.4 Setting Display and X11 Forwarding Configuration

If you are on a remote terminal, and the local node has only one visual (which is typical), then use the following syntax to set the DISPLAY environment variable:

Bourne, Korn, and Bash shells

$ export DISPLAY=hostname:0

C shell:

$ setenv DISPLAY hostname:0

For example, if you are using the Bash shell, and if your host name is node1, then enter the following command:

$ export DISPLAY=node1:0

To ensure that X11 forwarding will not cause the installation to fail, create a user-level SSH client configuration file for the Oracle software owner user, as follows:

1. Using any text editor, edit or create the software installation owner's ~/.ssh/config file.

2. Make sure that the ForwardX11 attribute is set to no. For example:

   Host *
         ForwardX11 no
   

2.13.5 Preventing Installation Errors Caused by Terminal Output Commands

During an Oracle Grid Infrastructure installation, OUI uses SSH to run commands and copy files to the other nodes. During the installation, hidden files on the system (for example, .bashrc or .cshrc) will cause makefile and other installation errors if they contain stty commands.

To avoid this problem, you must modify these files in each Oracle installation owner user home directory to suppress all output on STDOUT or STDERR (for example, stty, xtitle, and other such commands) as in the following examples:

• Bourne, Bash, or Korn shell:

  if [ -t 0 ]; then
     stty intr ^C
  fi
  

• C shell:

  test -t 0
  if ($status == 0) then
     stty intr ^C
  endif
  

  Note:

  When SSH is not available, the Installer uses the remsh commands instead of ssh and scp.

  If there are hidden files that contain stty commands that are loaded by the remote shell, then OUI indicates an error and stops the installation.