To create and verify a file system, perform the following steps:

1. Create an Oracle ADVM volume in a mounted disk group with the ASMCMD volcreate command.

   The compatibility parameters COMPATIBLE.ASM and COMPATIBLE.ADVM must be set to 11.2 or higher for the disk group to contain an Oracle ADVM volume. To use Oracle ACFS encryption, replication, security, or tagging, the disk group on which the volume is created for the file system must have compatibility attributes for ASM and ADVM set to 11.2.0.2 or higher.

   Start ASMCMD connected to the Oracle ASM instance. You must be a user in the OSASM operating system group.

   When configuring Oracle ADVM volume devices within a disk group, Oracle recommends assigning the Oracle Grid Infrastructure user and Oracle ASM administrator roles to users who have root privileges.

   To create a volume:

   ASMCMD [+] > volcreate -G data -s 10G volume1
   

   When creating an Oracle ADVM volume, a volume device name is created that includes a unique Oracle ADVM persistent disk group number. The volume device file functions in the same manner as any other disk or logical volume to mount file systems or for applications to use directly.

   The format of the volume name is platform-specific.

2. Determine the device name of the volume that was created.

   You can determine the volume device name with the ASMCMD volinfo command or from the VOLUME_DEVICE column in the V$ASM_VOLUME view.

   For example:

   ASMCMD [+] > volinfo -G data volume1
   Diskgroup Name: DATA
   
            Volume Name: VOLUME1
            Volume Device: /dev/asm/volume1-123
            State: ENABLED
        ... 
   
   SQL> SELECT volume_name, volume_device FROM V$ASM_VOLUME 
        WHERE volume_name ='VOLUME1';
   
   VOLUME_NAME        VOLUME_DEVICE
   -----------------  --------------------------------------
   VOLUME1            /dev/asm/volume1-123
   

3. Create a file system with the Oracle ACFS mkfs command.

   Create a file system using an existing volume device.

   For example:

   $ /sbin/mkfs -t acfs /dev/asm/volume1-123
   mkfs.acfs: version                   = 19.0.0.0.0
   mkfs.acfs: on-disk version           = 46.0
   mkfs.acfs: volume                    = /dev/asm/volume1-123
   mkfs.acfs: volume size               = 10737418240  (   10.00 GB )
   mkfs.acfs: Format complete.

   The root privilege is not required to run mkfs. The ownership of the volume device file dictates who can run this command.

4. Register the file system.

   In an Oracle Grid Infrastructure Clusterware configuration, you can run the srvctl add filesystem command to register and automount a file system. For example:

   # srvctl add filesystem -device /dev/asm/volume1-123 -path /acfsmounts/acfs1
          -user user1,user2,user3 -mtowner sysowner -mtgroup sysgrp -mtperm 755

   You can also register a file system with the acfsutil registry command. For example:

   $ /sbin/acfsutil registry -a /dev/asm/volume1-123 /acfsmounts/acfs1

   After registering an Oracle ACFS file system in the cluster mount registry, the file system is mounted automatically on each cluster member listed in the registry entry during the next registry check action. This automatic process runs every 30 seconds and eliminates the requirement to manually mount the file system on each member of the cluster. Registering an Oracle ACFS file system also causes the file system to be mounted automatically whenever Oracle Clusterware or the system is restarted.

   Note:

   • The srvctl add filesystem command is required when an Oracle Database home is installed on an Oracle ACFS file system. In this case, the file system should not be explicitly added to the registry with the Oracle ACFS registration command (acfsutil registry).
   • Oracle ACFS registration is not supported in an Oracle Restart (standalone) configuration, which is a single-instance (non-clustered) environment.
   • The root or asmadmin privileges are required to modify the registry. The Windows Administrator privilege is equivalent to the root privilege on Linux.

5. Mount or start the file system.

   If you have previously registered the file system, then start the file system with SRVCTL. For example:

   $ srvctl start filesystem -device /dev/asm/volume1-123

   If you have not previously registered the file system, then mount the file system with the Oracle ACFS mount command. For example:

   # /bin/mount -t acfs /dev/asm/volume1-123 /acfsmounts/acf1

   After an unregistered file system has been mounted, ensure that the permissions are set to allow access to the file system for the appropriate users. For example:

   # chown -R oracle:dba /acfsmounts/acfs1

   The root privilege is required to run the mount command and the Windows Administrator privilege is required to run the acfsmountvol command.

6. Create a test file in the file system.

   The user that creates the test file should be a user that is intended to access the file system. This test ensures that the appropriate user can write to the file system.

   For example:

   $ echo "Oracle ACFS File System" > /acfsmounts/acfs1/myfile
   

7. List the contents of the test file that was created in the file system.

   For example:

   $ cat /acfsmounts/acfs1/myfile
   Oracle ACFS File System

If the node is part of a cluster, perform the following steps on node 2 to view the test file you created on node 1.

1. Enable the volume that was previously created and enabled on node 1.

   Start ASMCMD connected to the Oracle ASM instance. You must be a user in the OSASM operating system group.

   For example:

   ASMCMD [+] > volenable -G data volume1
   

2. View information about the volume that you created on node 1.

   For example:

   ASMCMD [+] > volinfo -G data volume1
   

3. Mount the file system using the Oracle ACFS mount command.

   For example:

   # /bin/mount -t acfs /dev/asm/volume1-123 /acfsmounts/acfs1
   

   The root privilege is required run the mount command and the Windows Administrator privilege is required to run the acfsmountvol command.

   After the file system has been mounted, ensure that the permissions are set to allow access for the appropriate users.

4. List the contents of the test file you previously created on the file system.

   For example:

   $ cat /acfsmounts/acfs1/myfile
   Oracle ACFS File System
   

   The contents should match the file created previously on node 1.

To create and verify a snapshot on node 1:

The disk groups on which volumes are created for the primary and standby file systems must have compatibility attributes for ASM and ADVM set to 12.2 or higher. To use a snapshot as a storage location, or to use replication role reversal, the compatibility attributes for Oracle ASM and Oracle ADVM must be set to 18.0 or higher.

The steps to manage replication are:

1. Ensure that ssh has been configured for replication.

   • Ensure that host keys and user keys for the ssh command have been configured on your primary and standby clusters.

   • On Windows, ensure that Cygwin is installed and ssh is configured as required for replication.

2. Ensure that the snapshots needed by replication can be created at all times. At any given point replication, may need to be able to use two concurrent snapshots of the primary location, and one snapshot of the standby location.

3. Ensure that there is adequate network connectivity between the primary and standby sites. You should verify that the achievable network data transfer rate from primary to standby is substantially larger than the rate of change of data on the primary location.

   One way to estimate network data transfer rate is to start with an observed transfer rate, then reduce it to account for known sources of overhead. For example, you can calculate the elapsed time needed to FTP a 1 GB file from the primary location to the intended standby location, during a period when network usage is low. This provides an estimate of the maximum achievable transfer rate. This rate should be reduced to account for overheads inherent in replication transfers, as well as to allow for other demands on the network. For replication overhead, a reasonable approach is to reduce the measured rate by 20%, then by an additional 5% for each node in the primary cluster.

   To estimate the average rate of change on the primary, you can use the command acfsutil info fs with the -s option. This command should be run on each node where the file system that contains the primary location is mounted. The command displays the amount and rate of change to the file system on that node. To compute the total rate of change for the file system, the rate of changed for each node must be aggregated. A reasonable value to use for -s is 900, which would yield a 15 minute sampling interval.

   With the output from acfsutil info fs with the -s option, you can determine the average rate of change, the peak rate of change, and how long the peaks last. A conservative approach to using this data is to choose the peak rate of change as the target rate that must be accommodated.

   Because replication must transfer all data changed on the primary to the standby, obviously the achievable network transfer rate must be higher, ideally significantly higher, than the target rate of change on the primary. If this is not the case, you should increase network capacity before implementing replication for this primary location and workload.

   For example, assume you have a four node primary cluster and you determine that a 1 GB file can be transferred in 30 seconds, yielding a current FTP transfer rate of 33 MB per second. An estimate of the current replication transfer rate would be approximately 20 MB per second, calculated as follows:

   33 MB/sec * (1 – 0.2 – (4 * 0.05)) = 33 * 0.6 = ~20 MB/sec

   Also, you find that the average rate of change to the primary is 8 GB per hour, with a peak rate of 25 GB per hour. Using the peak rate, you can calculate a target rate of change of approximately 7 MB per second as follows:

   (25 GB/hour * 1024) / 3600 = ~7 MB/sec

   In the scenario that was discussed in this step, you can reasonably expect the network to be able to handle the additional workload from replication.

4. Ensure that there is adequate storage capacity on the primary and standby sites.

   Estimate the storage capacity needed for replication on the sites hosting the primary and standby locations. In the general case, the primary site must store two snapshots of the primary location on an ongoing basis and the standby site must store a single snapshot of the standby location. The space occupied by these snapshots mostly consists of user data or metadata preserved in the snapshot, that has since been modified which triggers a new copy of the data to be created.

   The space occupied by replication-related snapshots can be directly viewed using the command acfsutil snap info. On the primary, check for snapshots with the names starting with the string REPL. On the standby, look for snapshots for names starting with SDBACKUP.

   If you use interval-based replication, the -i option to acfsutil repl init primary, and if the replication operations are successfully completing within the specified interval, then the size of replication-related snapshots is related to the rate of change of the primary and the length of the interval. For example, with an average rate of change of 8 GB per hour and a two hour replication interval, you would expect that snapshot storage usage is in the range of 16 GB per snapshot.

   Snapshot size does vary with the rate of change of the primary. Another factor is that snapshot size depends in part on the number of files in the file system, as well as the rate of change. Potentially more importantly, if you use constant mode replication, the -C option to acfsutil repl init primary, or if replication operations are not completing successfully in the interval given with interval—based replication because the interval is too small, the size of replication-related snapshots is difficult to predict in advance. In these cases, observe the size of the snapshots being generated over time and adjust the file system size as needed with the acfsutil size command to accommodate normal storage needs in addition in the presence of the snapshots. When collecting this information, a good starting point is to accommodate space for the snapshots to contain the data that is multiple times larger than the collection period, at the average rate of change of the primary.

   While collecting this information, choose a conservative starting point for the amount of space to allow for replication snapshots. For example, you can compute the space needed to store changes to the file system over the collection period as described previously, then you can allocate several times that space for future snapshots.

5. Determine the user to be employed for replication and optionally set up tags.

   Choose or create the replication user who logs in with ssh to the standby cluster to apply data replicated from the primary location to the standby location. This user is defined only at the OS level and not within Oracle. The user should belong to the groups defined for Oracle AM administrator access.

   Note:

   The same user and group identities (including all uids and gids) must be specified for the replication user on both your primary cluster and your standby cluster.

   Optionally set tags on directories and files to replicate only selected files in an Oracle ACFS primary location. You can also add tags to files after replication has started.

6. Configure the site hosting the standby location.

   Before replicating an Oracle ACFS a primary storage location, configure the site hosting the standby location by performing the following:

   • To use the file system as a standby location, create a new standby file system of adequate size to hold the files replicated from the primary location, as well as a single replication snapshot, and mount the file system. For example:

     /standby/repl_data

   • To use a snapshot of an existing file system as a standby location, create a new read-write snapshot, and ensure that the file system is of adequate size to hold the files replicated from the primary location, as well as a single replication snapshot.

   • For either kind of standby location, run the acfsutil repl init standby command on the site hosting the standby location. For example:

     # /sbin/acfsutil repl init standby -u repluser /standby/repl_data

     Note:

     If the acfsutil repl init standby command is interrupted for any reason, the user must re-create the file system or snapshot used for the location, re-mount the file system if needed, and re-run the command.

     This command requires the name of the replication user and the standby location. The specified user is the user under which ssh, invoked from the primary cluster, logs in to the standby cluster to apply changes. This user is specified with the -u option. For example: -u repluser.

     If the standby location is a file system, it is named with its mount point. For example: /standby/repl_data.

     If the standby location is a read-write snapshot, it is named with the snapshot name and the mount point of the containing file system, with the two separated by the @ character. For example: drsnap1101@/standby/repl_data.

   The acfsutil repl init standby command requires root or system administrator privileges to run.

7. After the standby location has been set up, configure the site hosting the primary location and start replication.

   Run the acfsutil repl init primary command on the site hosting the primary location. For example:

   $ /sbin/acfsutil repl init primary -i 2h -s repluser@standby12_vip -m /standby/repl_data /acfsmounts/repl_data

   This command requires the following configuration information.

   • A replication interval, given with the option -i interval for interval mode or the option -C for constant mode replication. If an interval is specified, the option value is the minimum amount of time that elapses between replication operations. At the start of each operation, replication takes a new snapshot of the primary and compares it to the previous snapshot, if any. The changes needed to update the standby to match the primary are then sent to the standby. If -C is given instead of -i interval, a new replication operation is started as soon as the previous one completes.

     For example, to set up a replication interval of two hours, specify -i 2h.

   • The user name and network endpoint (VIP name or address, or host name or address) to be used to connect to the site hosting the standby location, specified with the —s option. For example: -s repluser@standby12_vip

   • If the primary location is a file system, then specify the name of the mount point of the file system. For example: /acfsmounts/repl_data

   • If the primary storage location is a snapshot, then specify the snapshot name plus the mount point of the containing file system, the two separated by the @ character. For example:  drsnap1101@/acfsmounts/repl_data

   • If the mount point, or snapshot name with the mount point, is different on the site hosting the standby location than it is on the site hosting the primary location, then specify the name of the standby location with the -m option. For example: -m /standby/repl_data

   The acfsutil repl init primary command requires root or system administrator privileges to run.

8. Monitor information about replication on the location.

   The acfsutil repl info command displays information about the state of the replication processing on the primary or standby location.

   For example, you can run the following on the site hosting the primary location to display configuration information.

   $ /sbin/acfsutil repl info -c -v /acfsmounts/repl_data

   You must have system administrator (the user root on non-Windows systems or local SYSTEM on Windows) or Oracle AM administrator privileges to run this command.

9. Manage the replication background process.

   Run the acfsutil repl bg command to start, stop, or retrieve information about the replication background process.

   For example, run the following command to display information about the replication process for the /acfsmounts/repl_data file system.

   $ /sbin/acfsutil repl bg info /acfsmounts/repl_data

   You must have system administrator or Oracle AM administrator privileges to run the acfsutil repl bg info command.

10. Pause replication momentarily if necessary.

    Run the acfsutil repl pause to momentarily stop replication. Run the acfsutil repl resume command as soon as possible to resume replication.

    For example, the following command pauses replication on the /acfsmounts/repl_data file system.

    $ /sbin/acfsutil repl pause /acfsmounts/repl_data

    The following command resumes replication on the /acfsmounts/repl_data file system.

    $ /sbin/acfsutil repl resume /acfsmounts/repl_data

    You must have system administrator or Oracle AM administrator privileges to run the acfsutil repl pause and acfsutil repl resume commands.

11. Failing over to a standby or turning a standby location into an active location.

    If the primary location is inaccessible, you can run acfsutil repl terminate standby command to turn the standby location into read-write storage. If the primary location still exists, you should terminate the primary first with the acfsutil repl terminate primary command.

    Before terminating replication with acfsutil repl terminate standby on the standby location, you can determine the point in time of the primary location that the standby location represents. This timestamp is displayed with acfsutil repl info -c as Last sync time with primary. If the failover action must be coordinated with Oracle Data Guard, you can use the timestamp to set back the database if needed, or perform other necessary actions that are based on the timestamp.

    The standby location may be in the process of being modified by replication. This could occur if:

    • The primary location is available and a replication operation is currently in progress.

    • The primary location is not available, but a replication operation was in progress when it became unavailable.

    To be sure of obtaining the contents of the standby location when it was last identical to a snapshot from the primary, follow one of these procedures.

    • If the primary location is available, run the acfsutil repl terminate primary command on the primary site to terminate replication. The command waits for any in-progress replication operation to complete before it returns. Then run acfsutil repl info -c to determine the point in time of the primary location represented on the standby. After you have this information, run acfsutil repl terminate standby on the standby site.

    • If the primary location is not available, you should first compare two date strings from the acfsutil repl info -c output; these are the dates from the Receiving primary as of line and the Last sync time with primary line. If these dates are identical, then the standby location contains the most recent available point-in-time image of the primary. If they are not equal, you must use the backup snapshot recorded by replication to recover the last point-in-time image captured on the standby. When both the primary location and the standby location are file systems, you can find this snapshot using the acfsutil snap info command. Search for a snapshot with a name of the form:

      SDBACKUP_tstamp1_REPL_0_tstamp2_0

      where tstamp1 represents the time at which the backup snapshot was created, and tstamp2 represents the point in time when the primary contents in this snapshot were recorded. There should be only one backup snapshot present. The date of the backup snapshot corresponds to the date in the Last sync time with primary line output by acfsutil repl info -c. If there is no backup snapshot available, then the contents of the primary were never successfully transferred to the standby.

      The date of the backup snapshot corresponds to the date in the Last sync time with primary line output by acfsutil repl info -c.

      To use the backup snapshot, you must terminate replication and ensure that the snapshot is preserved because it is deleted by default. To ensure the snapshot is preserved, add the -k option to the command line for acfsutil repl terminate standby. After replication has been terminated, you optionally can run the acfsutil snap remaster command to use the snapshot as the new contents of the standby file system.

Deregistering an Oracle ACFS File System

You can deregister an Oracle ACFS file system if you do not want the file system to be automatically mounted.

For example:

$ /sbin/acfsutil registry -d /acfsmounts/acfs1

If you deregister a file system, then you must explicitly mount the file system after Oracle Clusterware or the system is restarted.

For more information about the registry, refer to About the Oracle ACFS Mount Registry. For information about acfsutil registry, refer to acfsutil registry.

Dismounting an Oracle ACFS File System

You can dismount a file system without deregistering the file system or disabling the volume on which the file system is mounted.

For example, you can dismount a file system and run fsck to check the file system.

# /bin/umount /acfsmounts/acfs1

# /sbin/fsck -a -v -y -t acfs /dev/asm/volume1-123

After you dismount a file system, you must explicitly mount the file system.

Use umount on Linux systems or acfsdismount on Windows systems. For information about the commands to dismount a file system, refer to umount or acfsdismount.

Use fsck on Linux systems or acfschkdsk on Windows systems to check a file system. For information about the commands to check a file system, refer to fsck (offline mode) or acfschkdsk.

Disabling a Volume

To disable a volume, you must first dismount the file system on which the volume is mounted.

For example:

# /bin/umount /acfsmounts/acfs1

After a file system is dismounted, you can disable the volume and remove the volume device file.

For example:

ASMCMD> voldisable -G data volume1

Dismounting the file system and disabling a volume does not destroy data in the file system. You can enable the volume and mount the file system to access the existing data. For information about voldisable and volenable, refer to Managing Oracle ADVM with ASMCMD.

To permanently remove a volume and Oracle ACFS file system, perform the following steps. These steps destroy the data in the file system.

1. Deregister the file system with acfsutil registry -d.

   For example:

   $ /sbin/acfsutil registry -d /acfsmounts/acfs1
   acfsutil registry: successfully removed ACFS mount point
      /acfsmounts/acfs1 from Oracle Registry
   

2. Dismount the file system.

   For example:

   # /bin/umount /acfsmounts/acfs1
   

   You must dismount the file system on all nodes of a cluster.

   Use umount on Linux systems or acfsdismount on Windows systems.

3. Remove the file system with acfsutil rmfs.

   If you were not planning to remove the volume in a later step, this step is necessary to remove the file system. Otherwise, the file system is removed when the volume is deleted.

   For example:

   $ /sbin/acfsutil rmfs /dev/asm/volume1-123
   

4. Optionally you can disable the volume with the ASMCMD voldisable command.

   For example:

   ASMCMD> voldisable -G data volume1
   

5. Delete the volume with the ASMCMD voldelete command.

   For example:

   ASMCMD> voldelete -G data volume1