Oracle Sharding is tightly integrated with the Oracle replication and disaster recovery technologies Oracle Data Guard and Oracle GoldenGate.

Replication provides high availability, disaster recovery, and additional scalability for reads. A unit of replication can be a shard, a part of a shard, or a group of shards.

Replication topology in a sharded database is declaratively specified using GDSCTL command syntax. You can choose one of two technologies—Oracle Data Guard or Oracle GoldenGate—to replicate your data. Oracle Sharding automatically deploys the specified replication topology and enables data replication.

The availability of a sharded database is not affected by an outage or slowdown of one or more shards. Replication is used to provide individual shard-level high availability (Oracle Active Data Guard or Oracle GoldenGate). Replication is automatically configured and deployed when the sharded database is created. Optionally, you can use Oracle RAC for shard-level high availability, complemented by replication, to maintain shard-level data availability in the event of a cluster outage. Oracle Sharding automatically fails over database connections from a shard to its replica in the event of an unplanned outage.

• When To Choose Oracle GoldenGate for Shard High Availability
  When should Oracle GoldenGate be employed as your high availablility solution for Oracle Sharding?

Oracle Data Guard replication maintains one or more synchronized copies (standbys) of a shard (the primary) for high availability and data protection. Standbys may be deployed locally or remotely, and when using Oracle Active Data Guard can also be open for read-only access.

Oracle Data Guard can be used as the replication technology for sharded databases using the system-managed, user-defined, or composite method of sharding.

Oracle GoldenGate is used for fine-grained active-active replication where all shards are writable, and each shard can be partially replicated to other shards within a shardgroup.

In Oracle GoldenGate, replication is handled at the chunk level. For example, in Shardgroup 1 in the following figure, half of the data stored in each shard is replicated to one shard, and the other half to another shard. If any shard becomes unavailable, its workload is split between two other shards in the shardgroup. The multiple failover destinations mitigate the impact of a shard failure because there is no single shard that has to handle all of the workload from the failed shard.

With Oracle GoldenGate replication, a shardgroup can contain multiple replicas of each row in a sharded table; therefore, high availability is provided within a shardgroup, and there is no need to have a local replica of the shardgroup, as there is in the case of Data Guard replication. The number of times each row is replicated within a shardgroup is called its replication factor and is a configurable parameter.

To provide disaster recovery, a shardgroup can be replicated to one or more data centers. Each replica of a shardgroup can have a different number of shards, replication factor, database versions, and hardware platforms. However, all shardgroup replicas must have the same number of chunks, because replication is done at the chunk level.

Shardgroup 2 in the figure above contains the same data as Shardgroup 1, but resides in a different data center. Shards in both data centers are writable. The default replication factor, 2, is used for both shardgroups.

Note that because Shardgroup 2 contains only two shards and the replication factor is 2, the shards are fully replicated, and each of them contains all of the data stored in the sharded database. This means that any query routed to these shards can be executed without going across shards. There is only one failover destination in this shardgroup; if a shard goes down, the load on the other shard doubles.

Oracle Sharding is designed to minimize the number of conflicting updates performed to the same row on different shards. This is achieved designating a master chunk for each range of hash values and routing most of requests for the corresponding data to this chunk.

Sometimes it is impossible to avoid update conflicts because of state transitions, such as a chunk move or split, or a shard going up or down. The user may also intentionally allow conflicts in order to minimize transaction latency. For such cases Oracle GoldenGate provides automatic conflict detection and resolution which handles all kinds of conflicts including insert-delete conflicts.

Before creating any shards, there are some prerequisites:

• Register with scheduler (when using GDSCTL create shard)

• Prepare site-security wallets or client and server certificates.

• Install Oracle GoldenGate and add at least one secure deployment with sharding option, and start up GoldenGate services and servers.

• In each Oracle home, make a copy of the client wallets used to add GoldenGate deployments, and place it at $ORACLE_BASE/admin/ggshd_wallet/.

• Load PL/SQL packages from a GoldenGate install home. If you are creating shards using GDSCTL CREATE SHARD, this step is only applicable to the shard catalog. If you are using GDSCTL ADD SHARD, it applies to the shard catalog and all of the shards.

Run the following GDSCTL commands to deploy an example configuration shown in the figure above.

CREATE SHARDCATALOG -database host00:1521:shardcat -chunks 60
 -user 'gsmcatuser/gsmcatuser_password' 
 -repl OGG -sharding system -sdb orasdb
ADD GSM -gsm gsm1 -listener 1571 –catalog shard-dir1:1521:shardcat -localons 3841
ADD GSM -gsm gsm2 -listener 1571 –catalog shard-dir1:1521:shardcat -localons 3841
START GSM -gsm gsm1
START GSM -gsm gsm2
CONFIGURE -timeout 900
ADD REGION -region dc1
ADD REGION -region dc2
MODIFY GSM -gsm gsm1 -region dc1
MODIFY GSM -gsm gsm2 -region dc2
ADD SHARDGROUP -shardgroup shardgroup1 -region dc1 -repfactor 2 
ADD SHARDGROUP -shardgroup shardgroup2 -region dc2 -repfactor 2  


CREATE SHARD -shardgroup shardgroup1 -destination host01 -credential oracle_cred        
 -netparam /home/oracle/netca_dbhome.rsp -gg_service host01:9900/deployment_name
 -gg_password ggadmin_password -dbparamfile /home/oracle/dbparams01.tmp
 -dbtemplatefile /home/oracle/sharddb01.dbt
 
CREATE SHARD -shardgroup shardgroup1 -destination host02 -credential oracle_cred      
 -netparam /home/oracle/netca_dbhome.rsp -gg_service host02:9900/remote_scheduler_agent
 -gg_password ggadmin_password -dbparamfile /home/oracle/dbparams02.tmp
 -dbtemplatefile /home/oracle/sharddb02.dbt 

CREATE SHARD -shardgroup shardgroup1 -destination host03 -credential oracle_cred      
 -netparam /home/oracle/netca_dbhome.rsp -gg_service host03:9900/remote_scheduler_agent
 -gg_password ggadmin_password -dbparamfile /home/oracle/dbparams03.tmp
 -dbtemplatefile /home/oracle/sharddb03.dbt
 
CREATE SHARD -shardgroup shardgroup2  -destination host04 -credential oracle_cred      
-netparam /home/oracle/netca_dbhome.rsp -gg_service host04:9900/remote_scheduler_agent
 -gg_password ggadmin_password -dbparamfile /home/oracle/dbparams04.tmp
 -dbtemplatefile /home/oracle/sharddb04.dbt
 
CREATE SHARD -shardgroup shardgroup2  -destination host05 -credential oracle_cred      
-netparam /home/oracle/netca_dbhome.rsp -gg_service host05:9900/remote_scheduler_agent
 -gg_password ggadmin_password -dbparamfile /home/oracle/dbparams05.tmp
 -dbtemplatefile /home/oracle/sharddb05.dbt


DEPLOY

Note that the above example uses CREATE SHARD to create new shards during deployment. ADD SHARD is the alternative to CREATE SHARD, and the ADD SHARD method assumes the pre-existence of clean slate database instances ready to be converted into database shards.