7.4. AWS Configuration Example

This tutorial explains the simple way to try "Watchdog" on AWS and using the Elastic IP Address as the Virtual IP for the high availability solution.

Note: You can use watchdog with Pgpool-II in any mode: replication mode, master/slave mode and raw mode.

7.4.1. AWS Setup

For this example, we will use two node Pgpool-II watchdog cluster. So we will set up two Linux Amazon EC2 instances and one Elastic IP address. So for this example, do the following steps:

7.4.2. Pgpool-II configurations

Mostly the Pgpool-II configurations for this example will be same as in the Section 7.2, except the delegate_IP which we will not set in this example instead we will use wd_escalation_command and wd_de_escalation_command to switch the Elastic IP address to the maste/Active Pgpool-II node.

7.4.2.1. Pgpool-II configurations on Instance-1

use_watchdog = on
delegate_IP = ''
wd_hostname = 'instance-1-private-ip'
other_pgpool_hostname0 = 'instance-2-private-ip'
other_pgpool_port0 = 9999
other_wd_port0 = 9000
wd_escalation_command = '$path_to_script/aws-escalation.sh'
wd_de_escalation_command = '$path_to_script/aws-de-escalation.sh'
            

7.4.2.2. Pgpool-II configurations on Instance-2

use_watchdog = on
delegate_IP = ''
wd_hostname = 'instance-2-private-ip'
other_pgpool_hostname0 = 'instance-1-private-ip'
other_pgpool_port0 = 9999
other_wd_port0 = 9000
wd_escalation_command = '$path_to_script/aws-escalation.sh'
wd_de_escalation_command = '$path_to_script/aws-de-escalation.sh'
            

7.4.3. escalation and de-escalation Scripts

Create the aws-escalation.sh and aws-de-escalation.sh scripts on both instances and point the wd_escalation_command and wd_de_escalation_command to the respective scripts.

Note: You may need to configure the AWS CLI first on all AWS instances to enable the execution of commands used by wd-escalation.sh and wd-de-escalation.sh. See configure AWS CLI

7.4.3.1. escalation script

This script will be executed by the watchdog to assign the Elastic IP on the instance when the watchdog becomes the active/master node. Change the INSTANCE_ID and ELASTIC_IP values as per your AWS setup values.

aws-escalation.sh:

#! /bin/sh

ELASTIC_IP=35.163.178.3
  # replace it with the Elastic IP address you
  # allocated from the aws console
INSTANCE_ID=i-0a9b64e449b17ed4b
  # replace it with the instance id of the Instance
  # this script is installed on

echo "Assigning Elastic IP $ELASTIC_IP to the instance $INSTANCE_ID"
# bring up the Elastic IP
aws ec2 associate-address --instance-id $INSTANCE_ID --public-ip $ELASTIC_IP

exit 0
            

7.4.3.2. de-escalation script

This script will be executed by watchdog to remove the Elastic IP from the instance when the watchdog resign from the active/master node.

aws-de-escalation.sh:

#! /bin/sh

ELASTIC_IP=35.163.178.3
  # replace it with the Elastic IP address you
  # allocated from the aws console

echo "disassociating the Elastic IP $ELASTIC_IP from the instance"
# bring down the Elastic IP
aws ec2 disassociate-address --public-ip $ELASTIC_IP
exit 0
            

AWS Command References

"Configure AWS CLI", AWS Documentation: Configuring the AWS Command Line Interface.

"associate-address", AWS Documentation: associate-address reference.

"disassociate-address", AWS Documentation: disassociate-address reference.

7.4.4. Try it out

Start Pgpool-II on each server with "-n" switch and redirect log messages to the pgpool.log file. The log message of master/active Pgpool-II node will show the message of Elastic IP assignment.

LOG:  I am the cluster leader node. Starting escalation process
LOG:  escalation process started with PID:23543
LOG:  watchdog: escalation started
  Assigning Elastic IP 35.163.178.3 to the instance i-0a9b64e449b17ed4b
  {
  "AssociationId": "eipassoc-39853c42"
  }
LOG:  watchdog escalation successful
LOG:  watchdog escalation process with pid: 23543 exit with SUCCESS.
	  

Confirm to ping to the Elastic IP address.

[user@someserver]$ ping 35.163.178.3
PING 35.163.178.3 (35.163.178.3) 56(84) bytes of data.
64 bytes from 35.163.178.3: icmp_seq=1 ttl=64 time=0.328 ms
64 bytes from 35.163.178.3: icmp_seq=2 ttl=64 time=0.264 ms
64 bytes from 35.163.178.3: icmp_seq=3 ttl=64 time=0.412 ms
	  

Try to connect PostgreSQL by "psql -h ELASTIC_IP -p port".

[user@someserver]$ psql -h 35.163.178.3 -p 9999 -l
	  

7.4.5. Switching Elastic IP

To confirm if the Standby server acquires the Elastic IP when the Active server becomes unavailable, Stop the Pgpool-II on the Active server. Then, the Standby server should start using the Elastic IP address, And the Pgpool-II log will show the below messages.

LOG:  remote node "172.31.2.94:9999 [Linux ip-172-31-2-94]" is shutting down
LOG:  watchdog cluster has lost the coordinator node
LOG:  watchdog node state changed from [STANDBY] to [JOINING]
LOG:  watchdog node state changed from [JOINING] to [INITIALIZING]
LOG:  I am the only alive node in the watchdog cluster
HINT:  skipping stand for coordinator state
LOG:  watchdog node state changed from [INITIALIZING] to [MASTER]
LOG:  I am announcing my self as master/coordinator watchdog node
LOG:  I am the cluster leader node
DETAIL:  our declare coordinator message is accepted by all nodes
LOG:  I am the cluster leader node. Starting escalation process
LOG:  escalation process started with PID:23543
LOG:  watchdog: escalation started
  Assigning Elastic IP 35.163.178.3 to the instance i-0dd3e60734a6ebe14
  {
  "AssociationId": "eipassoc-39853c42"
  }
LOG:  watchdog escalation successful
LOG:  watchdog escalation process with pid: 61581 exit with SUCCESS.
	  

Confirm to ping to the Elastic IP address again.

[user@someserver]$ ping 35.163.178.3
PING 35.163.178.3 (35.163.178.3) 56(84) bytes of data.
64 bytes from 35.163.178.3: icmp_seq=1 ttl=64 time=0.328 ms
64 bytes from 35.163.178.3: icmp_seq=2 ttl=64 time=0.264 ms
64 bytes from 35.163.178.3: icmp_seq=3 ttl=64 time=0.412 ms
	  

Try to connect PostgreSQL by "psql -h ELASTIC_IP -p port".

[user@someserver]$ psql -h 35.163.178.3 -p 9999 -l