Reference: Distributed DDNS Data Node behavior - Adaptive Applications - BlueCat Gateway - 23.2.3

BlueCat Distributed DDNS Administration Guide

Product name
BlueCat Gateway

In the event of a network failure, the Distributed DDNS database cluster might split into several components. A component is a set of nodes that are connected to each other, but not to other nodes with the database cluster. If this occurs, only one component can continue to modify the database state to avoid history divergence, known as the Primary Component. When BlueCat Distributed DDNS is operating correctly, the Primary Component consists of all nodes within the cluster. When the cluster is split into multiple components, the database cluster uses a quorum algorithm to determine which component is selected as the Primary Component. This ensures that there is only one Primary Component in the cluster that can change the database state in the cluster to avoid data divergence between components.

In some situations, cluster failures can cause the data nodes to operate autonomously of the other nodes. This situation is known as split brain, where two data nodes can independently update the same row in the same table, resulting in data corruption. The database cluster avoids split-brain using Primary Components. In the event that a failure results in the splitting of clusters into two partitions of equal size, such as a two node cluster, neither partition becomes operational. If you configure a two node Distributed DDNS database cluster, automatic failure cannot occur due to the failure of one node causing the other node to turn into a non-operational state.

BlueCat recommends configuring at least three Distributed DDNS Data Nodes to make sure that database cluster quorum algorithm can determine a Primary Component and avoid split-brain situations. You can configure the Distributed DDNS database cluster as follows:
  • Single switch clusters with a minimum of three nodes

  • Clusters spanning multiple switches with a minimum of three switches

  • Clusters spanning multiple networks with a minimum of three networks

  • Clusters spanning multiple data centers with a minimum of three data centers

WAN replications

If you are configuring a Distributed DDNS database cluster over a WAN, you might experience transient network connectivity failures. To prevent the partitioning of the cluster, you can modify the configuration parameters to tolerate longer connectivity outages. The following parameters in the /var/lib/docker/volumes/mariadb-config/_data/my.cnf file allow for a tolerance of 4 minutes for connectivity outages:

evs.keepalive_period = PT3S
evs.suspect_timeout = PT4M
evs.inactive_timeout = PT5M
evs.install_timeout = PT5M

You can set the my.cnf options from the Distributed DDNS UI by modifying the parameters for wsrep_provider_options:

Note: You must apply these parameters to every node in the cluster for them to take effect.