In the event of a network failure, the Distributed DDNS database cluster might split into several components. A component is a set of nodes that are connected to each other, but not to other nodes with the database cluster. If this occurs, only one component can continue to modify the database state to avoid history divergence, known as the Primary Component. When BlueCat Distributed DDNS is operating correctly, the Primary Component consists of all nodes within the cluster. When the cluster is split into multiple components, the database cluster uses a quorum algorithm to determine which component is selected as the Primary Component. This ensures that there is only one Primary Component in the cluster that can change the database state in the cluster to avoid data divergence between components.
In some situations, cluster failures can cause the data nodes to operate autonomously of the other nodes. This situation is known as split brain, where two data nodes can independently update the same row in the same table, resulting in data corruption. The database cluster avoids split-brain using Primary Components. In the event that a failure results in the splitting of clusters into two partitions of equal size, such as a two node cluster, neither partition becomes operational. If you configure a two node Distributed DDNS database cluster, automatic failure cannot occur due to the failure of one node causing the other node to turn into a non-operational state.
Single switch clusters with a minimum of three nodes
Clusters spanning multiple switches with a minimum of three switches
Clusters spanning multiple networks with a minimum of three networks
Clusters spanning multiple data centers with a minimum of three data centers
WAN replications
If you are configuring a Distributed DDNS database cluster over a WAN, you might experience transient network connectivity failures. To prevent the partitioning of the cluster, you can modify the configuration parameters to tolerate longer connectivity outages. The following parameters in the /var/lib/docker/volumes/mariadb-config/_data/my.cnf file allow for a tolerance of 4 minutes for connectivity outages:
evs.keepalive_period = PT3S
evs.suspect_timeout = PT4M
evs.inactive_timeout = PT5M
evs.install_timeout = PT5MYou can set the my.cnf options from the Distributed DDNS UI by
modifying the parameters for wsrep_provider_options:
wsrep_provider_options="gmcast.listen_addr=tcp://[::]:4567;socket.ssl_cert=/etc/ddns_ssl/mysql/mariadb_server_cert.pem;socket.ssl_key=/etc/ddns_ssl/mysql/mariadb_server_key.pem;socket.ssl_ca=/etc/ddns_ssl/ca/ca_cert.pem;evs.keepalive_period=PT3S;evs.suspect_timeout=PT4M;evs.inactive_timeout=PT5M;evs.install_timeout=PT5M"