If there are issues with the Distributed DDNS data due to data loss or corruption, you can restore the database to an earlier instance based on configured backups. BlueCat recommends configuring database backups on a regular cadence to mitigate data loss due to a disaster recovery scenario.
You can restore database backups using two methods: restoring from a bootstrap node or restoring from a non-bootstrap node.
To determine which node is a bootstrap node, log in to the console of the Distributed
DDNS Data Node and run the following
command:
docker exec -it <node_name> bash -c "ps aux | grep mysqld"
The command displays a list of processes. The process mysqld --wsrep-new-cluster
indicates that the node is a boostrap node. The following example output indicates that
a node named node1 is the bootstrap node:
root@DBNodeContainer1:/home/mariadbcontainer1/Documents# docker exec -it node1 bash -c "ps aux | grep mysqld" mysql 286 0.5 10.1 1653768 102760 ? Sl 03:15 0:09 mysqld --wsrep-new-cluster
Note:
- When restoring the database from a backup, BlueCat recommends restoring from a bootstrap node.
- When restoring the database from a backup, the nodes restore serially and do
not restore simultaneously. You can view the log files during the
restoration process to verify whether individual nodes have started up and
successfully joined the cluster so that you can begin to restore another
node.To view the logs of a node, log in to the console of the Distributed DDNS Data Node and run the following command:
docker exec <node_name> tail -f /var/log/mysql/mariadb.err
The following example displays a node that has successfully joined the cluster:2020-11-03 07:51:45,836 DEBG 'mariadb' stdout output: 2020-11-03 07:51:45 2 [Note] WSREP: Server node192.168.56.124 synched with group 2020-11-03 07:51:45 2 [Note] WSREP: Server status change joined -> synced 2020-11-03 07:51:45 2 [Note] WSREP: Synchronized with group, ready for connections 2020-11-03 07:51:45 2 [Note] WSREP: Swsrep_notify_cmd is not defined, skipping notifications. 2020-11-03 07:51:45,837 DEBG 'mariadb' stdout output: 2020-11-03 07:51:45 0 [Note] WSREP: Donor monitor thread ended with total time 0 sec 2020-11-03 07:51:46,717 DEBG 'mariadb' stdout output: 2020-11-03 07:51:46 0 [Note] WSREP: (010c779a-b8d1, 'ssl://[::]:4567') turning message relay requesting off 2020-11-03 07:51:47,236 DEBG 'mariadb' stdout output: 2020-11-03 07:51:47 0 [Note] WSREP: async IST sender served 2020-11-03 07:51:47,253 DEBG 'mariadb' stdout output: 2020-11-03 07:51:47 0 [Note] WSREP: 1.0 (node192.168.56.125): State transferred from 0.0 (node192.168.56.124) complete. 2020-11-03 07:51:47,255 DEBG 'mariadb' stdout output: 2020-11-03 07:51:47 2 [Note] WSREP: Member 1.0 (node192.168.56.125) synched with group.