Viewing the Cloud Resolver logs - BlueCat Cloud Resolver - 1.7.2

BlueCat Cloud Resolver Administration Guide
Locale
English
Product name
BlueCat Cloud Resolver
Version
1.7.2
Cloud Resolver logs information to a logs file located at one of the following locations based on the distribution type of the Cloud Resolver host machine:
  • /var/log/syslog on Debian and Ubuntu.
  • /var/log/messages on RHEL and CentOS.

You can retrieve and view the contents of this file to diagnose and troubleshoot issues with Cloud Resolver.

The following section lists out common errors that might be found in the log file.

Cloud Resolver cannot resolve endpoints

Example error message
bluecat-cr-vm01 cloud-resolver[0]:  ERROR run:discover: cloud_resolver::services::server: 
Failed to Update Edge Domain List: error sending request for url (https://api-example.edge.bluec.at/v1/api/authentication/token): 
error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution

If you see this message, the host machine might not be able to resolve endpoints using DNS. To resolve this issue, ensure that you have the correct nameserver entries in the /etc/resolv.conf file so that the host machine can resolve endpoints.

Cloud Resolver API timeouts

Example error message
bluecat-cr-vm01 cloud-resolver[0]:  ERROR run:discover:discover:list_zones: cloud_resolver::azure::discover: error=error sending request for url 
(https://management.azure.com/subscriptions/abcdef12-98fe-6789-c321-aaabbbcccfff/providers/Microsoft.Network/privateDnsZones?api-version=2018-09-01): 
operation timed out

This message can appear if the connection between Cloud Resolver and the cloud environment times out due to connection issues or latency. If you see this message consistently, check the connection between Cloud Resolver and the cloud environment, as this may indicate a network issue.

In addition to the messages in syslog and messages, you can also retrieve the number of API failures by viewing the discover_live_errors and csp_api_total_errors values in the diagnostics endpoint.
  • The discover_live_errors displays the number of DNS zone discoveries that have failed since the last successful DNS zone discovery. If Cloud Resolver reconnects to the cloud environment and successfully issues an API call, the value resets to 0.
  • The csp_api_total_errors displays the total number of errors encountered by Cloud Resolver performing an API call against the cloud service provider.

For more information on the Cloud Resolver diagnostics endpoint, refer to Viewing the Cloud Resolver diagnostics endpoint.

Cloud Resolver port issues

Example error message
bluecat-cr-vm01 cloud-resolver.service[21147]:     Address already in use (os error 98)
This message appears when a port that is used by Cloud Resolver is being used by another process on the system. To resolve this issue, ensure that other services do not use one of the following ports used by Cloud Resolver:
  • 53—used for inbound and outbound DNS traffic.
  • 443—used for API access to the Cloud environment.
  • 8080—used for simple TCP checks.
  • 8090—used for verifying whether Kubernetes service is ready.
  • 9000—used for the diagnostics endpoint.
  • 9050—used for creating snapshots.
  • 9090—used for pulling data to Prometheus.

For more information on ports used by Cloud Resolver, refer to the "Network requirements" section of Prerequisites.

Cloud Resolver vault issues

Example error message
bluecat-cr-vm01 cloud-resolver.service[24181]: 2022-06-03T20:23:15.750054Z ERROR 
secret:edge_secrets:get_single_secret: cloud_resolver::azure: Error None for /CRS-EDGE-API-KEY/?api-version=7.2: 
builder error: relative URL without a base #033[3msecret_id#033[0m#033[2m=#033[0m"/CRS-EDGE-API-KEY"
bluecat-cr-vm01 cloud-resolver.service[24181]: 2022-06-03T20:23:15.750101Z ERROR 
secret:edge_secrets:get_single_secret: cloud_resolver::azure: error=builder error: 
relative URL without a base #033[3msecret_id#033[0m#033[2m=#033[0m"/CRS-EDGE-API-KEY"

This message appears if you have a vault entry in the Cloud Resolver configuration file that is misconfigured or not in use. If you have not configured a vault to store your Edge API key and Edge Secret key, remove the vault entry and restart the Cloud Resolver service. If you have configured vault, ensure that Cloud Resolver can access the vault endpoint to retrieve the keys.

Cloud Resolver cannot create a snapshot

Example error message
bluecat-cr-vm01 cloud-resolver.service[20962]:     Snapshot Directory doesn't exist

This message appears if Cloud Resolver cannot create a snapshot. To resolve this issue, ensure that the /var/lib/bluecat directory exists on the Cloud Resolver host machine and that the directory has write permissions. If you configured a custom Cloud Resolver directory, ensure that the directory exists and has write permissions.

Cloud Resolver fails to start after an operating system upgrade

Example error message
bluecat-cr-vm01 cloud-resolver.service[2536]: Cloud Resolver exited unexpectedly:
bluecat-cr-vm01 cloud-resolver.service[2536]:     Failed to create resolver: io error: No such file or directory (os error 2)

This message may appear if you have performed an operating system upgrade on the host machine running Cloud Resolver, such as upgrading to Ubuntu 22.04.2. In the example of Ubuntu 22.04, the operating system converts the /etc/resolv.conf file to a symbolic link. To resolve this issue, remove the file and create a new /etc/resolv.conf file with a nameserver within it. Once you have created the new /etc/resolv.conf file, restart Cloud Resolver.