If the GCP topic, sink, and subscription do not exist in GCP yet, and your GCP user account has write permissions for them, you can tell Cloud Discovery & Visibility (CDV) to automatically create them. To do so, on the GCP Visibility Options page, select the Enable Visibility after Discovery option.
If your GCP user account does not have the correct write permissions, to enable visibility, you must manually configure the GCP topic, sink, and subscription rules yourself.
Determine the CDV hash string
Names for the GCP topic, sink, and subscription that CDV uses include an MD5 hash string of the details that identify the project, Address Manager instance, and configuration. Before you manually create GCP topics, sinks, and subscriptions, you'll need to determine this hash string in order to create the names that CDV expects.
To determine the hash string, you can use the md5sum
command from the
command line or use the POST /visibility/default-queue-names-generator
Cloud Discovery & Visibility REST API endpoint. For more information about using
CDV's REST API endpoints, see REST API endpoints.
To determine the CDV hash string using the md5sum
command:
Append the Project ID, Address Manager URL, Address Manager username, and Configuration name together into a single string with no characters separating them.
Use the following command to generate an MD5 hash of that string:
echo -n <string> | md5sum
- Project ID:
eng-dev-cloud-integration-01
- Address Manager URL:
192.168.113.59
- Address Manager username:
anh
- Configuration name:
test10
The string that the MD5 hash should be based on is
eng-dev-cloud-integration-01192.168.113.59anhtest10
. The hash
string that CDV expects (generated with the md5sum
command) is
b30f604746001d56a0890d4488e48159
.
To create and configure the GCP topic, sink and subscription:
In Pub/Sub configuration on Google Cloud Platform, click Topics, then click Create Topic.
If you selected Override Queue and Notification Default Names in the Visibility options, create a topic with the custom name that you configured in the Pub/Sub Topic Name field.
If you did not enter a custom Pub/Sub topic name, create a topic with the name
BC-CDV-PUBSUB-TOPIC-<hash string>
.For example, with the sample hash used above, you would create a topic with a name ofBC-CDV-PUBSUB-TOPIC-b30f604746001d56a0890d4488e48159
.-
In Logging configuration on Google Cloud Platform, click Logs Router, then create a new sink:
- If you selected Override Queue and Notification Default
Names in the Visibility options, under Sink
details, enter the custom name that you configured in
the Logging Sink Name field.
If you did not enter a custom logging sink name, under Sink details, enter the name
BC-CDV-LOGGING-SINK-<hash string>
.For example, with the sample hash used above, you would create a sink with a name ofBC-CDV-LOGGING-SINK-b30f604746001d56a0890d4488e48159
. Under Sink destination, in Select sink service, select Cloud Pub/Sub topic. Then, click Select a Cloud Pub/Sub topic and choose the Pub/Sub topic that you created earlier.
- Under Choose logs to include in sink, create logs
with the following inclusion filters. (These logs will be included in
the logs routing sink.):
Type Value Virtual Network protoPayload.methodName =~ ("(?i)networks" OR "(?i)subnetworks" OR "(?i)managedZones") AND severity="NOTICE" AND cloudaudit.googleapis.com%2Factivity AND protoPayload.response.status="running"
Virtual Machine Create two logs with the following filters:
-
protoPayload.methodName=~ "(?i)instances" AND severity="NOTICE" AND cloudaudit.googleapis.com%2Factivity AND protoPayload.response.status="running"
-
protoPayload.methodName= "compute.instances.guestTerminate" AND severity="INFO”
Load Balancer protoPayload.methodName=~ ("(?i)backendServices" OR "(?i)targetPools" OR "(?i)urlMaps" OR "(?i)forwardingRules" OR "(?i)targetHttpProxies" OR "(?i)targetHttpsProxies" OR "(?i)targetSslProxies" OR "(?i)targetTcpProxies") AND severity="NOTICE" AND cloudaudit.googleapis.com%2Factivity AND protoPayload.response.status="running
Cloud DNS protoPayload.methodName =~ ("dns.managedZones.(?i)" OR "dns.changes.(?i)") AND severity="NOTICE" AND cloudaudit.googleapis.com%2Factivity OR protoPayload.response.status="running
Private Service Connect protoPayload.methodName =~ "(?i)forwardingRules" AND severity="NOTICE" AND cloudaudit.googleapis.com%2Factivity AND protoPayload.response.status="running"
Kubernetes protoPayload.methodName =~ "(?i)ClusterManage" AND severity="NOTICE" AND operation.last=true
-
- (Optional) Under Choose logs to filter out of
sink, select the logs to filter out of the sink.
When you're done, save and close the sink.
- If you selected Override Queue and Notification Default
Names in the Visibility options, under Sink
details, enter the custom name that you configured in
the Logging Sink Name field.
- If you selected Override Queue and Notification Default
Names in the Visibility options, create a Pub/Sub Subscription
with the custom name that you configured in the Pub/Sub Subscription
Name field.
If you did not enter a custom Pub/Sub Subscription name, create a Pub/Sub Subscription with the name
BC-CDV-PUBSUB-SUBSCRIPTION-<hash string>
.For example, with the sample hash used above, you would create a subscription with a name of
BC-CDV-PUBSUB-SUBSCRIPTIONb30f604746001d56a0890d4488e48159
. - Select the Cloud Pub/Sub topic and choose the Pub/Sub topic that you created earlier. (In our example,
BC-CDV-PUBSUB-TOPIC-b30f604746001d56a0890d4488e48159
. After you create the GCP topic, sink, and subscription, open Cloud Discovery & Visibility, go to the GCP Visibility page (click the GCP tab in the main banner, then click the Visibility tab). Then, click to select the Enable Visibility after Discovery option.
After visibility tasks have run once and the GCP topic, sink, and subscription are defined, your GCP administrator can revert your GCP account permissions to read-only if you were granted temporary write access.