Manually configuring SQS, SNS, and EventBridge rules in AWS environments - Adaptive Applications - BlueCat Gateway - 23.3.2

Cloud Discovery & Visibility Administration Guide
Locale
English
Product name
BlueCat Gateway
Version
23.3.2

If a Standard SQS queue, SNS topic, EventBridge rule, or associated subscription does not yet exist in your Amazon Web Services (AWS) environment, and your AWS user account has write permissions for them, Cloud Discovery & Visibility (CDV) will automatically create them when you enable visibility. (On the AWS Visibility Options page, select the Enable Visibility after Discovery option.)

If your AWS user account does not have the correct write permissions, to enable visibility, you must manually configure the queue, topic, and rules yourself.

Tip: When any of the required SQS, SNS, or EventBridge permissions do not exist, CDV displays an error indicating the specific missing permissions.

To determine the MD5 hash string for your objects

Names for the SQS Queue, SNS Topic, and EventBridge Rules required by CDV include an MD5 hash string, generated from details that identify the account, region, and configuration. Before you manually create AWS queues, topics, and rules, you'll need to generate this hash string in order to create the names that CDV expects.

To determine the hash string, you can use the md5sum command from the command line or use the POST /visibility/default-queue-names-generator Cloud Discovery & Visibility REST API endpoint. For more information about using CDV's REST API endpoints, see REST API endpoints.

To determine the CDV hash string using the md5sum command:

  1. Append the ARN username, Address Manager URL, Address Manager username, AWS region name, AWS account, and Configuration name together into a single string, separated by underscore characters (_).

  2. Use the following command to generate an MD5 hash of that string:

    echo -n <string> | md5sum

For example, say the system has the following details:

  • ARN username: dduong
  • Address Manager URL: 192.168.56.93
  • Address Manager username: dduong
  • AWS region name: us-west-1
  • AWS account: 938684065067
  • Configuration name: us-west-1
    Note: If the BlueCat Configuration field is empty, the configuration name is the AWS region name, in this case us-west-1.

The string that the MD5 hash should be based on is dduong_192.168.56.93_dduong_us-west-1_938684065067_us-west-1. The hash string that CDV expects (generated with the md5sum command) is 30d431e82a945cd7819fb3cce23c61a8.

To create and configure the AWS topics, queues, rules, and subscriptions:

Note: Manual creation of AWS queue, topic, and rules require use of an account that has write permissions for those objects. If you don't have write permissions yourself, contact an AWS administrator to grant you temporary write access, or ask them to configure the AWS SQS Standard queue, SNS topic, and EventBridge Rule as described below.
In Amazon Web Services, create the following items:
  • Regional Standard SNS Topic: Create a Standard SNS topic in your region. When configuring the topic, use the following values for each associated key:
    Key Value
    Type Standard
    Name

    If you selected Override Queue and Notification Default Names in the Visibility options, enter the custom name that you configured in the SNS Topic Name field.

    If you did not enter a custom SNS topic name, enter BC-CDV-SNS-EC2-TOPIC-<hash string>.

    For example, with the sample hash used earlier, the name would be BC-CDV-SNS-EC2-TOPIC-30d431e82a945cd7819fb3cce23c61a8:
    Encryption (Optional) Disabled
    Access policy (Optional)
    • Method: Basic
    • Define who can publish messages to the topic: Only the topic owner
    • Define who can subscribe to this topic: Only the topic owner
    Delivery retry policy (HTTP/S) (Optional) Use the default delivery retry policy:
    {
  "http": {
    "defaultHealthyRetryPolicy": {
      "numRetries": 3,
      "numNoDelayRetries": 0,
      "minDelayTarget": 20,
      "maxDelayTarget": 20,
      "numMinDelayRetries": 0,
      "numMaxDelayRetries": 0,
      "backoffFunction": "linear"
    },
    "disableSubscriptionOverrides": false
  }
}
    Delivery status logging (Optional) Default
    For example, with the sample hash used earlier, the topic would look like this:

  • Standard SQS Queue: Create a Standard SQS queue in your region. When configuring the queue, use the following values for each key:
    Key Value
    Type Standard
    Env Prod
    Queue name

    If you selected Override Queue and Notification Default Names in the Visibility options, enter the custom name that you configured in the SQS Name field.

    If you did not enter a custom SQS name, enter BC-CDV-<hash string>.

    For example, with the sample hash used earlier, the name would be BC-CDV-30d431e82a945cd7819fb3cce23c61a8
    Configuration
    Visibility timeout 30 seconds
    Message retention period 4 days
    Delivery delay 0 seconds
    Maximum message size 256 KB
    Receive message wait time 0 seconds
    Access policy - Define who can access your queue
    Method Basic
    Define who can send messages to the queue Only the queue owner
    Define who can receive messages from the queue Only the queue owner
    Optional Configuration
    Dead-letter queue Disable
    Delivery status logging - optional Default
    For example, with the sample hash used earlier, the queue would look like this:

  • EventBridge Rule for AWS VPC and EC2: Create an EventBridge Rule for AWS VPC, EC2, with a target of the SNS topic created earlier. Use the following values for each associated key:
    Key Value
    Event Pattern 
    {
    "source": [
        "aws.ec2",
        "aws.elasticloadbalancing",
        "aws.eks"
    ],
    "detail-type": [
        "AWS API Call via CloudTrail"
    ],
    "detail": {
        "eventSource": [
            "ec2.amazonaws.com",
            "elasticloadbalancing.amazonaws.com",
            "eks.amazonaws.com"
        ]
    }
}
    Targets Use the SNS topic you created earlier.
    Name

    If you selected Override Queue and Notification Default Names in the Visibility options, enter the custom name that you configured in the EventBridge Rule Name field.

    If you did not enter a EventBridge rule name, enter BC-CDV-EC2-CW-<hash string>.

    For example, with the sample hash used earlier, this would be BC-CDV-EC2-CW-30d431e82a945cd7819fb3cce23c61a8
    State Enabled
    For example, with the sample hash used earlier, the EventBridge Rule would look like this:

  • Subscription: Create a subscription with the SNS topic you created earlier, a protocol of Amazon SQS, and an endpoint of the ARN of the SQS queue you created earlier. Use the following values for each associated key:

    Key Value
    Topic ARN

    The topic ARN of the SNS topic you created earlier. Typically, this is: arn:aws:sns:<AWS region name>:<aws-account-id>:BC-CDV-SNS-EC2-TOPIC-<hash string>).

    For example, with the sample hash and account information earlier, this would be arn:aws:sns:us-west-1:938684065067:BC-CDV-SNS-EC2-TOPIC-30d431e82a945cd7819fb3cce23c61a8
    Protocol Amazon SQS
    Endpoint The SQS ARN of the SQS queue you created earlier. Typically, this is: arn:aws:sns:sqs:<AWS region name>:<aws-account-id>:BC-CDV-<hash string>

    For example, with the sample hash and account information earlier, this would be arn:aws:sqs:us-west-1:938684065067:BC-CDV-30d431e82a945cd7819fb3cce23c61a8
    Enable raw message delivery Unchecked
    Subscription filter policy - optional None
    Redrive policy (dead-letter queue) - optional Disabled
    For example, with the sample hash and details given earlier, the subscription would look like this:

  • (AWS Route 53 only) Standard SNS topic for Route 53: If you are configuring visibility for AWS Route 53, also create a Standard SNS topic in the us-east-1 region. Use the following values for each associated key:
    Key Value
    Type Standard
    Name

    BC-CDV-SNS-R53-TOPIC-<hash string>

    For example, with the sample hash used earlier, this would be BC-CDV-SNS-R53-TOPIC-30d431e82a945cd7819fb3cce23c61a8
    Encryption - optional Disabled
    Access policy - optional
    • Method: Basic
    • Define who can publish messages to the topic: Only the topic owner
    • Define who can subscribe to this topic: Only the topic owner
    Delivery retry policy (HTTP/S) - optional Use the default delivery retry policy
    {
  "http": {
    "defaultHealthyRetryPolicy": {
      "numRetries": 3,
      "numNoDelayRetries": 0,
      "minDelayTarget": 20,
      "maxDelayTarget": 20,
      "numMinDelayRetries": 0,
      "numMaxDelayRetries": 0,
      "backoffFunction": "linear"
    },
    "disableSubscriptionOverrides": false
  }
}
    Delivery status logging - optional Default
    For example, with the sample hash and details given earlier, the Route 53 SNS topic would look like this:

  • (AWS Route 53 only) EventBridge Rule for Route 53: If you're configuring visibility for Route 53, also create an EventBridge Rule with a target of the SNS topic you created for Route 53. Use the following values for each associated key:

    Key Value
    Event Pattern 
    {
    "source": [
        "aws.route53"
    ],
    "detail-type": [
        "AWS API Call via CloudTrail"
    ],
    "detail": {
        "eventSource": [
            "route53.amazonaws.com"
        ]
    }
}
    Targets

    BC-CDV-SNS-R53-TOPIC-<hash string>

    For example, with the sample hash used earlier, this would be BC-CDV-SNS-R53-TOPIC-30d431e82a945cd7819fb3cce23c61a8
    Name

    BC-CDV-CW-R53-RULE-<hash string>

    For example, with the sample hash used earlier, this would be BC-CDV-CW-R53-RULE-30d431e82a945cd7819fb3cce23c61a8
    State Enabled

    For example, with the sample hash given earlier, the AWS Route 53 EventBridge Rule would look like this:



  • (AWS Route 53 only) Subscription for the Route 53 rule: If you're configuring visibility for Route 53, also create a subscription for the Route 53 topic you created earlier, with a protocol of Amazon SQS and an endpoint set to the ARN of the SQS you created earlier. Use the following values for each key:

    Key Value
    Topic ARN The ARN of the SNS topic you created earlier. Typically, this is: arn:aws:sns:sqs:<AWS region name>:<aws-account-id>:BC-CDV-SNS-R53-TOPIC-<hash string>

    For example, with the sample hash and account information earlier, this would be arn:aws:sns:us-west-1:938684065067:BC-CDV-SNS-R53-TOPIC-30d431e82a945cd7819fb3cce23c61a8
    Protocol Amazon SQS
    Endpoint The ARN of the SQS queue you created earlier. Typically, this is: arn:aws:sqs:sqs:<AWS region name>:<aws-account-id>:BC-CDV-<hash string>

    For example, with the sample hash and account information earlier, this would be arn:aws:sqs:us-west-1:938684065067:BC-CDV-30d431e82a945cd7819fb3cce23c61a8
    Enable raw message delivery Unchecked
    Subscription filter policy - optional None
    Redrive policy (dead-letter queue) - optional Disabled

    For example, with the sample hash given earlier, this subscription would look like this:



After the objects have been created in AWS, you can select the Enable Visibility after Discovery option. If you were granted temporary write access in order to create these objects, your AWS administrator can revert your AWS account permissions after the visibility task has run once and the SQS, SNS, and EventBridge rules are defined in AWS.