(AWS) Deploying Cloud Discovery & Visibility in Anthos on Amazon Web Services - Adaptive Applications - BlueCat Gateway

Anthos is an enterprise container platform from Google, but can be used on non-Google platforms including Amazon Web Services (AWS). In general, to deploy Cloud Discovery & Visibility (CDV) workflow on Anthos on AWS, you must:

Acquire the Anthos cluster credentials.
Create an Elastic File System (EFS) to allow for persistent storage of CDV logs and database files.
Create a Kubernetes Secret object for your secret-key.
Create a Kubernetes Secret object for your quay.io credentials.
Deploy CDV with the created Secret object, using a YAML file. You will also expose the deployed CDV application to the network so that it can perform discovery and visibility jobs.

If needed, you can also easily remove the deployed CDV application from Anthos.

Prerequisites

You must have already configured your Anthos cluster. You can do so with Terraform. For more details, see Create a VPC and cluster with Terraform in the GKE Enterprise documentation.

1. Generate your cluster credentials

To generate your AWS cluster credentials, use the following command:

gcloud container aws clusters get-credentials <AWS cluster name> --location <Region name>

Where:

<AWS cluster name> is the name of the AWS cluster.
<Region name> is the cluster's region (such as us-east4).

2. Create an AWS Elastic File System to persist the CDV logs and database

We strongly recommend that you use persistent storage for CDV logs and database files so that the directories can be reused in the case of migration or redeployment. To set up persistent storage for logs and database files, you must create an AWS Elastic File System (EFS) for that EC2 instances on the VPC will connect to.

To do so:

In Amazon Web Services (AWS), go to the Amazon Elastic File System (Amazon EFS) page and click Create file system.
In the Create file system window, enter the following:
- Name: Enter a name for the file system. These instructions assume you will use "cdv".
- Virtual Private Cloud (VPC): Select the VPC whose EC2 instances you want to connect to your file system.
Important: Make sure that you note the ID of the newly-created EFS for later.
Identify the security group of the node pool to which CDV will be deployed.

To do so:
1. Within the AWS cloud environment, go to the AWS EC2 Console.
2. Search for and locate your cluster's node pool.
3. When viewing your cluster's node pool, click the Security tab, then note the ID and name of its Security group.
  
  Typically, security groups have names like sg-072a54f1d193d5289.
Set the cdv file system that you created earlier to use this security group.
1. In Amazon Web Services (AWS), go to the Amazon Elastic File System (Amazon EFS) page and click File systems.
2. Locate and click the cdv file system that you created earlier.
3. Click the Network tab, then click Manage.
4. In the Mount targets section, for each appropriate Availability zone:
  1. Click the Add mount target button.
  2. Select the Availability zone, Subnet ID, enter the IP address, and select the Security group that you identified earlier.
  3. When you're done with that zone, click Save and add mount targets for the next Availability zone (if needed).

Create an access point for logs and databases. For security purposes and system resiliency, we strongly recommend you use persistent storage for both logs and databases.

To set up this access point:

Set up the access points in Amazon Web Services (AWS) as follows:
1. In AWS, go to the Amazon Elastic File System (Amazon EFS) page and click File Systems.
2. Locate and click the cdv file system that you created earlier.
3. Click the Access points tab.
4. Click Create access point and create an access point for CDV's /logs directory.
  
  Note the Access Point ID of the new access point.
5. Click Create access point and create an access point for CDV's /database directory.
  
  Note the Access Point ID of the new access point.

Create a YAML file named persistent-storage-efs.yaml with the following content, replacing <Elastic File System ID>, <Log access point ID>, and <Database access point ID> with appropriate values:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-logs-efs
  labels:
    type: efs
spec:
  storageClassName: efs-sc
  claimRef:
    name: pv-claim-logs-efs
    namespace: default
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  accessModes:
    - ReadWriteMany
  csi:
    driver: efs.csi.aws.com
    volumeHandle: <Elastic File System ID>::<Log access point ID>
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv-claim-logs-efs
  namespace: default
spec:
  storageClassName: efs-sc
  volumeName: pv-logs-efs
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-database-efs
  labels:
    type: efs
spec:
  storageClassName: efs-sc
  claimRef:
    name: pv-claim-database-efs
    namespace: default
  capacity:
    storage: 50Gi
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  accessModes:
    - ReadWriteMany
  csi:
    driver: efs.csi.aws.com
    volumeHandle: <Elastic File System ID>::<Database Access Point ID>
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv-claim-database-efs
  namespace: default
spec:
  storageClassName: efs-sc
  volumeName: pv-database-efs
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi

Run the new YAML file (from wherever it was saved) as follows:
```
kubectl apply -f persistent-storage-efs.yaml
```

3. Create a Kubernetes Secret object for your SECRET_KEY

Secret objects hold sensitive information, like account passwords and other details. You will need to create a Secret object for the secret-key value.

To create the Kubernetes Secret object

Create a YAML file named secret-key.yaml with the following content, replacing <String with 32 characters> with a secret password string of 32 characters:
```
apiVersion: v1
kind: Secret
metadata:
  name: secret-key
type: Opaque
stringData:
  key: <String with 32 characters>
```
Run this file as follows:
```
kubectl apply -f secret-key.yaml
```

4. Create a Kubernetes Secret object for your quay.io credentials

You will also need a separate Secret object so that the deployment process can access Quay.io and download the CDV container. This object will contain encrypted password details for Docker. To get this token:

From Quay.io, go to your Docker Settings page.
Under Docker CLI Password, next to CLI Password, click Generate Encrypted Password.
Click Kubernetes Secret.
Click Download <Username>-secret.yml and save the file.
Open this file in a text editor. Within this file, change the metadata specification to quayio-access.

Save your changes when you're done.
Create the Kubernetes Secret object with the following command:
```
kubectl apply -f <Username>-secret.yaml
```

5. Deploy CDV using a YAML script

To deploy CDV, do the following:

If you haven't already done so, create a YAML script named cdv-development.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cdv-deployment
  labels:
    app: cdv
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cdv
  template:
    metadata:
      labels:
        app: cdv
    spec:
      volumes:
      - name: logs
        persistentVolumeClaim:
          claimName: pv-claim-logs-efs
      - name: database
        persistentVolumeClaim:
          claimName: pv-claim-database-efs
      containers:
      - name: cdv
        image: quay.io/bluecat/cloud_discovery_visibility:25.3
        env:
        - name: BAM_IP
          value: <BAM_IP>
        - name: SECRET_KEY
          valueFrom:
            secretKeyRef:
              name: secret-key
              key: key
        ports:
        - containerPort: 44300
        volumeMounts:
        - mountPath: /logs/
          name: logs
        - mountPath: /var/lib/postgresql/
          name: database
        imagePullPolicy: Always
      imagePullSecrets:
      - name: quayio-access

Note: This script assumes the name of the Secret object that you created to access quay.io is quayio-access. If it isn't, change the name specified in imagePullSecrets to the name you used.

To run the script and deploy CDV, run the following command:
```
kubectl apply -f cdv-deployment.yaml
```
Confirm that the CDV container is deployed. To do so, run the following command to list currently-deployed Kubernetes pods:
```
kubectl get pods --all-namespaces -o wide
```
Look for a namespace with the same name given as the metadata/name setting in the YAML file (cdv-development).
Expose the CDV application to the network, so that it can perform discovery and visibility operations. To do so, run the following command:
```
kubectl expose deployment cdv-deployment --type LoadBalancer --port 443 --target-port 44300
```
To confirm that the appropriate ports are successfully exposed for the container, run the following command:
```
kubectl get services
```
In the table that displays, check the list of ports for cdv-development in the PORT(S) column. Also note the IP address in the EXTERNAL-IP column.
You can now use the LoadBalancer DNS to access the CDV application UI. Typically, you can access it through the External IP address that you noted before:
```
https://<EXTERNAL-IP address>/
```
such as:
```
https://a2a8a43ea76564cf6b9c4f3a385fc4c3-268185863.us-east-2.elb.amazonaws.com/
```

Removing CDV from Anthos

To remove the deployed instance of CDV from the Anthos container, run the following commands:

kubectl delete service cdv-deployment

kubectl delete -f cdv-deployment.yaml