How to setup Astra Traffic Collector via Cloud Market Place

Here's how to configure Astra Traffic Collector for traffic monitoring, by choosing the VM images from respective cloud provider marketplace (AWS/GCP/Azure/DO). It also covers troubleshooting common issues encountered during configuration.

Traffic Collector integration

Quick Installation

Spin up a new instance of traffic collector by chosing the right image from marketplace.

SSH into the VM.

Locate the following two editable files under /opt/astra-traffic-collector/
.env
config_custom.yaml

Update the secrets by editing .env file:
Replace <collectorId> with the COLLECTOR_ID displayed during the creation of astra traffic collector integration.
Replace <clientId> with the CLIENT_ID displayed during the creation of astra traffic collector integration.
Replace <clientSecret> with the CLIENT_SECRET displayed during the creation of astra traffic collector integration.

COLLECTOR_ID=<collectorId>
CLIENT_ID=<clientId>
CLIENT_SECRET=<clientSecret>
TOKEN_URL=https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token

That's it!. You should now see the astra-traffic-collector container running. Create "sensor" integrations from here and integrate it with astra-traffic-collector to seamlessly monitor traffic.

Customizing the behavior of astra-traffic-collector

To customize the traffic collector edit the config_custom.yaml. Refer here

Start the traffic collector by running start traffic-collector service

Upgrade

Docker container upgrade

This process updates the running container to a new version of the collector while retaining any customization you've made via configuration files, such as custom config.yaml_.

Change directory where docker-compose.yaml is present: cd /opt/astra-traffic-collector/

Run docker-compose down on the docker-compose.yaml to stop the container

Edit the image version field in services.otel-sensor.image of docker-compose.yaml to the latest image version and run the docker-compose up command to spin up the new container.

Image upgrade

Refers to upgrading the virtual machine image used for the Traffic Collector. This process typically involves creating a new VM image with updated software. Customizations made on the existing VM will be lost as the VM is replaced with a new image.

You can use GUI and also use cli tools like awscli, gcloud, az to pull a new VM using the Image from the cloud provider's marketplace.

To upgrade the image using AWS CLI:

aws ec2 describe-images --filters "Name=name,Values=YourImageName"
aws ec2 terminate-instances --instance-ids i-1234567890abcdef0
aws ec2 run-instances --image-id ami-0123456789abcdef0

To upgrade the image using Google Cloud CLI:

gcloud compute instances stop INSTANCE_NAME
gcloud compute instances delete INSTANCE_NAME
gcloud compute instances create INSTANCE_NAME --image-family=IMAGE_FAMILY --image-project=IMAGE_PROJECT

To upgrade the image using Azure CLI:

az vm deallocate --resource-group myResourceGroup --name myVM
az vm delete --resource-group myResourceGroup --name myVM
az vm create --resource-group myResourceGroup --name myVM --image IMAGE_URN

To upgrade the image using DigitalOcean CLI:

doctl compute droplet-action shutdown DROPLET_ID
doctl compute droplet delete DROPLET_ID
doctl compute droplet create --image IMAGE_SLUG --size droplet_size --region region_slug --ssh-keys key_id

Troubleshooting

Unable to send traces from traffic collector to ga collector

Symptoms

No entries in inventory/ inventory not getting updated

Following or similar error seen in traffic-collector container log

error	exporterhelper/queue_sender.go:92	Exporting failed. Dropping data.	{"kind": "exporter", "data_type": "traces", "name": "otlp", "error": "not retryable error: Permanent error: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: failed to get security token from token endpoint (endpoint \"https://kc-admin.getastra.dev/realms/astra_api_scanner/protocol/openid-connect/token\"); oauth2: \"unauthorized_client\" \"Invalid client or Invalid client credentials\"", "dropped_items": 1}

Cause

Authenication fails with IAM server

Solution

Edit /opt/astra-traffic-collector/.env and update it with right credentials.

Run start traffic-collector service

Unable to see entries in inventory

Symptoms

No entries in inventory/ inventory not getting updated

No error in nginx/traffic-collector log

Cause

Unregistered hostname

Solution

Double check if the hostname is registered under Scope URI for Report in Target setup page

FAQ (Frequently Asked Questions)

Can I see what trace are sent from my environment?

Yes, one can see the traces sent by traffic-collector by inspecting docker logs. Run docker logs <traffic-collector-container name>

Updated on: 13/11/2024