Here's how to configure Astra Traffic Collector for traffic monitoring inside a Mac based VM. It also covers troubleshooting common issues encountered during configuration.
SSH into the VM.
Pre-requisite
Install Docker Desktop version 4.34 or later in your machine by following the official doc.
Follow these five steps to enable host network mode which is essential
Sign in to your Docker account in Docker Desktop.
Navigate to Settings.
Under the Resources tab, select Network.
Check the Enable host networking option.
Select Apply and restart.
Create a directory for the Astra Traffic Collector
||| Please choose /var or /tmp as working directory since Mac doesn't expose all path for volume mount.
mkdir -p /var/astra-traffic-collector && cd /var/astra-traffic-collector
** Create docker-compose.yaml file**
version: '3.3'
services:
my-service-init:
image: busybox:1.35.0-uclibc
user: root
volumes:
- "collector-message:/tmp/message_data"
command: chown -R 10001:10001 /tmp/message_data
otel-sensor:
platform: linux/amd64
image: docker.io/getastra/traffic-collector
container_name: astra-traffic-collector
volumes:
- "collector-message:/var/lib/otelcol/file_storage:z"
- "/var/astra-traffic-collector/config_custom.yaml:/etc/otelcol-contrib/config_custom.yaml:ro"
network_mode: host
env_file:
- .env
restart: always
depends_on:
- my-service-init
volumes:
collector-message:
Create .env file
Replace with the COLLECTOR_ID displayed during the creation of astra traffic collector integration.
Replace with the CLIENT_ID displayed during the creation of astra traffic collector integration.
Replace with the CLIENT_SECRET displayed during the creation of astra traffic collector integration.
COLLECTOR_ID=
CLIENT_ID=
CLIENT_SECRET=
TOKEN_URL=https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token
REMOTE_ADDR_IDENTIFIER_HEADER=x-forwarded-for
Create config_custom.yaml file
processors:
# https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/filterprocessor
filter/custom:
error_mode: ignore
traces:
span:
## allowing traces based on hostname regex pattern. Following will drop all traces originated from host other than: localhost*
# - IsMatch(attributes["url.host"], "localhost*") == false
## excluding traces based on hostname regex pattern. Following will drop all traces originated from host: localhost*
# - IsMatch(attributes["url.host"], "localhost*")
## excluding traces based on template regex pattern. Following will drop all traces having url_template: _wdt*
# - IsMatch(attributes["url.template"], "_wdt*")
## exclude traces with method set to OPTIONS. Comment below line to allow the traces with http method OPTIONS
- ConvertCase(attributes["http.method"], "upper") == "OPTIONS"
# https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor
transform/custom:
error_mode: ignore
trace_statements:
- context: span
statements:
## Templatize url path regex pattern by keyword: "slug". Following will templatize the url Path: /api/v1/chinchikrqwertyuiop/ -> /api/v1/{{slug}}/
- replace_pattern(attributes["url.template"], "chinchikrqwertyuiop", "{{slug}}")
## Redact MasterCard credit card number
#- replace_all_patterns(attributes, "value", "^5[1-5][0-9]{14}$", "{{card}}")
Manage the service by using docker compose command
To start the service, run: docker compose up -d
To stop the service, run: docker compose down
To check the status, run: docker ps -a
To check the logs, run: docker logs astra-traffic-collector
** Change to working directory **
cd /var/astra-traffic-collector
** Stop the astra-traffic-collector by running **
docker compose down
**To customize the traffic collector edit the config_custom.yaml. Refer here **
**Start the traffic collector by running **
docker compose up -d
| This process updates the astra-traffic-collector container to a new version of the collector while retaining any customization you've made via configuration files, such as custom_config.yaml
**Execute the following commands to stop, upgrade and then start the service **
cd /var/astra-traffic-collector/
docker compose down
docker-compose pull
docker compose up -d
**Unable to send traces from traffic collector to ga collector **
Symptoms
No entries in inventory/ inventory not getting updated
Following or similar error seen in traffic-collector container log
error exporterhelper/queue_sender.go:92 Exporting failed. Dropping data. {"kind": "exporter", "data_type": "traces", "name": "otlp", "error": "not retryable error: Permanent error: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: failed to get security token from token endpoint (endpoint \"https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token\"); oauth2: \"unauthorized_client\" \"Invalid client or Invalid client credentials\"", "dropped_items": 1}
Cause
Authenication fails with IAM server
Solution
Edit /var/astra-traffic-collector/.env and update it with right credentials.
Restart astra-traffic-collector
docker compose down
docker compose up -d
Unable to see entries in inventory
Symptoms
No entries in inventory/ inventory not getting updated
No error in nginx/traffic-collector log
Cause
Unregistered hostname
Solution
Double check if the hostname is registered under Scope URI for Report in Target setup page
How to restart astra-traffic-collector
Execute the following commands to restart astra-traffic-collector
cd /var/astra-traffic-collector/
docker compose down
docker compose up -d
Can I see what trace are sent from my environment?
Yes, one can see the traces sent by traffic-collector by inspecting docker logs. Run docker logs astra-traffic-collector