How to setup Astra Traffic Collector in Kubernetes

Here's how to configure Astra Traffic Collector for traffic monitoring, in Kubernetes environments such as EKS, GKE, AKE. It also covers troubleshooting common issues encountered during configuration.

Traffic Collector integration

Quick Installation

Create astra-collector namespace for traffic collector installation by running

kubectl create ns astra-collector

Add astra traffic collector helm repository by running

helm repo add getastra https://raw.githubusercontent.com/getastra/obs-deployments/gh-pages/

Update the repo

helm repo update

Create values.yaml
Replace <collectorId> with the COLLECTOR_ID displayed during the creation of astra traffic collector integration.
Replace <clientId> with the CLIENT_ID displayed during the creation of astra traffic collector integration.
Replace <clientSecret> with the CLIENT_SECRET displayed during the creation of astra traffic collector integration.

secret:
      name: astra-collector-secrets
      collectorId: <collectorId>
      clientId: <clientId>
      clientSecret: <clientSecret>
      tokenUrl: https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token
      remoteAddrIdentifierHeader: x-forwarded-for

Install the helm chart by running:

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

That's it!. You should now see the astra-traffic-collector pod running under astra-collector namespace. Create "sensor" integrations from here and integrate it with astra-traffic-collector to seamlessly monitor traffic.

Customizing the behavior of astra-traffic-collector

To have a customized config, create and edit config_custom.yaml. Refer here

Create configmap containing content of config_custom.yaml file, if it's updated in previous step. Run

kubectl create configmap astra-collector-custom-config --from-file=./config_custom.yaml -n astra-collector

To mount the customized configuration, create values.yaml with following
Replace <collectorId> with the COLLECTOR_ID displayed during the creation of astra traffic collector integration.
Replace <clientId> with the CLIENT_ID displayed during the creation of astra traffic collector integration.
Replace <clientSecret> with the CLIENT_SECRET displayed during the creation of astra traffic collector integration.

secret:
    name: astra-collector-secrets
    collectorId: <collectorId>
    clientId: <clientId>
    clientSecret: <clientSecret>
    tokenUrl: https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token
    remoteAddrIdentifierHeader: x-forwarded-for

volumes:
  - configMap:
      defaultMode: 444
      name: astra-collector-custom-config
    name: custom-config
  
volumeMounts:
  - name: collector-message
    mountPath: /var/lib/otelcol/file_storage
  - name: custom-config
    mountPath: /etc/otelcol-contrib/config_custom.yaml
    subPath: config_custom.yaml

Upgrade the traffic collector by giving updated values.yaml

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

Upgrade

Upgrading traffic collector helm chart to latest version:

Update the helm repo

helm repo update

Upgrade the chart to latest version

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

Troubleshooting

Unable to send traces from traffic collector to ga-collector

Symptoms

No entries in inventory/ inventory not getting updated

Following error is seen in astra-traffic-collector container log

error	exporterhelper/queue_sender.go:92	Exporting failed. Dropping data.	{"kind": "exporter", "data_type": "traces", "name": "otlp", "error": "not retryable error: Permanent error: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: failed to get security token from token endpoint (endpoint \"https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token\"); oauth2: \"unauthorized_client\" \"Invalid client or Invalid client credentials\"", "dropped_items": 1}

Cause

Authenication fails with IAM server

Solution

update the values.yaml with right credentials and then run helm upgrade:

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

Unable to see entries in inventory

Symptoms

No entries in inventory/ inventory not getting updated

No error in nginx/traffic-collector log

Cause

Unregistered hostname

Solution

Double check if the hostname is registered under Scope URI for Report in Target setup page

Add the hostname under extra hosts to be scanned if it's not registered in the first place

FAQ (Frequently Asked Questions)

Can I see what trace are sent from my environment?

Yes, one can see the traces sent by astra-traffic-collector by inspecting logs. Run kubectl logs astra-traffic-collector-0 -n astra-collector to see the logs.

How to regenerate client secret for astra-traffic-collector integration?

Go to integrations.
Click on "vertical three dots" of the astra-traffic-collector integration for which client secrets should be regenerated
Click on "Regenerate client secret"
Copy down the secrets shown
Update the values.yaml with new secrets. After updating your values.yaml should look something like this:

secret:
  name: astra-collector-secrets
  collectorId: <collectorId>
  clientId: <clientId>
  clientSecret: <clientSecret>
  tokenUrl: https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token
  remoteAddrIdentifierHeader: x-forwarded-for

#any other changes here

Run the helm upgrade command with latest valuues.yaml which has updated client secret

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

Updated on: 03/12/2024