How to setup Astra Traffic Collector in Kubernetes

Here's how to configure Astra Traffic Collector for traffic monitoring, in Kubernetes environments such as EKS, GKE, AKE. It also covers troubleshooting common issues encountered during configuration.

Traffic Collector integration

Quick Installation

Create a working directory and switch to it.

mkdir -p /opt/astra-traffic-collector && cd /opt/astra-traffic-collector

Create astra-collector namespace for traffic collector installation by running

kubectl create ns astra-collector

Add astra traffic collector helm repository by running

helm repo add getastra https://raw.githubusercontent.com/getastra/obs-deployments/gh-pages/

Update the repo

helm repo update

Create values.yaml
Replace <collectorId> with the COLLECTOR_ID displayed during the creation of astra traffic collector integration.
Replace <clientId> with the CLIENT_ID displayed during the creation of astra traffic collector integration.
Replace <clientSecret> with the CLIENT_SECRET displayed during the creation of astra traffic collector integration.

secret:
      name: astra-collector-secrets
      collectorId: <collectorId>
      clientId: <clientId>
      clientSecret: <clientSecret>
      tokenUrl: https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token
      remoteAddrIdentifierHeader: x-forwarded-for

volumes:
  - configMap:
      defaultMode: 444
      name: astra-collector-custom-config
    name: custom-config
  
volumeMounts:
  - name: collector-message
    mountPath: /var/lib/otelcol/file_storage
  - name: custom-config
    mountPath: /etc/otelcol-contrib/config_custom.yaml
    subPath: config_custom.yaml

Create config_custom.yaml file as shown below.

processors:
  
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/filterprocessor
  filter/custom:
    error_mode: ignore
    traces:
      span:
        ## allowing traces based on hostname regex pattern. Following will drop all traces originated from host other than: localhost*
        # - IsMatch(attributes["url.host"], "localhost*") == false
        ## excluding traces based on hostname regex pattern. Following will drop all traces originated from host: localhost*
        # - IsMatch(attributes["url.host"], "localhost*")
        ## excluding traces based on template regex pattern. Following will drop all traces having url_template: _wdt*
        # - IsMatch(attributes["url.template"], "_wdt*")
        ## exclude traces with method set to OPTIONS. Comment below line to allow the traces with http method OPTIONS
        - ConvertCase(attributes["http.method"], "upper") == "OPTIONS"
  
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor
  transform/custom:
    error_mode: ignore
    trace_statements:
      - context: span
        statements:
          ## Templatize url path regex pattern by keyword: "slug". Following will templatize the url Path: /api/v1/chinchikrqwertyuiop/ -> /api/v1/{{slug}}/
          - replace_pattern(attributes["url.template"], "chinchikrqwertyuiop", "{{slug}}")
          ## Redact MasterCard credit card number 
          #- replace_all_patterns(attributes, "value", "^5[1-5][0-9]{14}$", "{{card}}")

Create configmap containing content of config_custom.yaml file

kubectl create configmap astra-collector-custom-config --from-file=./config_custom.yaml -n astra-collector

Install the helm chart by running:

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

That's it!. You should now see the astra-traffic-collector pod running under astra-collector namespace. Create "sensor" integrations from here and integrate it with astra-traffic-collector to seamlessly monitor traffic.

Customizing the behavior of astra-traffic-collector

Edit /opt/astra-traffic-collector/config_custom.yaml. Refer here to know how to filter, redact and templatize

Update configmap containing content of config_custom.yaml file

kubectl create configmap astra-collector-custom-config --from-file=/opt/astra-traffic-collector/config_custom.yaml -n astra-collector --dry-run=client -o yaml | kubectl apply -f -
kubectl delete po astra-traffic-collector -n astra-collector

Upgrade

Upgrading traffic collector helm chart to latest version:

Update the helm repo

helm repo update

Upgrade the chart to latest version

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

Troubleshooting

Unable to send traces from traffic collector to ga-collector

Symptoms

No entries in inventory/ inventory not getting updated

Following error is seen in astra-traffic-collector container log

error	exporterhelper/queue_sender.go:92	Exporting failed. Dropping data.	{"kind": "exporter", "data_type": "traces", "name": "otlp", "error": "not retryable error: Permanent error: rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: failed to get security token from token endpoint (endpoint \"https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token\"); oauth2: \"unauthorized_client\" \"Invalid client or Invalid client credentials\"", "dropped_items": 1}

Cause

Authenication fails with IAM server

Solution

update the values.yaml with right credentials and then run helm upgrade:

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

Unable to see entries in inventory

Symptoms

No entries in inventory/ inventory not getting updated

No error in nginx/traffic-collector log

Cause

Unregistered hostname

Solution

Double check if the hostname is registered under Scope URI for Report in Target setup page

Add the hostname under extra hosts to be scanned if it's not registered in the first place

FAQ (Frequently Asked Questions)

Can I see what trace are sent from my environment?

Yes, one can see the traces sent by astra-traffic-collector by inspecting logs. Run kubectl logs astra-traffic-collector-0 -n astra-collector to see the logs.

How to regenerate client secret for astra-traffic-collector integration?

Go to integrations.
Click on "vertical three dots" of the astra-traffic-collector integration for which client secrets should be regenerated
Click on "Regenerate client secret"
Copy down the secrets shown
Update the values.yaml with new secrets. After updating your values.yaml should look something like this:

secret:
  name: astra-collector-secrets
  collectorId: <collectorId>
  clientId: <clientId>
  clientSecret: <clientSecret>
  tokenUrl: https://auth.getastra.com/realms/astra_api_scanner/protocol/openid-connect/token
  remoteAddrIdentifierHeader: x-forwarded-for

#any other changes here

Run the helm upgrade command with latest valuues.yaml which has updated client secret

helm upgrade --install traffic-collector getastra/traffic-collector-chart --namespace astra-collector --debug --values values.yaml

Updated on: 19/02/2025