Customizing Traffic Collector Configuration

Sensor can be customized to filter, templatize and redact the traces generated from traffic. These customization are present to give flexibility to the customer to define a rule to choose a trace to be monitored. Similarly, personally identified information in the trace can be redacted before it reaches Astra's server. This way it can be ensured that no sensitive data leaves customers environment. Lets look at configuring each of them in detail.

Following config_custom.yaml needs to be created locally. All the further suggested customization can be done to this file.

processors:
  
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/filterprocessor
  filter/custom:
    error_mode: ignore
    traces:
      span:
        ## allowing traces based on hostname regex pattern. Following will drop all traces originated from host other than: localhost*
        # - IsMatch(attributes["url.host"], "localhost*") == false
        ## excluding traces based on hostname regex pattern. Following will drop all traces originated from host: localhost*
        # - IsMatch(attributes["url.host"], "localhost*")
        ## excluding traces based on template regex pattern. Following will drop all traces having url_template: _wdt*
        # - IsMatch(attributes["url.template"], "_wdt*")
        ## exclude traces with method set to OPTIONS. Comment below line to allow the traces with http method OPTIONS
        - ConvertCase(attributes["http.method"], "upper") == "OPTIONS"
  
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor
  transform/custom:
    error_mode: ignore
    trace_statements:
      - context: span
        statements:
          ## Templatize url path regex pattern by keyword: "slug". Following will templatize the url Path: /api/v1/chinchikrqwertyuiop/ -> /api/v1/{{slug}}/
          - replace_pattern(attributes["url.template"], "chinchikrqwertyuiop", "{{slug}}")
          ## Redact MasterCard credit card number
          #- replace_all_patterns(attributes, "value", "^5[1-5][0-9]{14}$", "{{card}}")

Filtering

Filtering can be achieved based on a trace attribute: "url.host" or "url.template". Primarily, filtering can be divided into AllowListing and ExcludeListing.

AllowListing refers to allowing the traces orginated from the specfifc host/pattern.
ExcludeListing refers to excluding the traces matching specific host/pattern.

Steps to perform allow listing and deny listing

Edit the custom_config.yaml as shown below:

processors:
  filter/custom:
      error_mode: ignore
      traces:
        span:
          ## allowing traces based on hostname regex pattern. Following will drop all traces originated from host other than: localhost*
          - IsMatch(attributes["url.host"], "localhost*") == false
          ## excluding traces based on hostname regex pattern. Following will drop all traces originated from host: localhost*
          - IsMatch(attributes["url.host"], "localhost*")
          ## excluding traces based on template regex pattern. Following will drop all traces having url_template: _wdt*
          - IsMatch(attributes["url.template"], "_wdt*")

Templatizing

Templatizing is a useful way to templatize the API Path. Although Astra takes care of templatizing integer, UUID, ULID, Timestamp etc, still there is a flexibility for the customers to write their own templates for the pattern that they are aware of.

Steps to perform allow listing and deny listing

Edit the custom_config.yaml as shown below:

processors:
  transform/custom:
    error_mode: ignore
    trace_statements:
      - context: span
        statements:
          ## Templatize url path regex pattern by keyword: "slug". Following will templatize the url Path: /api/v1/chinchikrqwertyuiop/ -> /api/v1/{{slug}}/
          - replace_pattern(attributes["url.template"], "chinchikr*", "{{slug}}")
          ## Redact MasterCard credit card number
          #- replace_all_patterns(attributes, "value", "^5[1-5][0-9]{14}$", "{{card}}")

Redacting

Redacting can be achieved by finding the right regex pattern to be redacted. Redaction ensures that sensitive information are not leaving customers environment

Steps to perform allow listing and deny listing

Edit the custom_config.yaml as shown below:

processors:
  transform/custom:
    error_mode: ignore
    trace_statements:
      - context: span
        statements:
          ## Redact MasterCard credit card number
          - replace_all_patterns(attributes, "value", "^5[1-5][0-9]{14}$", "{{CreditCard}}")

Updated on: 05/12/2024