Introduction

This guide walks you through setting up Istio service mesh instrumentation to capture request and response flows using a Lua script embedded in an EnvoyFilter. This instrumentation allows you to send live traffic data to the Astra Traffic Collector (ATC) over OTLP/HTTP, providing deep observability into your HTTP transactions.

Prerequisites

• Access to a Kubernetes cluster where Istio is installed and configured as a sidecar service mesh.

• An active Astra Traffic Collector that is available and reachable within your environment. 📄 How to setup Astra Traffic Collector in Kubernetes

• An Istio instrumentation created in your Astra Dashboard to obtain a Sensor ID. 📄 How to Create Sensor Integration for API Observability

Illustration: High-level integration flow between Istio Service Mesh and Astra Traffic Collector

Instructions

Step 1: Configure OTLP/HTTP Receiver in Astra Traffic Collector

The traffic collector must be configured to receive telemetry data via OTLP over HTTP before proceeding.

📄 How To Set Up OTLP/HTTP Receiver in Astra Traffic Collector

Step 2: Enable External HTTP Calls in Istio

By default, Istio blocks outbound traffic to unknown external services. You must explicitly allow connections from Envoy to the collector.

Create a file named allow-otel-collector-calls.yaml and add the following YAML:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: allow-otel-collector-endpoint
  namespace: istio-system
spec:
  hosts:
  - "astra-traffic-collector.astra-collector"  # Replace with your collector's domain
  ports:
  - number: 4318
    name: http-otlp
    protocol: HTTP
  resolution: DNS
  location: MESH_INTERNAL

This ServiceEntry allows the Lua filter to make external HTTPS calls to your collector endpoint.

Note: If the Astra Traffic Collector is outside the Kubernetes cluster, set spec.location to MESH_EXTERNAL and change spec.ports.protocol to HTTPS.

Step 3: Create Istio Service Mesh Integration

This step uses an EnvoyFilter with a Lua script to intercept inbound and outbound HTTP traffic and capture both request and response bodies.

Create a file named mesh-integration.yaml and add the following YAML.

NOTE: Before filling in the values, copy the YAML into a text editor and search for the keyword "Astra otel configuration" — it appears in two places. Those are the locations where you need to make changes as described in the table below.

# -----------------------------------------------------------------------------
# EnvoyFilter for Astra OpenTelemetry Tracing
#
# This configuration injects a Lua filter into the Istio Service Mesh.
# The Lua script captures request/response metadata (headers, body, timing, etc.)
# and forwards them as OpenTelemetry (OTEL) spans to the configured collector.
#
# Why this is needed:
# - Istio does not natively capture full request/response bodies for tracing.
# - With this filter, Astra can provide deep observability into HTTP transactions.
# - Spans are enriched with HTTP, network, and custom attributes before being sent.
# -----------------------------------------------------------------------------

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: astra-istio-mesh-filter
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      # -----------------------------------------------------------------------
      # Select the Kubernetes Applications with Istio Service Mesh to apply this filter on.
      # Replace "productpage" with the actual Application label.
      #
      # Only the Applications with this label will have the Lua OTEL filter injected.
      # -----------------------------------------------------------------------
      # Astra otel configuration. Replace this "productpage" with the actual app selector that needs to be observed. 

      app: productpage
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.lua
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
            inlineCode: |

              -------------------------------------------------------------------
              -- Configuration Parameters
              -- Tune these values for body size limits, sampling, and collector URL
              -------------------------------------------------------------------

              local config = {
                -- Sampling ratio: value between 0 to 1
                -- 0 = no requests sampled
                -- 1 = all requests sampled (100%)
                -- 0.5 = 50% of requests sampled
                sampling_ratio = 1,

                -- Protocol scheme for the requests
                -- Must be either "http" or "https"
                -- Leave empty to auto-detect from request
                scheme = "",

                otel = {
                  -- Astra otel configuration
                  -- Example: "http://astra-traffic-collector.astra-collector:4318/v1/traces"
                  collectorUrl = "",
                  resource = {
                    -- replace your-sensor-id-uuid with actual sensorID
                    sensor_id = "your-sensor-id-uuid",
                  }
                }
              }

              -------------------------------------------------------------------
              -- Header Allowlist
              -- Only these headers will be recorded in spans.
              -- Covers common request/response, tracing, security, and CORS headers.
              -------------------------------------------------------------------

              local header_names = {
                -- Request headers
                "accept", "accept-encoding", "accept-language", "cache-control",
                "connection", "content-length", "content-type", "cookie",
                "host", "pragma", "referer", "user-agent", "origin", "authorization",
                "if-match", "if-none-match", "if-modified-since", "if-unmodified-since",
                "range", "te", "upgrade", "via", "warning",
                
                -- Response headers
                "age", "allow", "cache-control", "content-disposition", "content-encoding",
                "content-language", "content-length", "content-location", "content-range",
                "content-type", "date", "etag", "expires", "last-modified", "location",
                "server", "set-cookie", "vary", "connection", "keep-alive", "proxy-authenticate",
                "proxy-authorization", "transfer-encoding", "upgrade", "via", "warning",
                
                -- Tracing headers
                "x-request-id",
                
                -- Forwarded headers
                "forwarded", "x-forwarded-for", "x-forwarded-host", "x-forwarded-proto",
                "x-forwarded-port", "x-forwarded-client-cert", "x-real-ip",
                
                -- Envoy headers
                "x-envoy-upstream-service-time", "x-envoy-internal", "x-envoy-decorator-operation",
                "x-envoy-expected-rq-timeout-ms", "x-envoy-original-path", "x-envoy-attempt-count",
                "x-envoy-external-address", "x-envoy-force-trace", "x-envoy-original-dst-host",
                "x-envoy-max-retries", "x-envoy-retry-on", "x-envoy-retriable-status-codes",
                "x-envoy-retriable-header-names", "x-envoy-retry-grpc-on", "x-envoy-downstream-service-cluster",
                "x-envoy-downstream-service-node", "x-envoy-local-overloaded", "x-envoy-peer-metadata",
                "x-envoy-peer-metadata-id", "x-envoy-expected-rq-timeout-ms", "x-envoy-upstream-service-time",
                
                -- Istio headers
                "x-istio-attributes", "x-istio-request-id", "x-istio-auth-userinfo",
                
                -- CORS headers
                "access-control-allow-origin", "access-control-allow-methods", 
                "access-control-allow-headers", "access-control-max-age",
                "access-control-allow-credentials", "access-control-expose-headers",
                "access-control-request-method", "access-control-request-headers",
                
                -- Security headers
                "strict-transport-security", "x-content-type-options", "x-frame-options",
                "x-xss-protection", "content-security-policy", "referrer-policy",
                "cross-origin-resource-policy", "cross-origin-opener-policy",
                "cross-origin-embedder-policy",

                -- WebSocket headers
                "sec-websocket-key", "sec-websocket-version", "sec-websocket-protocol",
                "sec-websocket-accept", "sec-websocket-extensions"
              }

              -------------------------------------------------------------------
              -- should_sample()
              -- Randomly decides whether to trace this request based on sampling ratio.
              -------------------------------------------------------------------

              function should_sample()
                local random_value = math.random()
                return random_value <= (config.sampling_ratio or 1.0)
              end

              -------------------------------------------------------------------
              -- filter_headers()
              -- Filters request/response headers to only capture allowlisted ones.
              -------------------------------------------------------------------

              function filter_headers(headers)
                if not headers then return {} end
                local result = {}
                for _, name in ipairs(header_names) do
                  local val = headers:get(name)
                  if val then
                    result[name] = val
                  end
                end
                return result
              end

              -------------------------------------------------------------------
              -- random_hex()
              -- Generates a random hexadecimal string for trace/span IDs.
              -------------------------------------------------------------------

              function random_hex(length)
                local chars = "0123456789abcdef"
                local result = ""
                for i = 1, length do
                  local rand = math.random(1, #chars)
                  result = result .. string.sub(chars, rand, rand)
                end
                return result
              end

              -------------------------------------------------------------------
              -- envoy_on_request()
              -- Triggered at request ingress.
              -- Captures HTTP method, path, headers, body, and client info.
              -- Skips tracing for static paths or missing mandatory headers.
              -------------------------------------------------------------------

              function envoy_on_request(request_handle)
                local path = request_handle:headers():get(":path")
                if not should_sample() then
                  request_handle:logWarn("[Astra Instrumentation] Request not sampled due to sampling ratio. Path: " .. path)
                  return
                end

                if path:match("^/static") then
                  request_handle:logWarn("[Astra Instrumentation] Skipping static path: " .. path)
                  return
                end
              
                request_handle:logInfo("[Astra Instrumentation] Processing request for path: " .. path)

                local transaction = {
                  timestamp = os.date("!%Y-%m-%dT%H:%M:%S.000Z"),
                  protocol = "1.1",
                  peer_address = "",
                  trace_id = random_hex(32),
                  span_id = random_hex(16),
                  start_time = math.floor(os.time() * 1000000000),
                  request = {
                    target = "",
                    host = "",
                    method = "",
                    scheme = "",
                    headers = {},
                    body = "",
                    body_size = 0
                  },
                  response = {
                    status = "",
                    headers = {},
                    body = "",
                    body_size = 0
                  }
                }
                
                transaction.request.headers = filter_headers(request_handle:headers())
                
                -- Get request body
                local req_body = ""
                for chunk in request_handle:bodyChunks() do
                  local chunk_size = chunk:length()
                  if chunk_size > 0 then
                    req_body = req_body .. chunk:getBytes(0, chunk_size)
                  end
                end
 
                transaction.request.body = req_body
                transaction.request.body_size = #transaction.request.body

                transaction.request.method = request_handle:headers():get(":method") or ""
                transaction.request.target = request_handle:headers():get(":path") or ""
                transaction.request.host = request_handle:headers():get(":authority") or transaction.request.headers["host"] or ""
                transaction.request.scheme = config.scheme or request_handle:headers():get(":scheme") or "https"

                if transaction.request.headers["x-forwarded-proto"] == "https" then
                  transaction.request.scheme = "https"
                end

                if not transaction.request.method or not transaction.request.target or not transaction.request.host then
                  request_handle:logWarn("[Astra Instrumentation] Missing mandatory headers, skipping otel tracing. Path: " .. path)
                  transaction = nil
                  return
                end

                local forwarded_for = transaction.request.headers["x-forwarded-for"]
                if forwarded_for and forwarded_for ~= "" then
                  transaction.peer_address = string.match(forwarded_for, "^([^,]+)")
                elseif transaction.request.headers["x-envoy-external-address"] then
                  transaction.peer_address = transaction.request.headers["x-envoy-external-address"]
                elseif transaction.request.headers["x-real-ip"] then
                  transaction.peer_address = transaction.request.headers["x-real-ip"]
                end

                if transaction.request.headers["x-envoy-peer-metadata-id"] then
                  local meta_id = transaction.request.headers["x-envoy-peer-metadata-id"]
                  local service_name = string.match(meta_id, "sidecar~[^~]+~([^~]+)%.")
                  if service_name then
                    transaction.peer_address = service_name
                  end
                end

                -- Store the transaction in the dynamic metadata for later use in the response filter
                request_handle:streamInfo():dynamicMetadata():set("lua", "astra_transaction", transaction)
                request_handle:logDebug("[Astra Instrumentation] Transaction stored in metadata with trace_id: " .. transaction.trace_id .. ". Path: " .. path)
              end

              -------------------------------------------------------------------
              -- envoy_on_response()
              -- Triggered before sending response back.
              -- Captures response headers/body, finalizes span, and sends OTEL payload.
              -------------------------------------------------------------------

              function envoy_on_response(response_handle)
                -- Get the transaction from the dynamic metadata
                local transaction = response_handle:streamInfo():dynamicMetadata():get("lua")["astra_transaction"]

                -- If the transaction is not found or is nil, return
                if not transaction or transaction == nil then
                  response_handle:logWarn("[Astra Instrumentation] No astra_transaction found in metadata")
                  return
                end

                response_handle:logInfo("[Astra Instrumentation] Processing response for trace_id: " .. (transaction.trace_id or "unknown"))

                transaction.response.headers = filter_headers(response_handle:headers())
 
                -- Get response body (decompressor filter handles decompression)
                local resp_body = ""
                for chunk in response_handle:bodyChunks() do
                  local chunk_size = chunk:length()
                  if chunk_size > 0 then
                    resp_body = resp_body .. chunk:getBytes(0, chunk_size)
                  end
                end
 
                transaction.response.body = resp_body
                transaction.response.status = response_handle:headers():get(":status") or ""
                transaction.response.body_size = #transaction.response.body
                transaction.end_time = math.floor(os.time() * 1000000000)

                local otel_payload = {
                  resourceSpans = {
                    {
                      resource = {
                        attributes = {
                          { key = "service.name", value = { stringValue = "astra-istio-instrumentation" } },
                          { key = "service.version", value = { stringValue = "1.0.0" } }                
                        }
                      },
                      scopeSpans = {
                        {
                          scope = {
                            -- Name of your instrumented gateway. Helpful in identification of spans
                            name = "astra-istio-servicemesh"
                          },
                          spans = {
                            {
                              traceId = transaction.trace_id,
                              spanId = transaction.span_id,
                              name = "example-span",
                              kind = 1,
                              startTimeUnixNano = string.format("%.0f", transaction.start_time),
                              endTimeUnixNano = string.format("%.0f", transaction.end_time),
                              attributes = {
                                { key = "sensor.id", value = { stringValue = config.otel.resource.sensor_id } },
                                { key = "http.method", value = { stringValue = transaction.request.method } },
                                { key = "http.status_code", value = { intValue = tonumber(transaction.response.status) or 0 } },
                                { key = "http.target", value = { stringValue = transaction.request.target } },
                                { key = "http.host", value = { stringValue = transaction.request.host } },
                                { key = "http.flavor", value = { stringValue = transaction.protocol } },
                                { key = "http.scheme", value = { stringValue = transaction.request.scheme } },
                                { key = "net.sock.peer.addr", value = { stringValue = transaction.peer_address } },
                                { key = "http.request.headers", value = { stringValue = json_encode(transaction.request.headers) } },
                                { key = "http.response.headers", value = { stringValue = json_encode(transaction.response.headers) } },
                                { key = "http.request.body", value = { stringValue = transaction.request.body } },
                                { key = "http.response.body", value = { stringValue = transaction.response.body } }
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }

                local json_payload = json_encode(otel_payload)
                
                local protocol, host, port, path = string.match(config.otel.collectorUrl, "(%w+)://([^:/]+):?(%d*)(/?.*)")
                local cluster_name = "outbound|" .. port .. "||" .. host

                local headers = {
                  [":method"] = "POST",
                  [":path"] = path,
                  [":authority"] = host,
                  ["content-type"] = "application/json",
                  ["content-length"] = tostring(#json_payload)
                }
                
                local success, result = pcall(function()
                  local resp_headers, resp_body = response_handle:httpCall(
                    cluster_name,
                    headers,
                    json_payload,
                    5000
                  )
                
                  local status = resp_headers[":status"]
                  if status then
                    local status_code = tonumber(status)
                    if status_code and status_code >= 200 and status_code < 300 then
                      response_handle:logDebug("[Astra Instrumentation] OTEL trace sent successfully for trace_id: " .. transaction.trace_id .. ", status: " .. status)
                    else
                      response_handle:logErr("[Astra Instrumentation] OTEL collector returned error status: " .. status .. " for trace_id: " .. transaction.trace_id)
                    end
                  else
                    response_handle:logWarn("[Astra Instrumentation] OTEL call completed but no status code found")
                  end
                end)
              
                if not success then
                  response_handle:logErr("[Astra Instrumentation] httpCall failed: " .. tostring(result))
                end

                transaction = nil
              end

              -------------------------------------------------------------------
              -- JSON Encoding
              -- Proper JSON encoding implementation with escape handling
              -------------------------------------------------------------------

              local escape_char_map = {
                [ "\\" ] = "\\\\",
                [ "\"" ] = "\\\"",
                [ "\b" ] = "\\b",
                [ "\f" ] = "\\f",
                [ "\n" ] = "\\n",
                [ "\r" ] = "\\r",
                [ "\t" ] = "\\t",
              }

              local function escape_char(c)
                return escape_char_map[c] or string.format("\\u%04x", c:byte())
              end

              local function encode_nil(val)
                return "null"
              end

              local function encode_string(val)
                return '"' .. val:gsub('[%z\1-\31\\"]', escape_char) .. '"'
              end

              local function encode_number(val)
                -- Check for NaN, -inf and inf
                if val ~= val or val <= -math.huge or val >= math.huge then
                  return "null"
                end
                return string.format("%.14g", val)
              end

              local encode

              local function encode_table(val, stack)
                local res = {}
                stack = stack or {}

                -- Circular reference check
                if stack[val] then
                  return "null"
                end

                stack[val] = true

                local is_array = false
              
                if rawget(val, 1) ~= nil or next(val) == nil then
                  -- Check if it's a valid array
                  local n = 0
                  is_array = true
                  for k in pairs(val) do
                    if type(k) ~= "number" then
                      is_array = false
                      break
                    end
                    n = n + 1
                  end
                  if is_array and n ~= #val then
                    is_array = false
                  end
                end

                if is_array then
                  -- Encode as array
                  for i, v in ipairs(val) do
                    table.insert(res, encode(v, stack))
                  end
                  stack[val] = nil
                  return "[" .. table.concat(res, ",") .. "]"
                else
                  -- Encode as object
                  for k, v in pairs(val) do
                    if type(k) == "string" then
                      table.insert(res, encode_string(k) .. ":" .. encode(v, stack))
                    end
                  end
                  stack[val] = nil
                  return "{" .. table.concat(res, ",") .. "}"
                end
              end

              local type_func_map = {
                [ "nil"     ] = encode_nil,
                [ "table"   ] = encode_table,
                [ "string"  ] = encode_string,
                [ "number"  ] = encode_number,
                [ "boolean" ] = tostring,
              }

              encode = function(val, stack)
                local t = type(val)
                local f = type_func_map[t]
                if f then
                  return f(val, stack)
                end
                return "null"
              end

              function json_encode(val)
                return encode(val)
              end

Step 4: Apply the Configuration

Deploy both configuration files to your cluster in the following order:

# First, apply the ServiceEntry to allow external calls
kubectl apply -f allow-external-calls.yaml

# Then, apply the EnvoyFilter
kubectl apply -f mesh-integration.yaml

Expected Outcome

Once the configurations are applied, the selected application will begin capturing headers and bodies for its HTTP traffic and forwarding them to the Astra Traffic Collector. As and when the selected app receives HTTP traffic, you should be able to see the traces in the collector

1. 📄 Verifying Traces in Astra Traffic Collector

Troubleshooting & Best Practices

• Traces Not Appearing: Verify that the sensor_id and collectorUrl are correctly entered in the Lua script and that the Astra Traffic Collector is reachable from the Envoy proxy.

• Sampling Volume: You can tune the sampling_ratio value in the Lua script between 0 and 1 to control the volume of traffic forwarded for analysis. A value of 1 captures all requests; 0.5 captures 50%.

• Static Path Skipping: The Lua script automatically skips paths beginning with /static. Be aware of this if your application serves meaningful traffic under such paths.

• Missing Mandatory Headers: If a request is missing the method, path, or host headers, the Lua script will skip tracing for that request. This is logged as a warning in Envoy's logs.

• Collector Outside the Cluster: If your Astra Traffic Collector is deployed outside the Kubernetes cluster, ensure spec.location is set to MESH_EXTERNAL and spec.ports.protocol is set to HTTPS in the ServiceEntry.

• Namespace Mismatch: Ensure both the ServiceEntry and EnvoyFilter are deployed in the same namespace as your Istio gateway, typically istio-system.

• Apply Order: Always apply the ServiceEntry before the EnvoyFilter. Applying them in the wrong order may result in the Lua script failing to reach the collector on first execution.