Overview
Kong Instrumentation enhances API monitoring by capturing detailed HTTP traffic data through OpenTelemetry. It provides granular visibility into requests and responses while maintaining performance through selective data capture and size limits.
Architecture
Prerequisites
Kong Gateway installed and running
OpenTelemetry Collector configured
Access to Kong configuration (Kong Manager)
A service and route pre-configured to setup tracing
Ensure Kong Manager GUI is running and accessible (default port: 8002) If Kong Manager is not accessible:
# Update your Kong container configuration to expose port 8002
docker run -d --name kong \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
-p 8000:8000 \
-p 8443:8443 \
-p 8001:8001 \
-p 8002:8002 \ # Add this line if missing
kong:latest
Setting up OTLP/HTTP Receiver in Astra Traffic Collector
You can configure the Astra Traffic Collector to receive data over HTTP or HTTPS. For production environments, HTTPS is recommended for secure data transmission
Option 1: HTTP Configuration (Development/Testing)
Open config_custom.yaml in your traffic collector's instance and update the receivers section
receivers:
otlp:
protocols:
http:
endpoint: "0.0.0.0:4318" # HTTP receiver on port 4318
processors:
#...
save the file once edited and restart the collector using systemctl restart astra-traffic-collector.service
Option 2: HTTPS Configuration (Recommended for Production)
Update the same config_custom.yaml to use HTTPS with your cert and key files from the trusted authority:
receivers:
otlp:
protocols:
http:
endpoint: "0.0.0.0:4318" # HTTP receiver on port 4318
tls:
cert_file: "/etc/otelcol-contrib/"
key_file: "/etc/otelcol-contrib/"
The third field is ca_file, which is used when the certificate is self-signed or from an untrusted CA. If your certificate is issued by a trusted CA (e.g., Let's Encrypt), you don’t need to specify ca_file.
|| Ensure the certificate files are correctly placed in /etc/otelcol-contrib/ and have proper permissions. The private key should be readable only by the owner (for security reasons) & The certificate can be readable by others but should not be writable:
Next, modify your volume section in the docker-compose.yaml by adding the following lines in your Astra Traffic Collector to include volume mounts for the certificates:
volumes:
- /:/etc/otelcol-contrib/:ro
- /:/etc/otelcol-contrib/:ro
save the file once edited and restart the collector using systemctl restart astra-traffic-collector.service
Quick Installation
Set Required Environment Variable
# First, access your Kong container
docker exec -it kong bash
# Then set the environment variable for HTTP instrumentation
export KONG_TRACING_INSTRUMENTATIONS=http_client
Access Kong Manager Interface
Open your web browser
Navigate to
http://:8002From the main dashboard, click on "Plugins" in the left sidebar
Click the "New Plugin" button
In the plugin catalog, search for and select "OpenTelemetry"
Choose between the two Plugin Scopes
Global: Plugin will apply to all services, routes, and consumers - Recommended for system-wide tracing
Scoped: Plugin will apply to specific targets - Select specific Gateway Services - Select specific Routes - Select specific Consumers - Useful for granular control of tracing
Configure OpenTelemetry SettingsRequired Endpoints (at least one must be configured):
Traces Endpoint: Enter your OpenTelemetry collector endpoint
Example:
https://astra-traffic-collector-ip-address:4318/v1/traces
Logs Endpoint (optional): Enter if you want to collect logs
Protocol Configuration: - Select all applicable protocols: grpc, grpcs, http, https - These determine which types of requests will be traced
Propagation Settings: - Set Default Format to "w3c" (recommended standard) - Header Type: "w3c" - Configure any custom headers if needed
Resource Attributes: - sensor.version: Set to your current version to "1.0.0" - service.name: Name of the service should be set to "astra-otel-plugin" - sensor.id: Set sensor.id using the value provided at the time of creating the Apigee Integration (e.g., "05927cc2-cf2f-4508-a11e-3001964b9113")
**Advanced Settings (adjust based on your needs) **: Batch Span Count: 1 Connect Timeout: 1000 ms Max Batch Size: 200 Max Entries: 10000 Initial Retry Delay: 0.01 Max Retry Delay: 60 Max Retry Time: 60 Sampling Rate: Set between 0.0 and 1.0 Read/Send Timeouts: 5000 ms
Review and Save
Click "View Configuration" to review all settings
Verify all parameters are correctly set
Click "Save" to apply the configuration
The plugin will be immediately active based on your scope selection
Verify Configuration
Check the plugin appears in your plugins list
Status should show as "Enabled"
Make a test request through your Kong gateway
Verify traces are appearing in your OpenTelemetry collector
Troubleshooting
Verify your collector endpoint is accessible from Kong
Check Kong's error logs for connection issues
Ensure sampling rate is greater than 0
Verify protocols match your service configuration
|| After setting up the OpenTelemetry plugin, you need to configure the Kong Functions plugin to properly instrument the tracing:
Create Kong Functions Plugin
Return to Kong Manager (
http://:8002)Navigate to "Plugins"
Click "New Plugin"
Search and select "Kong Functions"
Choose the same scope as your OpenTelemetry plugin (Global or Scoped)
Configure Access Phase Function
Find the "Access" field and paste the following code
-- Access phase handler
local root_span = kong.tracing.active_span()
if not root_span then
kong.log.err("No active span found")
return
end
-- Get direct client IP (load balancer or actual client)
local peer_addr = kong.client.get_ip()
kong.log.debug("Direct client IP: ", peer_addr)
if peer_addr then
root_span:set_attribute("net.sock.peer.addr", peer_addr)
end
-- Add http.target
root_span:set_attribute("http.target", kong.request.get_path_with_query())
local function escape_json_value(v)
if type(v) == "string" then
return (v:gsub('"', '\\"'))
end
return v
end
local headers = kong.request.get_headers()
if headers then
local headers_str = "{"
local first = true
for k, v in pairs(headers) do
if not first then
headers_str = headers_str .. ", "
end
headers_str = headers_str .. '"' .. k .. '": '
if type(v) == "table" then
headers_str = headers_str .. '"' .. escape_json_value(table.concat(v, ",")) .. '"'
elseif type(v) == "number" then
headers_str = headers_str .. tostring(v)
else
headers_str = headers_str .. '"' .. escape_json_value(tostring(v)) .. '"'
end
first = false
end
headers_str = headers_str .. "}"
root_span:set_attribute("http.request.headers", headers_str)
end
-- Add request body in access phase
local ok, body = pcall(kong.request.get_raw_body)
if ok and body then
root_span:set_attribute("http.request.body", body)
end
Configure Body Filter Function
Find the "Body Filter" field and paste the following code:
-- Constants
local MAX_BODY_SIZE = 1024 * 1024 -- 1MB in bytes
-- Body filter phase handler
local root_span = kong.tracing.active_span()
if not root_span then
return
end
-- Get chunk and eof flag safely
local chunk, eof
if ngx.arg[1] ~= nil then
chunk = ngx.arg[1]
end
if ngx.arg[2] ~= nil then
eof = ngx.arg[2]
end
-- Initialize response_chunks and total size if needed
kong.ctx.shared.response_chunks = kong.ctx.shared.response_chunks or {}
kong.ctx.shared.response_size = kong.ctx.shared.response_size or 0
-- Track size and collect chunks for tracing only
if chunk and type(chunk) == "string" then
kong.ctx.shared.response_size = kong.ctx.shared.response_size + #chunk
if kong.ctx.shared.response_size <= MAX_BODY_SIZE then
table.insert(kong.ctx.shared.response_chunks, chunk)
end
end
-- Always pass through the original chunk unmodified
ngx.arg[1] = chunk
-- On EOF, set the complete body for tracing
if eof then
local body = ""
if kong.ctx.shared.response_size <= MAX_BODY_SIZE then
body = table.concat(kong.ctx.shared.response_chunks)
end
root_span:set_attribute("http.response.body", body)
kong.ctx.shared.response_chunks = nil
kong.ctx.shared.response_size = nil
end
Configure Header Filter Function
Find the "Header Filter" field and paste the following code:
-- Header filter phase handler
local root_span = kong.tracing.active_span()
if not root_span then
return
end
-- Function to escape JSON string values
local function escape_json_value(v)
if type(v) == "string" then
return (v:gsub('"', '\\"'))
end
return v
end
-- Add response headers in JSON-like string format
local headers = kong.response.get_headers()
if headers then
local headers_str = "{"
local first = true
for k, v in pairs(headers) do
if not first then
headers_str = headers_str .. ", "
end
headers_str = headers_str .. '"' .. escape_json_value(k) .. '": '
if type(v) == "table" then
headers_str = headers_str .. '"' .. escape_json_value(table.concat(v, ",")) .. '"'
elseif type(v) == "number" then
headers_str = headers_str .. tostring(v)
else
headers_str = headers_str .. '"' .. escape_json_value(tostring(v)) .. '"'
end
first = false
end
headers_str = headers_str .. "}"
root_span:set_attribute("http.response.headers", headers_str)
end
Save Configuration
Review all three function configurations
Click "Save" to apply the Kong Functions plugin