Here's how to configure AWS Rest and HTTP API Gateway with OpenTelemetry (Otel) for traffic monitoring. It also covers troubleshooting common issues encountered during configuration.

AWS APIGW integration

This integration is supported for the following type of AWS API Gateway:

• REST API Gateway

• HTTP API Gateway

|| AWS supports detailed tracing only for REST API Gateway and not for HTTP API Gateway. This means that limited information like URL Path, Status code, Domain and Protocol can be observed for HTTP API Gateway. Request/Response headers and body which can only be observed with detailed tracing can't be observed for HTTP API Gateway.

Quick Installation

1. Pre-requisites

• Configure tracing and logging for the AWS REST API Gateway by following this and for AWS HTTP API Gateway by following this

• Keep the AWS AccessKeyID and SecretAccessKey ready.

• Launch a VM (EC2 instance) with following minimal hardware configuration in the same VPC where the Astra-Traffic-Collector is running.

  • Memory: 2GB RAM, CPU: 2, Disk: 10GB

• SSH into your VM and install Docker and Docker Compose in your VM by following the official doc.

2. Create a directory for the Astra Log Parser

• Create logs folder and assign user role for volume mounting.

mkdir -p /opt/astra-log-parser/logs && chown -R 10001:10001 /opt/astra-log-parser/logs && cd /opt/astra-log-parser

3. ** Create docker-compose.yaml file**

version: '3.3'
services:
  astra-log-parser:
    image: getastra/log-parser
    container_name: astra-log-parser
    volumes:
      - /opt/astra-log-parser/logs:/astra/logs
    network_mode: host
    env_file:
      - .env
    restart: always

4. Create .env file

Replace Replace with the SENSOR_ID displayed during the creation of AWS API Gateway integration

• Replace host:port with the address of astra-traffic-collector.

• Replace with the AWS Access Key Id.

• Replace with the AWS Secret Access Key.

• Replace with the AWS Region.

• Replace with the AWS CloudWatch Log Group Name associated with API Gateway.

SENSOR_ID=
OTLP_ENDPOINT=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_REGION=
LOG_GROUP_NAME=

5. **Create linux service file under /etc/systemd/system/astra-log-parser.service ** The service file allows the Astra Log Parser to be managed as a systemd service, enabling easy start, stop, restart, and automatic start on boot.

[Unit]
Description=Start Astra Log Parser
After=docker.service
Requires=docker.service

[Service]
Type=simple
ExecStart=/bin/bash -c "docker compose -f /opt/astra-log-parser/docker-compose.yaml up"
ExecStop=/bin/bash -c "docker compose -f /opt/astra-log-parser/docker-compose.yaml down"
Restart=always

[Install]
WantedBy=multi-user.target

6. **Manage the service by using systemctl **

# Enable the service to start on boot
sudo systemctl enable astra-log-parser

# Manage the service
sudo systemctl start | stop | restart | status astra-log-parser

# Disable the service
sudo systemctl disable astra-log-parser

Upgrade

Docker container upgrade

1. **Execute the following commands to stop, upgrade and then start the service **

systemctl stop astra-log-parser
cd /opt/astra-log-parser/
docker-compose pull
systemctl start astra-log-parser

Troubleshooting

1. I do not see my astra-log-parser running

Symptoms

• Error seen when systemctl status astra-log-parser is executed

• Following or similar error seen when docker logs astra-log-parser is executed

astra-log-parser   | Error parsing UUID SENSOR_ID: invalid UUID length: 0
astra-log-parser   | LOG_GROUP_NAME empty
astra-log-parser   | AWS_ACCESS_KEY_ID empty
astra-log-parser   | AWS_SECRET_ACCESS_KEY empty
astra-log-parser   | AWS_REGION empty

Cause

• lack of required env variables

  Solution

• Edit /opt/astra-log-parser/.env and update it with right information.

• Restart astra-log-parser sudo systemctl stop astra-log-parsersudo systemctl start astra-log-parser

2. Unable to see entries in inventory

Symptoms

• No entries in inventory/ inventory not getting updated

• No error in astra-log-parser or astra-traffic-collector log

  Cause

• Unregistered hostname

  Solution

• Double check if the hostname is registered under Scope URI for Report in Target setup page

FAQ (Frequently Asked Questions)

1. Can I see what trace are sent from my environment?

Yes, one can see the traces sent by traffic-collector by inspecting docker logs. Run docker logs astra-traffic-collector