In this article we will see how to enable traffic monitoring for AWS API Gateway using cloudwatch logs.
Create an IAM role to send logs to CloudWatch
Open the AWS Identity and Access Management (IAM) console.
In the navigation pane, choose Roles.
In the Roles pane, choose Create role.
On the Create role page, enter the following information: - For Trusted entity, choose AWS Service. - For use case, choose API Gateway, and then choose the API Gateway radio button.
Choose Next.
Under Permissions policies, use the default AmazonAPIGatewayPushToCloudWatchLogs AWS managed policy that has all the required permissions.
Choose Next.
Under Name, review and create, enter the following information: - For Role name, enter a name for the role. - (Optional) For Role description, edit the description to your preferences. - (Optional) Add tags.
Choose Create role.
In the Roles pane, choose the role that you created.
In the Summary pane, note the role's ARN to use in the next section.
Add the IAM role in the API Gateway console
Open the API Gateway console.
In the APIs pane, choose the name of the REST API.
In the navigation pane, choose Settings.
Under Settings, for CloudWatch log role ARN, enter the IAM role's ARN.
Choose Save.
|| The CloudWatch Logs role is an AWS Region-level configuration that you use with all the APIs in the Region. If you create multiple APIs across different Regions, then complete the preceding steps in each Region.
Turn on API logging and stage the API
Create the logGroup and keep the log group name and ARN handy. Please ensure to create the log group name as: API-Gateway-Execution-Logs_gatewayID/stageName. Example: If the gatewayID is z3xiu18cq2 and stage name is test, then, create the log group name MUST be API-Gateway-Execution-Logs_z3xiu18cq2/test
Open the API Gateway console.
In the navigation pane, choose APIs, and then select your API.
In the navigation pane, choose Stages, and then select your stage.
Under Logs and tracing, choose Edit.
Under CloudWatch logs, select level to Error and info logs from the dropdown menu.
Select the Data tracing option.
Enable Custom Access Logging, complete the following steps:
For Access Log Destination ARN, enter the ARN of CloudWatch log group created earlier.
Enter following log format as it is:
($context.requestId) Access Log {"httpMethod":"$context.httpMethod", "path":"$context.path", "status":"$context.status", "protocol":"$context.protocol", "domain":"$context.domainName"}
Choose Save Changes.
Test your logging setup
| To send a new request to your API, use your client application or tools, such as the Postman.
Open the CloudWatch console.
In the navigation pane, under Logs, choose Log groups.
For Log groups, select the log group of the API that you want to debug - For REST APIs, the log group name will be like API-Gateway-Execution-Logs_gatewayID/stageName
In the Log streams list, select the log stream with the latest Last Event Time to view messages with the details of your request.
Create the logGroup and keep the log group name and ARN handy. Please ensure to create the log group name as: API-Gateway-Execution-Logs_gatewayID/stageName. Example: If the gatewayID is z3xiu18cq2 and stage name is test, then, create the log group name MUST be API-Gateway-Execution-Logs_z3xiu18cq2/test
Sign in to the API Gateway console at https://console.aws.amazon.com/apigateway
Choose an HTTP API
Under the Monitor tab in the primary navigation panel, choose Logging.
Select a stage to activate logging and choose Select.
Choose Edit to activate access logging.
Turn on Access logging
For Access Log Destination ARN, enter the ARN of CloudWatch log group created earlier.
Enter following log format as it is:
($context.requestId) Access Log {"httpMethod":"$context.httpMethod", "path":"$context.path", "status":"$context.status", "protocol":"$context.protocol", "domain":"$context.domainName"}
Choose Save.
Test your logging setup
| To send a new request to your API, use your client application or tools, such as the Postman.
Open the CloudWatch console.
In the navigation pane, under Logs, choose Log groups.
For Log groups, select the log group of the API that you want to debug
In the Log streams list, select the log stream with the latest Last Event Time to view messages with the details of your request.
It's a two step process which are detailed below:
Step 1: Create IAM User
Navigate to IAM -> Users -> Create User.
Give the name as "astra-log-parser" and click on Next.
In the Permissions Options section, select the Attach Policies Directly radio button.
In the Permissions Policy section, search for "cloudwatchreadonlyaccess" and select the CloudWatchReadOnlyAccess policy.
Click on Next.
Review and Create User and click on Create User.
Step 2: Create Access Key
Click on the newly created user "astra-log-parser" in IAM users.
Under the Summary section, click on Create Access Key.
In the Access key best practices & alternatives page:
Select the use case as "Application running on an AWS Compute service".
Tick the Confirmation checkbox.
Click on Next.
In the Set description tag - optional page:
Provide an optional description.
Click on Create access key.
In the Retrieve Access keys page:
Click on Show to view the secret access key. Note down the secret access key
||| If you lose or forget your secret access key, you cannot retrieve it. Instead, create a new access key and make the old key inactive.