This article discusses how to use Terraform to integrate the Amazon API gateway with the Cequence Unified API Protection (UAP) platform. You can also integrate using CloudFormation.

Integration architecture

The Amazon API gateway integration for the Cequence UAP platform is made of the following components.

aws cloudwatch.png

Amazon API Gateway

Receives incoming requests from clients and routes the requests to the appropriate API backend services or applications.

Generates detailed log entries for each request and response, including headers, body, query parameters, and other metadata relevant to Cequence UAP platform analysis.

Amazon CloudWatch

Receives log events from API Gateway and stores them in log groups.

Note: Amazon CloudWatch limitations limit batch sizes to 1MB and 5000 transactions per second, per region. Use the Service Quotas service to change the transaction-per-second limit. The batch size limit cannot be increased.

Amazon CloudWatch Events

A serverless event management service that enables the triggering of an AWS service at a scheduled interval. Amazon CloudWatch Events triggers an AWS Lambda function that captures API Gateway log events every minute.

AWS Lambda function

Triggered every minute by AWS CloudWatch Events. Pulls the API Gateway log entries from AWS CloudWatch Events, transforms the entries into the payload format used by the Cequence UAP platform, aggregates the entries over the last minute of activity, and posts the batch to the Cequence UAP platform for analysis.

The procedures in this article automate the Lambda function configuration by discovering all API gateways and their deployed stages, then configuring the dependencies above. A Terraform script that provides this automation can be applied to an individual AWS account.

Prerequisites

An AWS account with sufficient permissions to manage API Gateway, Lambda, IAM, CloudWatch, EventBridge Scheduler, and EC2.
Working REST and HTTP APIs deployed to various stages in API Gateway in an AWS account.
Cequence Bridge(v5.6.2) Deployed
Terraform release v1.9.5 or later.
AWS CLI release 2.14.0 or later
Current installations of node.js and npm.
Access to an AWS CLI user account with specific privileges and roles, which are described in a following section.
Access to the jq, ed, and zip UNIX tools.
An AWS policy named according to the following format.
```
iam_apigw_lambda_cloudwatch_eventbridge_cq_integration
```
Attach the permissions in the following image to the AWS CLI user account.

AWS CLI user privileges

Setting up the Amazon API Gateway integration requires the use of an AWS CLI user account with an access key, secret, and the privileges listed in the following JSON code. These restricted permissions apply to resources that are created by Terraform.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ScanRegions",
            "Effect": "Allow",
            "Action": "ec2:DescribeRegions",
            "Resource": "*"
        },
        {
            "Sid": "IAMRoleManagement",
            "Effect": "Allow",
            "Action": [
                "iam:CreateRole",
                "iam:GetRole",
                "iam:PassRole",
                "iam:DeleteRole",
                "iam:ListRolePolicies",
                "iam:PutRolePolicy",
                "iam:GetRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:AttachRolePolicy",
                "iam:DetachRolePolicy",
                "iam:ListAttachedRolePolicies",
                "iam:ListInstanceProfilesForRole"
            ],
            "Resource": "arn:aws:iam::*:role/*"
        },
        {
            "Sid": "LambdaFunctionManagement",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction",
                "lambda:DeleteFunction",
                "lambda:GetFunction",
                "lambda:UpdateFunctionConfiguration",
                "lambda:UpdateFunctionCode",
                "lambda:ListVersionsByFunction",
                "lambda:GetFunctionCodeSigningConfig"
            ],
            "Resource": "arn:aws:lambda:*:*:function:*"
        },
        {
            "Sid": "CloudWatchEventsManagement",
            "Effect": "Allow",
            "Action": [
                "events:PutRule",
                "events:PutTargets",
                "events:DescribeRule",
                "events:ListTagsForResource",
                "events:DeleteRule"
            ],
            "Resource": "arn:aws:events:*:*:*"
        },
        {
            "Sid": "SchedulerManagement",
            "Effect": "Allow",
            "Action": [
                "scheduler:CreateSchedule",
                "scheduler:DeleteSchedule",
                "scheduler:GetSchedule"
            ],
            "Resource": "arn:aws:scheduler:*:*:schedule/default/*"
        },
        {
            "Sid": "APIGatewayAccess",
            "Effect": "Allow",
            "Action": [
                "apigateway:GET",
                "apigateway:POST",
                "apigateway:PUT",
                "apigateway:PATCH",
                "apigateway:DELETE",
                "logs:CreateLogGroup",
                "logs:DeleteLogGroup",
                "logs:PutRetentionPolicy",
                "logs:DescribeLogGroups",
                "logs:ListTagsForResource",
                "logs:CreateLogDelivery",
                "logs:PutResourcePolicy",
                "logs:UpdateLogDelivery",
                "logs:DeleteLogDelivery",
                "logs:DescribeResourcePolicies",
                "logs:GetLogDelivery",
                "logs:ListLogDeliveries"
            ],
            "Resource": "*"
        }
    ]
}

The Terraform script requires the following roles and permissions.

API Gateway Role (aws_iam_role.api_gateway_cloudwatch)

Purpose: Allows API Gateway to push logs to CloudWatch
Trust Policy: Allows only apigateway.amazonaws.com to assume this role
Permissions: Uses AWS managed policy AmazonAPIGatewayPushToCloudWatchLogs
Used by: API Gateway account settings for CloudWatch logging

Lambda Role (aws_iam_role.lambda_role)

Purpose: Enables Lambda function to interact with CloudWatch Logs
Trust Policy: Allows only lambda.amazonaws.com to assume this role
Custom Policy Permissions:

logs:CreateLogGroup
logs:CreateLogStream
logs:PutLogEvents
logs:FilterLogEvents
logs:Unmask

Scope: Full access to CloudWatch logs resources (arn:aws:logs:*:*:*)

EventBridge Role (aws_iam_role.eventbridge_role)

Purpose: Allows EventBridge to invoke the Lambda function
Trust Policy: Allows only scheduler.amazonaws.com to assume this role
Custom Policy Permissions:
```
lambda:InvokeFunction
```
Scope: Limited to only the specific Lambda function ARN

Generating a client ID and client Secret

Several Cequence components must authenticate to the Cequence UAP platform in order to transmit and receive data. Create authentication credentials in the Cequence UAP platform to enable this authentication.

Log in to the UAP management portal UI.
The URL for the management portal is typically of the form https://ui.<your-tenant-name>.<domain>. Replace <your-tenant-name> with the name of your Cequence tenant organization. Replace <domain> with your domain name.
Select General Settings > User Management.
The User Management pane appears.
Click the Clients tab.
Click Add New Client.
The new client dialog box appears.
Type the client name in the Client Name field.
This name is the client ID. Note the client ID for later use.
Enable the Traffic Management toggle.
(Optional) To change the token lifespan from the default of 1800 seconds, type a whole number of seconds in Token Lifespan.
Click Save.
A dialog box with the client secret appears.
Click the blue Copy icon to copy the secret to the clipboard, then click Close.
The client is now set up. Note the client name for future use.
The client list appears.
Note the value of the client secret for later use. This value will not be shown again later on the UI for security reasons.

Installing with Terraform

The compressed archive file that contains the integration bundle contains a file named main.tf. Terraform uses this file during installation.

The main.tf configuration file is structured into two sections, a section for global AWS resources and a section for regional resources. This approach provides comprehensive API scanning and monitoring across regions, maintaining resource isolation and regional compliance.

The Global AWS resources section creates account-wide resources that are not regionn-specific. These resources are shared across all regions in the AWS account.
The Regional Resources section creates and deploys region-specific components. These resources enable the monitoring of APIs deployed across multiple AWS regions within the same account. Each region has an independent set of monitoring resources. The region specific components include:
- Cequence Lambda function
- EventBridge scheduler
- CloudWatch log groups
- API Gateway configurations

Download the compressed archive file that contains the plugin bundle.
Extract the archive file. The extraction creates the cequence directory structure.
```
tar -xvf cequence-aws-api-gateway-x.x.x.b7xxxxf34.tar.gz
```
Navigate to the scripts directory under cequence and grant execute permission to the scripts by running the following command.
```
cd cequence/scripts
chmod +x *.sh
```

Create a copy of the example environment properties file by running the following command.

cp .env.example .env

Note:- The .env file in the bundle contains variables to cater for both Terraform and CloudFormation. The following table defines the properties of the environment variables for terraform Only and leave the other variables as default.

Variable	Description
cequence_client_id	The client ID generated earlier in this article
cequence_client_secret	The client secret generated earlier in this article
cequence_auth_endpoint	The endpoint for Cequence Edge or Bridge authentication.
cequence_edge_endpoint	The endpoint for the Cequence Edge or Bridge.
cequence_aws_regions	all or a single region or list of comma separated regions
cequence_is_aws_organization_deployment	false for Terraform Deployment Currently only used for CloudFormation to create the bucket policy to allow access from all the accounts in the organization.
cequence_is_multi_account_deployment	false for Terraform Deployment. For Terraform Deployment only Single Region is supported
cequence_deployment_type	terraform
cequence_log_level	Supports info, debug and trace. Used for diagnostics and troubleshooting. Note:- trace is more verbose than debug. Only enable info in production.
cequence_auto_install_dependencies	true by default If you want to install the dependencies automatically, set this to true. Note:- Install JQ, AWS CLI, Terraform CLI, ED and ZIP
cequence_auto_uninstall_dependencies	false by default If you want to uninstall the dependencies automatically, set this to true Note:- Uninstall JQ, AWS CLI, Terraform CLI, ED and ZIP
cequence_rest_api_enabled	true If you want to enable the REST APIs, set this to true.
cequence_http_api_enabled	true If you want to enable the HTTP APIs, set this to true.
cequence_auto_discover_apis	true If false, use api-config.json to explicitly list the apis to be enabled for a single region
is_existing_http_log_group	false by default True if you already have access logs enabled to a log group.
cequence_http_log_group_name	cequence-http-api-access-logs Use your existing log group name here or we create a new one with this name
cequence_tf_auto_approve	Default is false. When set to true, the artifacts created or changed by an integration script are not presented for review.
cequence_res_content_types	skip if response does not match any of the below content types Leave default
cequence_static_file_extensions	skip if URI matches any of the below extensions Leave default

Note: Enable Specific APIs in a Single Region Only

If we want to enable the integration only for a small subset of APIs then provide the region names in the environment variable file.

Then create a copy of api-config-example.json to api-config.json

cp api-config-example.json api-config.json

And edit the api-config.json and add region name, api_id/stage_name and save the file in the below format

{
"api_gateway_rest_apis": {
"aws_region_name": [
"api_id1/stage_name",
"api_id2/stage_name"
]
},
"api_gateway_http_apis": {
"aws_region_name": [
"api_id1/stage_name",
"api_id2/stage_name"
]
}
}

Below Example has only REST APIs and no HTTP APIs

{
"api_gateway_rest_apis": {
"eu-west-2": [
"ligvxxxaxsa/test",
"ligvxxxxsa/uat"
]
},
"api_gateway_http_apis": {}
}

5. Edit the environment properties file to change the values of specific variables.
Replace the placeholders for the client ID and client Secret with the actual values. Type the URIs for the Cequence Bridge Authentication endpoint and Cequence Bridge Edge endpoint.

6. Enable the integration by running the following script.

./enable_aws_apigw_integration.sh

The script scans all APIs in the AWS account and enables the integrations for deployed REST and HTTP APIs in all regions. As a best practice, confirm that the terraform.tfstate files are kept current, then re-run this script on a regular interval to discover and catalogue new APIs.

Disabling the integration

You can take several approaches to disabling the integration.

Disabling REST API functionality only

Open the .env file in a text editor.
Change the value of the cequence_rest_api_enabled variable to false.
Save and exit the editor.
Run the following script.
```
./disable_aws_apigw_integration.sh
```

Disabling HTTP API functionality only

Open the .env file in a text editor.
Change the value of the cequence_http_api_enabled variable to false.
Save and exit the editor.
Run the following script.
```
./disable_aws_apigw_integration.sh
```

Disabling all functionality

Open the .env file in a text editor.
Change the value of the cequence_rest_api_enabled variable to false.
Change the value of the cequence_http_api_enabled variable to false.
Save and exit the editor.
Run the following script.
```
./disable_aws_apigw_integration.sh
```

Destroying artifacts created by the integration enabling script

To delete all resources and artifacts created by the enabling script, open the .env file in a text editor

and change the value of

cequence_rest_api_enabled to false
cequence_http_api_enabled to false

And run the disabling script with the --destroy flag.

./disable_aws_apigw_integration.sh --destroy

Notes

AWS API Gateway limits log events to 1024 bytes. Log events larger than 1024 bytes, such as request and response bodies, are truncated by API Gateway before submission to CloudWatch logs.
For REST APIs where AWS truncates the request and response body, Cequence sets a variable in the body as

{
"cq_truncated": true
}

3. Set the retention period for CloudWatch Log Groups associated with APIs deployed in AWS API Gateway to at least 1 day, or according to the your retention policy.

4. The Request and Response body for HTTP APIs are not logged by default in AWS. Only below variables are captured and send to UAP Platform

requestTimeEpoch, requestId, accountId, stage, instance_id, ip, host, http-version, http-method, uri-query-fragment, status-code

Refer AWS Documentation:- https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-logging-variable

Search

AWS API Gateway Integration using Terraform

AWS API Gateway Integration using Terraform

Integration architecture

Amazon API Gateway

Amazon CloudWatch

Amazon CloudWatch Events

AWS Lambda function

Prerequisites

AWS CLI user privileges

API Gateway Role (aws_iam_role.api_gateway_cloudwatch)

Lambda Role (aws_iam_role.lambda_role)

EventBridge Role (aws_iam_role.eventbridge_role)

Generating a client ID and client Secret

Installing with Terraform

Disabling the integration

Disabling REST API functionality only

Disabling HTTP API functionality only

Disabling all functionality

Destroying artifacts created by the integration enabling script

Notes