The procedure in this article can be used to integrate Amazon API gateway with Cequence UAP.

Integration architecture

The Amazon API gateway integration for the Cequence Unified API Protection (UAP) platform is made of the following components.

Amazon API Gateway

Receives incoming requests from clients and routes them to the appropriate API backend services or applications.

Generates detailed log entries for each request and response, including headers, body, query parameters, and other metadata relevant to Cequence UAP platform analysis.

Amazon CloudWatch

Receives log events from API Gateway and stores them in log groups.

Note: Amazon CloudWatch limitations limit batch sizes to 1MB and 5000 transactions per second, per region. Use the Service Quotas service to change the transaction-per-second limit. The batch size limit cannot be increased.

Amazon CloudWatch Events

A serverless event management service that enables the triggering of an AWS service at a scheduled interval. Amazon CloudWatch Events triggers an AWS Lambda function that captures API Gateway log events every minute.

Parameter Store

Stores an authentication token acquired from the Cequence UAP platform. The token is valid for the interval configured for TTL (time-to-live). Refresh the token when TTL expires.

AWS Lambda function

Triggered every minute by AWS CloudWatch Events. Pulls the API Gateway log entries from AWS CloudWatch Events, transforms the entries into the payload format used by the Cequence UAP platform, aggregates the entries over the last minute of activity, and posts the batch to the Cequence UAP platform for analysis.

Implementation Steps

Prerequisites

Before deploying the Cequence API Gateway CloudFormation stack, ensure you have the following prerequisites in place:

1. AWS Account and Access:
- An active AWS account
- AWS CLI installed and configured with appropriate credentials
- IAM user or role with sufficient permissions to create and manage the following AWS services:
- IAM roles and policies
- Lambda functions
- CloudWatch logs
- EventBridge rules
- Systems Manager Parameter Store
- CloudFormation stacks

2. AWS Region:
- Decide on the AWS region where you want to deploy the stack
- Ensure the chosen region supports all the required services (Lambda, API Gateway, EventBridge, etc.)

3. API Gateway:
- An existing API Gateway with a known ID
- The stage name for the API Gateway you want to monitor

4. Cequence API Access:
- Access to the Cequence API (https://bridge.yourdomain.com/api-transactions)
- Access to the Cequence Auth API (https://auth.yourdomain.com/auth/realms/cequence/protocol/openid-connect/token)
- Valid Client ID and Client Secret for Cequence API authentication

5. Lambda Function Code:
- Prepare the Lambda function code for the log aggregator
- Package the code into a ZIP file named `cequence_plugin.zip`
- Ensure the ZIP file is accessible when creating the CloudFormation stack

6. CloudFormation Template:
- The CloudFormation template file (YAML or JSON) saved locally or in an S3 bucket

7. Parameters File:
- The JSON parameters file with all required values filled in
- Ensure sensitive values (like Client Secret) are handled securely

8. Networking:
- If your Lambda function needs to access resources in a VPC, ensure you have the necessary VPC, subnets, and security groups set up

9. Limits and Quotas:
- Check that you have sufficient capacity within your AWS account limits for creating new Lambda functions, CloudWatch log groups, and other resources

10. Permissions:
- Ensure you have permissions to create and manage all the resources defined in the CloudFormation template
- If using custom resource names or ARNs, ensure they comply with your organization's naming conventions and policies

11. Python Runtime:
- The template uses Python 3.9 for the Lambda function. Ensure this runtime is available in your chosen region

12. Time Zone:
- Verify that the specified time zone (default: America/Los_Angeles) is appropriate for your use case

13. Monitoring and Alerting:
- Consider setting up additional CloudWatch alarms or other monitoring solutions to track the health and performance of the deployed resources

14. Backup and Disaster Recovery:
- Plan for backing up any critical data or configurations
- Consider how you would recover or redeploy the stack in case of issues

15. Cost Consideration:
- Review the AWS pricing for all services used in the template
- Estimate the cost of running this stack in your account

16. Compliance and Security:
- Ensure that this deployment aligns with your organization's compliance requirements and security policies
- Consider encrypting sensitive parameters using AWS KMS

17. Testing Environment:
- It's recommended to first deploy this stack in a non-production environment for testing

By ensuring all these prerequisites are met, you'll be well-prepared to deploy the Cequence API Gateway CloudFormation stack successfully. Remember to review and adjust any specific details to match your exact requirements and environment.

Required Permissions

IAM permissions:
* iam:CreateRole
* iam:PutRolePolicy
* iam:PassRole
Lambda permissions:
* lambda:CreateFunction
* lambda:AddPermission
CloudWatch Logs permissions:
* logs:CreateLogGroup
* logs:PutRetentionPolicy
EventBridge permissions:
* events:PutRule
* events:PutTargets
SSM permissions:
* ssm:PutParameter
CloudFormation permissions:
* cloudformation:CreateStack
* cloudformation:DescribeStacks
* cloudformation:DescribeStackEvents
* cloudformation:GetTemplate
* cloudformation:ListStackResources
S3 permissions (if you're uploading the Lambda function code to S3):
* s3:GetObject
* s3:PutObject

These permissions should be sufficient to create and manage the resources defined in the CloudFormation template. However, depending on your specific AWS environment and any additional security measures in place, you might need to adjust these permissions.

Installation Steps

1. Download the tarball to your working directly from here. 

2. Untar the downloaded file and change the directory to cq-aws-api-gateway-tf

tar -xvf cq-aws-api-gateway-tf.tar.gz
cd cq-aws-api-gateway-tf/

Step 1: Prepare the CloudFormation Template

1. Download the CloudFormation template file (YAML or JSON) to your local machine.
2. If you need to make any modifications to the template, do so now using a text editor.

Step 2: Prepare the Parameters File

1. Open the parameters file (`cequence-parameters.json`) in a text editor.
2. Fill in all the parameter values with your specific details. Pay special attention to:
- `ApiGatewayId`: Your API Gateway ID
- `Stage`: Your API Gateway stage name
- `ApiUrl`: Your Cequence API URL
- `AuthApiUrl`: Your Cequence Auth API URL
- `ClientId` and `ClientSecret`: Your Cequence API credentials
3. Save the file after making your changes.

Step 3: Prepare the Lambda Function Code

1. Ensure your Lambda function code is ready and packaged into a file named `cequence_plugin.zip`.
2. Place this ZIP file in a location accessible to the CloudFormation process (e.g., an S3 bucket or the same directory as your CloudFormation template).

Step 4: Deploy the CloudFormation Stack

You can deploy the stack using either the AWS Management Console or the AWS CLI.

Option A: Using AWS Management Console

1. Log in to the AWS Management Console and navigate to the CloudFormation service.
2. Click "Create stack" (with new resources).
3. Under "Specify template", choose "Upload a template file".
4. Click "Choose file" and select your CloudFormation template file.
5. Click "Next".
6. Enter a Stack name (e.g., "CequenceAPIGatewayStack").
7. Under "Parameters", you can either manually input the parameter values or choose "Upload a file containing the parameter values" and select your `cequence-parameters.json` file.
8. Click "Next".
9. On the "Configure stack options" page, add any tags if desired, and configure stack policies or permissions if needed.
10. Click "Next".
11. Review your stack details. Check the acknowledgment box if your stack creates IAM resources.
12. Click "Create stack".

Option B: Using AWS CLI

1. Open a terminal or command prompt.
2. Ensure your AWS CLI is configured with the correct credentials and region.
3. Run the following command, replacing the placeholders with your actual file paths:

aws cloudformation create-stack \
--stack-name CequenceAPIGatewayStack \
--template-body file://path/to/your/template.yaml \
--parameters file://path/to/cequence-parameters.json \
--capabilities CAPABILITY_NAMED_IAM

Note: The `CAPABILITY_NAMED_IAM` capability is required because the stack creates IAM resources.

4. Wait for the stack creation to complete. You can monitor the progress in the AWS CloudFormation console or using the following CLI command:

aws cloudformation wait stack-create-complete --stack-name CequenceAPIGatewayStack

Step 5: Verify the Installation

After the stack creation is complete, verify that all resources have been created correctly:

1. Check the Outputs tab in the CloudFormation console, or use the following CLI command:

aws cloudformation describe-stacks --stack-name CequenceAPIGatewayStack --query 'Stacks[0].Outputs'

2. Verify that the Lambda function has been created:

aws lambda get-function --function-name cequence_log_aggregator

3. Check that the EventBridge rule is created and enabled:

aws events describe-rule --name cequence-plugin-log-scheduler

4. Verify the CloudWatch log group:

aws logs describe-log-groups --log-group-name-prefix /aws/lambda/cequence_log_aggregator

5. Check the SSM Parameter:

aws ssm get-parameter --name test-cequence-auth-token --with-decryption

Step 6: Test the Deployment

1. Invoke the Lambda function manually to ensure it's working correctly:

aws lambda invoke --function-name cequence_log_aggregator --payload '{}' response.json

2. Check the CloudWatch logs for the Lambda function to ensure it ran successfully and is able to interact with the Cequence API.

Troubleshooting

If you encounter any issues during the installation:

1. Check the CloudFormation events in the AWS console or use the following CLI command:

aws cloudformation describe-stack-events --stack-name CequenceAPIGatewayStack

2. Review the CloudWatch logs for the Lambda function for any error messages.

3. Verify that all the prerequisites were met, especially regarding permissions and network access.

4. If you need to make changes, you can update the stack using the AWS console or the following CLI command:

aws cloudformation update-stack \
--stack-name CequenceAPIGatewayStack \
--template-body file://path/to/your/updated-template.yaml \
--parameters file://path/to/updated-cequence-parameters.json \
--capabilities CAPABILITY_NAMED_IAM

Remember to replace placeholder values (like file paths and stack names) with your actual values when using these commands.

Cleanup Steps

When you're ready to remove the resources created by this CloudFormation stack, follow these steps:

1. Delete the CloudFormation Stack:

aws cloudformation delete-stack --stack-name YourStackName

Replace `YourStackName` with the name you gave your stack when you created it.

2. Wait for Stack Deletion:

aws cloudformation wait stack-delete-complete --stack-name YourStackName

This command will wait until the stack deletion is complete.

3. Verify Resource Deletion:

After the stack deletion is complete, verify that the following resources have been deleted:
- SSM Parameter
- IAM Role
- Lambda Function
- CloudWatch Log Group
- EventBridge Rule

You can use the AWS CLI commands from the post-installation steps to check if these resources still exist.

4. Check for Any Remaining CloudWatch Log Groups:

aws logs describe-log-groups --log-group-name-prefix /aws/lambda/cequence_log_aggregator

If any log groups remain, delete them manually:

aws logs delete-log-group --log-group-name /aws/lambda/cequence_log_aggregator

5. Check for Any Remaining IAM Roles:

aws iam get-role --role-name cequence_lambda_execution_role

If the role still exists, delete it manually:

aws iam delete-role --role-name cequence_lambda_execution_role

6. Remove Local Files:
Delete any local copies of the CloudFormation template, parameters file, and Lambda function code.

7. Revoke Any Associated Permissions:
If you granted any specific permissions to IAM users or roles to manage this stack, consider revoking those permissions if they're no longer needed.

Note: Remember to replace `YourStackName` with the actual name of your CloudFormation stack in all commands. Always exercise caution when deleting resources, especially in a production environment, to ensure you don't accidentally remove critical infrastructure.