Amazon API Gateway is a fully managed AWS service that enables developers to create, publish, maintain, monitor, and secure APIs at any scale. Amazon API Gateway acts as a front door for applications that access data, business logic, or functionality from backend services.

Amazon API Gateway

Amazon API Gateway receives incoming requests from clients and routes the requests to the appropriate API backend services or applications. Amazon API Gateway also generates detailed log entries for each request and response, including headers, body, query parameters, and other metadata relevant to Cequence Unified API Protection (UAP) platform analysis.

Amazon CloudWatch

Amazon CloudWatch receives log events from API Gateway and stores the events in log groups.

Note: Amazon CloudWatch limits batch sizes to 1 MB and 5000 transactions per second, per region. Use the Service Quotas service to change the transaction-per-second limit. The batch size limit cannot be increased. See: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/cloudwatch_limits_cwl.html

Amazon EventBridge Scheduler

A serverless event management service that enables the triggering of an AWS service at a scheduled interval. EventBridge Scheduler triggers an AWS Lambda function that fetches API Gateway log events every minute.

AWS Lambda function

Triggered every minute by AWS EventBridge Scheduler. The Lambda function pulls API Gateway log entries from AWS CloudWatch, transforms the entries into the payload format used by the Cequence UAP platform, aggregates the entries over the last minute of activity, and posts the batch to the Cequence UAP platform for analysis.

Prerequisites

• An AWS account with sufficient permissions to deploy AWS CloudFormation StackSets and Lambda into multiple accounts, regions, or organizational units (OUs), or across the whole organization.
• For deploying across the organization, you need either a root account or a Delegated StackSet Administrator account (service-managed model). See: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacksets-orgs-delegated-admin.html
• For deploying to multiple accounts without organization-wide scope (self-managed model), you need trust relationships between an administrator account and target accounts. See: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacksets-prereqs-self-managed.html
• Working REST or HTTP APIs deployed to stages in API Gateway.
• The user who runs the enable/disable scripts must have permissions to:
  • Discover AWS regions.
  • Manage IAM roles and policies needed for the integration.
  • List and manage Lambda functions.
  • Manage CloudWatch Events and EventBridge Scheduler rules.
  • Read from and write to CloudWatch Logs.
  • Manage API Gateway (basic CRUD operations).
  • Manage S3 buckets and objects for the Cequence artifacts buckets.
  • Manage CloudFormation StackSets (create, update, and delete stack sets and stack instances).
  • Read AWS Organizations metadata when using organization-wide deployment.

How cross-account StackSets work

The following steps describe the conceptual flow for cross-account StackSet deployment.

1. Deploy baseline IAM roles into the accounts where StackSets will create and manage resources.
2. The administrator account assumes these roles to create resources in the target accounts.
3. Once assumed, the role has the permissions needed to create, update, or delete AWS resources as defined in the StackSet templates.

This follows AWS's recommended security model, where administrator accounts manage resources across multiple target accounts through specifically configured execution roles.

Enable Cequence and AWS API Gateway integration (REST and HTTP APIs)

At a high level, enabling the integration involves the following steps.

1. Obtain and unpack the Cequence bundle.
2. Prepare an environment configuration file (.env) that describes:
   • Where Cequence endpoints are hosted.
   • Which AWS regions and accounts are in scope.
   • Whether the deployment is organization-wide or limited to specific accounts.
   • Whether REST APIs, HTTP APIs, or both should be integrated.
   • How API auto-discovery and explicit API lists should be handled.
3. Run a script that:
   • Validates the environment.
   • Prepares S3 buckets for artifacts.
   • Uploads CloudFormation templates and Lambda artifacts.
   • Optionally creates and deploys a CloudFormation StackSet across the selected accounts and regions.

Environment configuration

The .env file is a central configuration file used by the integration scripts. The following sections describe the purposes of each configuration area.

Environment validation

The scripts check that a .env file exists and can be loaded, verify that at least one API type (REST or HTTP) is enabled, and verify that AWS CLI is properly configured.

Organization support

When cequence_is_aws_organization_deployment=true, the integration can be rolled out organization-wide using StackSets and AWS Organizations. Organizational unit (OU) targeting is controlled by a CSV file that can target either the entire organization (root OU or "all") or specific OUs by ID. Optional account-level filtering can further narrow down which accounts receive the integration within the targeted OUs, using intersection or difference modes.

Multi-account support

The integration supports both single-account and multi-account deployments. Multi-account deployment is supported for CloudFormation but not for Terraform. In self-managed multi-account mode, an administrator account hosts the StackSet, target accounts are listed in a CSV file, and cross-account trust is configured so that the administrator account can assume an execution role in each target account.

API configuration

You can enable or disable REST APIs and HTTP APIs independently. When auto-discovery is disabled, the integration can be restricted to a specific set of APIs and stages using a JSON configuration file. The configuration validates that listed regions are valid AWS regions and that API configuration structures are properly formatted.

Region management

You can either let the script auto-discover all enabled AWS regions or specify a list of target regions explicitly.

Deployment preparation

The scripts install any required local dependencies using a helper script, determine and prepare S3 buckets for CloudFormation artifacts (Lambda zips, templates, and API lists) per region, and validate that Cequence credentials and endpoints are not using placeholder values.

Key environment settings

Cequence credentials

Use these settings to toggle whether Cequence API calls require authentication. When authentication is enabled, configure the Client ID, Client Secret, and token endpoint for OAuth2. Also configure the Cequence Edge or Bridge endpoint that will receive batched API transaction data.

AWS regions and accounts

Specify whether to use all enabled regions or a comma-separated list of specific regions. Control whether the deployment is organization-wide using AWS Organizations or targeted to specific accounts using a multi-account configuration file. You can also specify an optional organization ID when automatic discovery of the organization is not possible.

Deployment type

Select the deployment mechanism used by the scripts. CloudFormation is the mechanism described in this article.

Logging and Lambda behavior

Configure the Lambda log level using info, debug, or trace. You can also configure caching behavior for REST API lists retrieved from S3.

API enablement and discovery

Enable or disable REST API and HTTP API integration independently. Toggle whether APIs are auto-discovered or explicitly listed using a configuration file. Indicate whether REST API execution logs are already enabled and whether HTTP APIs already have access logs configured, and provide an existing or desired log group name for HTTP API access logs.

S3 bucket strategy

Define an optional bucket name prefix for REST API lists, Lambda deployment zips, and CloudFormation templates. Toggle whether the script should manage (create and delete) these buckets or use externally managed buckets.

CloudFormation-specific controls

These settings control automatic confirmation for S3 uploads and automatic deletion of S3 artifacts when destroying the integration. You can configure the StackSet naming convention and choose whether the script automatically creates or updates the StackSet and deploys stack instances, or only uploads artifacts so that you can create the StackSet manually in the console. Additional settings include automatic confirmation for StackSet operations, the preferred region for StackSet operations and template hosting, the CloudWatch log retention period in days, and the scheduling expression for periodic API refresh using EventBridge Scheduler in rate(...) or cron(...) format.

Filtering and truncation behavior

Configure a regular expression that identifies the response content types that should be processed, such as XML, JSON, form-encoded, and HTML. Configure a separate regular expression that identifies static file extensions that should be excluded from processing, such as CSS, images, documents, archives, and media files.

Optional IAM role ARNs

When you manage IAM roles yourself rather than having CloudFormation or Terraform create the roles, you can provide a pre-created Lambda execution role ARN, a pre-created API Gateway CloudWatch logging role ARN, and a pre-created EventBridge Scheduler role ARN.

Regional enablement constraints

For CloudFormation deployments, every account in scope must have all target regions enabled. When any region is disabled in any targeted account (including accounts within targeted OUs), CloudFormation StackSet operations can fail with messages indicating that an account is not opted into specific regions. For information on enabling regions in accounts, see: https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-regions.html

CloudFormation deployment using a script

When you run the primary enable script for a CloudFormation deployment, the script performs the following steps.

1. Reads and validates your .env configuration.
2. Installs any required local dependencies if configured to do so.
3. Prepares and uploads CloudFormation templates and Lambda artifacts into the appropriate S3 buckets for each enabled region.
4. Creates or updates a StackSet with:
   • Parameters controlling which APIs to enable (REST or HTTP).
   • Cequence endpoints and credentials.
   • Logging configuration, including log groups, retention period, and log level.
5. Deploys StackSet instances across the specified accounts and regions according to the chosen permission model (service-managed or self-managed).
6. Waits for StackSet operations to complete and reports success or failure.

After completion, you can inspect the StackSet and its instances in the AWS CloudFormation console to verify that the integration is active in the intended accounts and regions.

CloudFormation deployment using the console

You can configure the script to only upload artifacts to S3 and then create and manage the StackSet through the AWS console. The following sections describe the high-level steps.

Create the StackSet

Open the CloudFormation console in the account that will administer the StackSet (root, delegated administrator, or a self-managed admin account with the appropriate trust relationships). Choose the permission model, the IAM roles, and the template source as described below.

• Choose the permission model:
  • Service-managed for AWS Organizations-based deployments.
  • Self-managed for manually specified accounts or OUs with explicit trust.
• Choose the IAM roles:
  • Administration role, typically named along the lines of AWSCloudFormationStackSetAdministrationRole.
  • Execution role, typically named along the lines of AWSCloudFormationStackSetExecutionRole.
• Point the StackSet template source at the S3 URL created by the script.

Configure StackSet parameters

When creating the StackSet, provide the following parameters.

• A StackSet name and description.
• Parameters that describe:
  • The S3 bucket prefix where Cequence artifacts are stored.
  • Whether REST and HTTP APIs are enabled, and any optional API lists for single-account mode.
  • Whether REST execution logs and HTTP access logs are already set up.
  • The log group name for HTTP API access logs, when using a specific group name.
  • Whether logging on HTTP API $default stages should be automatically enabled.
  • The log level for Lambda functions.
  • Regular expressions for supported content types and excluded file extensions.
  • Cequence authentication settings, including whether authentication is needed, client ID and secret, authentication endpoint, and Edge or Bridge endpoint.
  • Optional IAM role ARNs, when you choose to manage those roles yourself.

StackSet options and deployment strategy

Add tags to identify the deployment, such as tags indicating the deployment is a Cequence integration and whether CloudFormation is the deployment method. Enable execution and acknowledge IAM-related capabilities, including creation of roles with custom names and macro or auto-expand capabilities when required. Then choose your deployment scope and concurrency settings as follows.

• Choose the deployment scope:
  • Service-managed:
    • Entire organization.
    • Specific organizational units, with optional further account filtering.
  • Self-managed:
    • List of specific accounts.
    • Organizational units configured with IAM trust relationships.
• Select regions, either specific regions or all enabled regions.
• Set concurrency and failure tolerance options, including maximum concurrent accounts, failure tolerance, and regional concurrency mode.

Review and deploy

Review your configuration and submit. Monitor the StackSet operations and verify that stack instances are successfully created in the target accounts and regions.

Testing the integration

The following steps describe how to verify that the integration is functioning correctly.

1. Invoke your REST or HTTP APIs to generate traffic that is logged to CloudWatch.
2. Wait for the scheduled Lambda to run (typically every minute).
3. Inspect the Lambda's CloudWatch log group for summary statistics of each run, including:
   • How many log groups and streams were processed.
   • Total events fetched from CloudWatch.
   • Parsing success and failure counts.
   • Number and sizes of batches sent to Cequence Edge.
   • Network and processing latencies.
4. Verify that transactions appear in the Cequence UAP platform dashboards as expected.

Managing regions over time

AWS StackSets impose some constraints around region changes. Adding new regions to an existing StackSet using a template update is limited. In some cases you may need to delete the existing StackSet and create a new StackSet with the expanded region list.

When the integration is controlled by the script, the typical flow to add regions is as follows.

1. Temporarily disable REST and HTTP enablement in the configuration to roll back resources.
2. Run a script that deletes StackSet instances and the StackSet itself.
3. Update the region list in the configuration.
4. Re-run the enable script to create a new StackSet spanning the updated regions.

Disabling or deleting the integration

Disabling using a script-managed deployment

You can disable only REST or only HTTP APIs by toggling the respective flags in the configuration and re-running the enable flow. To disable both REST and HTTP, set both API enablement flags to false and run a disable flow that removes StackSet instances and, optionally, buckets and other artifacts depending on your destruction settings.

Disabling using a console-managed deployment

From the AWS CloudFormation console, you can disable the integration using the following steps.

1. Open the StackSets view and select the StackSet you created for the integration.
2. Use the override parameters action to specify which accounts and regions should be updated.
3. Set the REST and HTTP enablement parameters to false.
4. Submit the update and monitor operations until completion.

Deleting the integration

Deleting the integration consists of two parts.

1. Remove all StackSet instances and the StackSet:
   • Delete stack instances in all target accounts and regions from the console or using automation.
   • Once instances are removed, delete the StackSet definition.
   • Monitor operation status to verify that the deletion completes successfully.
2. Delete Cequence artifacts from S3 regional buckets:
   • Optionally configure the integration to delete S3 buckets and artifacts that were created for deployment.
   • Note that when many regions are enabled, bucket deletion across all regions can take noticeable time.

When using the console exclusively, the S3 cleanup step can also be performed manually per bucket. CloudFormation does not automatically delete external buckets that were only referenced as artifact stores.

HTTP and REST API limitations

HTTP APIs

The request and response bodies for HTTP APIs are not logged by this integration. The following attributes are captured and sent to the Cequence UAP platform.

requestTimeEpoch, requestId, accountId, stage, instance_id,
ip, host, http-version, http-method, uri-query-fragment, status-code

For reference on HTTP API logging variables, see: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-logging-variables.html

HTTP APIs that have access logs already enabled in a format other than JSON are not currently supported by this integration.

Payload and header behaviors

Truncated REST API bodies

When REST API bodies exceed certain size thresholds (greater than 1 KB for this integration's processing window), the body content may be truncated. In such cases, instead of the full body, a small indicator body is sent to the Cequence UAP platform to signal truncation, in a format appropriate to the content type, such as form-encoded, JSON, XML, SOAP, or HTML with a "body truncated" marker.

HTTP API request and response bodies

For HTTP APIs, where neither request nor response bodies are available, the integration sends a minimal payload indicating that only endpoint information is being captured, such as {"cq-endpoints-only": "true"}.

Discovery-only headers

For REST APIs where bodies cannot be captured due to truncation, the integration adds a cq-discovery-only=true header to indicate that only discovery-level metadata (endpoints, status codes, and similar attributes) is available. For HTTP APIs, which do not include request or response bodies in logs for this integration, the same cq-discovery-only=true header is applied to indicate discovery-only coverage.

Troubleshooting and operational considerations

• Template updates vs. replacements. Parameter-only changes and incremental template updates can typically be handled with standard StackSet updates. Significant architectural changes, permission model shifts, or repeated update failures may require replacing the StackSet (delete and recreate).
• Region selection on retries. When retrying or updating a previously failed operation for certain regions, ensure that all relevant regions are included in the update operation. Omitting regions can leave the regions in a pending or inconsistent state.
• Role visibility issues. When configured IAM roles for StackSets are not visible in the console dropdowns, review the trust policies and permissions to verify that the roles meet AWS CloudFormation's requirements for StackSet administration and execution roles.
• Template versioning and history. AWS maintains operation history and retains access to previous template versions used in StackSet operations. Stack instances preserve their current state until explicitly updated or replaced. Rollback operations may rely on cached versions of previous templates.

When to update vs. replace a StackSet

Use Update StackSet for the following scenarios.

• Changing parameter values.
• Adding or modifying resources without fundamentally altering the naming or permission model.
• Iterative improvements to template logic.

Consider Replace (delete and recreate) for the following scenarios.

• Resource name changes that force replacements.
• Fundamental architecture or permission model changes.
• Complex cross-region dependency shifts.
• Persistent or repeated update failures that are difficult to resolve in place.
• Major template restructuring.

Key behavior of updates

Only resources that change are updated or replaced. Unchanged resources remain untouched. CloudFormation determines which resources must be replaced vs. updated in place.