You can integrate the Amazon API gateway with the Cequence Unified API Protection (UAP) platform with CloudFormation or with Terraform.

Integration architecture

The Amazon API gateway integration for the Cequence Unified API Protection (UAP) platform is made of the following components.

Amazon API Gateway

Receives incoming requests from clients and routes them to the appropriate API backend services or applications.

Generates detailed log entries for each request and response, including headers, body, query parameters, and other metadata relevant to Cequence UAP platform analysis.

Amazon CloudWatch

Receives log events from API Gateway and stores them in log groups.

Note: Amazon CloudWatch limitations limit batch sizes to 1MB and 5000 transactions per second, per region. Use the Service Quotas service to change the transaction-per-second limit. The batch size limit cannot be increased.

Amazon CloudWatch Events

A serverless event management service that enables the triggering of an AWS service at a scheduled interval. Amazon CloudWatch Events triggers an AWS Lambda function that captures API Gateway log events every minute.

Parameter Store

Stores an authentication token acquired from the Cequence UAP platform. The token is valid for the interval configured for TTL (time-to-live). Refresh the token when TTL expires.

AWS Lambda function

Triggered every minute by AWS CloudWatch Events. Pulls the API Gateway log entries from AWS CloudWatch Events, transforms the entries into the payload format used by the Cequence UAP platform, aggregates the entries over the last minute of activity, and posts the batch to the Cequence UAP platform for analysis.

Before you start

Required Access and Permissions

• AWS account with sufficient permissions to Deploy AWS CloudFormation StackSets and Lambda functions across multiple accounts/regions or OUs(Organisation Units)
• Working REST or HTTP APIs deployed to various stages in API Gateway other than default stage
• Cequence Bridge release 5.6.2 or later

Cequence Credentials

• Client ID
• Client Secret
• Auth Endpoint
• Bridge/Edge Endpoint

Creating a Cequence client ID and secret

Several Cequence components must authenticate to the Cequence UAP platform in order to transmit and receive data. Create authentication credentials in the Cequence UAP platform to enable this authentication.

1. Log in to the UAP management portal UI.
   The URL for the management portal is typically of the form https://ui.<your-tenant-name>.<domain>. Replace <your-tenant-name> with the name of your Cequence tenant organization. Replace <domain> with your domain name.
2. Select General Settings > User Management.
   The User Management pane appears.
3. Click the Clients tab.
4. Click Add New Client.
   The new client dialog box appears.
5. Type the client name in the Client Name field.
   This name is the client ID. Note the client ID for later use.
6. Enable the Traffic Management toggle.
7. (Optional) To change the token lifespan from the default of 1800 seconds, type a whole number of seconds in Token Lifespan.
8. Click Save.
   A dialog box with the client secret appears.
9. Click the blue Copy icon to copy the secret to the clipboard, then click Close.
   The client is now set up. Note the client name for future use.
   The client list appears.
10. Note the value of the client secret for later use. This value will not be shown again later on the UI for security reasons.

Items to download

Download the compressed archive file that contains the required files.

After downloading, extract the compressed archive file.

tar -xvf cequence-aws-api-gateway*
cd cequence

What you'll need

Set up the following roles in your Amazon console. Role creation is discussed in Amazon's documentation.

Note: Use the templates provided in the cequence/docs/stacksets-roles-templates directory of the extracted archive bundle file to create the roles. Do not use the default AWS templates.

• For StackSet administration, create an IAM role named AWSCloudFormationStackSetAdministrationRole in the administrator account. Configure the trust policy for the CloudFormation service and grant permissions to assume the execution role in the target accounts.
• In each target account, create a service role named AWSCloudFormationStackSetExecutionRole. Configure a trust policy to trust the administrator account. Log in as an Administrator to each target AWS account and create the role using CloudFormation and Create. Use cequence/docs/stacksets-roles-templates/AWSCloudFormationStackSetExecutionRole.yml as the template file. Use the AWS account ID of your StackSet administrator as the value of the AdministratorAccountIds parameter.

Required IAM role and permissions

The user account deploying the CloudFormation Template and running the scripts to enable or disable the integration must have the following IAM role and permissions. Create an AWS policy named iam_apigw_lambda_cloudwatch_eventbridge_cq_integration with the following permissions.

Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ScanRegions",
            "Effect": "Allow",
            "Action": "ec2:DescribeRegions",
            "Resource": "*"
        },
        {
            "Sid": "IAMRoleManagement",
            "Effect": "Allow",
            "Action": [
                "iam:CreateRole",
                "iam:GetRole",
                "iam:PassRole",
                "iam:DeleteRole",
                "iam:ListRoles",
                "iam:ListRolePolicies",
                "iam:PutRolePolicy",
                "iam:GetRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:AttachRolePolicy",
                "iam:DetachRolePolicy",
                "iam:ListAttachedRolePolicies",
                "iam:ListInstanceProfilesForRole"
            ],
            "Resource": "arn:aws:iam::*:role/*"
        },
        {
            "Sid": "LambdaFunctionManagement",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction",
                "lambda:DeleteFunction",
                "lambda:GetFunction",
                "lambda:UpdateFunctionConfiguration",
                "lambda:UpdateFunctionCode",
                "lambda:ListVersionsByFunction",
                "lambda:GetFunctionCodeSigningConfig",
                "lambda:ListFunctions"
            ],
            "Resource": "*"
        },
        {
            "Sid": "CloudWatchEventsManagement",
            "Effect": "Allow",
            "Action": [
                "events:PutRule",
                "events:PutTargets",
                "events:DescribeRule",
                "events:ListTagsForResource",
                "events:DeleteRule"
            ],
            "Resource": "arn:aws:events:*:*:*"
        },
        {
            "Sid": "SchedulerManagement",
            "Effect": "Allow",
            "Action": [
                "scheduler:CreateSchedule",
                "scheduler:DeleteSchedule",
                "scheduler:GetSchedule",
                "scheduler:ListSchedules"
            ],
            "Resource": "arn:aws:scheduler:*:*:schedule/*/*"
        },
        {
            "Sid": "APIGatewayAccess",
            "Effect": "Allow",
            "Action": [
                "apigateway:GET",
                "apigateway:POST",
                "apigateway:PUT",
                "apigateway:PATCH",
                "apigateway:DELETE",
                "logs:CreateLogGroup",
                "logs:DeleteLogGroup",
                "logs:PutRetentionPolicy",
                "logs:DescribeLogGroups",
                "logs:ListTagsForResource",
                "logs:CreateLogDelivery",
                "logs:PutResourcePolicy",
                "logs:UpdateLogDelivery",
                "logs:DeleteLogDelivery",
                "logs:DescribeResourcePolicies",
                "logs:GetLogDelivery",
                "logs:ListLogDeliveries"
            ],
            "Resource": "*"
        },
        {
            "Sid": "S3Access",
            "Effect": "Allow",
            "Action": [
                "s3:CreateBucket",
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:PutBucketPolicy",
                "s3:GetBucketPolicy",
                "s3:DeleteBucketPolicy",
                "s3:DeleteBucket",
                "s3:GetBucketLocation",
                "s3:GetBucketVersioning",
                "s3:PutBucketVersioning",
                "s3:DeleteObjectVersion",
                "s3:ListBucketVersions"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Sid": "CloudFormation",
            "Effect": "Allow",
            "Action": [
                "cloudformation:*"
            ],
            "Resource": "*"
        },
        {
            "Sid": "OrganizationRead",
            "Effect": "Allow",
            "Action": [
                "organizations:DescribeOrganization"
            ],
            "Resource": "*"
        }
    ]
}

Setting up the integration

Using CloudFormation to integrate the Cequence UAP platform with the AWS API Gateway takes place over several phases.

Set up the environment variables file

1. Extract the downloaded compressed archive file.

   tar -xvf cequence-aws-api-gateway*

2. Navigate to the cequence/scripts folder. Make a copy of the .env.example file named .env.

   cd cequence/scripts
   cp .env.example .env

3. Edit the .env file in the cequence/scripts directory. 

   Note: The .env file in the bundle contains variables applicable to deployments that use either Terraform or CloudFormation. The following variables affect CloudFormation deployments. Leave all other variables at their default values.

   # --------------------------
   # Account Info
   # --------------------------
   
   # If accounts are managed using AWS Organization, set this to true.
   # Currently only used for CloudFormation to create the bucket policy to allow access from all the accounts in the organization.
   # If this is true, cequence_is_multi_account_deployment can be false
   cequence_is_aws_organization_deployment=true
   
   # If cequence_is_aws_organization_deployment is set to true, this can be false
   # Multi account deployments are done using CloudFormation via AWS Console and not via scripts at the moment.
   # Currently only used for CloudFormation to create the bucket policy to allow access from all the accounts required.
   # If true, use accounts-config.csv to explicitly list the accounts to enable
   cequence_is_multi_account_deployment=false
   
   # --------------------------
   # Cequence Deployment Type
   # --------------------------
   # cloudformation or terraform
   # Currently CloudFormation is supported only via AWS Console. We can use scripts to upload the files to S3 and create the stacksets using the AWS Console.
   
   cequence_deployment_type=cloudformation
   
   # To upload the files to S3, set this to true. The account running the script is used to create the buckets. So, run it from the account managing the stacksets.
   cequence_cfn_auto_upload_to_s3=true
   
   # During disable with destroy, set this to true to also delete the S3 buckets created.
   cequence_cfn_auto_delete_from_s3=false
   
   
   # --------------------------
   # AWS Regions
   # --------------------------
   # The regions where we want to enable the Integration. For cloudformation, we create the buckets in all these regions. Say "all" for us to auto discover all enabled regions.
   cequence_aws_regions=all # "all" or a csv of regions like "us-east-1,ap-southeast-1"
   
   
   # --------------------------
   # Dependencies
   # --------------------------
   
   # If you want to install the dependencies automatically, set this to true.
   cequence_auto_install_dependencies=true
   
   
   # --------------------------
   # API Config
   # --------------------------
   
   # If you want to enable the REST APIs, set this to true.
   cequence_rest_api_enabled=true
   
   # If you want to enable the HTTP APIs, set this to true.
   cequence_http_api_enabled=true

4. Make all scripts executable and run the enable script.

   chmod +x *.sh
   ./enable_aws_apigw_integration.sh

   A successful run shows the following output on the console.

   --------------------------------------------------
   Upload successful!
   Bucket: cequence-aws-api-artifacts-339712954593-us-west-2
   --------------------------------------------------
   
   
   --------------------------------------------------
   S3 bucket operations completed successfully!
   We can use any enabled region we uploaded to. Below is an example URL to use in AWS CloudFormation Console.
   --------------------------------------------------
   S3 template URL to use in AWS CloudFormation Console: https://s3.amazonaws.com/cequence-aws-api-artifacts-<aws-account-id>-us-east-1/cloudformation/template.yaml
   --------------------------------------------------

5. Extract the S3 template URL from the script output in the previous step. Note this URL for use in the following procedure.

Preparing the template and resources

1. In the AWS CloudFormation console, click Stacksets > Create StackSet.
2. In Permissions, choose exactly one option.
   • IAM execution role name as AWSCloudFormationStackSetAdministrationRole
   • IAM admin role ARN
3. In Prepare template, select Template is ready.
4. Then provide the S3 location of the template YAML file extracted from the previous section
5. Click Next.
   The StackSet definition wizard advances to the next step.
6. Type a name for the StackSet, such as aws-cequence-apigw-integration.
7. Type a description for the Stackset.
   An example description is "This StackSet facilitates the deployment and creation of AWS artifacts that enable Cequence and AWS API Gateway integration across regions and accounts."
8. Specify values for the parameters added from your template, then click Next.
   The StackSet definition wizard advances to the next step.

Cequence API configuration parameters

Lambda configuration parameters

Content type configuration (Optional)

Optionally, you can configure regular expressions to define supported response content or unsupported static file extensions.

Regular Expressions for Supported Response Content Types

^[ ]application/xml[ ];?.$|^[ ]application/soap\\+xml[ ];?.$|^[ ]text/xml[ ];?.$|^[ ]application/soap[ ];?.$|^[ ]application/json[ ];?.$|^[ ]application/x-www-form-urlencoded[ ];?.$|^[ ]text/html[ ];?.*$

Regular Expressions for Unsupported Static File Extensions

/\\.(css|swf|bmp|bin|csv|oga|jsonld|eot|opus|mpkg|xul|tif|midi|ico|ics|html|jar|3g2|ogv|otf|zip|cda|ogx|rar|7z|tar|png|webp|webm|woff|pptx|mpeg|doc|odp|weba|odt|ods|aac|tiff|gif|vsd|js|mid|arc|avi|sh|epub|bz|jpeg|woff2|bz2|3gp|azw|htm|jpg|xlsx|rtf|svg|ttf|wav|docx|xhtml|mp4|mp3|txt|git|gz|pdf|ppt|abw|mjs|csh|php|xls|ts)$/i

Cequence Credentials

• Client ID
• Client Secret
• Auth Endpoint
• Bridge/Edge Endpoint

Optional IAM Role ARNs

The integration uses Amazon Resource Names (ARNs) with IAM roles. Optionally, specify the following ARNs.

• Cequence Lambda Role ARN. Creates a new role if this isn't provided in the params
• API Gateway CloudWatch Role ARN. Creates a new role if this isn't provided in the params
• EventBridge Scheduler Role ARN. Creates a new role if this isn't provided in the params

Configuring StackSet options

1. In tags, click Add new tag to add new tags.

   cequence_integration: true
   cequence_created: true
   cequence-deployment-type: cloudformation

2. Set Execution Configuration to Active.
3. Select the checkboxes to acknowledge required capabilities.
   • AWS CloudFormation might create IAM resources with custom names.
   • AWS CloudFormation might require CAPABILITY_AUTO_EXPAND.
4. Click Next.
   The StackSet definition wizard advances to the next step.

Setting Deployment options

1. Select Deploy new stacks to add stacks to the StackSet.
2. Choose Deploy stacks in accounts and provide target account IDs.
3. Choose a deployment region.
   • Choose Specific Regions and select target regions.
   • Alternately, choose All regions. 
     Note: Select only the enabled AWS regions. Disabling regions while All is selected causes a Cloudformation error.
4. Click Next.
   The StackSet definition wizard advances to the next step.
5. Set the value of Maximum concurrent accounts to the total number of accounts you want to deploy the template on.
6. Set the value of Failure tolerance to one less than the total number of accounts you want to deploy the template on.
7. Set Regional Concurrency to Parallel.
8. Set Concurrency mode to Soft failure tolerance.

Reviewing and Deploying

1. Review all configurations.
2. Click Submit.
3. Monitor the deployment's progress. Look for a change in status from Running to Succeeded.

Testing the Integration

1. Make API calls to your configured endpoints
2. Allow CloudWatch logs to populate (1-2 minutes)
3. Check the Lambda log group for statistics
4. Click on a log stream
5. Expand statistics at the end with message "LAMBDA FINISHED PROCESSING. Stats of the run:"

Example Lambda statistics

// Example Lambda Stats for Async Mode
// Duration: 444914.77 ms	Billed Duration: 444915 ms	Memory Size: 512 MB	Max Memory Used: 301 MB	Init Duration: 574.73 ms

{
  // Number of APIs given to the lambda in env var or csv file
  // "api_id/stage" is one Log Group
  "log_group_count": 25,

  // Lambda triggered time minus configured interval time
  "cloudwatch_filter_start_time": 1728637887330,

  // Lambda triggered time
  "cloudwatch_filter_end_time": 1728580281330,

  // Interval time in milliseconds
  "log_fetch_interval_mill_sec": 60000,

  // Number of events to fetch from CloudWatch. Max is 10000
  "log_fetch_limit": 10000,

  // Statuses got from CloudWatch SDK when fetching logs for all Log Groups
  "cloudwatch_statuses": {
    "200": 800
  },

  // Total number of attempts made by the SDK to fetch logs from CloudWatch
  "total_cloudwatch_fetch_attempts": 1011,

  // Average async CloudWatch latency in milliseconds per Log Group
  "average_async_cloudwatch_latency": 7957.141444114738,

  // Total async CloudWatch latency in milliseconds
  "total_async_cloudwatch_latency": 8044670,

  // Number of times the SDK had to retry fetching logs from CloudWatch
  "total_cloudwatch_retries": 193,

  // Number of times the SDK successfully fetched logs from CloudWatch
  "total_cloudwatch_successes": 800,

  // Number of times the SDK failed to fetch logs from CloudWatch
  "total_cloudwatch_failures": 0,

  // Total number of events fetched from CloudWatch for all Log Streams in all Log Groups
  // We get 9 events per Log Stream. A Log Stream is created per API call
  "total_events_fetched": 3780000,

  // Total number of pages fetched from CloudWatch for all Log Groups
  "total_pages": 800,

  // Additional pages fetched by the SDK when more than 1 page is available for a Log Group
  "additional_pages_fetched": 775,

  // Total number of Log Streams fetched from CloudWatch for all Log Groups
  "total_streams_fetched": 224600,

  // Total number of Log Streams that are incomplete
  // In a page, CloudWatch might not return all events of a Log Stream
  "total_incomplete_streams_count": 1100,

  // Total number of Log Streams that are truncated
  // If headers or body size is more than 1KB, truncated event is added with [TRUNCATED] message
  "total_truncated_streams_count": 0,

  // Total number of complete untruncated Log Streams
  "total_complete_streams_count": 223500,

  // Total number of transactions parsed from all Log Streams based on Cequence format
  "total_transactions_parsed_from_streams": 223500,

  // Total number of transactions failed to parse from all Log Streams based on Cequence format
  "total_failed_to_parse_streams": 0,

  // Total batches sent to Cequence Edge
  "total_batches": 750,

  // Total successful batches sent to Cequence Edge
  "successful_batches": 750,

  // Total failed batches sent to Cequence Edge
  "failed_batches": 0,

  // Latency to get the access token if not found in local cache
  "cequence_auth_latency": 573,

  // Number of times Lambda retried to get the access token
  "total_cequence_auth_retries": 1,

  // Number of times Lambda retried to send all the batches to Cequence Edge
  "total_cequence_edge_retries": 750,

  // Minimum number of transactions sent across all batches
  "min_batch_sent": 17,

  // Maximum number of transactions sent across all batches
  "max_batch_sent": 374,

  // Average number of transactions sent across all batches
  "average_batch_size": 298,

  // Average async latency to send all the batches to Cequence Edge
  "average_async_cequene_edge_latency": 5382.534666666666,

  // Total async latency to send all the batches to Cequence Edge
  "total_async_cequence_edge_latency": 4036901,

  // Total async network time to send all the batches to Cequence Edge
  // Total async CloudWatch latency + Total async Cequence Edge latency
  "total_async_network_time": 12082144,

  // Estimated Lambda execution time in milliseconds
  "estimated_lambda_execution_time": 444184,

  // Latencies per batch sent to Cequence Edge in format "latency/batch_size"
  "cequence_edge_latencies_per_batch": "10882/374,9905/374,193/17,...",

  // Latencies per page fetched from CloudWatch for all Log Groups
  "cloudwatch_latencies": "987,1440,..."
}

Now the transactions should start to appear on the UAP Sentinel & Spartan Dashboard

Adding More AWS Regions For API Discovery

AWS Stackset does not allow adding new regions to the same stackset
when we edit/override an existing Cloudformation Template.

In this scenario, the existing Stackset needs to be deleted, and a new Stackset needs to be created.

Disable The Integration

To Disable the integration

1. Go to the CloudFormation console
2. Select "StackSets" from the left navigation >> Choose and click the stackset
3. Click Action and Override stackset parameters
4. Provide the account number or the Organization units and select the regions where the integration needs to be disabled. Leave defaults and click next
5. Now select CequenceHttpApiEnabled and CequenceRestApiEnabled from the list of Parameters. Click on Override stackset value
6. Choose false and Click Save Changes
7. The new override value should be updated
8. Click Next
9. Review the changes and Submit
10. Navigate to the Operations Tab and verify that the operation has succeeded

Updates and Maintenance

Update the Parameters of the template

To update the Parameters of the template, use the override action and update the values

Template Updates

When you update to a new template version instead of updating parameter values, use the Replace option instead of the current template. Because AWS caches templates, this is necessary even when you use the same S3 link. AWS only fetches the template again upon replacement.

Update the template to a new version

To update the template to the new version, re-run steps 2 to 5 under the "Setting up the Environment Files section" to upload the new template to S3 bucket. Then, extract the S3 bucket URL

Navigate to the CloudFormation Template. Select the operations tab and use the edit action and replace the yaml with the s3 bucket extracted link

Note:- 

Update the StackSet under the following circumstances.

• Parameter changes
• Adding or modifying resources
• Template logic improvements
• Minor architectural changes

Replace the StackSet by deleting the original StackSet and recreating a new StackSet under the following circumstances.

• The resource name changes
• Fundamental architecture changes
• Permission model changes
• Cross-region dependency changes
• Consistent update failures
• Template structure overhaul

Update behaviors

• Normal updates only update changed resources. Unchanged resources are not affected.
• CloudFormation automatically determines which resources need replacement.
• AWS caches old templates and configurations.
• Previous templates remain accessible in the operation history.
• Stack instances retain their current state until explicitly updated.

Deleting the Integration

Using AWS Console

1. Go to the CloudFormation console.
2. Select "StackSets" from the left navigation.
   Note: Delete all stack instances before deleting the StackSet.
3. Navigate to the Stack Instances Tab > Select Actions > Delete Stacks From StackSet.
4. Provide the Account Numbers and select Add All Regions.
5. Click Next and Review.
6. Click Submit.
7. Navigate to the Operations tab and click Verify.
   Wait for the status to change to Succeeded. The change takes a few minutes depending on the number of accounts and regions enabled.
8. Select the StackSet.
9. Click Actions > Delete StackSet.
10. Confirm deletion.

The process removes the StackSet definition and all associated stack instances across the specified accounts and regions.

Important Notes

• The deletion is asynchronous - monitor the operation status.
• Confirm that you have proper permissions, including the StackSet admin role.
• Consider the impact on resources managed by the StackSet instances.

Delete the Cequence Bundle From S3  Buckets

Set the below variables in the .env file in the cequence/scripts folder.

# During disable with destroy, set this to true to also delete the S3 buckets created.
cequence_cfn_auto_delete_from_s3=true

# If you want to enable the REST APIs, set this to true.
cequence_rest_api_enabled=false

# If you want to enable the HTTP APIs, set this to true.
cequence_http_api_enabled=false

Run the disable script with the destroy flag.

cd cequence/scripts
./disable_aws_apigw_integration.sh --destroy

Note: This process can take some time when many regions are enabled.

Known limitations of HTTP API logging

HTTP APIs have restricted logging capabilities compared to REST APIs. Details on HTTP API access log customization are available in Amazon's documentation.

Available HTTP API Log Variables

• requestTimeEpoch
• requestId
• accountId
• stage
• instance_id
• ip
• host
• http-version
• http-method
• uri-query-fragment
• status-code

Note: Request and response body content is not logged for HTTP APIs.

For detailed HTTP API access log customization, refer to: Customize HTTP API access logs - Amazon API Gateway

In case of problems

• Template caching: Use template replacement instead of update for version changes.
• Permission errors: Verify all required IAM roles and policies are correctly configured.
• Deployment failures: Check CloudFormation stack events for detailed error messages.
• Lambda execution issues: Review Lambda function logs and increase log level to "debug" if needed.
• Cross-region deployment issues: Ensure StackSet execution roles exist in all target regions.

Notes

1. AWS API Gateway limits log events to 1024 bytes. Log events larger than 1024 bytes, such as request and response bodies, are truncated by API Gateway before submission to CloudWatch logs.

2. For REST APIs where AWS truncates the request and response body, Cequence sets the following as the body.

   { "cq_body_truncated": "true" }

   For REST APIs where the Cequence plugin is unable to capture the request and response if they are truncated, below header is set from the plugin and sent to the Cequence UAP Platform.

3. For all HTTP APIs, the following header is set from the plugin and sent to the Cequence UAP Platform since the request and response bodies are unavailable.

   "cq-discovery-only" = "true"

4. Set the retention period for CloudWatch Log Groups associated with APIs deployed in AWS API Gateway to at least 1 day, or according to the your retention policy.
5. The Request and Response body for HTTP APIs are not logged by default in AWS. Only the following variables are captured and send to UAP Platform.
   • requestTimeEpoch
   • requestId
   • accountId
   • stage
   • instance_id
   • ip
   • host
   • http-version
   • http-method
   • uri-query-fragment
   • status-code

Refer to AWS Documentation.