The IBM DataPower API gateway acts as a proxy between consumer applications and backend APIs. The API gateway secures, protects, and manages APIs by intercepting API requests and applying policies, such as throttling and security.

You can extend the gateway's functions to change the API flow by implementing custom gateway scripts and global policies in order to meet the requirements of your specific use case.

In order to integrate with the Cequence Unified API Protection (UAP) platform, the IBM DataPower API gateway must send all request and response data to a Cequence API Edge server. This process uses synchronously triggered custom scripts to process request and response data, then asynchronously format and post that data to the Cequence Bridge, which transmits the data to the Cequence Edge server. The Edge server analyzes the transaction data to provide actionable insights into your API traffic. Configuration of traffic filtering and sensitive data masking for the Cequence Bridge is discussed in a separate article.

Using the procedures in this article requires a bundle of files provided by Cequence. You can download this bundle from this page.

High level architecture

The client sends a request to the IBM DataPower API Gateway.
The request script intercepts the request and captures the request payload and metadata.
The request script forwards the request to the Backend/Target Applications.
The backend or target application generates a response, and any subsequent policies are executed.
The response script intercepts the response in order to capture the response payload and metadata.
The response script forwards the response to the clients or caller.
The script posts the combined request and response payload, along with the associated metadata, to the Cequence Bridge.
The Cequence Bridge forwards this information to the Cequence UAP Edge server for threat analysis, detection, and mitigation.

Bridge availability

If the Bridge becomes unavailable, there is a chance that DataPower might restart if the requests are too high. Hence, it is advised to configure Bridge with high availability and always keep it up and running.

Reference: https://www.ibm.com/support/pages/node/399915

Integration setup

Setting up the Cequence UAP platform to integrate with the IBM DataPower API gateway is a multistep process.

Create a Cequence policy
Upload your TLS certificate
Create a TLS client profile for Cequence
Configure the TLS client profile for the gateway

Pre-Requisites

IBM DataPower API gateway release 10.62 or 10.5.0.18 or later
Cequence UAP platform release 7.3.X or later
Client ID (Key) and Client Secret generated from the Cequence UAP platform UI
Cequence Sensor Bridge deployed in close proximity to the IBM DataPower API Gateway
Endpoints for the Cequence Bridge.
The plugin is supported for Multi-Protocol Gateway (MPGW) in DataPower

Generating a Client ID and Client Secret

For instructions on generating a Client ID and Client Secret, see the documentation on creating API keys.

Installing the integration

This procedure installs the software to integrate the Cequence UAP platform with the IBM DataPower API gateway. Other procedures later in this article configure the integration.

Download the archive file cequence-datapower-v1.zip from the Downloads page.

Copy the downloaded archive file to a directory named cequence. Create this folder if necessary.

cp <$DOWNLOAD_DIR>/cequence-datapower-v1.zip $CEQUENCE_DP_HOME 
mkdir -p $HOME/cequence && cd "$_"

Uncompress the archive file, then navigate to the cequence-datapower-v1 directory. The directory name may vary to match changes in the file name.
```
unzip cequence-datapower-v1.zip 
cd cequence-datapower-v1
```
Set the current directory as the value of the CEQUENCE_DP_HOME variable.
```
export CEQUENCE_DP_HOME=$(pwd)
```
Copy the environment template to a new file.
```
cd $CEQUENCE_DP_HOME 
cp .env_example .env
```
Open the .env file in a text editor and set the value of the CEQUENCE_BRIDGE_URL variable to the URL of the Bridge endpoint.
Skip this step unless you require x-cequence performance headers that show latency. Set the value of the CEQUENCE_ENABLE_CUSTOM_RESPONSE_HEADERS variable to true.
Run the setup script.
```
cd $CEQUENCE_DP_HOME/scripts 
chmod +x *.sh 
./setup.sh
```
The setup script generates the gateway scripts required for the Cequence Policy. The scripts are written to the $CEQUENCE_DP_HOME/generated folder.

The integration software is now installed. The directory structure and contents of the generated folder are as follows.

└── generated 
   ├── cequence-policy.zip 
   ├── cequence-tls-client-profile.zip 
   ├── cequence.crt 
   ├── cequence.js 
   └── postman-echo.crt

Creating a Cequence Policy

You can create a policy for the Cequence UAP platform by importing a compressed file with the necessary items or with manual configuration. The cequence-policy.zip file is generated by the setup script in the CEQUENCE_DP_HOME/generated/ directory.

Importing the compressed file

Log in to the IBM DataPower API gateway.
From the left sidebar, click Import, then select Import Configuration.
Click Choose file.
Select the cequence-policy.zip file and click Next.
Click Import.
Click Done.

You can confirm successful file import by checking the Multi-protocol Gateway policy section of the Gateway Policy page for the new policy.

Manually configuring the policy

Skip this procedure if you have successfully imported the compressed file.

In the left sidebar, click File Management.
Only perform this step if the cequence directory does not exist. Click Actions > Create subdirectory.
In the Name field, type cequence, then click Confirm Create.
Click Continue.
Expand the local section.
Next to the cequence directory, click Actions > Upload Files.
Select the cequence.js file in the $CEQUENCE_DP_HOME/generated directory.
Click Upload.
Click Continue.
In the left sidebar, click Multi-Protocol Gateway Policy.
Click Add New Policy.
In the Name field, type cequence-policy, then click New Rule.
In the Name field, type cequence_rule, then double-click Match action of the rule.
Select __default-accept-service-providers__ from the dropdown, then click Done.
Drag a GatewayScript to the right of the Match rule. The GatewayScript icon is a set of braces, {}.
Double-click the GatewayScript.
From the dropdown, select INPUT.
Select local:///cequence, then select the uploaded script.
In the Output field, type CEQUENCE_SCRIPT_OUTPUT, then click Done.
Click Apply Policy. The IBM DataPower API Gateway automatically adds a Results Action to the end of the policy action queue.
At the right side of the queue, double-click the last Results Action.
From the dropdown, select INPUT, then Done.
Click Apply Policy.
Drag an Advanced action to the right of the GatewayScript action. The Advanced action icon is a black diamond with a white curved vertical line.
Double-click the Advanced action.
Select Set Variable, then click Next.
In the context field, type NULL.
In the Variable name field, type the following.
```
context/CEQUENCE_BRIDGE_OUTPUT/_roottree
```
In the Variable value field, type the following.
```
var://context/cequence/bridgePayload
```
Click Done.
Drag an Advanced action to the right of the Advanced action you configured previously.
Double-click the Advanced action.
Select Set Variable, then click Next.
In the context field, type NULL.

In the Variable name field, type the following.

context/CEQUENCE_BRIDGE_OUTPUT/_extension/header/Content-Type

In the Variable value field, type the following.
```
application/json
```
Click Done.
Drag an Advanced action to the right of the Advanced action you configured previously.
Double-click the Advanced action.
Select Conditional, then click Next.
From the Input drop-down, select CEQUENCE_SCRIPT_OUTPUT
In the Match Condition field, type the following.
```
/cequence/processResultsAction/text() = 'true'
```
From the Action drop-down, select Results Asynchronous.
Click Create Action.
From the Input drop-down, select CEQUENCE_BRIDGE_OUTPUT.
From the Destination drop-down, select var://.
In the Destination field, type the following.
```
context/cequence/bridgeUrl
```
Click Done.
In the Match Condition field, type the following.
```
/cequence/processParseAction/text() = 'true'
```
From the Action drop-down, select Parse.
Click Create Action.
From the Input drop-down, select INPUT.
From the Output drop-down, select OUTPUT.
Click Done, then click Done again.
Click Apply Policy.
Click Save Config.
Click Close.

Uploading the Cequence security certificate

The IBM DataPower API gateway cannot import the Cequence TLS certificate from an archive file. This procedure uploads the security certificate.

If the settings of your existing TLS client profile's validation credentials are set to Disable Server Certificate Validation, you can skip this procedure and go directly to Configuring the Cequence Plugin.

When you change the security certificate, perform this procedure again to add the new certificate.

When the validation credential settings for your existing TLS profile are configured to check for dates, renew the Cequence certificate before expiry, then re-upload the certificate.

When you renew or update the Cequence Bridge security certificate, run the setup.sh script to generate a new cequence.crt file. Alternately, you can export the certificate from your browser.

In the left sidebar, click File Management.
Next to the certificate, click Actions, then Upload Files.
Choose the following file and click Upload.
```
CEQUENCE_DP_HOME/generated/cequence.crt
```
If you are renewing or changing an existing certificate, click Overwrite Existing Files. Otherwise, skip this step.
Click Continue.
Click Save Config.
Click Close.

Creating the Cequence TLS Client Profile

You can create a Cequence TLS client profile manually or by importing a compressed archive file.

Before you begin

Verify that a valid Cequence security certificate is present.

Configure the TLS client profile by importing a compressed archive file

On the left sidebar, click Import Configuration.
Click Choose File.

Select the following file, then click Next.

CEQUENCE_DP_HOME/generated/cequence-tls-client-profile.zip

Click Import
Click Done

Display the list of TLS client profiles to confirm that the profile exists.

Manually configuring the TLS client profile

Skip this procedure if you have successfully configured a TLS client profile by importing a compressed archive file.

On the left sidebar, click TLS client profile.
Click Add.
In the Name field, type cequence-tls-client-profile.
Toggle Validate server certificate to off.
Click Apply.
Click Save Config.
Click Close.

Configuring the TLS client profile for the gateway

The asynchronous actions configured in the Cequence policy rule send payloads to Cequence over HTTPS. Using HTTPS requires a TLS client profile to establish secure connection.

Gateway APIs without an existing TLS client profile use HTTP. Configure these APIs to use the cequence-tls-client-profile configured previously.

When using IBM DataPower version 10.5.0.18 and you need to select the TLS client profile, navigate to the multi-protocol gateway, select the gateway where you want to update the configuration, select the Proxy settings tab from the header, and choose the TLS Client Profile from the dropdown.

Configuring existing TLS client profiles

This procedure is only necessary when an API is using an existing TLS profile with server certificate validation. This procedure adds the Cequence certificate to the existing validation credentials of the existing TLS profile.

In the IBM DataPower API gateway dashboard, navigate to the TLS profile in use.
Click Edit.
From the Certificates dropdown, select cequence-cert.
Click Add.
(Optional) To enable expired certificates, disable the Use CRL and Check Dates toggles.
Click Apply.
Click Apply.
Click Apply.
Click Save Config.
Click Close.

Configuring the Cequence plugin

When you use the default policy for your gateway, set the processing policy as cequence-policy. When you have an existing policy you want to use for the gateway, update the rules.

For example, a gateway with a default, non-custom policy appears similar to the following image.

By contrast, a gateway with an existing custom policy appears similar to the following image.

Replacing a default policy

These steps assume you are logged in to the IBM DataPower console.

Click Control Panel > Multi-Protocol Gateway.
Click the name of the gateway with the default policy you want to change.
From the dropdown, select cequence-policy.
Click Apply.

Repeat these steps for each multi-protocol gateway without an existing custom policy.

Modifying an existing custom policy

Click Control Panel > Multi-Protocol Gateway.
Click the name of the Gateway with the default policy you want to change.
Click the three-dot icon next to the existing processing policy. The configuration window appears.
When the policy has request and response rules, edit each rule. For each request rule, place the Cequence Processing rule at the start of the request rule, after the matching action. For each response rule, place the Cequence Processing rule at the end of the response rule. For each Both Directions rule, add the Cequence Processing rule at the start. These placements enable the capture of the request and response payloads as received by the Datapower gateway.
Double click Advanced.
Click Call Processing Rule, then click Next.
From the Input dropdown, select Input.
From the Processing rule dropdown, select cequence_rule.
From the Output dropdown, select NULL.
Click Done.
Click Apply.

Clearing a "PIPE Output not followed by PIPE Input" error

When the end of the response rule does not contain a default results action, you may see an error message similar to PIPE Output not followed by PIPE Input on action. To clear this error, set the output for the script before the Processing rule.

Click Control Panel > Multi-Protocol Gateway.
Click the name of the Gateway with the default policy you want to change.
Click the three-dot icon next to the existing processing policy. The configuration window appears.
Double click Advanced.
Double-click the double braces ({}) icon. The script editor pane appears.
From the Output dropdown, select OUTPUT.
Click Done. A new Default Results action appears at the end of the rule timeline.
Delete the new Default Results action.
Click Apply Policy, then close the popup.
Click Apply.

Testing the Plugin Setup

To test that the integration is operating correctly, first send some traffic through the system.

Log in to the Cequence UAP platform. The dashboard appears.
In the left hand navigation, in the Runtime Inventory section, click API Inventory. The API Inventory page appears in the Dashboard main area.
At the top of the API Inventory page, click the Active API Endpoints tab. Typically this is the active tab by default, so you may not need to perform this step.
At the top of the list of endpoints, click the vertical three-dot menu next to More. A list of other time frames appears.
Select 15 Min. The API Inventory displays API endpoints that have been active in the last 15 minutes.

Check the displayed results to confirm the traffic you sent appears in the inventory of active API endpoints.

Automated deployment with DCM and SOMA

DCM is a package for IBM DataPower configuration management. This implementation automates the deployment of Cequence security policies to IBM DataPower Multi-Protocol Gateways.

Prerequisites

Required software

Software	Version	Purpose
JDK/JRE	1.6+	Java runtime for DCM
Apache Ant	1.8.1+	Build automation
libxml2-utils	Any	Provides xmllint for XML parsing
jq	Any	JSON processor for config parsing
curl	Any	HTTP client for SOMA API calls
openssl	Any	SSL certificate fetching
zip	Any	Package creation

DataPower prerequisites

Before running the automation, you must configure the following on your IBM DataPower appliance.

Enable XML Management Interface

The automation communicates with DataPower via the XML Management Interface (SOMA API on port 5550).

In the DataPower WebGUI, search for XML Management in the left sidebar.
Select XML Management Interface.
Click Enable if not already enabled.
Click Apply.

The XML Management Interface listens on port 5550 by default. Verify that this port is accessible from your deployment machine.

Create deployment policy (Optional)

Deployment policies control how objects are imported into DataPower. The automation requires a deployment policy named cequence-deployment-policy. The automated script handles this creation, but you can create it manually if needed.

In the DataPower WebGUI, search for Deployment policy in the left sidebar.
Select Deployment Policy.
Click Add button.
Enter the following.
- Name: cequence-deployment-policy
Click Apply.

Deployment policies use matching statements and clause types to control the inclusion of configuration data among imported configuration packages. The policy contains a sequence of clauses to include, exclude, or change configuration data during import. For this integration, the default (empty) policy allows all Cequence objects to be imported as-is.

Configuration

Environment variables

Copy the example file.

cp .env_example .env

Sample file:

CEQUENCE_BRIDGE_URL="https://bridge.org.cequence.ai/api-transactions"

# Set to true if you want to enable x-cequence response headers
CEQUENCE_ENABLE_CUSTOM_RESPONSE_HEADERS="false"

skip_interactive_mode="false"

# readlink -f $(which java) | sed 's|/bin/java$||'
JAVA_HOME="/usr/lib/jvm/java-17-openjdk-amd64"

# readlink -f $(which ant) | sed 's|/bin/ant$||'
ANT_HOME="/usr/share/ant"

DataPower configuration

Copy the example file.

cp config_example.jsonc config.jsonc

Sample file:

{
  "system": {
    "action" : "install", // install, backup, restore, save, rollback
    "xml_management_interface": {
      "local_address": "127.0.0.1", // IP address of the host machine
      "local_port": 5550 // Port number for the XML management interface
    },
    "domains": ["default", "domain1"], // List of domains to install
    "backup_creds": "cred1", // Credentials for backup defined in the credentials section (credentials.cred1)
    "save_creds": "cred1",
    "restore_creds": "cred1",
    "restore_filename":"backup.xml",
    "backup_filename":"backup.xml",
    // Targeted rollback state (persistent, gitignored). Used by install/rollback actions.
    "state_dir": "dcm/state",
    "logtype" : "info" // "info" to clear the screen or give as "verbose" to not clear the screen and show all logs
  },
  "credentials": {
    "cred1": {
      "username": "admin",
      "password": "admin"
    },
    "cred2": {
      "username": "",
      "password": ""
    }
  },
  "domains": {
    "default": {
      "creds": "cred1"
    },
    "domain1": {
      "creds": "cred1"
    }
  },
  "cequence_on_call_default_rule_positions": {
    "request-rule": 0, // Start position for request rules
    "response-rule": -1, // End position for response rules
    "bidirectional-rule": 0 // Start position for bidirectional rules
  },
  "custom_positions_to_cequence_on_call_processing_rules": {
    // "postman-echo-req":1 // Example rule name with a custom position of 1 i.e., second position as we start from 0. If there are only 2 existing objects, use 0.
  }
}

Installation

Generate Cequence Policy Packages

cd scripts
./setup.sh

This creates the following in the generated/ folder (gitignored):

cequence-policy.zip - Main Cequence policy package
cequence-tls-client-profile.zip - TLS client profile
cequence-cert.zip - Cequence certificate package
cequence.js - Generated gateway script
cequence.crt - Fetched SSL certificate

Deploy to DataPower

# Set action to "install" in config.jsonc, then run:
./dcm.sh

# Or overwrite the action by running:
./dcm.sh --action install

What it does:

Builds environment and device properties files.
Exports existing MultiProtocolGateway objects.
Creates a backup of current configuration.
Imports Cequence objects (policy, TLS profile, cert).
Bridges existing gateways with Cequence policy.
Inserts cequence_on_call processing action into existing policies.
Adds Cequence certificate to validation credentials.
Saves the configuration.

Sample output:

./dcm.sh
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                          Automation Utility for IBM Datapower Configuration  [install]
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1. Build Environment files  [  ]
2. Build ant file for DCM import cequence objects  [  ]
3. DCM Export Objects from IBM Datapowers  [  ]
4. SOMA Backup IBM Datapower configuration  [  ]
5. DCM Import Cequence Objects to Datapower [  ]
6. Local modify Default Processing Policy with Cequence Processing Poicy [  ]
7. local modify Proxy with Cequence TLS Profile [  ]
8. DCM import Modified Processing Policy & TLS Client Profile to IBM Datapower [  ]
9. DCM Insert Cequence On Call Processing Action into Existing Processing Policy [  ]
10. DCM Add Cequence cert to Validation Credentials On Call Processing Action into Existing Processing Policy [ ]
11. SOMA Restore IBM Datapower Configuration [N/A ]
12. SOMA Save IBM Datapower Configuration [ ]



Install complete

Rollback

Removes Cequence integration from DataPower.

# Set action in config.jsonc
"action": "rollback"

# Or run
./dcm.sh --action rollback

What it does:

Deletes Cequence objects from DataPower via DCM ant scripts.
Removes cequence.js file from DataPower.
Removes local://cequence directory from DataPower.

Sample output:

./dcm.sh --action rollback
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                          Automation Utility for IBM Datapower Configuration  [rollback]
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1. Build Environment files  [  ]
2. Build ant file for DCM import cequence objects  [✗ ]
3. DCM Export Objects from IBM Datapowers  [✗ ]
4. SOMA Backup IBM Datapower configuration  [  ]
5. DCM Import Cequence Objects to Datapower [✗ ]
6. Local modify Default Processing Policy with Cequence Processing Poicy [✗ ]
7. local modify Proxy with Cequence TLS Profile [✗ ]
8. DCM import Modified Processing Policy & TLS Client Profile to IBM Datapower [✗ ]
9. DCM Insert Cequence On Call Processing Action into Existing Processing Policy [✗ ]
10. DCM Add Cequence cert to Validation Credentials On Call Processing Action into Existing Processing Policy [✗ ]
11. SOMA Restore IBM Datapower Configuration [N/A ]
12. SOMA Save IBM Datapower Configuration [  ]



Rollback complete

Verify installation

At IBM DataPower Portal

You can open your configured multi protocol gateway and look for below components.

Processing Policy → cequence-policy
TLS client profile → cequence-tls-client-profile

Here you should see options selected that are related to Cequence, as mentioned above.

With UAP

Send traffic and navigate to the Cequence UAP dashboard to check the APIs discovered.

Automation troubleshooting

Common issues

Connection refused on port 5550.
- Verify that DataPower XML Management Interface is enabled.
- Verify the IP address and port in config.jsonc.
- Check firewall rules.
Authentication failed.
- Verify credentials in config.jsonc.
- Verify that the user has admin privileges on DataPower.

Rolling back an integration

To roll back DataPower integration for APIs that did not have a previous custom policy, revert the policy to default.

Log in to the DataPower gateway.
Click Control Panel > Multi-Protocol Gateway.
Select the gateway to roll back.
Set the Processing Policy to default.
Click Apply.

Repeat for all applicable gateways.

For APIs that had an existing custom policy, delete the call processing rule for each rule in the policy.

Log in to the DataPower gateway.
Click Control Panel > Multi-Protocol Gateway.
Select the gateway to roll back.
Click the three-dot icon next to the existing processing policy. The configuration window appears.
Double click Advanced.
Double click the Call Processing Rule.
Click Delete.
Click Apply.

Rolling back the TLS client profile

When a gateway has an HTTP backend and uses the Cequence TLS profile, you can roll back the TLS profile.

Log in to the DataPower gateway.
Click Control Panel > Multi-Protocol Gateway.
Select the gateway to roll back.
Set the TLS client profile to none.
Click Apply.

Rolling back HTTPS backend gateways

For gateways with HTTPS backends that have had their validation credentials modified by adding the Cequence certificate, rolling back involves removing this certificate.

Log in to the DataPower gateway.
Click Control Panel > Multi-Protocol Gateway.
Select the gateway to roll back.
Click the three dots next to the validation credentials. The credential configuration window appears.
Click X next to the Cequence certificate.
Click Apply. The view returns to the credential profile.
Click Apply. The view returns to the gateway profile.
Click Apply.

Deleting the Cequence policy

Log in to the DataPower gateway.
Click Control Panel > Multi-Protocol Gateway Policy.
Click cequence-policy.
Click Delete Policy.

Deleting the Cequence TLS Client Profile

Log in to the DataPower gateway.
Search for the TLS client profile.
Select the TLS profile.
Click the Cequence profile.
Click Delete.

Deleting the Cequence validation credentials

Log in to the DataPower gateway.
Search for the Validation credentials.
Select the credentials.
Select cequence-creds.
Click Delete.

Delete the Cequence certificate

Log in to the DataPower gateway.
Search for the Cequence certificate.
Select the certificate.
Click cequence-cert.
Click Delete.

Removing the Cequence files

Log in to the DataPower gateway.
Search for file management.
Enter the file management page.
Select cequence.crt and cequence.js.
Delete the selected files.
Delete the cequence directory.

Related to