Current release: v8.1.7

Cequence UAP 8.1 delivers major infrastructure improvements centered around upgrading to Elasticsearch 8.18, a search and analytics engine that powers UAP's data storage and retrieval capabilities. This release enhances system stability and performance while introducing a more efficient audit logging system using Apache Kafka, a distributed streaming platform for handling real-time data feeds. The update also includes improved configuration management tools, user interface enhancements, strengthened security through vulnerability fixes, and enhanced defensive capabilities.

Major Infrastructure Changes

Elasticsearch 8.18 Upgrade

• Core System Upgrade: Elasticsearch, UAP's primary data storage and search engine that handles transaction logs, detection data, and system analytics, has been upgraded from version 7.17.21 to 8.18.4. This major version jump delivers improved performance, enhanced security features, and better compatibility with future updates.
• Streamlined Configuration: Unnecessary Elasticsearch features have been disabled to optimize system stability. The XPack ML (machine learning) module and GeoIP downloads (geographic IP location services) have been removed. These features were consuming resources without providing value to UAP's core functionality, particularly in air-gapped environments where external downloads are restricted. Note: This optimization primarily benefits on-premises deployments; SaaS users benefit automatically without needing to manage these settings.
• Updated Index Templates: All index templates (data storage structures) have been modified to ensure proper compatibility with Elasticsearch 8.18.4's new requirements.

ARM64 Architecture Support

UAP 8.1 introduces multi-architecture support through a phased deployment approach. ARM64 is an alternative processor architecture (compared to traditional x86/AMD64) that can offer better performance-per-watt and reduced infrastructure costs. The system now includes ARM64-compatible container images for over 25 services, with some components remaining AMD64-only due to technical requirements or third-party dependencies that haven't yet been ported to ARM64. This architecture-aware deployment enables organizations to leverage modern ARM-based infrastructure where available while maintaining compatibility with existing systems.

User Interface Enhancements

Transaction View Improvements

The View Transactions page has received multiple fixes to improve usability. Horizontal scrolling issues that previously made it difficult to view complete transaction data have been resolved, and duplicate User Agent columns that cluttered the interface have been eliminated.

Detection Dashboard Updates

• Improved URI Display: The dashboard now shows actual URIs (the specific web addresses accessed) instead of parameterized versions. For example, URIs like /api/user/12345 will now be displayed as-is rather than /api/user/{id}, making it easier to investigate specific transactions.
• Eliminated Duplicate Filters: Duplicate User Agent filter options have been removed from the filtering interface.
• Better Space Utilization: Screen real estate is now used more efficiently throughout the dashboard, allowing more relevant information to be displayed without scrolling.
• Fixed Selection Issues: Problems with pivot selection and display functionality have been resolved, ensuring that selected items remain properly highlighted and functional.

Security and Infrastructure Updates

Component Security Updates

Several third-party components have been updated to address known security vulnerabilities. Note: These components are primarily relevant to on-premises deployments. SaaS users benefit from these updates automatically without needing to understand the underlying infrastructure:

• Redis: An in-memory data structure store used for caching and session management
• Kube Prometheus Stack: A collection of Kubernetes monitoring components that track system health and performance
• Elasticsearch exporter: A tool that extracts metrics from Elasticsearch for monitoring purposes

Air-Gapped Environment Improvements

Air-gapped environments are isolated networks without internet connectivity, commonly used in high-security deployments. These improvements primarily benefit on-premises deployments in restricted network environments:

• Monaco Editor Fix: Monaco is a code editor component (the same editor that powers Visual Studio Code) embedded in UAP's interface for editing rules and configurations. The package has been updated to eliminate external CDN (Content Delivery Network) dependencies. Previously, Monaco attempted to download fonts and resources from Microsoft's servers, which would fail in isolated environments. All resources are now bundled locally.
• PGBouncer Repository Change: PGBouncer is a lightweight connection pooler for PostgreSQL databases that helps manage database connections efficiently. UAP no longer pulls PGBouncer from the Apache repository, instead using a more reliable source. This change resolves availability issues where the Apache repository might be unreachable or slow.

Policy Engine Enhancement

A new configuration flag has been added to control caching behavior in the Policy Engine, which evaluates security rules and makes decisions about allowing or blocking traffic. This caching control allows administrators to enable or disable result caching, which can be useful during staged upgrades or when troubleshooting policy behavior. Cached results improve performance but might delay policy updates from taking effect.

Enhanced Defensive Capabilities

Mitigator Dataset Improvements

New APIs and distribution service support have been added for mitigator datasets. Mitigator datasets are collections of known malicious indicators (IP addresses, fingerprints, patterns) used to identify and block threats. The new APIs allow for more efficient distribution and updating of these datasets across the UAP infrastructure.

Even Rotation Rules Enhancement

The system's compound pivots now implement rules for source fingerprint-based Even Rotation rules. Even Rotation is a technique where attackers distribute their requests across multiple source addresses to avoid detection. The new rules improve UAP's ability to identify this behavior by analyzing patterns across multiple source fingerprints simultaneously, enhancing the accuracy and effectiveness of threat detection.

Bot Detection Cleanup

Unsupported encryption functionality has been removed from the BotAnalyzer component. The BotAnalyzer identifies automated bot traffic by analyzing behavioral patterns. The removed encryption features were legacy code from version 7.6 that was no longer used but could cause confusion during configuration. Note: Users upgrading from versions prior to 7.6 who may have configured these encryption features should review their BotAnalyzer settings, though most configurations should be unaffected as this functionality has been deprecated for several releases.

Hierarchical Data Extraction Change

In the 8.1 release, hierarchical data extraction is active by default. Hierarchical data extraction allows UAP to understand and analyze nested data structures in API requests and responses, such as JSON objects with multiple levels. This feature improves threat detection in modern API-based applications by enabling deeper inspection of complex data payloads. You can disable this behavior manually if it interferes with your specific use case or if you need to maintain compatibility with existing rule sets.

Upgrade Path and Rollback

Only Cequence UAP platform instances at release 8.0 can upgrade directly to release 8.1. Other clusters must upgrade to release 8.0 before upgrading to release 8.1. This staged upgrade path is required due to the significant infrastructure changes, particularly the Elasticsearch version jump.

Rollback of the 8.1 release of the Cequence UAP platform can only roll back to the 8.0.3-ES8 version. This specific version requirement exists because of the Elasticsearch upgrade - earlier versions use incompatible data formats. Details on upgrade and rollback procedures are in a separate article.

8.1.6 new features and improvements

Enhanced User Preferences - Implemented backend support for storing App tags and Rules preferences, including both System and custom rules.

Persistent Sorting Preferences - Detection and Mitigation sort order preferences are now saved in user preferences.

Audit Log Filtering - Added username filter capability to audit logs in the new UI.

Resolved Issues

Release 8.1.7

CEQASP-10481 Update Auto Refresh interval to 1 minute in the UI - Detection and Mitigation Dashboard

Release 8.1.6

Custom Metrics Management - Resolved issue where Custom Metrics index was being incorrectly imported or exported.

Resource Discovery Stability - Fixed ConcurrentModificationException that occurred in Resource Discovery 8.x and could cause processing failures.

Mitigation Events Filtering - Corrected broken filters for Mitigation Action in the Mitigation Events stream.

Sensitive Data Exposure Resolution - Fixed white screen issue that appeared when users clicked Filters Sort icon and Clear link.

Detection Pivot Details Display - Resolved white screen problem on fresh installations when users accessed detection pivot details.

Security Enhancement - Removed unnecessary salt from hashing function to improve security implementation.

Release 8.1.5

CEQASP-9678: Extended datasets delete endpoint now performs exact filename matching instead of broad pattern matching.

CEQASP-9290: PGBouncer container images are pulled from internal registry instead of Apache repository.

CEQASP-10153 Airflow Elasticsearch user secrets are created only when required.

Release 8.1.4

CEQASP-9937: Exposes Prometheus endpoint to fetch traffic metrics.

Release 8.1.3

CEQASP-9987: Data export event criteria now accepts user input in value textboxes.

CEQASP-9832: Mitigation page sorting now functions correctly in ascending order.

CEQASP-9768: Filter application on ISP, Organization, and Country fields in Threat Protection Detection now behaves correctly.

CEQASP-9739: Audit logs now consistently display user login information.

CEQASP-9681: Long values now display correctly when hovering over pie charts.

Release 8.1.2

CEQASP-9606: The baselining feature now uses an updated payload structure for machine learning model inference API calls, improving data processing efficiency.

CEQASP-9776: The Mitigation Dashboard list view now renders correctly instead of displaying a blank white screen when clicked.

CEQASP-9798: The Sensitive Data Expressions include and exclude fields now accept camel case and uppercase characters in the user interface.

Release 8.1.1

CEQASP-9552: Application tags now handle trailing slashes consistently with traffic filters, ensuring uniform matching behavior across both features.

CEQASP-9527: Sensitive query parameters in HTTP URLs now mask correctly, protecting confidential data in displayed results.

CEQASP-9524: The Show Usernames popup now displays the number of IP addresses associated with each username.

CEQASP-9517: The filter panel now maintains stable behavior when deleting the first filter in a list, eliminating unexpected interface responses.

CEQASP-9502: Mitigation filters now include a dropdown selector for policy selection, improving configuration workflow.

CEQASP-9493: Transaction filters now function identically across all pages, providing consistent filtering capabilities throughout the interface.

CEQASP-9487: The transactions tab interface now renders correctly without display anomalies.

CEQASP-9304: The Traffic Source column now appears in the Show/Hide Column List for the Transactions Detection table, allowing proper column visibility control.

CEQASP-8915: Traffic Metrics queries against polygraph indices in version 8.0.0 now execute successfully instead of failing across all shards.

CEQASP-7100: Red Eye Rule R58 now operates correctly during peak evening hours (4-10pm PST), eliminating false positive triggers during high-traffic periods.

Release 8.1.0 - Bug Fixes

User Interface Fixes:

CEQASP-7148: The filter by country feature in both transaction tabs now displays help text indicating that only two-letter country codes (ISO 3166-1 alpha-2 format) are permitted.

CEQASP-7159: Fixed a vulnerability where a newline character would bypass the IP overlap check for IP Dataset in the user interface, potentially allowing conflicting IP ranges to be configured.

CEQASP-8431: In Threat Protection Rules Custom Rules, the Save button now correctly remains disabled when a user clicks on "Change rule state" without selecting any rule, preventing accidental empty submissions.

CEQASP-8592: The Detection Dashboard Actions feature for 'Allow IPs' now displays a specific error message when attempting to add duplicate entries, replacing the previous generic error that didn't explain the issue.

CEQASP-8593: The Detection Dashboard Actions for 'Allow ISP' and 'Allow Organisation' now properly validate and reject duplicate values when they should not be permitted.

CEQASP-8833: The Sensitive Data Expressions page now displays a proper heading in the user interface, improving navigation and clarity.

CEQASP-8919: The Data Export Event Criteria window no longer shows unwanted white space that previously affected layout consistency.

CEQASP-9106: In UAP 8x, when creating an App Tag, the Host field now correctly validates that it must contain a "." character, ensuring proper domain formatting.

CEQASP-9155: The Show Usernames pop-up now correctly displays the count of usernames, providing proper context for the displayed data.

CEQASP-9197: The "Hide unselected" feature in transaction filters now functions properly, correctly hiding non-selected items from view.

CEQASP-9477: Fixed a UI layout bug where the Cancel button would overlap the + button when trying to add multiple IP addresses or fingerprints in the policy criteria, making the interface difficult to use.

Detection and Dashboard Fixes:

CEQASP-7365: The "Hide unselected" feature now properly hides the specifications as expected, improving the ability to focus on selected items.

CEQASP-7707: The Detection Dashboard now correctly loads values when grouped by User, and no longer returns an HTTP 500 error when attempting to sort by fingerprint.

CEQASP-8758: The bottom right pane of the Detection page now loads content progressively as expected, improving perceived performance and user experience.

CEQASP-8830: In Threat Protection Rules System Rules, the collapsed table background now matches the Figma design specifications, ensuring visual consistency.

CEQASP-8935: The "Hide Unselected" feature in Mitigation Event Criteria in Data Export now works properly, correctly filtering the displayed data.

CEQASP-9045: In the Detection dashboard, "Select all" now correctly selects only pivots in the current page as expected, and the pagination controls properly show "Deselect All" when appropriate.

CEQASP-9060: Filtering with "include exact match" now works correctly for User Agent, properly matching only exact strings rather than partial matches.

CEQASP-9157: In the Detection Dashboard, selecting all pivots after deselecting no longer produces an error, allowing for proper bulk selection management.

CEQASP-9158: Rules in the right side pivot details of the Detection dashboard are now consistently displayed, fixing intermittent display issues.

CEQASP-9202: Duplicate User Agent entries have been removed from the Detection Filter, eliminating confusion and improving filter usability.

CEQASP-9342: The View Transactions page layout has been fixed to properly handle multiple user agent fields, and horizontal scroll functionality has been restored for viewing wide data sets.

System and Backend Fixes:

CEQASP-7363: The BFF (Backend For Frontend) service now correctly loads IP addresses in the Policy Engine Cache, ensuring policy rules are properly applied.

CEQASP-8920: In Deceptive Mitigation Routing, the HTTP Status code can now be changed while creating a policy, providing proper flexibility in response configuration.

CEQASP-9028: The Executive Dashboard 'request volume processed' metric now uses the correct calculation and has been reworded for clarity.

CEQASP-9153: Fixed an issue where newly created issues were immediately and incorrectly updated to "resolved" status from "Open" by the system.

CEQASP-9201: The /sitemap/stats API endpoint no longer fails under the Live Activity Page, restoring proper statistics display.

CEQASP-9240: In UAP 8.0.1, the Mitigation page now correctly sorts in descending order by default instead of ascending order, showing most recent items first.

CEQASP-9253: The Risk Details timestamp has been fixed to properly align with the Kafka window timestamp, ensuring accurate time-based correlation.

CEQASP-9305: Elasticsearch no longer attempts to pull GeoIP data on startup, preventing unnecessary network requests and startup delays, particularly in air-gapped environments.

CEQASP-9359: Traffic Sources are now properly displayed when Double Aggregation is enabled, fixing a visibility issue in traffic analysis.

CEQASP-9376: The Host information is now correctly included in Pivot Details URI, providing complete context for analysis.

CEQASP-9387: The Monaco editor package now loads all dependencies from local sources instead of external CDNs, ensuring functionality in air-gapped environments.

Stories:

Model and Data Handling Improvements:

CEQASP-6059: The data model has been updated to set new "ML-generated" attributes for Data Extractions, Auth Expressions, Custom Rules, and Policies, enabling better tracking of machine learning-generated configurations versus manually created ones.

CEQASP-7042: Validation for total count has been implemented in mitigation and detection dashboards, ensuring data accuracy and preventing display errors.

CEQASP-8597: Real-life authentication expression scenarios have been implemented, improving the system's ability to handle complex authentication patterns found in production environments.

CEQASP-8730: Transactions from Allow/Deny IP/Fingerprint rules are no longer unnecessarily passed to the aggregator stream and subsequent joining processes, improving system performance by reducing unnecessary data processing.

Performance and Infrastructure Enhancements:

CEQASP-7842: The Distribution Service has been improved for better boot-up performance and more efficient concurrent mitigator data fetch operations, reducing system startup time and improving responsiveness under load.

CEQASP-8439: The Policy Engine has been enhanced to better handle concurrent mitigator data fetch operations, improving performance when multiple threat lists are being updated simultaneously.

CEQASP-9074: Index templates have been updated for compatibility with Elasticsearch 8.18, ensuring proper data storage and retrieval in the new version.

CEQASP-9287: Audit logging in the api-gateway has been reimplemented using Kafka instead of Elasticsearch. This change improves performance by using a more appropriate technology for high-volume event streaming and reduces load on Elasticsearch.

CEQASP-9348: A configuration flag has been added to enable or disable caching support in the Policy Engine, providing administrators with fine-grained control over performance versus real-time policy updates.

CEQASP-9382: Client login event audit logging has been disabled for successful logins to reduce log volume while maintaining security visibility for failed authentication attempts.

CEQASP-9407: XPack ML (Machine Learning) and GeoIP Downloads have been disabled inside Elasticsearch, removing unnecessary features that consume resources without providing value to UAP's core functionality.

User Interface Improvements:

CEQASP-8237: Users can now navigate back and forth between individual transaction details without having to leave the details pane, improving workflow efficiency during investigation.

CEQASP-8404: The UI has been updated to support lower-resolution screens with a minimum resolution of 1366 x 768 pixels, ensuring usability on a wider range of devices.

CEQASP-8845: The wording of sampling numbers in the UI has been improved for better clarity and user understanding.

CEQASP-9007: Various improvements have been made to the Transactions page, enhancing overall usability and performance.

CEQASP-9049: Detection Dashboard checkbox states are now properly preserved across page refreshes and navigation, maintaining user selections.

CEQASP-9195: In the Detection dashboard, the URI now displays the actual raw URI instead of the parameterized URI, providing more specific information for analysis.

Security and Compliance Updates:

CEQASP-9090: Security vulnerabilities identified by Sentinel scanning have been fixed for version 8.1, ensuring compliance with security standards.

CEQASP-9252: Unsupported encryption functionality has been removed from BotAnalyzer, eliminating code that has been deprecated since version 7.6 and reducing potential confusion.

New Capabilities:

CEQASP-9203: Compound Pivots and New Event Rotation Rules have been created to enhance threat detection capabilities, particularly for distributed attack patterns.

CEQASP-9259: A new API has been added for the mitigator to fetch datasets, enabling more efficient threat data distribution.

CEQASP-9263: Support for new mitigator datasets has been added via the distribution service, expanding the types of threat indicators that can be managed.

CEQASP-9308: The Event Rotation Rule upgrade scenario is now properly handled, ensuring smooth transitions when upgrading from previous versions.

Upgrade Considerations

The material in this section addresses several different upgrade scenarios and requirements for successfully transitioning to UAP 8.1.

Component Updates

This table lists the necessary updates you must perform before upgrading your Cequence UAP platform instance to the 8.1 release. Note: These components are part of the underlying Kubernetes infrastructure and are primarily relevant for on-premises deployments:

Keycloak Update

When you update the Cequence UAP platform to release 7.7.2 and later, make the following changes in the keycloak/values.yaml file to ensure the correct theme version is deployed:

keycloakTheme:
  image:
    tag: 8.1.0

Compatibility Matrix

The Cequence UAP platform release 8.1 requires the following minimum versions of other Cequence components to ensure proper integration and functionality:

On-Premises Deployments

The following information is specific to on-premises deployments. SaaS customers do not need to manage these components directly:

Helm Charts are package managers for Kubernetes applications, allowing consistent deployment and management of UAP components in on-premises environments.