This topic covers the integration of Cequence Sensor with AWS VPC Traffic Mirroring. For testing, this document assumes that you are using one application to receive traffic. Be sure to use an instance type that supports VPC traffic mirroring. VPC Traffic Mirror Support
Highlights
-
Installs sensor in a lightweight standalone container
-
Seamless integration with AWS VPC Traffic Mirroring and other packet providers
-
Key insights within minutes
Prerequisites for this integration
-
Red Hat Linux 8.4
- The user should be familiar with Red Hat Linux
-
Amazon Web Services
- Active AWS account
- Cequence Sensor must be installed in AWS
- Active AWS account
-
Active HTTP traffic sources
- SSL Decryption is not supported. For SSL decryption, we recommend using an Application gateway or 3rd party NVA/Packet Broker.
- Works with the current UAP Platform and older versions.
If you do not already have the Cequence Sensor installed and verified, please follow the link to complete the installation. Cequence Sensor Installation
Cequence Sensor
Investigate suspicious API behavior, mitigate bot attacks, and gain strategic insights within your AWS environment using VPC Traffic Mirroring with the Cequence Sensor. If full prevention is required, a complete installation of Defender is recommended. For a more detailed overview, please visit the Cequence Sensor - Installation to RHEL 8.4 Installation Guide.
Architecture
VPC Traffic Mirror Setup
Identify the Elastic Network Interfaces
- Identify the ENI of the application you would like to mirror traffic from
- You can do this within AWS by going to EC2>Instances>VM Traffic Source>
- Then select the networking tab and make note of the ENI for the traffic source
- You can do this within AWS by going to EC2>Instances>VM Traffic Source>
- Identify the ENI of the Cequence Sensor VM you will forward traffic to
- You can do this within AWS by going to EC2>Instances>Cequence Sensor>
- Then select the networking tab and make note of the ENI for the traffic source
- You can do this within AWS by going to EC2>Instances>Cequence Sensor>
- Once you have identified the ENI's of the traffic source and sensor, you can use this integration guide or follow the "Getting Started with Traffic Mirroring" document from AWS to complete the necessary setup in AWS
Sensor ENI
Traffic Source ENI
Create a Traffic Target
When you create your "target" you will need to configure your destination as the Cequence Sensor Instance. You will configure your "source" as the source of your traffic. In this case, the source is the Apache server. For more detail please see the Getting Started with Traffic Mirroring configuration guide by AWS.
Create a Traffic Filter
The traffic filter is a policy that you configure to allow or deny traffic. For this case, I will be allowing all traffic. You can configure this based on your traffic flow requirements. For more detail please see the Getting Started with Traffic Mirroring configuration guide by AWS.
Create a Traffic Mirror Session
Create your traffic mirror session using the "Target" and "Filter" and the ENI traffic source. Once you have completed this step you have completed the necessary AWS configurations. For more detail please see the Getting Started with Traffic Mirroring configuration guide by AWS.
Troubleshooting
The next section will walk you through simple steps to verify that your Cequence Sensor setup is fully operable.
- Make sure you are using a well-known CA for Bot Defense and/or Sentinel
- The Cequence Sensor doesn't support mirroring Syslog
Not Seeing Traffic in your Cequence Appliance?
Do a tcpdump on the sensor virtual machine to see if traffic is ingressing and egressing
[ec2-user@ip-10-0-0-104 ~]$ sudo tcpdump -i eth0 -nn port not 22 and port not 53
First, locate the sensor information. Then bash into the sensor container by using the following steps
[ec2-user@ip-10-0-0-104 ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
470c181af869 cequence/x-zeek-sensor "/bin/sh -c /opt/vel…" 6 weeks ago Up 6 weeks sensor
Use docker exec to gain access to the sensor virtual machine
[ec2-user@ip-10-0-0-104 ~]$ docker exec -it sensor bash
Is the Sensor Connected to Kafka? Once shelled into the Sensor navigate into the Sensor-Kafka-Connector Logs folder
[root@sensor] /# cd /opt/velocity-manager/sensor-kafka-connector/logs
Search the directory and you should see connector.log files. If you do not, then the sensor connector isn't running.
[root@sensor] logs# ls
connector-helper.pid connector.log connector.log.1.gz connector.log.2.gz connector.log.3.gz logrotate-state start.log
Try to start the connector manually
[root@sensor] logs# /opt/velocity-manager/bin/sensor-kafka-connector start
If it has started you will see the following information or something similar
[root@sensor] /# ps ax | grep connector
1076890 ? Sl 0:00 /opt/velocity-manager/sensor-kafka-connector/bin/defender-connector sensor /opt/velocity-manager/sensor-kafka-connector/conf/sensor-kafka-connector.json
Tail the connector logs to see if data is being written.
[root@sensor] logs# tail -f connector.log
===========================================================================
2022-01-26T22:19:59.344Z info
==============================connector stats==============================
reader: read: 98, rate: 98
pipeline: sensor-sentinel, received: 98, filtered out: 3
datasink: sentinel-api-edge, type: http, received: 95, written: 95, errors: 0, rate: 95, valid: true
===========================================================================
If you don't see a connector process go to the following directory to check to see if you have a connector configuration
[root@sensor] conf# ls
cqai-filters-advanced-default.json cqai-filters-simple-default.json logrotate.conf sensor-kafka-connector.json
cqai-filters-advanced.json cqai-filters-simple.json sensor-kafka-connector-default.json
Exit the sensor container
[root@sensor] conf# exit
exit
Then in the AWS instance navigate to
[ec2-user@ip-10-0-0-104 ~]$ cd /opt/velocity-manager-host/config/
Open the config.json and check the configuration. Here you can change the Cequence appliance destination
[root@sensor] conf# vi sensor-kafka-connector.json
If you make changes to the sensor-kafka-connector.json file be sure to restart velocity-manager-compose
[ec2-user@ip-10-0-0-104 config]# ../bin/velocity-manager-compose restart
The sensor has started but still not receiving data
Once your Sensor connector is working if you are still not sending traffic to your Cequence Security platforms check the Zeek logs. Run the following commands then do a curl to the device being monitored and this log should populate with more data.
[root@sensor] logs# cd /opt/zeek/logs/current
[root@sensor] current# tail -f http.log
Pfring(Optional)
If you are using pfring, be sure pfring is installed install properly. If so, you should see a zeek-pfring folder, and pfring folder
[root@sensor] current# cd /opt/
[root@sensor] opt# ls
cequence collectd graphite packet-stats pfring redis velocity-manager zeek zeek-pfring
Then make sure you see the zeek config files in the zeek-pfring/bin directory to ensure pfring is functional
[root@sensor] zeek-pfring# cd /opt/zeek-pfring/bin/
[root@sensor] bin# ls
bifcl binpac bro bro-config bro-cut broctl capstats paraglob-test trace-summary zeek zeek-config zeek-cut zeek-wrapper zeekctl
If you are still experiencing challenges please contact Cequence Support or Submit a request on this Zendesk article.
Version History
Date | Version | Notes |
Nov 4, 2023 | 2 | Updated with newer RHEL version. |
Mar 10, 2022 | 1 | Initial version. |