Check Point - Acceleration

SecureXL: Security Acceleration

SecureXL is a technology interface that accelerates multiple, intensive security operations, including operations carried out by Check Point's Stateful Inspection Firewall.
SecureXL accelerates Firewall and VPN performance by remembering certain attributes of packets and packet flows that have already been validated by the Firewall/VPN application. Then, validation of related packets and connections is delegated to the SecureXL API; this validation is done at the hardware interrupt level.

Packets attempting to establish a new TCP connection (or a comparable UDP) are handled in the slow path. Once the first packet is seen by the Firewall and suitable connections information is off-loaded to an appliance OS, further packets are handled at the OS's interrupt-level code.

SecureXL improved non-encrypted firewall traffic throughput and encrypted VPN traffic throughput. Packet acceleration is also referred to as throughput acceleration as it matches on the familiar 5-tuple of source address, destination address, source port, destination port and protocol. However, only packets during the specific TCP/UDP connection can be accelerated.

Session rate acceleration: SecureXL also reduces the overhead in establishing certain kinds of new connections, improving new connection rate(connections per second), connection setup/teardown rate (sessions per second) and throughput in certain high-connection rate traffic environment.

From a 5-tuple validation, the source port of a flow may be masked off, effectively providing a global match for source port. These new connection setup packets matching 4 out of 5 tuples avoid a round trip to the firewall application and limit the computing overhead. Security is not impacted because the OS continues to track the state of the new connection using Stateful inspection.

Application Layer Protocol (example HTTP)

Web pages consist of multiple components. Using HTTP 1.0, each component is downloaded from S2C using a separate TCP connection. This action involves substantial overhead in connection setup and tear-down and protective firewall connection tracking.

Once a connection involving a flow to port 80 is approved by the Firewall application for the web client (resulting from the first HTTP request), a template is created and stored. All subsequent connection setups carrying those additional requests can share that same template approval because it's ok that the source port differs. Establishing those subsequent connections does not involve a round trip to the Firewall, resulting in faster processing through the server Firewall.

HTTP 1.0 creates a separate connection for each HTTP component. The newer HTTP 1.1 improves the protocol's performance by permitting not only parallel, but also persistent and pipelined server connections. The server may keep the connection alive after sending the end of a component which avoids the need to create a new connection to send the next component.

FTP and many types of VOIP have handlers which precludes acceleration. There are several factors that preclude a packet from being accelerated (SDF, QoS, connections that have a Handler, multicast packets, etc.)
There are factors that can preclude templating if all other parameters are met for acceleration: time objects, dynamic objects, domain objects, source port ranges, IPS features not supported in Acceleration, NAT, Encrypted connections.

Once templating is disabled in the Rule Base, all connections matching rules lower in the Rule Base cannot be templated. Use fwaccel stat to determine at which rule templating is disabled and move the most used rules above that rule for session acceleration.

CoreXL: Multicore Acceleration 

Check Point - Clustering

Virtual Routing Redundancy Protocol 

VRRP cluster can be used for High Availability or Load Sharing. The check point implementation of VRRP includes additional functionality called Monitored Circuit VRRP which prevents black holes. 

You cannot deploy a standalone deployment (Security Gateway and SMS on the same computer) in a Gaia VRRP cluster.

A VRRP router might participate in more than one VRID. The VRID mappings and priorities are different for each VRID. 

Monitored Circuit VRRP eliminates black holes caused by asymmetric routes that can be created if only one interface on the master fails as opposed to the entire platform. Monitored Circuit VRRP monitors all of the VRRP-configured interfaces on the platform. If an interface fails, the master release its priority over all of the VRRP-configured interfaces. To release the priority, Gaia subtracts the Priority Delta from the priority to calculate the Effective Priority. Make sure to calculate the Priority delta value so that Gaia releases priority over all interfaces on a virtual router to let failover occur when one interface fails.

If the platforms run firewall software, you must configure the firewall policies to accept VRRP packets. The Multicast address assigned for VRRP is 224.0.0.18. If the policy does not accept packets to 224.0.0.18, Firewall platforms in the same VRRP group take on Master state.

With Monitored Circuit VRRP, some ethernet switches might not recognize the VRRP MAC address after a master to backup change. This is because many switches cache the MAC address related to the Ethernet device attached to a port. When the change to a backup router occurs, the MAC address for virtual router shifts to a different port and switches that cache the MAC address might not change to the correct port during VRRP change. To prevent this, replace the switch with a hub; disable MAC address caching or set the address ageing value sufficiently low; also enable portfast.

Cluster XL

ClusterXL provides both load sharing and high availability solutions.ClusterXL must be installed in a distributed configuration in which the SMS and the cluster members are on different machines. ClusterXL is part of the standard security gateway installation.

A Critical Device is a device that is critical to the operation of the cluster member. It is also known as a Problem Notification (PNote). It can be hardware or a process. The fwd and cphad processes as well as the Security Policy itself are predefined as critical devices. Use cphaprob command to add.

Cluster Control Process (CCP) is used specifically for clustered environments to allow gateways to report their own states and learn about the states of other members in the cluster. It is essential means by witch State Synchronization works to provide failover in the event an active member goes down.
There is no need to add a rule to the Rule Base that accepts CCP. When clustering is configured on the gateways, an implied rule is created making this provision.

ClusterXL uses unique physical IP and MAC addresses for the cluster members and virtual IP addresses to represent the cluster itself. Virtual IP addresses do not belong to an actual machine interface.

Cluster Synchronization  

In order to make sure each Gateway cluster member is aware of the connections going through the other members, a mechanism called State Synchronization exists witch allows status information about connections on the Security Gateways to be shared between the members. Every IP based service including TCP and UDP, recognized by the security gateway is synchronized. State synchronization is used both by ClusterXL and by third-party OPSEC-certified clustering products. 2 modes
   Full Synchronization - Transfers all Firewall Kernel table information from one cluster member to another. It's handled by the fwd daemon, using an encrypted TCP connection. Full synchronization is used for initial transfers of state information for thousands of connections. If a cluster member is brought up after failing down, it will perform full sync. Once all members are synchronized, only updates are transferred via delta sync.

   Delta Synchronization - Transfers changes in the Kernel tables between cluster members. Delta sync is handled by the Firewall Kernel using UDP Multicast or Broadcast on port 8116.

A user authenticated connection through a cluster member will be lost if the cluster member fails. However, a Client Authenticated or Session Authenticated connection will not be lost.
When failover, accounting information that was accumulated on the failed member but not yet reported to the SMS is lost.

Checkpoint recommends securing the synchronization interfaces by using a dedicated syn network or connecting the physical network interfaces of the cluster members directly.

ClusterXL: Load Sharing

Machines in a ClusterXL load sharing configuration must be synchronized. Machines in a ClusterXL HA configuration do not have to be synchronized but connections wll be lost upon failover if they are not. Multicast and unicast are 2 available modes in a load sharing environment.

Multicast Load Sharing 

Every member of the cluster receives all of the packets sent to the cluster IP address. ClusterXL decision algorithm on all cluster members decides which cluster member should perform enforcement processing on the packet. Only that machine processes the packet and sends the packet to its destination The other machines drop the packet.

Unicast Load Sharing

In this mode, one machine called the Pivot machine receives all traffic from a router with a Unicast configuration and redistributes the packets to the other machines in the cluster. The Pivot machine is chosen automatically by ClusterXL.

The Pivot is the only machine that communicates with the router and the router uses only the Pivot's Unicast MAC address to communicate to the cluster.

Sticky connections - A connection is sticky when all of its packets are handled, in either direction, by a single cluster member. In HA mode, all connections are routed through the same cluster member. In load sharing mode, this is not the case but certain connections can be made sticky by enabling the Sticky Decision Function (SDF).

Perform a Manual Failover of the Firewall Cluster

The best practice method for initiating a manual failover: run below command on an active cluster member that creates a problem notification entry with no refresh time in a problem state.
     cphaprob -d STOP -s problem -t 0 register
Running the command cphaprob list on this machine will show an entry named STOP. To remove the problematic STOP entry from the cluster member,
     cphaprob -d STOP unregister

Also can be done from expert mode
     clusterXL_admin down
     clusterXL_admin up

A manual failover can also be induced from the Gateways status screen in Smart View Monitor via Stop Cluster member.

ClusterXL CCP on the cluster members uses Multicasts by default as it's more efficient. If the connecting switch is incapable of forwarding Multicast, change the CCP mode to Broadcast
     cphaconf set_ccp broadcast
     cphaconf set_ccp multicast   //to change back to multicast

--------------------------------------------------------------------------------------------------------------------

Management High Availability

The SMS consists of several databases with information on different aspects of the system such as objects, users and policy information. In the absence of SMS, essential operations performed by the gateways, such as fetching of the Security Policy and the retrieval of the CRL, cannot take place.

In Management HA, the Active SMS always has one or more backup Standby SMS. These standby SMS must all be of the same operating system and version. In a Management HA deployment, the first installed SMS is specified as the Primary SMS.

The Secondary SMS is created with empty databases that are filled with information received from the Active SMS. Secondary SMS is ready once

• It is represented on the Primary SMS by a network object
• SIC has been initialized between it and the Primary SMS
• Manual synchronization has been completed with the Primary SMS for the first time

All management operations are done by the Active SMS. The transition from Standby to Active must be initiated manually. The Standby SMS are synchronized to the Active SMS so they are kept up-to-date with all changes in the databases and Security Policy Security Gateways can fetch the Security Policy and retrieve a CRL from both SMS.

In order for Management HA to function properly, there must be a backup of Database (such as Objects and Users), Certificate information such as Certificate Authority data and CRL, and the installed Security Policy.

Manual or Automatic synchronization. Synchronization status can be viewed in the Management High Availability Servers window or in SmartView Monitor depending on whether you are connected to the Active or Standby SMS. The possible statuses are : never been synchronized, synchronized, lagging (the peer SMS has not been synchronized since the Active SMS has changes applied to it), advanced (the peer SMS is more up-to-date), collision (the active SMS and its peer have different installed policies and/or databases).

Checkpoint - some commands

To change default shell for admin account to bash and allow WinSCP connections,
   chsh -s /bin/bash admin

To return the setting back to default shell,
   chsh -s /etc/cli.sh admin

--------------------------------------------------

fw stat  -- run on gateway
fw unloadlocal   -- run on gateway to uninstall the firewall policy
fw fetch localhost -- run on gateway

fwm load Standard A-GW-01 --on management server

cpinfo
cpinfo -o <output file name>.txt
fw ctl pstat --monitor health and sync status on GW

fw monitor
fw monitor -e "accept src=10.1.1.201 or dst=10.1.1.201;" -o monitor2.out

Checkpoint - fw ctl pstat

Only run on Security gateway.

monitor the heath of your CP box., especially Syc Status.

Sync:
Version: new
Status: Able to Send/Receive sync packets
Sync packets sent:
total : 466729198, retransmitted : 241305, retrans reqs : 6089, acks : 809
Sync packets received:
total : 77283541, were queued : 6715, dropped by net : 6079
retrans reqs : 37462, received 175 acks
retrans reqs for illegal seq : 0
dropped updates as a result of sync overload: 0
Delta Sync memory usage: currently using XX KB mem
Callback statistics: handled 138 cb, average delay : 2, max delay : 34
Number of Pending packets currently held: 1
Packets released due to timeout: 18


Explanation:

Version: new
This line must appear if synchronization is configured (versions above 4.1)

Status: Able to Send/Receive sync packets
If sync is unable to either send or receive packets, there is a problem

Sync packets sent:
total : 466729198, retransmitted : 241305, retrans reqs : 6089, acks : 809
TOTAL number of sync packets is non-zero and increasing
RETRANS REQS may increase under load

Sync packets received:
total : 77283541, were queued : 6715, dropped by net : 6079
QUEUED value never decreases - A non-zero value does not indicate a problem
DROPPED BY NET number may indicate network congestion

The “dropped by net” counter is incremented when the cluster member receives a sync packet with a sequence number which is higher than the expected seq num. This means packets with lower seq where lost somewhere along the way, and we need to find out where.

retrans reqs : 37462, received 175 acks
RETRANS REQS growing very fast may indicate that the load is becoming too high

retrans reqs for illegal seq : 0
May indicate a sync problem

dropped updates as a result of sync overload: 0
In a heavily loaded system, the cluster member may drop synchronization updates sent from another cluster member

Delta Sync memory usage: currently using XX KB mem
This statistic only appears for a non-zero value.
It requires memory only while full sync is occurring at other times, Delta sync requires no memory

Callback statistics: handled 138 cb, average delay : 2, max delay : 34
This statistic only appears for a non-zero value.
AVERAGE DELAY should be 1-5 packets, otherwise indicates an overload of sync traffic

Number of Pending packets currently held: 1
This statistic only appears for a non-zero value.

Packets released due to timeout: 18
This statistic only appears for a non-zero value.
If the it is large (more than 100 pending packets), and the "Number of Pending packets currently held" is small, you should take action to reduce the number of pending packets.
To tackle this problem, try google "Reducing the Number of Pending Packets".

Checkpoint - tcpdump and fw monitor

What’s the difference between tcpdump and fw monitor ?

Tcpdump displays traffic coming or leaving to/from a firewall interface while fw monitor would also tell you how the packet is going through the firewall including routing and NAT decisions.

Let’s get deeper

FW MONITOR

It captures traffic at 4 important points in the firewall namely i, I, o & O. You would see them in the capture in the same sequence.
i – Preinbound, just where the packet is received on the interface. If you see only this then the packet is dropped by address spoofing or the access rule.
I – Postinbound, where the packet has gone across the incoming interface. If you don’t see the next line in capture after this at ‘o’, you could infer that it’s a routing issue.
For both i & I the interface is the incoming, where the packet enters the firewall.
o – Preoutbound, the place where the packet is received at the exit interface within the firewall. If this is the point beyond which the capture is not seen, then you may need to check NAT.
O- Postoutbound, If you see this then make you are sure that the packet has left the firewall and the ACL, route and NAT all are working as expected.

fw monitor -e "accept src=10.1.1.201 or dst=10.1.1.201;" -o monitor2.out

TCPDUMP.

It captures at position i & O of firewall monitor, and you can be sure the traffic has left the firewall. This is similar to the way captures work on a Cisco PIX/ASA.

So, which one you use ?

Consider you run tcpdump and see the incoming traffic but don’t see the traffic leaving the exit interface. You can guess it’s a routing or a NAT issue. But to make a sure shot without wasting time by looking in routes or the NAT rules you could run fw monitor and know what the issue is.