Switch Stack

All stack members must run the same Cisco IOS software image to ensure compatibility among stack members.
From the stack master, you configure:
  • System-level (global) features that apply to all stack members
  • Interface-level features for each stack member
The stack master contains the saved and running configuration files for the switch stack. The configuration files include the system-level settings for the switch stack and the interface-level settings for each stack member. Each stack member has a current copy of these files for back-up purposes.

Switch Stack Membership
A standalone switch is a switch stack with one stack member that also operates as the stack master.
Adding powered-on switches (merging) causes the stack masters of the merging switch stacks to elect a stack master from among themselves. The reelected stack master retains its role and configuration as do its stack members.
Make sure that you power off the switches that you add to or remove from the switch stack.
After adding or removing stack members, make sure that the switch stack is operating at full bandwidth(40 Gb/s). Press the Mode button on a stack member until the Stack mode LED is on. The last two right port LEDs on all switches in the stack should be green.

A new, out-of-the-box switch (one that has not joined a switch stack or has not been manually assigned a stack member number) ships with a default stack member number of 1. When it joins a switch stack, its default stack member number changes to the lowest available member number in the stack.

A higher priority value for a stack member increases the probability of it being elected stack master and retaining its stack member number. The priority value can be 1 to 15. The default priority value is 1. Use "show switch". We recommend assigning the highest priority value to the switch that you prefer to be the stack master. To change the priority value for a stack member, use the switch stack-member-number priority new priority-value global configuration command. The new priority value takes effect immediately but does not affect the current stack master.

The stack master is elected or reelected based on one of these factors and in the order listed:
1. The switch that is currently the stack master.
2. The switch with the highest stack member priority value.
3. The switch that has the configuration file.
4. The switch with the lowest MAC address.
After a new stack master is elected and the previous stack master becomes available, the previous stack master does not resume its role as stack master.

Switch Stack Configuration Files
The stack master has the saved and running configuration files for the switch stack. All stack members periodically receive synchronized copies of the configuration files from the stack master.

A new, out-of-box switch joining a switch stack uses the system-level settings of that switch stack. If a switch is moved to a different switch stack, that switch loses its saved configuration file and uses the system-level configuration of the new switch stack.
  • If an interface-specific configuration does not exist for that member number, the stack member uses its default interface-specific configuration.
  • If an interface-specific configuration exists for that member number, the stack member uses the interface-specific configuration associated with that member number.
If you replace a failed member with an identical model, the replacement member automatically uses the same interface-specific configuration as the failed switch. must have the same stack member number as the failed switch.

Offline configuration to Provision a Stack Member
The configuration that you create on the switch stack is called the provisioned configuration. The switch that is added to the switch stack and that receives this configuration is called the provisioned switch. You manually create the provisioned configuration through the switch stack-member-number provision type global configuration command.

When you configure the interfaces associated with a provisioned switch (for example, as part of a VLAN), the switch stack accepts the configuration, and the information appears in the running configuration. The interface associated with the provisioned switch does not appear in the display of the specific feature(show vlan). The switch stack retains the provisioned configuration in the running configuration whether or not the provisioned switch is part of the stack.

If you remove a provisioned switch from the switch stack, the configuration associated with the removed stack member remains in the running configuration as provisioned information. (use the no switch stack-member-number provision global configuration command.) You can display the stack protocol version by using the show platform stack manager all privileged EXEC command.


Stack Protocl Version
A switch with the same Cisco IOS software version as the stack master can immediately join the switch stack. If an incompatibility exists, the fully functional stack members generate a system message that describes the cause of the incompatibility on the specific stack members. Switches with different major version numbers are incompatible and cannot exist in the same switch stack. Switches with the same major version number but with a different minor version number are considered partially compatible. When connected to a switch stack, a partially compatible switch enters version-mismatch (VM) mode and cannot join the stack as a fully functioning member.

If the auto-upgrade feature is enabled on the new switch, the stack master automatically upgrades the new switch with the same software image running on a compatible stack member.

Connectivity to Stack Member
To debug a specific stack member, you can access it from the stack master by using the session stack-member-number privileged EXEC command. Only the show and debug commands are available in a CLI session to a specific stack member.
The switch stack is managed through a single IP address. The IP address is a system-level setting and is not specific to the stack master or to any other stack member. Stack members retain their IP addresses when you remove them from a switch stack (so, change the IP addresses of any switches that you remove from the switch stack to avoid IP conflict.)

Setting the Stack Port Speed to 10 Gbps
In a mixed stack of Catalyst 2960-X and 2960-S switches, you must set the stack port speed to 10 Gbps. This task is required in a mixed stack configuration and must be run on a Catalyst 2960-X switch in the switch stack before you add a 2960-S switch to the stack. Otherwise, the switches will not stack.

How Does NAT-T work with IPSec?

Reference URL: https://supportforums.cisco.com/document/64281/how-does-nat-t-work-ipsec

http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_conn_dplane/configuration/xe-3s/asr1000/sec-ipsec-data-plane-xe-3s-asr1000-book/sec-ipsec-nat-transp.html

Background:

ESP  encrypts all critical information, encapsulating the entire inner TCP/UDP datagram within an ESP header. ESP is an IP protocol in the same sense  that TCP and UDP are IP protocols (OSI Network Layer 3), but it does not  have any port  information like TCP/UDP (OSI Transport Layer 4).  This is a difference from  ISAKMP which uses UDP port 500 as its transport layer.

PAT (Port Address Translation) is used to provide many hosts access to the internet through the same publically routable ip address.  PAT works by building a database that binds each local host's ip address to the publically routable ip address using a specific port number.  In this manner, any packet sourced from an inside host will have its IP header modified by the PAT devcie such that the source address and port number are changed from the RFC 1918 address/port to the publically routable ip address and a new unique port.  Referencing this binding database, any return traffic can be untranslated in the same manner.

Q1: Why can't an ESP packet pass through a PAT device?

It is precisely because ESP is a protocol without ports that prevents it from passing through PAT devices.  Because there is no port to change in the ESP packet, the binding database can't assign a unique port to the packet at the time it changes its RFC 1918 address to the publically routable address.  If the packet can't be assigned a unique port then the database binding won't complete and there is no way to tell which inside host sourced this packet.  As a result there is no way for the return traffic to be untranslated successfully.

Q2: How does NAT-T work with ISAKMP/IPsec?

NAT Traversal performs two tasks:

1. Detects if both ends support NAT-T 
2. Detects NAT devices along the transmission path (NAT-Discovery)

Step one occurs in ISAKMP Main Mode messages one and two.  If both devices support NAT-T, then NAT-Discovery is performed in ISKAMP Main Mode messages (packets) three and four.  THe NAT-D payload sent is a hash of the original IP address and port. Devices exchange two NAT-D packets, one with source IP and port, and another with destination IP and port. The receiving device recalculates the hash and compares it with the hash it received; if they don't match a NAT device exists.  

If a NAT device has been determined to exist, NAT-T will change the ISAKMP transport with ISAKMP Main Mode messages five and six, at which point all ISAKMP packets change from UDP port 500 to UDP port 4500.  NAT-T encapsulates the Quick Mode (IPsec Phase 2) exchange inside UDP 4500 as well.  After Quick Mode completes data that gets encrypted on the IPsec Security Association is encapsulated inside UDP port 4500 as well, thus providing a port to be used in the PAT device for translation. 

To visualize how this works and how the IP packet is encapsulated:

1. Clear text packet will be encrypted/encapsulated inside an ESP packet
2. ESP packet will be encapsulated inside a UDP/4500 packet.

NAT-T  encapsulates ESP packets inside UDP and assigns both the Source and Destination ports as 4500.  After this encapsulation there is enough information for the PAT database binding to build successfully.  Now ESP packets can be translated through a PAT device.

When a packet with source and destination port of 4500 is sent through a PAT device (from inside to outside), the PAT device will change the source port from 4500 to a random high port, while keeping the destination port of 4500. When a different NAT-T session passes through the PAT device, it will change the source port from 4500 to a different random high port, and so on. This way each local host has a unique database entry in the PAT devices mapping its RFC1918 ip address/port4500 to the public ip address/high-port.

Q3: What is the difference between NAT-T and IPSec-over-UDP ?

Although both these protocols work similiar, there are two main differences.

• When NAT-T is enabled, it encapsulates the ESP packet with UDP only when it encounters a NAT device. Otherwise, no UDP encapsulation is done. But, IPSec Over UDP, always encapsulates the packet with UDP.

• NAT-T always use the standard port, UDP-4500. It is not configurable. IPSec over UDP normally uses UDP-10000 but this could be any other port based on the configuration on the VPN server.

Check Point - Acceleration

SecureXL: Security Acceleration

SecureXL is a technology interface that accelerates multiple, intensive security operations, including operations carried out by Check Point's Stateful Inspection Firewall.
SecureXL accelerates Firewall and VPN performance by remembering certain attributes of packets and packet flows that have already been validated by the Firewall/VPN application. Then, validation of related packets and connections is delegated to the SecureXL API; this validation is done at the hardware interrupt level.

Packets attempting to establish a new TCP connection (or a comparable UDP) are handled in the slow path. Once the first packet is seen by the Firewall and suitable connections information is off-loaded to an appliance OS, further packets are handled at the OS's interrupt-level code.

SecureXL improved non-encrypted firewall traffic throughput and encrypted VPN traffic throughput. Packet acceleration is also referred to as throughput acceleration as it matches on the familiar 5-tuple of source address, destination address, source port, destination port and protocol. However, only packets during the specific TCP/UDP connection can be accelerated.

Session rate acceleration: SecureXL also reduces the overhead in establishing certain kinds of new connections, improving new connection rate(connections per second), connection setup/teardown rate (sessions per second) and throughput in certain high-connection rate traffic environment.

From a 5-tuple validation, the source port of a flow may be masked off, effectively providing a global match for source port. These new connection setup packets matching 4 out of 5 tuples avoid a round trip to the firewall application and limit the computing overhead. Security is not impacted because the OS continues to track the state of the new connection using Stateful inspection.

Application Layer Protocol (example HTTP)

Web pages consist of multiple components. Using HTTP 1.0, each component is downloaded from S2C using a separate TCP connection. This action involves substantial overhead in connection setup and tear-down and protective firewall connection tracking.

Once a connection involving a flow to port 80 is approved by the Firewall application for the web client (resulting from the first HTTP request), a template is created and stored. All subsequent connection setups carrying those additional requests can share that same template approval because it's ok that the source port differs. Establishing those subsequent connections does not involve a round trip to the Firewall, resulting in faster processing through the server Firewall.

HTTP 1.0 creates a separate connection for each HTTP component. The newer HTTP 1.1 improves the protocol's performance by permitting not only parallel, but also persistent and pipelined server connections. The server may keep the connection alive after sending the end of a component which avoids the need to create a new connection to send the next component.

FTP and many types of VOIP have handlers which precludes acceleration. There are several factors that preclude a packet from being accelerated (SDF, QoS, connections that have a Handler, multicast packets, etc.)
There are factors that can preclude templating if all other parameters are met for acceleration: time objects, dynamic objects, domain objects, source port ranges, IPS features not supported in Acceleration, NAT, Encrypted connections.

Once templating is disabled in the Rule Base, all connections matching rules lower in the Rule Base cannot be templated. Use fwaccel stat to determine at which rule templating is disabled and move the most used rules above that rule for session acceleration.

CoreXL: Multicore Acceleration 

Check Point - Clustering

Virtual Routing Redundancy Protocol 

VRRP cluster can be used for High Availability or Load Sharing. The check point implementation of VRRP includes additional functionality called Monitored Circuit VRRP which prevents black holes. 

You cannot deploy a standalone deployment (Security Gateway and SMS on the same computer) in a Gaia VRRP cluster.

A VRRP router might participate in more than one VRID. The VRID mappings and priorities are different for each VRID. 

Monitored Circuit VRRP eliminates black holes caused by asymmetric routes that can be created if only one interface on the master fails as opposed to the entire platform. Monitored Circuit VRRP monitors all of the VRRP-configured interfaces on the platform. If an interface fails, the master release its priority over all of the VRRP-configured interfaces. To release the priority, Gaia subtracts the Priority Delta from the priority to calculate the Effective Priority. Make sure to calculate the Priority delta value so that Gaia releases priority over all interfaces on a virtual router to let failover occur when one interface fails.

If the platforms run firewall software, you must configure the firewall policies to accept VRRP packets. The Multicast address assigned for VRRP is 224.0.0.18. If the policy does not accept packets to 224.0.0.18, Firewall platforms in the same VRRP group take on Master state.

With Monitored Circuit VRRP, some ethernet switches might not recognize the VRRP MAC address after a master to backup change. This is because many switches cache the MAC address related to the Ethernet device attached to a port. When the change to a backup router occurs, the MAC address for virtual router shifts to a different port and switches that cache the MAC address might not change to the correct port during VRRP change. To prevent this, replace the switch with a hub; disable MAC address caching or set the address ageing value sufficiently low; also enable portfast.

Cluster XL

ClusterXL provides both load sharing and high availability solutions.ClusterXL must be installed in a distributed configuration in which the SMS and the cluster members are on different machines. ClusterXL is part of the standard security gateway installation.

A Critical Device is a device that is critical to the operation of the cluster member. It is also known as a Problem Notification (PNote). It can be hardware or a process. The fwd and cphad processes as well as the Security Policy itself are predefined as critical devices. Use cphaprob command to add.

Cluster Control Process (CCP) is used specifically for clustered environments to allow gateways to report their own states and learn about the states of other members in the cluster. It is essential means by witch State Synchronization works to provide failover in the event an active member goes down.
There is no need to add a rule to the Rule Base that accepts CCP. When clustering is configured on the gateways, an implied rule is created making this provision.

ClusterXL uses unique physical IP and MAC addresses for the cluster members and virtual IP addresses to represent the cluster itself. Virtual IP addresses do not belong to an actual machine interface.

Cluster Synchronization  

In order to make sure each Gateway cluster member is aware of the connections going through the other members, a mechanism called State Synchronization exists witch allows status information about connections on the Security Gateways to be shared between the members. Every IP based service including TCP and UDP, recognized by the security gateway is synchronized. State synchronization is used both by ClusterXL and by third-party OPSEC-certified clustering products. 2 modes
   Full Synchronization - Transfers all Firewall Kernel table information from one cluster member to another. It's handled by the fwd daemon, using an encrypted TCP connection. Full synchronization is used for initial transfers of state information for thousands of connections. If a cluster member is brought up after failing down, it will perform full sync. Once all members are synchronized, only updates are transferred via delta sync.

   Delta Synchronization - Transfers changes in the Kernel tables between cluster members. Delta sync is handled by the Firewall Kernel using UDP Multicast or Broadcast on port 8116.

A user authenticated connection through a cluster member will be lost if the cluster member fails. However, a Client Authenticated or Session Authenticated connection will not be lost.
When failover, accounting information that was accumulated on the failed member but not yet reported to the SMS is lost.

Checkpoint recommends securing the synchronization interfaces by using a dedicated syn network or connecting the physical network interfaces of the cluster members directly.

ClusterXL: Load Sharing

Machines in a ClusterXL load sharing configuration must be synchronized. Machines in a ClusterXL HA configuration do not have to be synchronized but connections wll be lost upon failover if they are not. Multicast and unicast are 2 available modes in a load sharing environment.

Multicast Load Sharing 

Every member of the cluster receives all of the packets sent to the cluster IP address. ClusterXL decision algorithm on all cluster members decides which cluster member should perform enforcement processing on the packet. Only that machine processes the packet and sends the packet to its destination The other machines drop the packet.

Unicast Load Sharing

In this mode, one machine called the Pivot machine receives all traffic from a router with a Unicast configuration and redistributes the packets to the other machines in the cluster. The Pivot machine is chosen automatically by ClusterXL.

The Pivot is the only machine that communicates with the router and the router uses only the Pivot's Unicast MAC address to communicate to the cluster.

Sticky connections - A connection is sticky when all of its packets are handled, in either direction, by a single cluster member. In HA mode, all connections are routed through the same cluster member. In load sharing mode, this is not the case but certain connections can be made sticky by enabling the Sticky Decision Function (SDF).

Perform a Manual Failover of the Firewall Cluster

The best practice method for initiating a manual failover: run below command on an active cluster member that creates a problem notification entry with no refresh time in a problem state.
     cphaprob -d STOP -s problem -t 0 register
Running the command cphaprob list on this machine will show an entry named STOP. To remove the problematic STOP entry from the cluster member,
     cphaprob -d STOP unregister

Also can be done from expert mode
     clusterXL_admin down
     clusterXL_admin up

A manual failover can also be induced from the Gateways status screen in Smart View Monitor via Stop Cluster member.

ClusterXL CCP on the cluster members uses Multicasts by default as it's more efficient. If the connecting switch is incapable of forwarding Multicast, change the CCP mode to Broadcast
     cphaconf set_ccp broadcast
     cphaconf set_ccp multicast   //to change back to multicast

--------------------------------------------------------------------------------------------------------------------

Management High Availability

The SMS consists of several databases with information on different aspects of the system such as objects, users and policy information. In the absence of SMS, essential operations performed by the gateways, such as fetching of the Security Policy and the retrieval of the CRL, cannot take place.

In Management HA, the Active SMS always has one or more backup Standby SMS. These standby SMS must all be of the same operating system and version. In a Management HA deployment, the first installed SMS is specified as the Primary SMS.

The Secondary SMS is created with empty databases that are filled with information received from the Active SMS. Secondary SMS is ready once

• It is represented on the Primary SMS by a network object
• SIC has been initialized between it and the Primary SMS
• Manual synchronization has been completed with the Primary SMS for the first time

All management operations are done by the Active SMS. The transition from Standby to Active must be initiated manually. The Standby SMS are synchronized to the Active SMS so they are kept up-to-date with all changes in the databases and Security Policy Security Gateways can fetch the Security Policy and retrieve a CRL from both SMS.

In order for Management HA to function properly, there must be a backup of Database (such as Objects and Users), Certificate information such as Certificate Authority data and CRL, and the installed Security Policy.

Manual or Automatic synchronization. Synchronization status can be viewed in the Management High Availability Servers window or in SmartView Monitor depending on whether you are connected to the Active or Standby SMS. The possible statuses are : never been synchronized, synchronized, lagging (the peer SMS has not been synchronized since the Active SMS has changes applied to it), advanced (the peer SMS is more up-to-date), collision (the active SMS and its peer have different installed policies and/or databases).

Wireshark notes - 4 - Tips

Try to keep my Wireshark trace files to 100 MB size maximum.

Define a useful naming scheme for your trace files as soon as possible. Consider including capture location, capture purpose and any notes about the trace file in your trace file names.
sw1-msmith-slowsalesforce.pcapng
sw1-msmith-backgroundidle.pcapng
local-gspicer-slowbrowse.pcapng
local-gspicer-uploadstuck.pcapng
fs2-disconnects.pcapng
rtr2side1-slowpath.pcapng
rtr2side2-slowpath.pcapng

Tips for Analyzing TCP-Based Applications

-Look at the TCP handshake to get a snapshot of round trip time.

   If capturing at the client, measure the time between the SYN and the SYN/ACK.

   If capturing at the server, measure the time between the SYN/ACK and ACK.

-Open SYN and SYN/ACK packets and examine TCP peer capabilities (TCP Options).

   Decent MSS size?

   SACK supported by both?

   Window Scaling supported by both?

   Decent scaling factor?

-Launch the IO Graph and look for drops in throughput.

   Add the Bad TCP coloring rule filter to the IO Graph to correlate drops in throughput with TCP issues (the Golden Graph).

-Open the Expert Infos to view detected problems.

   Focus on Errors, Warnings and Notes.

   Expand sections and click on packets to jump to that location in the trace file and explore further.

-View and sort the TCP Delta column (tcp.time_delta).

   Sort the column from high to low and examine delays.

   Do not get distracted by "normal delays" (refer to Do not Focus on "Normal" or Acceptable Delays).

 -View and sort the Calculated window size field to look for issues.

   Do not worry about FIN or RST packets with Window 0 values.

   Look for low window size values and delays in close proximity.

Tips for Locating the Cause of Intermittent Problems

Consider using a Ring Buffer during the capture process. To capture intermittent problems, set up a capture machine close to one of the machines that experiences the problem. Start capturing traffic to a file set and define the number of files to be saved by the Ring Buffer. Do not set an auto stop condition—stop the capture as soon as possible after the problem occurs.

When you stop capturing the last file is displayed. Work backwards through this file and then the

other files in the file set to locate the problem. Select File | File Set | List Files to view and navigate

between files in the file set.

Tips for Detecting WLAN Problems

You need to capture the 802.11 Management, Control and Data frames, the 802.11 header, and have a pseudoheader applied. Management and Control frames are necessary to identify problems with associating and authenticating to a WLAN. Data frames provide us with the actual throughput rates on a WLAN.

Tips for Sanitizing Trace Files

Security rule: Never share trace files that may contain confidential information. Use TraceWrangler that was created specifically to sanitize .pcapng files.

Tips for When you get stuck

Search www.ietf.org, www.wiresharkbook.com/resources.html, and also consider asking for help at ask.wireshark.org. 

Wireshark notes - 3 - Application Errors and Advanced IO Graph

dns.flags.rcode > 0
http.response.code >= 400 or
http.response.code > 399

HTTP response code
1xx: Informational—Request received, continuing process
2xx: Success—The action was successfully received, understood, and accepted
3xx: Redirection—Further action must be taken in order to complete the request
4xx: Client Error—The request contains bad syntax or cannot be fulfilled
5xx: Server Error—The server failed to fulfill an apparently valid request


SMB response code(NT status) of 0 indicates the request was successful.
smb.nt_status > 0 || smb2.nt_status > 0


SIP is a request/response-based application. SIP can run over UDP or TCP. When SIP is configured to run over TCP, we hope to see an ACK to our SIP request in a reasonable amount of time and then a successful response. SIP response codes are
1xx: Provisional — request received, continuing to process the request.
2xx: Success — the action was successfully received, understood, and accepted.
3xx: Redirection — further action needs to be taken in order to complete the request.
4xx: Client Error — the request contains bad syntax or cannot be fulfilled at this server.
5xx: Server Error — the server failed to fulfill an apparently valid request.
6xx: Global Failure — the request cannot be fulfilled at any server.

sip.Status-Code >= 400 or
sip.Status-Code > 399

----------------------------------------------------------------------------------------------------

A picture is worth a thousand packets

Use an IO Graph to compare the throughput of separate conversations

Use an IO Graph to compare application throughput based on port numbers in use Consider using Advanced IO Graphs when you need the Calc functions (such as MIN, AVG, MAX)

When the application runs over TCP and you have the option of using an application name filter (such as http), it is recommend you use a port-based filter (such as tcp.port==80) instead in order to include the TCP overhead (such as TCP handshake packets, ACKs, FINs, and RSTs) in your graph.

The Advanced IO Graph offers Calc functions for summing the contents of a field, counting the occurrences of a field and more.
  -Use Calc: SUM(*) to add the contents of a numerical field, such as tcp.len, which does not exist in a packet, but is Wireshark's field to count just data bytes in packets.
  -Use Calc: COUNT FRAMES(*) to count the occurrence of specific type of frame or Expert Infos item such as tcp.analysis.retransmission.
  -Use Calc: COUNT FIELDS(*) to count the occurrence of a field, such as the IP ID (ip.id) field which occurs twice in some ICMP packets.
  -Use Calc: MIN(*), AVG(*) and MAX(*) to graph the minimum, average and maximum value of a numerical field, such as the tcp.window_size field.
  -Use Calc: LOAD(*) to graph response time fields, such as smb.time.

There is no field in a packet called tcp.len, but Wireshark uses this value to define the number of data bytes in each TCP segment. Tcp.len value does not count header values.

---------------------------------------------------

Detect Consistently low throughput due to low packet sizes

Low packet sizes may be caused by an application that intentionally wants to transfer smaller amounts of data. Low packet sizes can also be an indication of a low Maximum Segment Size (MSS) setting. For MSS setting, check tcp handshake.

Identify Queuing Delays along a Path

Interconnecting devices can inject delays by queuing (holding the packets temporarily before forwarding them) along a path. Consider using a traffic generator to detect queuing along a path. A tool such as iPerf/jPerf can be used to transmit traffic at a steady rate.

Correlate drops in Throughput with TCP Problems (the Golden Graph)

This graph can determine if throughput issues are related to network problems such as lost packets or

zero window sizes. This is a great graph to build whenever anyone complains about slow performance of a TCP-based application.

Graph Time Delays

This is a great way to identify slow responses for an application that does not have a delta time function.

Graph High TCP Delta Time (TCP-Based Application)

Some TCP-based applications (such as HTTP and SMB) have a delta time tracking function in Wireshark. If the application does not have the delta time tracking function built into the dissector, you can still graph high delta times using tcp.time_delta.

Graph Other Network Problems

You can graph window size issues based on the TCP analysis flag (tcp.analysis.zero_window) or the actual Calculated window size file value.

You can graph packet loss and recovery processes using the TCP analysis flags for each part of the process. 

Although TCP time-sequence graph can be very busy, it can depict not only packet loss but it can also depict selective ACKs.

The above graph clearly depicts the points in the trace where Wireshark noticed packet loss. In addition, the graph depicts the packet loss recovery process by graphing Duplicate ACKs and Retransmissions.

Wireshark notes - 2

Expert Info Messages

Previous Segment not Captured

tcp.analysis.lost_segment

Packet loss recovery method #1 - Fast Recovery
If the receiver supports Fast Recovery and notices the jump in sequence number value, it will immediately begin sending Duplicate Acknowledgments requesting sequence number 7,920. Upon receipt of 4 identical ACK (can be more than 4), sender retransmit the packet.

Packet loss recovery method #2 - Sender Retransmission Timeout (RTO)

If the sender notices that a data packet has not been acknowledged within its Retransmission Timeout (RTO) timer value, it will retransmit the packet.

To determine how many packets were lost, add 3 colums - sequence number, next sequence number and acknowledgement number.

Location of the capture

Since we know the sequence number of the packet that is missing, we can use that information to

determine if we see the original and the Retransmission or just the Retransmission. That will tell us if

we are upstream or downstream from packet loss. Filter tcp.seq==9164761.

Duplicate ACKs

tcp.analysis.duplicate_ack Duplicate ACKs are an indication that a host supports Fast Recovery and noticed that a packet arrived with a sequence number beyond the calculated next sequence number. Duplicate ACKs are usually a sign of packet loss, but Duplicate ACKs can also be an indication of out-of-order packets.

If the packet with the missing sequence number arrives within 3 ms, Wireshark marks that packet as

Out-of-Order (tcp.analysis.out_of_order). If the packet with the missing sequence number arrives later than 3 ms. later, Wireshark will indicate that the packet is a either a Retransmission or a Fast Retransmission.

These Duplicate ACKs complain about a missing sequence number. If SACK is in use we should see

only the missing packets being retransmitted. The SACK Left Edge and SACK Right Edge fields in

the TCP Options area acknowledge other data packets received while the Acknowledgment Number

field still indicates the desired missing sequence number.

If SACK is not in use we may see many unnecessary retransmissions as the sender retransmits every

data packet starting at the missing sequence number.

Out-of-Order Packets

tcp.analysis.out_of_order  Out-of-order packets may not affect performance if there is very little time(1-3 ms) between their expected arrival and their actual arrival. If out-of-order packets arrive after quite a delay, or there are many out-of-order packets, there may be a noticeable degradation in performance. TCP cannot pass received data up to the application until all the bytes are in the correct order.

Determining if a packet is Out of order, Retransmission or Fast Retransmission

Fast Retransmission

tcp.analysis.fast_retransmission  Fast Retransmissions are triggered by receipt of three identical ACKs (the original ACK and two Duplicate ACKs).

Retransmission 

tcp.analysis.retransmission  Standard Retransmissions are not triggered by Duplicate ACKs. Standard Retransmissions are triggered by a Retransmission Time Out (RTO) at the sender. The RTO timer is used to ensure data delivery continues even if the TCP peer stops communicating (with ACKs). When the RTO timer expires without receiving an ACK for the data packet, the sender retransmits the unacknowledged data packet.

You do not want to spend time troubleshooting Retransmissions or Fast Retransmissions when these
packets are actually Out-of-Order packets that did not arrive within 3 ms of the higher Sequence
Number field value.

Remember, Duplicate ACKs lead to Fast Retransmissions. An expired RTO at the sender leads to
Retransmissions. Each of these is an indication of packet loss which typically occurs at interconnecting devices. Capturing at different points on the network can help you find the point of
packet loss.
Applications cannot pick up data from the buffer until all sequential bytes have been received. Outof-
Order problems typically aren't felt by network users unless there is a large gap in time between
the expected arrival time and actual arrival time.

ACKed Unseen Segment

tcp.analysis.ack_lost_segment  This Expert Infos warning indicates that Wireshark sees an ACK, but it did not see the data packet that is being acknowledged.

Zero Window

tcp.analysis.zero_window  Each side of a TCP conversation advertises its receive buffer space in the Window Size Value field (tcp.window_size_value). When a receiving application cannot pull data out of the receive buffer fast enough, this advertised Window Size value can drop to zero

The Window Size Value field indicates the actual Window Size being advertised. When Window

Scaling is in use, Wireshark multiplies the Scaling Factor by the advertised Window Size Value field

to provide the scaled Window Size (Calculated window size field). The TCP FIN or RST packets would not be colored by the Bad TCP.

Window Full

tcp.analysis.window_full  Window Full is an indication that the target will be out of receive buffer space when the data packet arrives. Use 'Bytes in Flight" to watch a stuck application.

Zero Window Probe and Zero Window Probe ACK

tcp.analysis.zero_window_probe, tcp.analysis.zero_window_probe_ack  Zero Window Probe packets may be sent by a host to a TCP peer that is advertising a Zero Window condition in the hope of eliciting a Window Update response. Keep in mind that a host may send a TCP Keep Alive (decrementing the sequence number by 1) instead of a Zero Window Probe packet.