Demystifying TCP Options and TCP/IP Fingerprinting

The TCP/IP protocol stack carries more information than just source, destination, and payload. Hidden within the TCP header, especially the TCP Options field, lies a rich set of parameters that describe how an operating system’s TCP stack behaves. These subtle patterns—like the order and values of TCP options—form a unique TCP/IP fingerprint, enabling us to identify the client’s OS or device type (Linux, Windows, Android, iOS, etc.) even without user-agent strings or active probes.

Understanding the TCP/IP Stack

The TCP/IP stack defines how data is transmitted over the internet, layer by layer:

IP layer: Handles addressing, routing, and fragmentation.
TCP layer: Ensures reliable, ordered delivery of packets using sequence numbers, acknowledgments, and flow control.
Application layer: Uses TCP sockets for protocols like HTTP, TLS, SMTP, etc.

Each OS implements its own version of the TCP/IP stack—with slightly different default configurations. These subtle differences, observable in packet headers, are what make fingerprinting possible.

When we say TCP/IP stack fingerprinting, we are examining the implementation details of the TCP/IP stack in the OS. Different OSes set defaults differently for TCP header fields, option ordering, and behavior (like how it handles TTL, window size, IP ID, etc.).

TCP Options

TCP has a flexible header, which can include extra parameters called options. They are mostly negotiated during connection setup (SYN/SYN-ACK packets). They allow clients and servers to optimize the connection. Common options include:

a) MSS (Maximum Segment Size)

Indicates the largest payload (in bytes) the sender can handle in a single TCP segment.
Helps prevent fragmentation at the IP layer.
Example: MSS=1460 means each TCP packet can carry up to 1460 bytes of application data.

b) SACK Permitted (Selective Acknowledgment)

Allows the receiver to inform the sender which segments were successfully received, even if some were lost.
Without SACK, TCP can only acknowledge the highest contiguous sequence number, leading to unnecessary retransmissions.
Option flag in SYN/SYN-ACK: SACK Permitted = 1 → OS supports selective acknowledgment.

SACK (Selective Acknowledgment) is a TCP feature that allows the receiver to inform the sender exactly which pieces of data (segments) have arrived successfully, even when some in between were lost.

It improves efficiency during packet loss — especially on high-latency or high-bandwidth networks — by avoiding retransmission of data that was already received.

Without SACK (Classic TCP)
TCP uses a cumulative acknowledgment (ACK):

ACK number = “I’ve received all bytes up to this sequence number minus 1.”
If packet #3 is lost but #4 and #5 arrive, the receiver cannot say “I got 4 and 5”.
It must keep re-ACKing “I got up to #2” → causing duplicate ACKs → sender retransmits #3, #4, #5 unnecessarily.

Result → Wasted bandwidth and slower recovery.

With SACK Enabled

The receiver can now send a SACK block inside the ACK, like:

This tells the sender: “I got segments 3000–3500 and 4000–4500, but I’m missing 2000–3000.”
The sender only retransmits the missing part.

Negotiation: “SACK Permitted” Option

Appears only in SYN and SYN-ACK packets during connection setup.
Option Kind = 4, Length = 2 [04][02]
It doesn’t enable SACK by itself — it signals that both sides support it.
Once both ends advertise SACK Permitted, subsequent ACKs can carry SACK blocks.

Scenario	Without SACK	With SACK
1 packet lost among 10	Retransmit all after loss	Retransmit only the missing one
Efficiency	Low	High
Throughput recovery	Slow	Fast
Bandwidth use	Wasteful	Optimal

c) Timestamps (TSval, TSecr)

· TCP Timestamps are part of the TCP TimeStamp Option (RFC 1323), used for:

o Round-trip time measurement (RTT)

o PAWS (Protect Against Wrapped Sequence numbers) to prevent old packets from being interpreted as new

· Fields:

o TSval = current timestamp of the sender

o TSecr = echoed timestamp from the other side (if ACKing a packet)

TCP Timestamp Option (TSopt):

· TSval → Timestamp Value

o This is the current time value of the sender, usually in milliseconds or some clock units.

o It’s used by the receiver to calculate Round-Trip Time (RTT) for the connection.

· TSecr → Timestamp Echo Reply

o This echoes back the TSval received from the other side in the previous packet.

o Helps the sender measure RTT and is used for PAWS (Protection Against Wrapped Sequence numbers) to detect old/reordered packets.

In short:

· TSval = “my timestamp right now”

· TSecr = “your timestamp that I am echoing back”

It’s part of TCP extensions defined in RFC 1323 for high-speed networks.

What TSecr actually is

· TSecr = Timestamp Echo Reply

· It’s a 4-byte value inside the TCP Timestamp option.

· Its job is to echo back the TSval sent by the other side in a previous TCP packet.

How it helps measure RTT

· Suppose Client sends TSval = 1000 in a SYN or data packet.

· Server receives it and responds with TSecr = 1000 (echoing client’s TSval) and its own TSval = 2000.

· When client receives this response, it knows how long the packet took to travel → Round Trip Time (RTT) can be calculated as: RTT ≈ CurrentTime - TSval_sent

How it helps PAWS (Protection Against Wrapped Sequence numbers)

· Problem: In high-speed TCP connections, the sequence numbers (32-bit) can “wrap around” quickly.

· PAWS uses timestamps to ignore old or duplicated packets:

o If a packet has TSval older than previously seen TSval, it’s considered stale/replayed and discarded.

Simplified analogy

· TSval = “I sent this packet at time X”

· TSecr = “I received your packet that had time X”

· By comparing these, TCP knows:

o How fast packets are traveling (RTT)

o Which packets are old or duplicate (PAWS)

Explanation:

· TSval (Timestamp Value) is set by the sender (Client) when sending a packet.

· TSecr (Timestamp Echo Reply) is set by the receiver (Server) to echo back the sender’s TSval.

· When the client receives the response, it compares TSecr vs current time to calculate RTT.

· This also helps PAWS detect old/replayed packets in high-speed connections.

The proper formula in practice is:

· Client sends packet at TSval = 1000

· Server receives and responds with TSecr = 1000 (echoed) and TSval_server = 2000

· When client receives the response, it can calculate RTT ≈ ClientReceiveTime - TSval_sent

Key point:

· The echoed TSecr is used to validate the packet is not stale (PAWS)

· RTT is measured using local clock difference: ClientReceiveTime - TSval_sent

d) NOP (No-Operation)

· A 1-byte placeholder used to align TCP options on a 32-bit boundary.

· TCP requires options to be 4-byte aligned; NOPs act as padding without affecting functionality.

e) Window Scale

· TCP Window defines how many bytes can be in-flight (unacknowledged) at a time.

· The window size field in the TCP header is only 16 bits → max 65,535 bytes.

· Window scaling extends this using a scaling factor (RFC 1323):

o Window Scale = 7 → multiply the window size by 2^7 = 128

o Example: TCP header shows Window=16,384 and scale=7 → actual window = 16,384 × 128 = 2,097,152 bytes (~2 MB)

Window Size vs Window Scale (WS)

TCP Window Size

· Defined in the TCP header: a 16-bit field (0–65535) that tells the sender how many bytes the receiver can accept at the moment.

· Essentially, it’s the receiver’s buffer space advertised to the sender.

· Max value without scaling = 65,535 bytes (~64 KB).

Example:

· A Linux client sends Window Size = 29200 → sender can send up to 29,200 bytes before waiting for ACK.

Window Scale (WS)

· Defined in TCP Options, negotiated only during handshake (SYN/SYN-ACK).

· It’s a multiplicative factor that allows the 16-bit window size to represent a larger buffer.

· Value = N, scale = 2^N

· Effective maximum window = Window Size × 2^WS

Example:

· Window Size = 29200

· WS = 7 → Effective window = 29200 × 2^7 = 29200 × 128 = 3,737,600 bytes (~3.56 MB)

Key Differences

Feature	Window Size	Window Scale (WS)
Field	TCP header (16-bit)	TCP Option (3 bytes)
Purpose	Advertise receive buffer	Extend window beyond 64 KB limit
Negotiated	Each packet (can vary)	Only during handshake
Maximum without scaling	65,535 bytes	N/A
Maximum with scaling	65,535 × 2^WS	Depends on WS value (0–14)
Affects	Flow control	Flow control for high-speed/long-delay networks

Example SYN Packet with WS

Window Size: Base buffer advertised by receiver (16-bit field in TCP header).

WS (Window Scale): Multiplier negotiated in SYN/SYN-ACK to expand max buffer beyond 65,535 bytes.

Effective Receive Window: How much data the sender can transmit before waiting for an ACK.

What “before waiting for ACK” means in TCP flow control

In TCP, the sender must not overwhelm the receiver. The receiver advertises a window size (the Window Size field in the TCP header) which tells the sender: “I can currently accept up to X bytes of data without acknowledgment.”

Example: Linux Client

Window Size = 29200 bytes
Sender starts transmitting data to the receiver.

Behavior:

Sender can send 29,200 bytes in total without receiving any ACKs.
Once those 29,200 bytes are sent, the sender must pause.
Sender waits until it receives an ACK from the receiver acknowledging that some or all of the data has been received.
As ACKs arrive, the window “slides”, allowing the sender to transmit more bytes.

Why This Matters

Prevents buffer overflow on the receiver.
Implements flow control in TCP.
In high-speed networks, the Window Scale (WS) may be used to allow the sender to transmit much more data before waiting for ACKs.

Visual Example:

✅ “Before waiting for ACK” = the maximum number of bytes the sender can transmit without receiving confirmation that the receiver got the data.

Window Size & Sending Data

Window Size = 29,200 bytes means:

· The sender can transmit up to 29,200 bytes in total before it must wait for acknowledgments (ACKs) from the receiver.

· These 29,200 bytes can be sent as multiple TCP segments (chunks), not necessarily as a single large segment.

· TCP segments are typically smaller than the window (often 1460 bytes if MSS = 1460).

Example (MSS = 1460 bytes)

Without Window Scaling

Window Size = 29,200 bytes
Means: the sender can have up to 29,200 bytes “in flight” (sent but not yet ACKed).
If it sends 3 segments of 1,460 bytes each → 3 × 1,460 = 4,380 bytes.
→ Still 29,200 − 4,380 = 24,820 bytes of window left to send before needing ACKs.
Once the receiver ACKs some data, the window slides forward, allowing more data to be sent.

With Window Scaling (WS = 7)

WS = 7 → scaling factor = 2⁷ = 128
Effective Window = 29,200 × 128 = 3,737,600 bytes
That means the sender can have up to 3.7 MB of data in flight before needing ACKs.

With WS = 7, the sender can transmit 3.7 MB worth of TCP segments (in multiple chunks) without waiting for acknowledgments, as long as the total unacknowledged data ≤ 3,737,600 bytes.

f) Option Ordering

The order in which TCP options appear in the SYN packet is unique per OS and kernel.
Example: Windows may order as [MSS, WS, SACK, TS]; Linux as [MSS, SACK, TS, WS].
This ordering is a signature in TCP/IP fingerprinting.

Why TCP options matter

They improve connection efficiency: prevent fragmentation, allow large windows, reduce retransmission overhead.
They reveal OS fingerprinting information because each OS sets defaults differently (MSS, WS, TS, option order).
Security/monitoring tools use these fields to passively identify the client OS without sending probes.

Quick Example SYN Packet (simplified):

· This tells the server:

· Max segment: 1460 bytes

· Can scale window by 128×

· Supports selective acknowledgment

· Includes timestamp for RTT

· And to a fingerprinting tool, this combination often uniquely identifies the client OS.

Let's go in depth

Tuesday, October 14, 2025

Demystifying TCP Options