CVE-2019-11478: Sack Panic

CVE-2019-11477, nicknamed SACK Panic, is a remote denial of service in the Linux TCP stack where the whole disaster comes down to a __u16 counter that can only count to 65,535. Convince the kernel it’s holding more than that, and it panics on purpose. In this post I want to trace exactly how a remote attacker, just someone downloading data from you the victim can push that counter over the edge.

Think of data as parcels. A big crate holds many parcels. On the side of the crate is a paper label saying how many parcels are inside. This is the story of the whole bug.

Intuition behind CVE 2019-11477

In CVE 2019-11477, victim is the sender, the one who sends data. It’s where the kernel panic occurs and the machine crashes, so all the buggy code runs here. Attacker is the machine that receives data. It asks for tiny parcels, and sends back tricky replies.

We all know that when the victim sends data over TCP, it can’t send it all in one lump. TCP cuts the stream into small pieces called segments which in our story are our parcels. We also know that an MSS (Maximum Segment Size) is the biggest a single segment (our parcel) is allowed to be. The most important point here is that in TCP, the receiver announces the MSS, and the sender must obey it. Neither side accepts anything bigger than it. If the attacker (receiver) says that only send me 48-byte parcels, then the victim (sender) now has to cut all its outgoing data into tiny 48-byte segments. After some TCP header space, the real data in each is as small as 8 bytes. This way the attacker just forced the victim to make tiny parcels/segments and remember it’s going to be 8 bytes per segment.

The network can drop parcels. So after the victim sends a parcel, it keeps a copy in case it must resend. These copies wait in a list called the retransmit queue. The attacker’s tricks will target this queue.

What an skb is (the crate)

The Linux kernel stores each parcel/segment of network data in a structure called an sk_buff (skb). An skb is our crate. it holds the data, plus labels and bookkeeping.

struct sk_buff {
	union {
		struct {
			/* These two members must be first to match sk_buff_head. */
			struct sk_buff		*next;
			struct sk_buff		*prev;

			union {
				struct net_device	*dev;
				/* Some protocols might use this space to store information,
				 * while device pointer would be NULL.
				 * UDP receive path is one user.
				 */
				unsigned long		dev_scratch;
			};
		};
		struct rb_node		rbnode; /* used in netem, ip4 defrag, and tcp stack */
		struct list_head	list;
		struct llist_node	ll_node;
	};

	union {
		struct sock		*sk;
		int			ip_defrag_offset;
	};

	union {
		ktime_t		tstamp;
		u64		skb_mstamp_ns; /* earliest departure time */
	};
	/*
	 * This is the control buffer. It is free to use for every
	 * layer. Please put your private variables there. If you
	 * want to keep them across layers you have to do a skb_clone()
	 * first. This is owned by whoever has the skb queued ATM.
	 */
	char			cb[48] __aligned(8);

	union {
		struct {
			unsigned long	_skb_refdst;
			void		(*destructor)(struct sk_buff *skb);
		};
		struct list_head	tcp_tsorted_anchor;
#ifdef CONFIG_NET_SOCK_MSG
		unsigned long		_sk_redir;
#endif
	};

#if defined(CONFIG_NF_CONNTRACK) || defined(CONFIG_NF_CONNTRACK_MODULE)
	unsigned long		 _nfct;
#endif
	unsigned int		len,
				data_len;
	__u16			mac_len,
				hdr_len;

	/* Following fields are _not_ copied in __copy_skb_header()
	 * Note that queue_mapping is here mostly to fill a hole.
	 */
	__u16			queue_mapping;

/* if you move cloned around you also must adapt those constants */
#ifdef __BIG_ENDIAN_BITFIELD
#define CLONED_MASK	(1 << 7)
#else
#define CLONED_MASK	1
#endif
#define CLONED_OFFSET		offsetof(struct sk_buff, __cloned_offset)

	/* private: */
	__u8			__cloned_offset[0];
	/* public: */
	__u8			cloned:1,
				nohdr:1,
				fclone:2,
				peeked:1,
				head_frag:1,
				pfmemalloc:1,
				pp_recycle:1; /* page_pool recycle indicator */
#ifdef CONFIG_SKB_EXTENSIONS
	__u8			active_extensions;
#endif

	/* Fields enclosed in headers group are copied
	 * using a single memcpy() in __copy_skb_header()
	 */
	struct_group(headers,

	/* private: */
	__u8			__pkt_type_offset[0];
	/* public: */
	__u8			pkt_type:3; /* see PKT_TYPE_MAX */
	__u8			ignore_df:1;
	__u8			nf_trace:1;
	__u8			ip_summed:2;
	__u8			ooo_okay:1;

	__u8			l4_hash:1;
	__u8			sw_hash:1;
	__u8			wifi_acked_valid:1;
	__u8			wifi_acked:1;
	__u8			no_fcs:1;
	/* Indicates the inner headers are valid in the skbuff. */
	__u8			encapsulation:1;
	__u8			encap_hdr_csum:1;
	__u8			csum_valid:1;

	/* private: */
	__u8			__pkt_vlan_present_offset[0];
	/* public: */
	__u8			vlan_present:1;	/* See PKT_VLAN_PRESENT_BIT */
	__u8			csum_complete_sw:1;
	__u8			csum_level:2;
	__u8			dst_pending_confirm:1;
	__u8			mono_delivery_time:1;	/* See SKB_MONO_DELIVERY_TIME_MASK */
#ifdef CONFIG_NET_CLS_ACT
	__u8			tc_skip_classify:1;
	__u8			tc_at_ingress:1;	/* See TC_AT_INGRESS_MASK */
#endif
#ifdef CONFIG_IPV6_NDISC_NODETYPE
	__u8			ndisc_nodetype:2;
#endif

	__u8			ipvs_property:1;
	__u8			inner_protocol_type:1;
	__u8			remcsum_offload:1;
#ifdef CONFIG_NET_SWITCHDEV
	__u8			offload_fwd_mark:1;
	__u8			offload_l3_fwd_mark:1;
#endif
	__u8			redirected:1;
#ifdef CONFIG_NET_REDIRECT
	__u8			from_ingress:1;
#endif
#ifdef CONFIG_NETFILTER_SKIP_EGRESS
	__u8			nf_skip_egress:1;
#endif
#ifdef CONFIG_TLS_DEVICE
	__u8			decrypted:1;
#endif
	__u8			slow_gro:1;
	__u8			csum_not_inet:1;
	__u8			scm_io_uring:1;

#ifdef CONFIG_NET_SCHED
	__u16			tc_index;	/* traffic control index */
#endif

	union {
		__wsum		csum;
		struct {
			__u16	csum_start;
			__u16	csum_offset;
		};
	};
	__u32			priority;
	int			skb_iif;
	__u32			hash;
	__be16			vlan_proto;
	__u16			vlan_tci;
#if defined(CONFIG_NET_RX_BUSY_POLL) || defined(CONFIG_XPS)
	union {
		unsigned int	napi_id;
		unsigned int	sender_cpu;
	};
#endif
	u16			alloc_cpu;
#ifdef CONFIG_NETWORK_SECMARK
	__u32		secmark;
#endif

	union {
		__u32		mark;
		__u32		reserved_tailroom;
	};

	union {
		__be16		inner_protocol;
		__u8		inner_ipproto;
	};

	__u16			inner_transport_header;
	__u16			inner_network_header;
	__u16			inner_mac_header;

	__be16			protocol;
	__u16			transport_header;
	__u16			network_header;
	__u16			mac_header;

#ifdef CONFIG_KCOV
	u64			kcov_handle;
#endif

	); /* end headers group */

	/* These elements must be at the end, see alloc_skb() for details.  */
	sk_buff_data_t		tail;
	sk_buff_data_t		end;
	unsigned char		*head,
				*data;
	unsigned int		truesize;
	refcount_t		users;

#ifdef CONFIG_SKB_EXTENSIONS
	/* only useable after checking ->active_extensions != 0 */
	struct skb_ext		*extensions;
#endif
};

One skb can hold a lot of data. It can point to 17 chunks of memory, each up to 32 KB. So one crate can hold about:

So one crate maxes out around 544 KB. The 17 fragments are not in struct sk_buff. They live in a separate struct called skb_shared_info, which sits in memory right after the packet data buffer. In the sk_buff look at these two fields near the end:

sk_buff_data_t  tail;
sk_buff_data_t  end;          // marks the end of the data buffer
unsigned char   *head, *data;

The skb_shared_info struct is stored at skb->endwhich is just past the linear data area. The kernel reaches it with the macro skb_shinfo(skb):

#define skb_shinfo(SKB)  ((struct skb_shared_info *)(skb_end_pointer(SKB)))

So sk_buff holds pointers to the buffer, the fragment array lives at the tail of that buffer, not as a named field in the struct. The 17 fragment array is in struct skb_shared_info (defined in the same include/linux/skbuff.h):

struct skb_shared_info {
    __u8        flags;
    __u8        meta_len;
    __u8        nr_frags;                 // how many fragments are in use
    __u8        tx_flags;
    unsigned short  gso_size;
    unsigned short  gso_segs;
    ...
    skb_frag_t  frags[MAX_SKB_FRAGS];     //17-slot fragment array
};

• frags[MAX_SKB_FRAGS] is the array. Each skb_frag_t points to a page (a chunk of memory) plus an offset and length. That’s one fragment holding up to ~32 KB.

typedef struct skb_frag_struct skb_frag_t;

struct skb_frag_struct {
	struct {
		struct page *p;
	} page;
#if (BITS_PER_LONG > 32) || (PAGE_SIZE >= 65536)
	__u32 page_offset;
	__u32 size;
#else
	__u16 page_offset;
	__u16 size;
#endif
};

page.p is the pointer to the physical memory page holding the data.

The relationship

Getting back to our topic, TCP has to cut our data into segments no bigger than the MSS since that’s the biggest size our segments or in easy words our parcel can be. Say MSS is 1460 bytes and we want to send 65,536 bytes. That’s about 45 segments.

The naive way is that the kernel builds 45 separate little packets, and pushes each one down through the whole network stack (routing, checksums, queueing, driver) 45 times. That per-packet overhead is expensive when you do it millions of times a second.

What Generic Segmentation Offload does instead

Generic Segmentation Offload lets the kernel build one big skb (our crate) holding all 65,536 bytes, and carry it through most of the stack as a single unit. The cutting into 45 real MSS sized segments is delayed until the very last moment, often done by the network card hardware. One trip through the stack instead of 45. This makes it much faster. So now a single skb no longer means one packet on the wire. It means one buffer that will become many packets. In our example, one skb stands for 45 segments.

Since one skb now represents multiple segments, the kernel has to remember how many. It needs that number for congestion control, for counting what’s in flight, for retransmission accounting, and so on. It can’t just look at the skb and assume “1 packet” anymore.

That number is stored in a field called tcp_gso_segs (inside the skb’s TCP control block). “gso_segs” literally reads as “GSO segments” = how many segments this GSO skb represents. It does not store bytes, and it does not store the parcels themselves. The actual data lives in the crate’s (sk_buff) memory fragments. tcp_gso_segs is only the label on the side saying “how many segments this data will become when it’s split for the wire.”

tcp_skb_pcount(skb)    returns TCP_SKB_CB(skb)->tcp_gso_segs// "how many parcels"

The companion field tcp_gso_size stores the size of each parcel (the 8-byte payload figure). So the pair is:

• tcp_gso_size = how big each parcel is (bytes per segment)
• tcp_gso_segs = how many parcels (the count that overflows)

And roughly tcp_gso_segs = payload_bytes ÷ tcp_gso_size.

The limit on tcp_gso_segs

tcp_gso_segs is a u16 which is a 16-bit number. It can only hold 0 to 65,535. Go one higher and an overflow occurs means it wraps back to 0.

How tcp_gso_segs label gets its number

When the kernel fills a crate (sk_buff), it works out the parcel/segment count by dividing the data size (the maximum an sk_buff can hold) by the MSS:

/* net/ipv4/tcp_output.c — tcp_set_skb_tso_segs() */
tcp_skb_pcount_set(skb, DIV_ROUND_UP(skb->len, mss_now));  /* count = bytes / mss */

With MSS forcing 8-byte parcels/segments a crate (sk_buff) holding N bytes is about N ÷ 8 parcels/segments. In simple words this means that an sk_buff can hold a maximum of N/8 segments. So normally the victim builds each crate with under 64 KB of data. At 8 bytes/parcel:

So a single fresh crate never overflows the label (tcp_gso_segs). The attacker needs a way to cram many crates into one to overflow tcp_gso_segs, let’s see how it does that using SACK.

SACK

SACK, or Selective ACKnowledgment is the tool that merges crates. It is a TCP feature where the receiver tells the sender exactly which pieces it got. The attacker (receiver) sends these SACK messages and the victim (sender) reads them and reshuffles its retransmit queue.

Here’s how the the SACK option looks like in bytes

• ptr[0] = Kind = 5 (says “this is a SACK option”).
• ptr[1] = Length (total bytes of the option).
• then the blocks, each an 8-byte (start, end) pair.

Let’s take a deeper look at what SACK really is…

What a SACK message actually is…

A SACK is not a separate packet. It’s an option field carried inside a normal TCP ACK packet’s header. When you (receiver) send an ACK, you can attach a “SACK option” that lists ranges of bytes you’ve received. You can enter multiple ranges and each range is called a SACK block, and it’s just two 32-bit numbers:

• left edge = first byte of the range you got
• right edge = first byte you didn’t get (one past the end)

For example if we say that “I have 0–1000 and 2001–3000” is sent as:

• the normal ACK field says 1001 (meaning “I have everything up to 1000 in order”).
• plus one SACK block: left = 2001, right = 3001 (the out-of-order piece means nothing after 1000 and 2000 was received, that is a hole)

We aren’t going to make a range for 0 to 1000 since those are the first 1000 bytes of our data which the receiver acknowledged on receiving. An acknowledgment is always for the data which has been continuously received till that one point. In our case all the bytes from 0 to 1000 reached successfully without break so 1001 is going to be our acknowledgement.

A TCP header has only 40 bytes for options, so you can fit at most about 3 to 4 SACK blocks in one ACK. That’s the whole SACK message, an ACK number plus a short list of (left, right) pairs.

What the sender does with those numbers

We know that the sender keeps its sent data as a row of skbs (crates):

Each crate has a status flag. Lets see where that is. We have already seen the struct sk_buff. Recall that in that struct we have:

/*
	 * This is the control buffer. It is free to use for every
	 * layer. Please put your private variables there. If you
	 * want to keep them across layers you have to do a skb_clone()
	 * first. This is owned by whoever has the skb queued ATM.
	 */
	char			cb[48] __aligned(8);

That cb[48] is a 48 byte scratch area the struct deliberately leaves blank. The comment above it says each layer is free to use it for its own private data. TCP uses it to store its own struct laid on top of those 48 bytes. That struct is struct tcp_skb_cb, defined in include/net/tcp.h, and you reach it with the macro:

#define TCP_SKB_CB(__skb) ((struct tcp_skb_cb *)&((__skb)->cb[0]))

So TCP_SKB_CB(skb) just means that interpret this skb’s cb[48] bytes as a tcp_skb_cb. Inside struct tcp_skb_cb there’s a field called sacked which is the status flag:

struct tcp_skb_cb {
    __u32   seq;          /* starting byte number of this crate */
    __u32   end_seq;      /* one past the last byte */
    ...
    __u8    sacked;       /* State flags for SACK. */
    ...
    __u16   tcp_gso_segs; /*  the parcel-count label that overflows */
    __u16   tcp_gso_size;
    ...
};

sacked is a bitfield. It’s the bit that marks “receiver confirmed this via SACK” is TCPCB_SACKED_ACKED.

enum tcp_skb_cb_sacked_flags 
{
.....
TCPCB_SACKED_ACKED = (1 << 0);
.....	
};

That’s what the merge guard checks in /net/ipv4/tcp_input.c:

if ((TCP_SKB_CB(prev)->sacked & TCPCB_TAGBITS) != TCPCB_SACKED_ACKED)
    goto fallback;

Note that both the things we’ve been calling the crate’s “status flag” (sacked) and the crate’s “parcel count label” (tcp_gso_segs) do not appear by name in the sk_buff! They live inside the generic cb[48] blob at line 34, viewed through struct tcp_skb_cb.

When a SACK block arrives naming a byte range, the sender finds the crate covering that range and stamps them SACKED_ACKED (“receiver confirmed this one”). The SACK “I have 2001–3000” as seen in the example above makes the sender do:

You might be thinking why isn’t 0 to 1000 SACKed as it did get the acknowledgment! The point is it got acknowledged by a different mechanism we already did discuss, and that mechanism does something stronger than SACK. Lets look at it closely again:

Two different kinds of “I got it”

Cumulative ACK (the number 1001 in the example) confirms the solid run from the start 0 to 1000. When bytes are cumulatively ACKed, the sender doesn’t just mark them, it throws those crates away entirely. They’re fully confirmed, in order, done. No reason to keep them in the retransmit queue at all.

SACK confirms only the out-of-order chunks past a hole (1001 to 2000) like 2001 to 3000. These the sender cannot throw away yet, because there’s still a hole before them (1001–2000 missing). Until that hole is filled, the whole region stays “in progress.” So the sender keeps those crates around but stamps them SACKED = “receiver has this, don’t resend it, but I can’t retire it yet.”

So why isn’t 0–1000 marked SACKED?

Because it’s already gone which is better than SACKED. It left the queue the moment the cumulative ACK covered it. There’s nothing left to stamp. SACKED is a holding state that only makes sense for data stuck behind a hole. 0–1000 has no hole before it, so it graduated straight to fully acknowledged and freed.

The reason this matters for the bug is that the crates that pile up and get merged are exactly the SACKED ones, the ones stuck behind holes that can’t be freed. The attacker keeps a huge run of tiny crates in that stuck-but-flagged state, which is what feeds the merging. So a SACK message’s real job is to mark specific crates as confirmed.

Why marking leads to merging

This is the key. Merging (shift) does not happen because of the gap. It happens when two crates next to each other are both stamped SACKED_ACKED.

The sender’s logic (tcp_shift_skb_data) is: “If this crate is SACKed, and the crate right before it (prev) is also SACKed, then keeping them as two separate crates is wasteful so pour this one into prev and free the empty crate.” Look at the guard in the code as already seen… It refuses to merge unless prev is SACKed:

if ((TCP_SKB_CB(prev)->sacked & TCPCB_TAGBITS) != TCPCB_SACKED_ACKED)
    goto fallback;   // prev not SACKed so don't merge

So two confirmed neighbors get collapsed into one:

That’s the natural, harmless purpose that shrink the retransmit queue by fusing runs of confirmed crates.

Now how the attacker exploits this

With normal size segments there are only a handful of crates, so merging a few of them is nothing. The attack changes two things:

1. Tiny segments → an enormous number of crates. Because you forced 8-byte parcels, the sender’s data is chopped into a huge number of tiny crates:

You might think that why are we forcing 8 bytes alone for tcp_gso_segs. Because 16 or 24 wouldn’t overflow. 8 is essentially the only value small enough. Here’s the math. One crate maxes out at 17 fragments × 32 KB = 557,056 bytes (~544 KB). Divide that by the segment size to get the parcel count (tcp_gso_segs):

So a crate full of 16-byte parcels only reaches ~34,816 which is nowhere near the 65,535 limit. Same for 24. They can’t overflow the label no matter how much you merge, because the crate runs out of physical space (544 KB) long before the count reaches 65,535.

The threshold

To push the count past 65,535, the segment size has to be:

So the parcel size must be 8 bytes or less for a full crate to exceed the limit. 8 works (69,632), but 9 already fails (557,056 / 9 ≈ 61,895, under the limit).

Why 8 is the floor

The attacker can’t go below 8 either. The smallest MSS Linux accepts is 48, and after TCP option space:

So 8 bytes is the smallest possible segment payload, and it happens to be just barely small enough to cross 65,535. That’s not a coincidence the attacker chose, it’s the only value that works, and it’s exactly the minimum the protocol allows.

So in one line a full crate is ~544 KB, so you need parcels ≤ 8.5 bytes to fit more than 65,535 of them; 16 or 24 are too big and top out well under the limit, and 8 is both small enough and the smallest the protocol permits.

We know that crafted SACKs mark long runs of them as confirmed. You send SACK blocks (and the sender also splits/marks crates while processing them) so that a long, continuous run of adjacent tiny crates all become SACKED_ACKED. Now the sender sees a giant run of confirmed neighbors and does exactly what it’s designed to do, pour them all, one after another, into a single prev crate. That’s the chaining you saw in the code (“hole filled allows collapsing with the next as well”) after each merge it grabs the next neighbor and pours that in too:

prev swallows thousands of 8-byte crates. Its tcp_gso_segs (parcel count label) climbs 60,000… 65,000… then one more pour pushes it past 65,535 and the label rolls over to a tiny wrong number. The safety check sees the impossible label and the kernel crashes.

“if receiver says I received 0–1000 and 2001–3000, why does it merge everything?”

That small example alone wouldn’t merge much, only where two adjacent crates are both SACKed. The attack isn’t one SACK like that rather it’s a stream of crafted SACKs that mark a long unbroken run of thousands of adjacent tiny crates as confirmed.

So the attacker keeps triggering merges. The one big crate fills toward its physical ceiling:

But the label can only count to 65,535. So the crate now really holds ~69,632 parcels, and the label can’t say that so it wraps:

The crate physically holds ~65,700 parcels, but the label reads 164.

Codeflow

We will be analyzing the code for Linux kernel version 5.1.10 which is vulnerable. We first land on tcp_v4_rcv() (net/ipv4/tcp_ipv4.c) which is TCP’s front door for every incoming IPv4 packet. The IP layer hands it a packet and says this one is TCP so deal with it. Its job is to figure out which connection the packet belongs to and what state that connection is in.

Inside tcp_v4_rcv it calls tcp_v4_do_rcv. Once it has the socket and sees the state is ESTABLISHED, it calls tcp_rcv_established():

if (sk->sk_state == TCP_ESTABLISHED) {
    //some code
    tcp_rcv_established(sk, skb);   
    return 0;
}

From tcp_rcv_established we go to tcp_ack at three points but our CVE, the SACK packet reaches tcp_ack only through the step5 call:

step5:
    if (tcp_ack(sk, skb, FLAG_SLOWPATH | FLAG_UPDATE_TS_RECENT) < 0)
        goto discard;

It doesn’t take the earlier two calls (tcp_ack(sk, skb, 0) and tcp_ack(sk, skb, FLAG_DATA)) which live inside the big header prediction if at the top of the function:

if ((tcp_flag_word(th) & TCP_HP_BITS) == tp->pred_flags &&
    TCP_SKB_CB(skb)->seq == tp->rcv_nxt &&
    !after(TCP_SKB_CB(skb)->ack_seq, tp->snd_nxt)) {

tp->pred_flags is a precomputed snapshot of what a “normal, expected” header looks like, including its expected length. A SACK packet fails this match for two reasons:

• its header is longer (the SACK option adds bytes), so the flags/length word won’t equal pred_flags.
• the comment above the check: ```c /* pred_flags is 0xS?10 « 16 + snd_wnd
  • if header_prediction is to be made
  • ‘S’ will always be tp->tcp_header_len » 2
  • ’?’ will be 0 for the fast path, otherwise pred_flags is 0 to
  • turn it off (when there are holes in the receive
  • space for instance)
  • PSH flag is ignored. */ ```

note pred_flags is deliberately set to 0 whenever there are holes in the receive space and SACK only appears because there are holes. Either way the if is false, so the entire block (and both tcp_ack calls inside it) is skipped.

With that block skipped, execution falls to:

slow_path:
	if (len < (th->doff << 2) || tcp_checksum_complete(skb))
		goto csum_error;
	if (!th->ack && !th->rst && !th->syn)
		goto discard;
	if (!tcp_validate_incoming(sk, skb, th, 1))
		return;

step5:
	if (tcp_ack(sk, skb, FLAG_SLOWPATH | FLAG_UPDATE_TS_RECENT) < 0)
		goto discard;

The attacker’s packets are valid ACKs with correct sequence numbers so the prior checks pass and takes the step5 tcp_ack.

Every incoming acknowledgment on an established connection passes through tcp_ack. For this attack, the packet arriving at tcp_ack is a duplicate ACK with SACK blocks attached. Two things about it are deliberate. First, its cumulative acknowledgment number doesn’t move. The attacker has left a gap early in the stream and never fills it, so the “receiver” is permanently stuck acknowledging only the bytes before that gap. Second, its SACK blocks keep reporting data past the gap, as if the receiver has been happily collecting everything on the far side of the hole.

That combination is the engine of the attack. The frozen acknowledgment means the sender can never retire the segments sitting at the gap and they stay parked in the retransmit queue instead of being freed. The SACK blocks, meanwhile, keep marking the parked segments as received, which is exactly the state that makes the kernel try to merge them. So every one of these packets does the same thing, it leaves the queue full and nudges the kernel toward merging more of it. It does that by clearing three checkpoints.

The stale ACK guard. The function first pulls out two values that both point at the same place at the front edge of the hole:

u32 prior_snd_una = tp->snd_una;  // oldest byte the sender sent but isnt' yet acknowledged
u32 ack = TCP_SKB_CB(skb)->ack_seq;     // what this packet acknowledges

Then it checks whether the ACK is stale:

if (before(ack, prior_snd_una))
    goto old_ack;

Say the sender has transmitted bytes 0–5000, the receiver got 0–1000, and 1001 onward is “missing.” The receiver’s cumulative ACK can’t move past the gap, so it stays at 1001, the sender’s oldest unconfirmed byte is also 1001. That makes ack == prior_snd_una, and before(1001, 1001) is false, so the packet is not stale and we don’t branch to old_ack. This is exactly why the attacker keeps the ACK frozen. An advancing ACK would let the sender retire (free) those queued segments, and freed segments can’t be merged. A frozen ACK keeps them parked in the retransmit queue, available to merge.

The branch selector. Next, tcp_ack decides between a cheap “window just moved forward” path and the detailed path that actually processes SACK:

if ((flag & (FLAG_SLOWPATH | FLAG_SND_UNA_ADVANCED)) == FLAG_SND_UNA_ADVANCED) {
    /* pure forward advance — no SACK work */
} else {
    /* detailed path: SACK handled here */

Our packet took the slow path into tcp_ack (it carries options), so FLAG_SLOWPATH is set and since it acknowledged no new data, FLAG_SND_UNA_ADVANCED is unset. The condition wants the opposite combination, so it fails and execution drops into the else, the branch that handles SACK.

The call that starts the overflow. Inside that branch:

if (TCP_SKB_CB(skb)->sacked)
    flag |= tcp_sacktag_write_queue(sk, skb, prior_snd_una, &sack_state);

On an incoming ACK, TCP_SKB_CB(skb)->sacked holds the byte offset to the SACK option in the header (zero when there’s no SACK). The attacker’s packet carries SACK, so it’s non-zero and the call fires tcp_sacktag_write_queue.

tcp_sacktag_write_queue turns SACK numbers into queue work. This is where the SACK option stops being bytes in a packet and becomes real work on the retransmit queue. It runs in a few stages.

1. Find the SACK option in the packet

const unsigned char *ptr = (skb_transport_header(ack_skb) + TCP_SKB_CB(ack_skb)->sacked);
struct tcp_sack_block_wire *sp_wire = (struct tcp_sack_block_wire *)(ptr + 2);

• skb_transport_header(ack_skb) points at byte 0 of the TCP header. TCP_SKB_CB(ack_skb)->sacked is the saved offset to where the SACK option sits inside it. Added together, ptr lands on the option.
• ptr + 2 skips the 2 header bytes (Kind, Length recall SACK layout) to reach the first (start, end) pair, and the cast lets the code index it as sp_wire[0], sp_wire[1], …

tcp_sack_block_wire is one block in wire layout:

struct tcp_sack_block_wire {
    __be32 start_seq;
    __be32 end_seq;
};

Then the blocks (ranges) are counted and each block is read and validated into sp[].

for (i = 0; i < num_sacks; i++) {
    sp[used_sacks].start_seq = get_unaligned_be32(&sp_wire[i].start_seq);
    sp[used_sacks].end_seq   = get_unaligned_be32(&sp_wire[i].end_seq);
    //some code...
}

The blocks are then sorted using a small bubble sort into ascending sequence order, so the retransmit queue (itself ordered by sequence) can be walked front-to-back in one pass.

while (i < used_sacks) {
    u32 start_seq = sp[i].start_seq;
    u32 end_seq   = sp[i].end_seq;
    ...
walk:
    skb = tcp_sacktag_walk(skb, sk, next_dup, state, start_seq, end_seq, dup_sack);
advance_sp:
    i++;
}

For each validated block, tcp_sacktag_walk is called with that block’s start_seq andend_seq. For one SACK range, tcp_sacktag_walk walks the queued segments it covers, tags each as SACKed, and before tagging tries to merge each into its neighbor. The merge attempt is the only part that matters here:

tmp = tcp_shift_skb_data(sk, skb, state, start_seq, end_seq, dup_sack);

As the walk sweeps the attacker’s long run of tiny, adjacent, SACKed segments, tcp_shift_skb_data fires over and over, exactly the condition that makes merging actually happen. (dup_sack just flags a DSACK which is a duplicate report and isn’t used by this attack. The attacker sends normal SACKs.)

Intcp_shift_skb_data the merging happens. First the target (prev) neighbor gets picked:

prev = skb_rb_prev(skb);                  // crate before this one
if (!prev)
    goto fallback;
if ((TCP_SKB_CB(prev)->sacked & TCPCB_TAGBITS) != TCPCB_SACKED_ACKED)
    goto fallback;                        // prev must be purely SACK-confirmed
if (!tcp_skb_can_collapse_to(prev))
    goto fallback;

prev is the accumulator and everything gets poured into it. Only adjacent confirmed crates fuse. The attacker’s long SACKed run means this passes repeatedly, so prev keeps accepting more. Then we do in tcp_shift_skb_data:

....
len    = skb->len;                 // bytes to move
pcount = tcp_skb_pcount(skb);      // segments in this crate
mss    = tcp_skb_seglen(skb);      // bytes per segment (8 in the attack)
...
if (!skb_shift(prev, skb, len)) // physically move data into prev 
    goto fallback;
if (!tcp_shifted_skb(sk, prev, skb, state, pcount, len, mss, dup_sack))  // fix up accounting
    goto out;

skb_shift moves the payload’s page fragments into prev, it doesn’t copy bytes, it just re-points fragment entries out of skb’s array into prev’s:

With MAX_SKB_FRAGS = 17 and each fragment up to ~32 KB, prev tops out at ~544 KB. At 8 bytes per segment that’s 544 KB ÷ 8 ≈ 69,632 segments which is more than the 16-bit tcp_gso_segs can hold. In 5.1.10 this skb_shift is unguarded. It keeps packing fragments regardless of the resulting count. After one merge it pulls the next SACKed neighbor into the same prev and repeats. That’s the engine that lets one prev swallow crate after crate, driving the count upward. The checks keep the run uniform (same SACKed state, same segment size).

Then in tcp_shifted_skb lies our buggy code. It does:

tcp_skb_pcount_add(prev, pcount);
BUG_ON(tcp_skb_pcount(skb) < pcount);   // panics once prev's count has wrapped
tcp_skb_pcount_add(skb, -pcount);

• tcp_skb_pcount_add(prev, pcount) does prev->tcp_gso_segs += pcount, where pcount is the number of segments of the sk_buff going to be added to prev. This is the 16-bit add that overflows. As the merges chain, prev’s count climbs past 65,535 and wraps to a small value.
• BUG_ON(...) with the count wrapped, skb’s stored count now reads below pcount, the assertion is true, and the kernel panics. That panic is the remote DoS. The fix later softened this to WARN_ON_ONCE and, more importantly, stopped the overflow upstream with the capped tcp_skb_shift.

### 13. The crash

After skb_shift has moved the data, tcp_shifted_skb runs a sanity check on the segment counts:

tcp_skb_pcount_add(prev, pcount);
BUG_ON(tcp_skb_pcount(skb) < pcount);
tcp_skb_pcount_add(skb, -pcount);

Two buffers are involved:

• prev collects the merged data. Its 16-bit counter (tcp_gso_segs) goes up on every merge.
• skb is the buffer being emptied. The BUG_ON reads its counter.

The two values being compared:

• pcount is the number of segments being moved out of skb in this merge.
• tcp_skb_pcount(skb) is the segment count stored in skb.

The check says “the buffer I’m draining must say it holds at least as many segments as I’m removing.” Normally that is always true. If it is false, the kernel assumes memory is corrupt and calls BUG_ON.

Step 1: the overflow (no crash yet).
Over many ACKs, one buffer B is prev in merge after merge. It ends up holding about 66,000 segments of 8 bytes each. The 16-bit counter wraps and stores 66,000 − 65,536 = 464 (example numbers). The BUG_ON only reads skb, never prev, so nothing crashes. B now has a stored count that is much smaller than its real data.

Step 2: the crash.
A later ACK carries a SACK block that covers only part of B. Now B is the skb being emptied. tcp_shift_skb_data computes pcount = len / mss from the number of bytes the SACK covers. Say that is 8,192:

This only happens when the SACK covers part of the buffer. If it covers the whole buffer, pcount = tcp_skb_pcount(skb). The two values are then identical and the check is false.

Why the whole machine goes down.
BUG() executes an invalid instruction (ud2), which raises an invalid-opcode exception. The kernel prints the kernel BUG at ... message and starts an oops. TCP receive runs in softirq context, so there is no process to kill. The kernel calls panic("Fatal exception in interrupt"). The attacker needs no login and no exploit code, only a tiny MSS and crafted SACKs on an ordinary connection.

##14. The fix

Add a check before merging: if pouring the parcels would push the label past 65,535, don’t merge.

/* net/ipv4/tcp_input.c — new tcp_skb_shift(), commit 3b4929f65b0d */
int tcp_skb_shift(struct sk_buff *to, struct sk_buff *from,
                  int pcount, int shiftlen)
{
    if (unlikely(to->len + shiftlen >= 65535 * TCP_MIN_GSO_SIZE))
        return 0;                               /* refuse: too many bytes */
    if (unlikely(tcp_skb_pcount(to) + pcount > 65535))
        return 0;                               /* refuse: label would overflow */
    return skb_shift(to, from, shiftlen);       /* only now pour the parcels */
}

And the deadly assertion was softened so a mismatch just logs a warning instead of killing the box:

- BUG_ON(tcp_skb_pcount(skb) < pcount);
+ WARN_ON_ONCE(tcp_skb_pcount(skb) < pcount);