Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Size of Packets

potaroo.net

Post reply on HN
Print view View original

11–20 of 73 posts

Re: The Size of Packets

#11

Post by whiatp »

cryptonector wrote: post #3

> Path MTU discovery has not been enthusiastically embraced Ugh. I don't understand this. Especially passive PMTUD should just be rolled out everywhere. On Linux it still defaults to disabled! https://sourcegraph.com/search?q=context%3Aglobal+repo%3A%5E...

PMTU just doesn't feel reliable to me because of poorly behaved boxes in the middle. The worst offender I've had to deal with was AWS Transit Gateway, which just doesn't bother sending ICMP too big messages. The second worst offender is, IMO (data center and ISP) routers that generate ICMP replies in their CPU, meaning large packets hit a rate limited exception punt path out of the switch ASIC over to the cheapest CPU they could find to put in the box. If too many people are hitting that path at the same time, (maybe) no reply for you.

More rare cases, but really frustrating to debug was when we had an L2 switch in the path with lower MTU than the routers it was joining together. Without an IP level stack, there is no generation of ICMP messages and that thing just ate larger packets. The even stranger case was when there was a Linux box doing forwarding that had segment offload left on. It was taking in several 1500 byte TCP packets from one side, smashing them into ~9000 byte monsters, and then tried to send those over a VPNish network interface that absolutely couldn't handle that. Even if the network in the middle bothered to generate the ICMP too big message, the source would have been thoroughly confused because it never sent anything over 1500.

Re: The Size of Packets

#12

Post by ikiris »

Veserv wrote: post #5

MTU discovery would be so much easier if the default behavior was truncate and forward when encountering a oversized packet. The endpoints can then just compare the bytes received against the size encoded inside of the packet to trivially detect truncation and thus get the inbound MTU size. This allows you to do MTU discovery as a endpoint protocol with all the authentication benefits that provides and allows you to…

And how do you tell the difference between cut off packets, and a mtu drop? What about crcs / frame checks? Do you regenerate the frames? Do you do this at routed interfaces? What if there's just layer 2 only involved?

Re: The Size of Packets

#13

Post by nullc »

Is there any convenient way to tell linux distributions that the local subnet can handle 9k jumbos (or whatever) but that anything routed out must be 1500?

I currently have this solved by just sticking hosts on two vlans, one that has the default route and another that only has the jumbo capable hosts. ... but this seems kinda stupid.

Re: The Size of Packets

#14

Post by lucb1e »

nayuki wrote: post #7

> The speed of light in glass or fiber-optic cable is significantly slower, at approximately 194,865 kilometers per second. The speed of voltage propagation in copper is 224,844 kilometres per second. If I understand correctly, the speed of light in an electrical cable doesn't depend on the metal that carries current, but instead depends on the dielectric materials (plastic, air, etc.) between the two conductors?

Huh? Maybe I'm completely misreading the question, but when they say fiber-optic cable, they do mean optic. It's not an "electrical cable"; there is no metal needed in optic communication cables (perhaps for stiffness or whatnot, but not for the communication)

Re: The Size of Packets

#15

Post by lucb1e »

beeburrt wrote: post #6

That font size is tiny. If this is your site, maybe consider a larger font size

It's the same size as HN: 12px. HN looks larger to me for some reason, but I can't figure out why: when I overlay a quote someone posted here over the website with half transparency in GIMP, the text is clearly the same height. Some letters are wider, some narrower, but the final length of the 8 words I sampled is 360px on HN vs. 358px on that website (so differences basically cancel out)

This is on Firefox/Debian, in case that means something for installed fonts. I see that site's CSS specifies Verdana and Arial, names that sound windowsey to me but I have no idea if my system has (analogous versions to) those

Re: The Size of Packets

#16

Post by LegionMammal978 »

ikiris wrote: post #12
Veserv wrote: post #5

MTU discovery would be so much easier if the default behavior was truncate and forward when encountering a oversized packet. The endpoints can then just compare the bytes received against the size encoded inside of the packet to trivially detect truncation and thus get the inbound MTU size. This allows you to do MTU discovery as a endpoint protocol with all the authentication benefits that provides and allows you to…

And how do you tell the difference between cut off packets, and a mtu drop? What about crcs / frame checks? Do you regenerate the frames? Do you do this at routed interfaces? What if there's just layer 2 only involved?

> And how do you tell the difference between cut off packets, and a mtu drop?

You don't, apart from enforcing a bare-minimum MTU for sanity's sake. If your jumbo-size packets are getting randomly cut off by a middlebox, then they probably aren't stable at that size anyway.

Re: The Size of Packets

#17

Post by Veserv »

ikiris wrote: post #12
Veserv wrote: post #5

MTU discovery would be so much easier if the default behavior was truncate and forward when encountering a oversized packet. The endpoints can then just compare the bytes received against the size encoded inside of the packet to trivially detect truncation and thus get the inbound MTU size. This allows you to do MTU discovery as a endpoint protocol with all the authentication benefits that provides and allows you to…

And how do you tell the difference between cut off packets, and a mtu drop? What about crcs / frame checks? Do you regenerate the frames? Do you do this at routed interfaces? What if there's just layer 2 only involved?

Packets do not get “cut-off” normally. That is kind of the point. Some protocols allow transparent fragmentation, but the fragments need to encode enough information for reconstruction, so you can still detect “less data received than encoded on send”.

You do not need bit error detection because you literally truncated the packet. The data is already lost. But in the process you learned it was due to MTU limits which is very useful. Protocols are already required to be robust to garbage that fails bit error detection anyways, so it is not “required” to always have valid integrity tags. You could transparently re-encode bit error detection on the truncated packet if you so desire to ensure data integrity of the “MTU resulted in truncation” packet that you are now forwarding, but again, not necessary.

Any end-to-end protocol that encodes the intended data size in-band can use this technique across truncating transport layers. And any protocol which does so already requires implementations to not blindly trust the in-band value otherwise you get trivial buffer overflows. So, all non-grossly insecure client implementations should already be able to safely handle MTU truncation if they received it (they would just not be able to use that for MTU discovery until they are updated). The only thing you need is routers to truncate instead of drop and then you can slowly update client implementations to take advantage of the new feature since this middlebox change should not break any existing implementations unless they are inexcusably insecure.

Re: The Size of Packets

#19

Post by fbouynot »

nullc wrote: post #13

Is there any convenient way to tell linux distributions that the local subnet can handle 9k jumbos (or whatever) but that anything routed out must be 1500? I currently have this solved by just sticking hosts on two vlans, one that has the default route and another that only has the jumbo capable hosts. ... but this seems kinda stupid.

Yes you can set your interface MTU at 9000 and assign a 1500 MTU to the routes themselves.

Re: The Size of Packets

#20

Post by ikiris »

Veserv wrote: post #17
ikiris wrote: post #12

Earlier quoted context omitted.

And how do you tell the difference between cut off packets, and a mtu drop? What about crcs / frame checks? Do you regenerate the frames? Do you do this at routed interfaces? What if there's just layer 2 only involved?

Packets do not get “cut-off” normally. That is kind of the point. Some protocols allow transparent fragmentation, but the fragments need to encode enough information for reconstruction, so you can still detect “less data received than encoded on send”. You do not need bit error detection because you literally truncated the packet. The data is already lost. But in the process you learned it was due to MTU limits which…

I don’t think you understand what normally looks like if you start forwarding damaged frames like this because you can’t tell the difference. That was the point.
Post reply on HN