Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TCP is harder than it looks

jsnell.iki.fi

Post reply on HN
Print view View original

21–30 of 66 posts

Re: TCP is harder than it looks

#21

Post by viraptor »

acqq wrote: post #20
philh wrote: post #9

Earlier quoted context omitted.

The other node doesn't depend on the settings, it depends on not receiving a duplicate SYN.

Note what he actually writes: "Our TCP implementation has a variable base SYN retransmit timout, and in this case it was roughly 500ms. So most of the time the page load would fail with our TCP stack, but succeed with an off the shelf one that had a SYN retransmit timeout of 1 second." If his timeout were 1 second, his connection with the other node would work. It's his own choice to insist on "variable base SYN retr…

I don't think it was an issue with the timeout. The other side simply could not handle retransmitted SYN. At all. The 500ms comes up only because that happened to be their RTT. If they used 1 second, this host would work, but another one using the same stack but with 1 second delay would continue to fail.

It's not about the other host not having an ideal implementation, or about testing ideal cases. The other host had a bug which cannot cope with some situations. There's nothing they could do about it, apart from tweaking the timeout to some bigger value.

Re: TCP is harder than it looks

#22

Post by abecedarius »

MrBuddyCasino wrote: post #19
michaelx386 wrote: post #13

Earlier quoted context omitted.

Great story with the added bonus of showing me the GNU Units command. What a fantastic program, no more Googling when I need to convert stuff.

Indeed, I just checked OSX - sadly, units doesn't know about millilightseconds ("586 units, 56 prefixes").

"milli lightsecond" should do it.

Re: TCP is harder than it looks

#23

Post by desas »

heymishy wrote: post #15
longwave wrote: post #12

Earlier quoted context omitted.

I realise you're trying to be facetious, but probably this one: http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-so...

thats a great set of TCP timeout stories, enjoyed reading. In what circles do you travel that you comea cross multiple niche stories like this?

Not the OP but both have been posted on HN before..

Re: TCP is harder than it looks

#24

Post by qznc »

MrBuddyCasino wrote: post #18
longwave wrote: post #12

Earlier quoted context omitted.

I realise you're trying to be facetious, but probably this one: http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-so...

That was quite impressive, not sure I would have found that out. I know I'm stretching my luck, but since those were great reads - do you have a 3rd best TCP timeout story? :)

Here is one on IPv6 not TCP: https://news.ycombinator.com/item?id=4709952

Re: TCP is harder than it looks

#25

Post by michaelx386 »

longwave wrote: post #12
MrBuddyCasino wrote: post #10

Earlier quoted context omitted.

Good read. I guess now we'd be interested in hearing your 2nd best TCP timeout story.

I realise you're trying to be facetious, but probably this one: http://mina.naguib.ca/blog/2012/10/22/the-little-ssh-that-so...

What makes this story so great is how determined they were to find the real problem even though it was already "fixed" by changing the routes. Who knows how much anguish this person has spared other people by not giving up until the problem was truly solved. Doing it with such skill and ingenuity was the icing on the cake, I could read these kind of stories all day.

Re: TCP is harder than it looks

#26

Post by jsnell »

viraptor wrote: post #21
acqq wrote: post #20

Earlier quoted context omitted.

Note what he actually writes: "Our TCP implementation has a variable base SYN retransmit timout, and in this case it was roughly 500ms. So most of the time the page load would fail with our TCP stack, but succeed with an off the shelf one that had a SYN retransmit timeout of 1 second." If his timeout were 1 second, his connection with the other node would work. It's his own choice to insist on "variable base SYN retr…

I don't think it was an issue with the timeout. The other side simply could not handle retransmitted SYN. At all. The 500ms comes up only because that happened to be their RTT. If they used 1 second, this host would work, but another one using the same stack but with 1 second delay would continue to fail. It's not about the other host not having an ideal implementation, or about testing ideal cases. The other host ha…

Exactly. And note that we're not talking outlandish latencies here. A 2G connection, or a combination of a 3G connection + OS X + being on the wrong side of the globe would've done it. Or a single well placed packet loss (losing the SYNACK).

Often when you have these kinds of incompatibility issues, they are possible to fix with no cost at all. There's no harm in rearranging or re-aliging your TCP options for maximum compatibility. Or it's possible to notice from either the handshake or later connection behavior that the other end is dodgy, and conditionally disable whatever feature might cause trouble. But you can't possibly do that when we have no information about the other end. That's why I was particularly annoyed by this issue.

(And I certainly wasn't expecting everything to work perfectly, like "acqq" claims. That expectation would be quickly beaten out of anyone dealing with arbitrary TCP traffic.)

Re: TCP is harder than it looks

#27

Post by lukego »

jsnell wrote: post #26
viraptor wrote: post #21

Earlier quoted context omitted.

I don't think it was an issue with the timeout. The other side simply could not handle retransmitted SYN. At all. The 500ms comes up only because that happened to be their RTT. If they used 1 second, this host would work, but another one using the same stack but with 1 second delay would continue to fail. It's not about the other host not having an ideal implementation, or about testing ideal cases. The other host ha…

Exactly. And note that we're not talking outlandish latencies here. A 2G connection, or a combination of a 3G connection + OS X + being on the wrong side of the globe would've done it. Or a single well placed packet loss (losing the SYNACK). Often when you have these kinds of incompatibility issues, they are possible to fix with no cost at all. There's no harm in rearranging or re-aliging your TCP options for maximum…

Could you map all such buggy devices on the internet and treat their known addresses specially? (Scan the whole public IPv4 range every month to find them?)

Just an idea :-)

Re: TCP is harder than it looks

#28

Post by acqq »

viraptor wrote: post #21
acqq wrote: post #20

Earlier quoted context omitted.

Note what he actually writes: "Our TCP implementation has a variable base SYN retransmit timout, and in this case it was roughly 500ms. So most of the time the page load would fail with our TCP stack, but succeed with an off the shelf one that had a SYN retransmit timeout of 1 second." If his timeout were 1 second, his connection with the other node would work. It's his own choice to insist on "variable base SYN retr…

I don't think it was an issue with the timeout. The other side simply could not handle retransmitted SYN. At all. The 500ms comes up only because that happened to be their RTT. If they used 1 second, this host would work, but another one using the same stack but with 1 second delay would continue to fail. It's not about the other host not having an ideal implementation, or about testing ideal cases. The other host ha…

I worked for many years with the locations on the remote islands with the high RTT. You wouldn't believe how much software of the biggest industry players failed to work under that circumstances.

My favorite case was when one company first claimed "OK on another side is probably the device from the competition." It was their device on another side too. They "debugged" the case for a bigger part of the year and the case remained unsolved.

I'm absolutely not saying that the far side implementation was OK. I concur that the TCP is damn hard. It's just that you can't expect that "everybody else" is perfect. You have to plan to handle the special cases or not be surprised that you can't handle them.

Re: TCP is harder than it looks

#29

Post by viraptor »

acqq wrote: post #28
viraptor wrote: post #21

Earlier quoted context omitted.

I don't think it was an issue with the timeout. The other side simply could not handle retransmitted SYN. At all. The 500ms comes up only because that happened to be their RTT. If they used 1 second, this host would work, but another one using the same stack but with 1 second delay would continue to fail. It's not about the other host not having an ideal implementation, or about testing ideal cases. The other host ha…

I worked for many years with the locations on the remote islands with the high RTT. You wouldn't believe how much software of the biggest industry players failed to work under that circumstances. My favorite case was when one company first claimed "OK on another side is probably the device from the competition." It was their device on another side too. They "debugged" the case for a bigger part of the year and the ca…

> You have to plan to handle the special cases or not be surprised that you can't handle them.

What do you propose then? You send valid traffic and get a completely broken response - I don't see any space left for handling special cases here. Like they said in the article, it's the first packet and you have no information about the other side yet. This is not something you can avoid/workaround.

Re: TCP is harder than it looks

#30

Post by viraptor »

lukego wrote: post #27
jsnell wrote: post #26

Earlier quoted context omitted.

Exactly. And note that we're not talking outlandish latencies here. A 2G connection, or a combination of a 3G connection + OS X + being on the wrong side of the globe would've done it. Or a single well placed packet loss (losing the SYNACK). Often when you have these kinds of incompatibility issues, they are possible to fix with no cost at all. There's no harm in rearranging or re-aliging your TCP options for maximum…

Could you map all such buggy devices on the internet and treat their known addresses specially? (Scan the whole public IPv4 range every month to find them?) Just an idea :-)

It's potentially a "scan all ports on every public ipv4 address" job instead. Each port may be redirected to a different machine.
Post reply on HN