Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Falsehoods programmers believe about TCP

lwn.net

Post reply on HN
Print view View original

211–220 of 247 posts

Re: Falsehoods programmers believe about TCP

#211

Post by m3047 »

saghm wrote: post #2

> remember, all of the following statements are false at least some of the time, but for some of these, perhaps not very often > 5. There is a such thing as a TCP packet > 6. There is no such thing as a TCP packet I don't understand this at all. Either the concept of a TCP packet exists, or the concept does not exist. Even it's not being used in certain scenarios, I don't see how you can argue that "there's no such t…

If what was originally a packet is fragged, the TCP headers may (likely) not be in all the frags. So if you're looking with wireshark and you think you can filter with TCP flags good for you, 90% of the world feels that way.

I work with DNS a lot and when a protocol which is datagram-oriented is translated to a stream-oriented medium and somebody wants to potentially handle multiple requests in that stream because "efficiency"... it's so important... they need a way to distinguish those embedded datagrams: "we should do it they way they did in HTTP, with a Content-Length: header, yea THAT's the ticket!" I'm sure that's what they were thinking.

Then along come the 90% and when they try to process DNS requests in TCP streams it's "what's this two bytes in front of the request? I dunno, but I just skip over it" and I suppose it works well enough, because with frags they'll drop the tail of the requests because "corruption" anyways, and who on earth sends multiple requests in a single packet, amiright?

You may think I'm kidding, but here's some really clever (really!) eBPF code: https://gist.github.com/oghie/b4e3accf1f87afcb939f884723e2b4...

If you want more on this topic: http://consulting.m3047.net/dubai-letters/dnstap-vs-pcap.htm...

Re: Falsehoods programmers believe about TCP

#212

Post by wizzwizz4 »

krick wrote: post #188
lovecg wrote: post #67

Earlier quoted context omitted.

I believe the article that started it all is https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... - crucially every entry is self-explanatory, which is a point that a lot of the subsequent “Falsehood…” list authors miss.

Every point is not self-explanatory, and some are clearly true (while the assumption is they must be false). For instance, it surely is true that any name will fit in under a Terabyte of text, if it can be encoded at all (and assuming the contrary is counter-productive). Claiming you should not assume any name can be spelled in Unicode is absurd as well. And, yes, it's perfectly fine to assume that if your system mus…

> People’s names fit within a certain defined amount of space.

For any given fixed size that people use in practice, there is a name that does not fit. This is saying "use a variable-length field for names, because there are always edge-cases".

> Jullien was born in Sisteron, Alpes-de-Haute-Provence, and was baptised Louis George Maurice Adolphe Roche Albert Abel Antonio Alexandre Noë Jean Lucien Daniel Eugène Joseph-le-brun Joseph-Barême Thomas Thomas Thomas-Thomas Pierre Arbon Pierre-Maurel Barthélemi Artus Alphonse Bertrand Dieudonné Emanuel Josué Vincent Luc Michel Jules-de-la-plane Jules-Bazin Julio César Jullien. His father was Antonio Jullien, a violinist. The explanation of his unusual number of names is that when the time came for the baby to be baptised, his father had been invited to play at a concert given by the Sisteron Philharmonic Society, and considered it only polite to ask one of the members of the orchestra to be godfather: but since every member wished to be considered for the privilege, he was christened with the names of all thirty-six members of the society.

https://en.wikipedia.org/wiki/Louis-Antoine_Jullien

Re: Falsehoods programmers believe about TCP

#213

Post by dathinab »

While I often do like "falsehoods believe about " format it doesn't always fit in well (and if placed alone without explanation often can at most help you to know where you have knowledge gaps but not which).

A common problem are points which aren't really falsehoods, but where people frequently take false conclusions from it.

E.g. if you ask if TCP is reliable, especially in a non CS paper context, the answer is yes. That is iff you take a reasonable definition of reliable (which doesn't expect literally impossible things) and a reasonable interpretation of mostly. And just listing it as a falsehood fails to point out that there are two potential issues with your understanding while making creating the risk of someone with expertise in that sub-field of IT potentially thinking TCP is quite unreliable when it isn't. I mean the most common usage of the word reliable is a gradient with its meaning in a yes/no question being a short form of "reliable _enough_". Furthermore for most use-cases the "unreliable" aspect of TCP isn't even the main relevant misunderstanding people can have with "TCP is mostly reliable" (through for some use case it is)

The main troublesome misinterpretation is what mostly means. I.e. if you would give it a regious definition it would be "if sampling typical devices used in typical situations across some target audience then for most target audiences (weighted by audience relevance) most of the sampled devices will in a sufficient large long term moving average be reliable enough" or something like that.

What that mainly means:

- even if it's mostly reliable there will be devices for which it is reliable unreliable and anything in between

- similar even if it's mostly reliable for a device that isn't necessary all the time

- nor do we do statements about the patterns when the mostly doesn't apply, i.e. for a device TCP might be mostly reliable except every Sunday 3am for 30s, would still be mostly

- there are use-cases where unreliability is much more common

- there are audiences for which unreliability is much more common

etc.

Similar for point 5,6 about TCP packages, they are definitively a thing and there is no falsehood there. The falsehood is in believing you can reliable control them, that your OS or some middle ware isn't messing with them (e.g. splitting/combining/rewriting). So in some situations it's best to pretend there are non, but in some other situations you have to care and this might differ for different parts of the same protocol. So point 5 and 6 make sense, but don't point in a helpful direction.

to be clear doesn't mean lists are bad, or this list being particular bad, but I which they had more references/details even if short and compact and more clearly separate things too

Re: Falsehoods programmers believe about TCP

#214

Post by dathinab »

saghm wrote: post #2

> remember, all of the following statements are false at least some of the time, but for some of these, perhaps not very often > 5. There is a such thing as a TCP packet > 6. There is no such thing as a TCP packet I don't understand this at all. Either the concept of a TCP packet exists, or the concept does not exist. Even it's not being used in certain scenarios, I don't see how you can argue that "there's no such t…

It's a typical problem of this lists they don't always point to the underlying issue they want to list.

5.,6. are likely about how all kinds of things infer with your packages mainly merging/splitting but also potentially messing with them in other ways (and in general the package->frame mapping). So if you want to build anything which relies on TCP packages being a thing it likely will not work, at the same time you can't always ignore package size either (e.g. for flushing byte streams, higher level protocol chunking etc.). Through as long as you don't create transport protocols write a network stack or fine tune your TCP stack or similar you likely don't have to care about it. I mean you also probably shouldn't use TCP directly but something more high level in most situations (e.g. HTTP, gRPC, etc.).

Re: Falsehoods programmers believe about TCP

#215

Post by dathinab »

devman0 wrote: post #95
Rauchg wrote: post #83

Earlier quoted context omitted.

Falsehoods programmers believe: the OSI model

Falsehoods programmers believe: the OSI model is useless

It's not useless, just bad at it's job.

I mean in practice it's so broken, imprecise and messy to a point it's often more misleading then helpful and IMHO should have been replaced in teaching with something better well over a decade ago.

And to be clear I'm not saying it's bad because some small implementation details don't fit, it's bad in it's job of being a high level abstraction where you ignore many implementation details.

Re: Falsehoods programmers believe about TCP

#216

Post by peter_d_sherman »

>"7. If we fail to connect to a well-known remote host, then we must be offline."

Now that is a very interesting one!

It's sort of related to the question:

"How much of the Internet is accessible from any given point (location, locality, etc.) at any given point of time?"

Which is sort of unknowable, at least, without attempting to connect with every possible connection point on the Internet, which (if it could be done) would still consist of a range of time, and every point in time following that point would bring changes, perhaps small relative to the whole -- but accruing over time -- more and more, as more time elapses...

Observation: That same (or possibly similar!) phenomena would seem to be at play with respect to the measurment (observation) of quantum systems, i.e., the more certain you are of position, the less certain you are of velocity, and vice-versa...

Well, the more you measure the connectivity to all points of the Internet at one point in time, the less certain you might be of the state of the entire system as more time elapses from that point in time...

But now, why?

Observation: Generally speaking, the larger a system is, the more degrees of freedom it has, in attempting to "lock down" (know by observation, be "certain" of) the entire state of that system at one point of time, the more the parts of the system with degrees of freedom (how many degrees of freedom does the entire Internet have?) will change/evolve/move/"be subject to change" as more time evolves the state of the system... in other words, if you can know position (instantaneous state) with certainty then you can't know velocity (where it's heading to and/or future state and/or that which predicts future state) with certainty!

Sort of like you can know the instantaneous state of the Stock Market and its history... but no one can exactly predict its future (it has many, many degrees of freedom, all of which are subject to change in various unpredictable and bizarre ways!)

Which brings us back to #7:

>"7. If we fail to connect to a well-known remote host, then we must be offline."

We might be offline... but then again, we might not be! (Ping, ICMP, UDP, Telnet and Gopher anyone?)

But then again, we might be!

The Internet's online/offline status (is it really off if it is off? Is it really on if its on?) -- is much like some modern relationships, that is, "It's complicated!" :-)

The Internet is a Black Box!

It's Schrodinger's Internet!

You know, "if a TCP packet travelling at 99.44% of the speed of light on a westbound train track meets a UDP packet travelling at 99.43% of the speed of light on an eastbound train track, then when do they meet?"

You know, "solve for x..."

You know, "assume that the speed of light is constant and that quantum effects are not present!" :-)

Re: Falsehoods programmers believe about TCP

#217

Post by lowbloodsugar »

solatic wrote: post #22

Related: you can get at most once delivery or at least once delivery; you cannot get exactly once delivery. If I had a dollar for every junior who thought that a lack of exactly once delivery guarantees was a bug...

Let me get some popcorn before reading these comments.

Re: Falsehoods programmers believe about TCP

#218

Post by lisper »

sethammons wrote: post #193
lisper wrote: post #149

Earlier quoted context omitted.

OK, I don't dispute that, but that is a very different claim than "you can't get exactly-once delivery." You can (if you have at-least-once delivery).

Read the two general's problem. It is proven that exactly once _delivery_ is a physical and mathematical impossibility. Like you said, you can simulate it with exactly once _processing_. But to do that, you have to know to do that. Others are saying "you can't divide by zero" and you are saying "yeah, but if I detect a zero and the do something different, then it is the same thing." No, knowing you have to do somethi…

What makes you thin the 2GP is relevant here? 2GP has to do with coordination and consensus, not exactly-once-delivery.

> you can simulate it with exactly once _processing_

What exactly do you think is the difference between "simulating" exactly-once delivery and actually having exactly-once delivery? What do you think "delivery" means in the context of a computer network?

Re: Falsehoods programmers believe about TCP

#219

Post by lisper »

shepherdjerred wrote: post #178
lisper wrote: post #157

Earlier quoted context omitted.

OK, but you have to do a little extrapolating here because the claim is not that you can do exactly-once under all circumstances. That is obviously false because you can't do exactly-once in a situation where all comms are down indefinitely. My claim is that if I have at-least-once then I can build exactly-once out of that.

This isn't about indefinite communication loss. Obviously no progress is possible in that case. The two generals' problem has nothing to do with a permanent failure. I think there is a lot of literature out there if you're really interested in understanding and I'm happy to provide more links if you'd like.

What makes you think the 2GP is relevant here? The 2GP has to do with coordination and consensus, not exactly-once delivery.

Re: Falsehoods programmers believe about TCP

#220

Post by lisper »

sethammons wrote: post #194
lisper wrote: post #152

Earlier quoted context omitted.

> But not 100%. Why not? > At some point, a counter move on the delivery has to be stored... -somewhere-. What is a "counter move on the delivery"?

Read the two general's problem. It is a mathematical impossibility. The counter is the delivery counter. Delivery, delivery + 1

What makes you think the 2GP is relevant here? The 2GP has to do with coordination and consensus, not exactly-once delivery.
Post reply on HN