Understanding IP, TCP, and HTTP • Hacker News Board

Re: Understanding IP, TCP, and HTTP

#41

Post by peterwwillis » Sat, Mar 08, 2014, 8:29 AM UTC

zAy0LfpBZLC8mAC wrote: post #36

peterwwillis wrote: post #34

Earlier quoted context omitted.

It's nice to put disclaimers in there, but if it's the first time a person has heard the information, the disclaimer is basically ignored. Because what are you going to do when you have to troubleshoot a tcp connection or write an application? Go back and find a book on tcp and learn the whole thing from the beginning? Unlikely, as you already have what you consider to be knowledge about tcp. Even if you don't consid…

I think there are arguments for both sides - if you do know something very well, you might overlook pitfalls when explaining it to someone who doesn't have your background, for example, while someone who has only just got the grasp of it might be well aware of what might confuse a newbie, so I think there is some value in unexperienced people writing about what they learn and how they learn it, with appropriate discl…

I think it's more likely that mistakes will happen if someone believes they know what they are doing. But how do they know if they know what they are doing or not? That's where "sensible person" becomes subjective to me.

I wrote an IP stack, of sorts, and used Wikipedia to do it. I'm aware that it's probably crappy, but only because it was basically designed to be. If I had tried to design it well, I might lead myself to believe I had done it correctly, for example because I found no problems with it in my testing. But as you're aware, there's plenty of problems with tcp/ip stacks that only come up as edge cases. So even if I was being sensible I might end up with shitty code and push it into a product, and then we're screwed. But if I had learned the stack correctly I couldn't be in that mess.

A kind of solution lies in forums like HN, though. Sure, the posts are fallible and are often upvoted merely because they are perceived as authoritative. But we have the comments section, and knowledgeable persons who can speak up and educate. So it may not matter at all who's teaching, as long as somebody picks up the slack.

Re: Understanding IP, TCP, and HTTP

#42

Post by mattikl » Sat, Mar 08, 2014, 8:35 AM UTC

ohblahitsme wrote: post #35

Jugurtha wrote: post #31

Earlier quoted context omitted.

It's the way I learn, too.. But it doesn't make me write things as a guy who's knowledgeable. I'm a total noob, yet the first few paragraphs made me cringe because I felt there were some odd things. I had a weird feeling about it. It wouldn't have bothered me if there wasn't this "A periodical about best practices and advanced techniques in Objective-C".. Or using the word "great contributors", etc. I mean, one has t…

Could you post an example of the "I'm learning and journaling my progress" writing style? I'd like to start doing this and I don't want to come off as an expert on things I'm just learning.

One thing is to not publish it -- a learning journal is probably much more important for you to write than for anyone to read. Then give yourself a couple of years or decades of learning time, and if you still want to write about it, what you wrote as a beginner will give you valuable insights into the beginner's mind, things you have probably forgotten.

And of course you can publish it (might be good for feedback), just state that it's a learning journal, not "best practices".

Re: Understanding IP, TCP, and HTTP

#43

Post by jokoon » Sat, Mar 08, 2014, 8:52 AM UTC

I'm still curious about an explanation why do we have both TCP and UDP.

For example if you do peer to peer, you need low latency, and UDP is best for that.

I think it's because TCP is hardware optimized, but it's designed to transmit a file in a stream, so if a packet is corrupt, it just waits to send that packet. In that fashion, TCP tend to be slower, but on average it's more efficient for single files or webpages.

You don't have good granularity with TCP, but if you want to work with UDP, you need to add redundancy and other mechanisms to make sure all is good.

ENet is an example of using UDP for gaming, so the goal is to have the lowest latency possible.

Re: Understanding IP, TCP, and HTTP

#44

Post by zAy0LfpBZLC8mAC » Sat, Mar 08, 2014, 9:02 AM UTC

peterwwillis wrote: post #41

zAy0LfpBZLC8mAC wrote: post #36

Earlier quoted context omitted.

I think there are arguments for both sides - if you do know something very well, you might overlook pitfalls when explaining it to someone who doesn't have your background, for example, while someone who has only just got the grasp of it might be well aware of what might confuse a newbie, so I think there is some value in unexperienced people writing about what they learn and how they learn it, with appropriate discl…

I think it's more likely that mistakes will happen if someone believes they know what they are doing. But how do they know if they know what they are doing or not? That's where "sensible person" becomes subjective to me. I wrote an IP stack, of sorts, and used Wikipedia to do it. I'm aware that it's probably crappy, but only because it was basically designed to be. If I had tried to design it well, I might lead mysel…

I guess my point is: The reasonable (and responsible) thing to do when you actually build something (rather than just learn about something out of curiosity or to be able to use the understanding in troubleshooting) is to read the primary sources, the standard documents, and in particular to be aware that whatever you learned from hearsay is not reliable enough to actually build a product on if there is an option to get your hands on the primary source. Especially with internet technology, we are in the great position that W3C recommendations and RFCs are freely available for everyone, so there isn't really much of a reason not to read them.

That might not be quite enough for a really good implementation, but overall software quality would be a hell of a lot better if everyone did that, it's just amazing when you look just at websites and also emails, how many people just make up how they think things work rather that reading the standards that are only a google search away.

Re: Understanding IP, TCP, and HTTP

#45

Post by danieleggert » Sat, Mar 08, 2014, 9:10 AM UTC

zAy0LfpBZLC8mAC wrote: post #24

cbsmith wrote: post #22

Earlier quoted context omitted.

I'm also going to cut him some slack with the bit about giving each segment a unique number. While formally the sequence number identifies each byte of data, it really is about providing heuristics to identify and correct for out of order, fragmented, missing & duplicate segments. It is important that it be about bytes, particularly for things like SACK, but if you are trying to simplify things you might describe it…

Sure, nothing wrong with simplifying things, but "Both ends are sending sackOK. This will enable Selective Acknowledgement. It switches the sequence numbers and acknowledgment number to use byte range instead of TCP segment numbers." is just flat-out wrong, and in particular suggests that "numbered segments" is not a simplification but an actual fact about how the thing works.

Thanks for taking the time to point this out. I'll update the article.

Re: Understanding IP, TCP, and HTTP

#46

Post by danieleggert » Sat, Mar 08, 2014, 9:11 AM UTC

zAy0LfpBZLC8mAC wrote: post #3

How I love it when people without deep knowledge of some subject write authoritative sounding articles. Without guarantee of completeness, to avoid the spread of misinformation: - IPv6 fragmentation has nothing to do with some "minimum payload size" (whatever that is) - there simply is no fragmentation being done by routers, the sender still can fragment however it pleases, and presumably will do so whenever it has t…

Thanks for taking the time to comment on the article. I'll update the article later today to correct these. Your help is appreciated.

Re: Understanding IP, TCP, and HTTP

#47

Post by frankzinger » Sat, Mar 08, 2014, 9:15 AM UTC

zAy0LfpBZLC8mAC wrote: post #38

frankzinger wrote: post #37

Earlier quoted context omitted.

TCP/IP Illustrated has a new edition (volume 1 only for now, AFAICT) which has been updated with much more modern content, including lots of info about IPv6: http://www.informit.com/store/tcp-ip-illustrated-volume-1-th...

Well, yeah, it's a book by the same name, but is it the same quality (well, it's not just the same name, of course, but a new author obviously can make a big change in quality, in either direction ;-)? In any case, my recommendation was referring to the old edition by Stevens alone, no clue about the new one, though at least the newly covered material seems appropriate to me. (For anyone who might not be aware: W. Ri…

I have not worked through the entirety of the original, which I also own, but the new one seems pretty good to me so far. But you may have a different opinion as somebody who knows much more than me about TCP/IP.

Re: Understanding IP, TCP, and HTTP

#48

Post by zAy0LfpBZLC8mAC » Sat, Mar 08, 2014, 9:37 AM UTC

jokoon wrote: post #43

I'm still curious about an explanation why do we have both TCP and UDP. For example if you do peer to peer, you need low latency, and UDP is best for that. I think it's because TCP is hardware optimized, but it's designed to transmit a file in a stream, so if a packet is corrupt, it just waits to send that packet. In that fashion, TCP tend to be slower, but on average it's more efficient for single files or webpages.…

Bittorrent is also peer to peer, and it doesn't need low latency. Really, it's about latency, nothing to do with peer to peer.

TCP has head-of-queue blocking, as it guarantees complete and in-order delivery, so when a packet gets lost in transit, it has to wait for a retransmit of the missing packet, whereas UDP delivers packets to the application as they arrive, including duplicates and without any guarantee that a packet arrives at all or which order they arrive (it really is essentially IP with port numbers and an (optional) payload checksum added), but that is fine for telephony, for example, where it usually simply doesn't matter when a few milliseconds of audio are missing, but delay is very annoying, so you don't bother with retransmits, you just drop any duplicates, sort reordered packets into the right order for a few hundred milliseconds of jitter buffer, and if packets don't show up in time or at all, they are simply skipped, possible interpolated where supported by the codec.

Also, a major part of TCP is flow control, to make sure you get as much througput as possible, but without overloading the network (which is kinda redundant, as an overloaded network will drop your packets, which means you'd have to do retransmits, which hurts throughput), UDP doesn't have any of that - which makes sense for applications like telephony, as telephony with a given codec needs a certain amount of bandwidth, you can not "slow it down", and additional bandwidth also doesn't make the call go faster.

In addition to realtime/low latency applications, UDP makes sense for really small transactions, such as DNS lookups, simply because it doesn't have the TCP connection establishment and teardown overhead, both in terms of latency and in terms of bandwidth use. If your request is smaller than a typical MTU and the repsonse probably is, too, you can be done in one roundtrip, with no need to keep any state at the server, and flow control als ordering and all that probably isn't particularly useful for such uses either.

And then, you can use UDP to build your own TCP replacements, of course, but it's probably not a good idea without some deep understanding of network dynamics, modern TCP algorithms are pretty sophisticated.

Also, I guess it should be mentioned that there is more than UDP and TCP, such as SCTP and DCCP. The only problem currently is that the (IPv4) internet is full of NAT gateways which make it impossible to use protocols other than UDP and TCP in end-user applications.

Re: Understanding IP, TCP, and HTTP

#49

Post by sajal83 » Sat, Mar 08, 2014, 10:05 AM UTC

> The improvements of using HTTP pipelining can be quite dramatic over high-latency connections – which is what you have when your iPhone is not on Wi-Fi. In fact, there’s been some research that suggests that there’s no additional performance benefit to using SPDY over HTTP pipelining on mobile networks

Excellent summary but i think pipeline has been oversimplified. HTTP pipelining is a FIFO queue. The responses have to be delivered in the same order as the requests. So if the first(or an early) response took longer to generate, all other requests in the pipeline have to wait. Something that SPDY is not susceptible to.

Re: Understanding IP, TCP, and HTTP

#50

Post by wtracy » Sat, Mar 08, 2014, 10:59 AM UTC

ohblahitsme wrote: post #35

Jugurtha wrote: post #31

Earlier quoted context omitted.

It's the way I learn, too.. But it doesn't make me write things as a guy who's knowledgeable. I'm a total noob, yet the first few paragraphs made me cringe because I felt there were some odd things. I had a weird feeling about it. It wouldn't have bothered me if there wasn't this "A periodical about best practices and advanced techniques in Objective-C".. Or using the word "great contributors", etc. I mean, one has t…

Could you post an example of the "I'm learning and journaling my progress" writing style? I'd like to start doing this and I don't want to come off as an expert on things I'm just learning.

I'll throw this out there: http://www.wtracy.com/blog/quaternions.html