Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is it just me or is networking really hard?

gafferongames.com

Post reply on HN
Print view View original

171–180 of 188 posts

Re: Is it just me or is networking really hard?

#171

Post by Const-me »

ilurk wrote: post #112

I know the article is language agnostic but I don't think anyone would develop their networking modules (in the context of videogames or finance) in anything but C++ (or Erlang/Elixir?). So could the local experts give their opinion on C++ networking libraries? I know there are a few [1] but how do they compare? I've only used Qt networking and POSIX sockets (although for C), but the application didn't have much of l…

You shouldn’t just compare C++ libraries. First, define your requirements, both functional (what should your networking do) and technical (latency, throughput, available bandwidth, operating systems of both clients and servers). Second, pick a protocols stack. Only then, look for those C++ libraries.

If on step #2 you’ll choose e.g. ProtoBuff + UDP, you’ll have totally different set of available libraries than if you choose SOAP + HTTPS. Besides, very few of those libraries work well on multiple platforms, e.g. libuv does, while libevent does not.

In addition, C++ is not the only choice. I once developed networking module for a game in C#. Have a few friends who are doing the same in finance industry.

Re: Is it just me or is networking really hard?

#172

Post by plonh »

ggambetta wrote: post #58
exDM69 wrote: post #23

Earlier quoted context omitted.

To be fair, Gaffer's original articles are well written, thoughtful and informative too. This is him ranting about the unfounded, negative and incorrect feedback he's been receiving about them from people who are willfully ignorant about networking. He could or should have replied with less attitude and more facts, though. I understood what and why he's saying what he is, but I didn't enjoy the negativity either.

His articles are excellent , and I've learned a lot from them myself. But I cannot sympathise with lowering the level of a technical discussion to personal insults.

I can't sympathize with a technical discussion Mandarin. Lucky it is a big internet with lots of blogs.

Re: Is it just me or is networking really hard?

#173

Post by plonh »

brudgers wrote: post #87
exDM69 wrote: post #23

Earlier quoted context omitted.

To be fair, Gaffer's original articles are well written, thoughtful and informative too. This is him ranting about the unfounded, negative and incorrect feedback he's been receiving about them from people who are willfully ignorant about networking. He could or should have replied with less attitude and more facts, though. I understood what and why he's saying what he is, but I didn't enjoy the negativity either.

An explanation for a behavior is not an excuse for it. Likewise my synpathy for the frustrations that led to someone else's behavior don't make the behavior right.

Indeed, it is wrong for a novice to ask a professional how he accomplished his work, and then call the answer "silly" without any evidence or reasoning in support.

Re: Is it just me or is networking really hard?

#174

Post by gafferongames »

angersock wrote: post #170
ggambetta wrote: post #58

Earlier quoted context omitted.

His articles are excellent , and I've learned a lot from them myself. But I cannot sympathise with lowering the level of a technical discussion to personal insults.

I don't know if this is a general failing of people's critical thinking abilities, a sloppiness in their reading, or something else, but I have noticed over the past (feels like) few years a growing problem: A lot of folks conflate personal insults ("you, ggambetta, are clearly a dunderhead when it comes to reading comments") with general sentiments about a group or abstract population ("Hacker News posters tend to b…

Spot on and this was firmly written with tongue in cheek! It's funny because it's true though. I harbor no ill will to anyone. Also, a nice ending to this whole thing, the original poster found the article, and halfway through reading it, realized it was about them and posted a comment!

Re: Is it just me or is networking really hard?

#175

Post by plonh »

a8da6b0c91d wrote: post #75
akerl_ wrote: post #55

Earlier quoted context omitted.

I'd argue that the cases where a hash table's performance characteristics come into play are few and far between for most developers, and when they do, it's way more relevant that the person have a general idea how to profile perf in their code and say "yup, this is the spot that I need to worry about optimizing first". By marking down people who don't know how to implement a hash table, you're cutting out a large ch…

> "yup, this is the spot that I need to worry about optimizing first". This almost never actually happens. In practice slow software of any size and complexity is sprinkled with many small inefficiencies that add up and no single thing makes a huge difference. To be performant you have to write with performance in mind from the get-go and not make many small mistakes.

And "what implementation to use for a map" is a decision nobody makes at first, as it depends a lot on hownit ends up used. In reality there are a few worst spots in our code. Being 2x too slow isn't big a deal except in the rare cases where your whole business is turning compute resources into money withn no other expenses. Investing programmer energy in features and correctness is a better use than premature optimization, say, 97% of the time.

Re: Is it just me or is networking really hard?

#176

Post by Retra »

kybernetikos wrote: post #116
mikehollinger wrote: post #48

Earlier quoted context omitted.

this. I'll chime in as someone with a skill set that is completely not special (linux application development, and hardware / firmware / software integration), but in certain circles, is unique since it doesn't involve JavaScript (yet). I think the question is too vague, but I've asked a similar question: To kernel hackers (not application developers) - Tell me what you know about the Linux boot process To anyone wit…

> to anyone with a 5th-gen language on their resume: how's a hash table implemented? use of a feature or function without understanding why and how it works can lead to really interesting performance problems, even if the code is functionally correct) Someone recently argued something similar with me, although they were even more emphatic about the importance of taking an interest in how the datastructures you're usi…

People vastly underestimate the amount of useful information that exists that you could learn. If someone asked me why I don't know some micro-fact, I'd want to respond by saying "because I've learned these other 10 things instead."

If you spend all your time learning some particular domain of knowledge, you're not going to be prepared to ask questions about the domains you don't know. And most people will have a hard time seeing the value in those things they didn't learn. Especially when they're playing an authority figure.

Re: Is it just me or is networking really hard?

#177

Post by Dylan16807 »

pyre wrote: post #167
Dylan16807 wrote: post #146

Earlier quoted context omitted.

Even looking at it like that, it's wrong. TCP has a lot of tunable parameters where the choices are baked into it, and it assumes that extra round trips are no big deal. Crypto does not have all that many tradeoffs. You just need expert implementation. It's dangerous and unnecessary to do it yourself. For something that is low-bandwidth and latency-sensitive, you can take TCP and make minor adjustments and come out w…

I was using "never implement your own crypto" as an example. I think that "never implement your own crypto" is a more important rule to follow than anything regarding TCP. On the other hand, if you don't know much about networking, trying to reimplement TCP over UDP isn't likely to gain you better performance. As with most things, there is no substitute for knowing what are doing.

I know it was an example, but the crypto saying is actually a good rule of thumb, while "don't do TCP" is a pretty bad rule of thumb.

You don't need to know that much about networking to get something that's better than TCP in this specific use case. It may be unoptimized and terrible but your goal is latency. Just pretend that 5% of packets will get lost and do something to compensate, and you beat TCP on imperfect networks, even as a novice.

Re: Is it just me or is networking really hard?

#178

Post by gafferongames »

nacs wrote: post #161
samplonius wrote: post #134

Earlier quoted context omitted.

Most video streaming today is not served by UDP, but by TCP. Only live video uses UDP. Stored video (ex. Netflix) is all sent via TCP.

> Only live video uses UDP Not necessarily. One of the largest live video streaming services, Twitch.tv, uses TCP for streaming.

Twitch is not interactive so it can just buffer for a few seconds more and handle the typical worst case of TCP. It'd probably still be better over UDP but I'm not qualified to say that for sure. I'm sure the twitch.tv guys could chime in on their choice and inform us of the pros and cons for their situation. The correct choice is application specific.

Re: Is it just me or is networking really hard?

#179

Post by aavegmittal »

chetanahuja wrote: post #117

"You can’t even imagine a way that reliability could be implemented on top of UDP that beats TCP? What total bullshit" This. This "TCP already does it best in every situation" is a common trope usually spouted by people who know nothing about what TCP does and why. The easiest way to get over this misconception is to ask yourself, can you think of a protocol that not only works well over all sorts of networks, all th…

Totally agreed. TCP is a great general purpose protocol which has been working "fine" in all cases. If it doesn't work well in a particular situation, then I believe its our fault that we are trying to fit it there just because it works.

Here is an example of what a custom protocol built over UDP taking in consideration the very specific use case of mobile devices accessing wireless networks, could do to achieve better throughputs than TCP, details here:

https://packetzoom.com/blog/lessons-learned-tcp-slow-start-d...

Re: Is it just me or is networking really hard?

#180

Post by zurn »

Dylan16807 wrote: post #152
zurn wrote: post #115

Earlier quoted context omitted.

Sure there are people with rotten last mile conditions, but healthy networks are the normal case. You get subpar download speeds with http and choppy skype/facetime/ssh if your network drops packets frequently. It's bad for business from the ISP POV. Using UDP will not work around the conditions you describe, game will still be laggy and choppy.

Downloads will work just fine if you're dropping .1% of packets. Skype is very resilient to dropped packets. Console use isn't quite so fast that a half-second hiccup is a big problem. But a naive game implementation over TCP will stutter badly and obviously every time.

Yep, and that's different from the hopelessly choppy last mile. The question here is if there's a big difference between a good (not naive) implementation over TCP vs various DIY UDP schemes, and whether that difference justifies the big complexity increases in game design due to trying to cope with out of order/lost events.
Post reply on HN