Live data from Hacker News

Is it just me or is networking really hard?

gafferongames.com

71–80 of 188 posts

Re: Is it just me or is networking really hard?

#71
I know this attitude all too well. I've replied to a few questions on gamedev.stackexchange with pointers to the usual articles on game state synchronization and lag compensation, and I get lukewarm reactions of "too complicated" type. Meanwhile mr rpc-over-tcp get tens of up votes and accepted answer.

Their loss I suppose.

Re: Is it just me or is networking really hard?

#72
The single problem with TCP this rightly points out is head-of-line blocking. But overcoming that still leaves you with the problem of coping with out of order or missing events at app level, with clients and servers seeing events in differing sequences (fex packet sequence of move-duck-fire, now shuffle the move around...). I witnessed one game engine project implement custom networking on UDP and then disable unreliable & out of order messages because of game logic headaches.

And the payoff is quite small, since packet loss is rare in healthy networks and TCP handles the rare loss pretty efficiently based on ack clocking, not timeouts (fast retransmit, basically same as his idea of "redundantly sending un-acked data").

Re: Is it just me or is networking really hard?

#73
You know what I found that is actually quite amazing: Video encoders for streams of linear/live video feed for broadcast go over UDP to the next device in the chain. An encoder will often output a stream of video over UDP to devices that will then split the video into multiple renditions.

Re: Is it just me or is networking really hard?

#74
post #44

>You have whole industries (gaming, video streaming, VoIP) that will laugh at you if you send time critical data over TCP and rightly so Skype uses TCP, no? Laugh, indeed.

This might finally explain why the video quality is so bad then. If you tried any commercial video call software it's quite hard to look back to skype.

Re: Is it just me or is networking really hard?

#75
post #55

Earlier quoted context omitted.

this. I'll chime in as someone with a skill set that is completely not special (linux application development, and hardware / firmware / software integration), but in certain circles, is unique since it doesn't involve JavaScript (yet). I think the question is too vague, but I've asked a similar question: To kernel hackers (not application developers) - Tell me what you know about the Linux boot process To anyone wit…

I'd argue that the cases where a hash table's performance characteristics come into play are few and far between for most developers, and when they do, it's way more relevant that the person have a general idea how to profile perf in their code and say "yup, this is the spot that I need to worry about optimizing first". By marking down people who don't know how to implement a hash table, you're cutting out a large ch…

> "yup, this is the spot that I need to worry about optimizing first".

This almost never actually happens. In practice slow software of any size and complexity is sprinkled with many small inefficiencies that add up and no single thing makes a huge difference. To be performant you have to write with performance in mind from the get-go and not make many small mistakes.

Re: Is it just me or is networking really hard?

#76
post #71

I know this attitude all too well. I've replied to a few questions on gamedev.stackexchange with pointers to the usual articles on game state synchronization and lag compensation, and I get lukewarm reactions of "too complicated" type. Meanwhile mr rpc-over-tcp get tens of up votes and accepted answer. Their loss I suppose.

Often, "to complicated" simply means "no time and no patience to RTFM".

Re: Is it just me or is networking really hard?

#77
post #72

The single problem with TCP this rightly points out is head-of-line blocking. But overcoming that still leaves you with the problem of coping with out of order or missing events at app level, with clients and servers seeing events in differing sequences (fex packet sequence of move-duck-fire, now shuffle the move around...). I witnessed one game engine project implement custom networking on UDP and then disable unrel…

How many networks are actually healthy, though? Networked gaming is less about a hardwired LAN and more about reliably synchronizing state using your crap wifi in a oversaturated band when Comcast hasn't repaired the infrastructure I your neighborhood in years.

In other words, a healthy network is the edge case.

Re: Is it just me or is networking really hard?

#78
post #46

I sometimes ask the question "how does the internet work?" in interviews. The vague and wrong answers I hear are astonishing. I'm looking for a basic understanding of the layering of TCP, or RDP, or ICMP (or anything actually) on top of IP. I don't get that level of insight much. The people who do have that insight usually can go deeper -- slow-start, BGP, etc. I wonder if we have a modern version of C.P. Snow's mid-…

This goes on the list of interview questions that's great at letting the interviewer claim pretty much any candidate they want to discard is unworthy of the position. It's the tech community's variant of a voting literacy test. If you ask me "how does the internet work" and what you really want is details on the underlying protocols, at around the range of TCP/IP, why aren't you actually asking for that? Your questio…

I disagree. It is like FizzBuzz for sysadmins. Being able to answer does not mean you should get the job, but an utter failure to answer should absolutely disqualify you.

Once you make that basic determination, other questions should be used to judge actual fit for the job at hand.

Re: Is it just me or is networking really hard?

#79

I sometimes ask the question "how does the internet work?" in interviews. The vague and wrong answers I hear are astonishing. I'm looking for a basic understanding of the layering of TCP, or RDP, or ICMP (or anything actually) on top of IP. I don't get that level of insight much. The people who do have that insight usually can go deeper -- slow-start, BGP, etc. I wonder if we have a modern version of C.P. Snow's mid-…

You have to stop at some point. Or do you go in depth into the workings of >10Gbps transceivers? the quantum mechanics of optical fibers? how to they manufacture top-notch photodiodes anyway? You can literally spend a lifetime in these depths.

What are some good resources for learning about >10Gbps transceivers (and high-speed communication in general)? I'm not sure where to start a Wikipedia crawl.

Re: Is it just me or is networking really hard?

#80
post #65

Earlier quoted context omitted.

this. I'll chime in as someone with a skill set that is completely not special (linux application development, and hardware / firmware / software integration), but in certain circles, is unique since it doesn't involve JavaScript (yet). I think the question is too vague, but I've asked a similar question: To kernel hackers (not application developers) - Tell me what you know about the Linux boot process To anyone wit…

> use of a feature or function without understanding why and how it works can lead to really interesting performance problems, even if the code is functionally correct Which you shouldn't worry about until a profile shows it's an issue. Otherwise, having a basic understanding of data structure usage is the actual critical bit

Given the choice between two functionally correct data structures with different performance profiles, you should pick the one more appropriate to the task at hand. A lot of performance issues can be proactively avoided with a bit of analytical thinking, but you do need to know how your data structures work and what their trade-offs are.

Profilers are wonderful tools, but in the 15 years I've been professionally writing software the number of people I know that have ever used one numbers in the low 10s. Of those, the number that can derive meaningful data from a profile is even smaller. And finally, those that can spot "death from a thousand cuts" situations (quite common) from the proverbial method that takes 20% of all CPU time is even smaller yet.

Profiling has a place, of course. But it's tricky to get right, incredibly time consuming without sampling, and often doesn't give you a complete picture. It's the performance equivalent of a debugger and really should be a fallback, not the only tool in your arsenal.

Post reply on HN