Live data from Hacker News

Beej's Guide to Network Programming

beej.us

51–60 of 91 posts

Re: Beej's Guide to Network Programming

#51
post #23

There is also a C guide [1] which has a pretty interesting email policy... I’m generally available to help out with email questions so feel free to write in, but I can’t guarantee a response. I lead a pretty busy life and there are times when I just can’t answer a question you have. When that’s the case, I usually just delete the message. It’s nothing personal; I just won’t ever have the time to give the detailed ans…

Why not like ...just leave the message alone lol? We don't have 64 Kilobytes of limited memory on our email servers anymore.

When you know you definitly do not have time for an answer deleting it might unload your mental burden. Keeping hundreds of unamswered messages around might feel like an unresolved issue.

Re: Beej's Guide to Network Programming

#52

All Beej's guides are fantastic, but if you're interested in Network programming you could do way worse than follow this up with his guide to network concepts. https://beej.us/guide/bgnet0/ If you search Algolia for Beej you'll see his material has been on hacker news numerous times.

Is this better as a preface to Network Programming or a suffix?

Yes, it seems reading about the concepts should be done first.

Re: Beej's Guide to Network Programming

#53
post #38

Open question: in what fields are you working where you really need to work with actual sockets? Most of the network programming I do fall into one of two categories: 1. A library exists to abstract over the bytes-into-sockets layer, meaning I don't need to deal with it; or 2. A library probably exists to abstract over the bytes-into-sockets layer, but I've chosen to ignore it either because I'm truly ignorant of it,…

It's useful to understand the socket API so that you can do things like read the output from strace, notice that accept() returns EAGAIN too often, and know what that indicates.

Re: Beej's Guide to Network Programming

#54

If you're trying to learn networking, better to start off with the foundational knowledge rather than jump straight into the code. Right now I'm going through the networking section on https://teachyourselfcs.com/ which is pretty good so far.

Contrary to popular belief, formal education wasn't useless here. All the foundational knowledge was covered in depth in my mandatory CS undergrad networking class. So this is a neat quick guide to all the stuff CS graduates already have learned once about but might've forgotten the details or don't know the hands on code.

Re: Beej's Guide to Network Programming

#55
post #29
post #23

There is also a C guide [1] which has a pretty interesting email policy... I’m generally available to help out with email questions so feel free to write in, but I can’t guarantee a response. I lead a pretty busy life and there are times when I just can’t answer a question you have. When that’s the case, I usually just delete the message. It’s nothing personal; I just won’t ever have the time to give the detailed ans…

I love this level of honesty. It is true, but people never say it

I had a work colleague with a similarly refreshing Out-Of-Office E-mail autoresponder that included something like: "I'm on vacation. When I get back I'm not going to have time to read all my missed E-mails. So your mail is going to be ignored and deleted with the other 2,000 unread mails. If it's important, E-mail me again after I return on [date]."

That's how you do it!

Re: Beej's Guide to Network Programming

#56
post #38

Open question: in what fields are you working where you really need to work with actual sockets? Most of the network programming I do fall into one of two categories: 1. A library exists to abstract over the bytes-into-sockets layer, meaning I don't need to deal with it; or 2. A library probably exists to abstract over the bytes-into-sockets layer, but I've chosen to ignore it either because I'm truly ignorant of it,…

at knownow i wrote an event-driven nonblocking http server in python with asyncore which could efficiently do comet over hundreds of concurrent http connections; existing http libraries were all blocking at the time

i also wrote a similar event-driven nonblocking http client in c for benchmarking and load-testing both that server and a different compatible one being written by another team in c++. this was nonblocking because it was important for the client to be faster than the server. at the time there was no libevent/libev/libuv so i wrote my own

at satellogic i maintained the upper layers of our cubesat space protocol stack, which provided a socket-like interface to our coap stack, which i also maintained part of

i also had to tweak kernel socket buffering settings for rabbitmq there

for wercam i'm experimenting with different approaches to ipc for efficiency; the data being transferred are pixel buffers so i don't think protobuf/thrift/etc. is going to help

for the mail server i share with some friends i wrote a milter in python to smtp-reject outside email to certain recipients unless it matched a whitelist; milter doesn't speak pb either and i couldn't find a python library

i wrote a multicast file transfer program in python (bccpo/bccpi) for copying files across the lan without having to fiddle around with ip addresses; the data being transferred is just (multicast) the ip and port and (unicast) the filename, file size, and file contents, so again i'm not sure pb would simplify anything. the hard part of getting it working was just multicast

i wrote a chat client for icb in python with asyncore, and again i couldn't find a suitable library

even when i'm using a library, it's often pretty useful to be able to read strace output or use select or poll with the file descriptor it's trying to hide from me

i don't think any of these qualify as either your category 1 or 2

i've also done a huge number of things that fall into category 2, of course, like writing an irc client in bash, writing a web server in assembly, and writing a chat server in three 80-column lines of c, writing a mud, and so on

Re: Beej's Guide to Network Programming

#57
post #38

Open question: in what fields are you working where you really need to work with actual sockets? Most of the network programming I do fall into one of two categories: 1. A library exists to abstract over the bytes-into-sockets layer, meaning I don't need to deal with it; or 2. A library probably exists to abstract over the bytes-into-sockets layer, but I've chosen to ignore it either because I'm truly ignorant of it,…

1. Writing the libraries you describe 2. Speaking unusual protocols for which (good) libraries don’t exist. 3. I think interacting with the sockets api can also be relevant for tweaking various options to get better performance (though often there are Linux defaults that can be tweaked instead)

I'm fully aware of the hypotheticals – what I'm curious about is which your personal experiences are with e.g. those three points you list.

Re: Beej's Guide to Network Programming

#58
post #38

Open question: in what fields are you working where you really need to work with actual sockets? Most of the network programming I do fall into one of two categories: 1. A library exists to abstract over the bytes-into-sockets layer, meaning I don't need to deal with it; or 2. A library probably exists to abstract over the bytes-into-sockets layer, but I've chosen to ignore it either because I'm truly ignorant of it,…

At least on Unix and Windows file read and write methods work for sockets too, so streaming bytes isn't different. How would you implement, say, netcat? Launch a jango rest service with kubernetes, which will run in a wm if it's a wrong OS, create a certificate with letsencrypt and access it with a react electron app?

Why would I implement netcat when netcat is already implemented? That's sort of why I'm asking – at least in the circles I move, all the low hanging fruit are already picked, and picking the high fruit seems rare, so I'm interested in which people do and what those fruits are!

Re: Beej's Guide to Network Programming

#59
post #38

Open question: in what fields are you working where you really need to work with actual sockets? Most of the network programming I do fall into one of two categories: 1. A library exists to abstract over the bytes-into-sockets layer, meaning I don't need to deal with it; or 2. A library probably exists to abstract over the bytes-into-sockets layer, but I've chosen to ignore it either because I'm truly ignorant of it,…

Had plenty of use cases where even zmq was overkill. E.g. interfacing a research machine with an OEM machine or a machine to a standalone embedded target, sometimes it makes sense to just serialize, send over a socket, ethernet crossover cable and be done.

Your description makes it sound sort of like temporary code to perform a one-time (or few-times) job, and not something that's running and maintained for years. Is that a fair characterisation or am I misunderstanding?

Re: Beej's Guide to Network Programming

#60

All Beej's guides are fantastic, but if you're interested in Network programming you could do way worse than follow this up with his guide to network concepts. https://beej.us/guide/bgnet0/ If you search Algolia for Beej you'll see his material has been on hacker news numerous times.

Anyone knows of a well-rounded guide that explains the structure of the Internet at large? I know everything listed in the one you linked, but I have a nebulous idea of how anything works after my packet exits my gateway.

So, BGP, autonomous systems, peering agreements and other site-to-site routing protocols that keep the entire Internet infrastructure ticking along.

Post reply on HN