Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Old box, dumb code, few thousand connections, no big deal

rachelbythebay.com

Post reply on HN
Print view View original

21–30 of 288 posts

Re: Old box, dumb code, few thousand connections, no big deal

#21

Post by wondringaloud »

sjburt wrote: post #11
thenewwazoo wrote: post #7

But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…

If “real engineering” worked the way you described, we would test planes by filling them full of passengers and flying them around the world. We don’t. We built wind tunnel models, we taxi them around at higher and higher speeds, we put them in machines that wiggle the wings at high loads. The first flight is a little hop and then right back down. Months later there might be a big ceremony with VIPs where the new pla…

She developed a prototype based on a hunch and a whim. Great work, no doubt, but engineering isn't based on intuition and some experience. Real engineering has a goal or specification in mind, and then proves through modeling, analysis, and _finally_ testing that it meets those specs.

Whipping something up and then seeing what it's capable of doesn't qualify.

Re: Old box, dumb code, few thousand connections, no big deal

#22

Post by ivalm »

What WSGI do people recommend for python? I've been using gunicorn but this made me think of alternatives. Quick google search found this benchmark [0], is it really that bjoern is much quicker? It seems all other WSGI are ~ equivalent.

[0] - https://www.appdynamics.com/blog/engineering/a-performance-a...

Re: Old box, dumb code, few thousand connections, no big deal

#23

Post by cgh »

icedchai wrote: post #17
cgh wrote: post #16

Earlier quoted context omitted.

I think it's basically just the equivalent of select(2)? Once upon a time, all network servers were written this way. They were pretty fast, too.

Most select(2) based servers were single threaded, however. Not that this is necessarily a bad thing.

Yes, you are right, of course. I shamefully misread the article. It's closer to listen/accept/fork.

Re: Old box, dumb code, few thousand connections, no big deal

#24

Post by modal-soul »

thenewwazoo wrote: post #7

But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…

Saw an interesting talk just about this topic [1]

The gist of it was: Other engineering disciplines use the techniques you've mentioned because of the the costs, both time and money, associated with getting it wrong.

Software engineering lends itself to different methods of development and construction, as the costs associated with getting it wrong or making changes after the fact are much lower. (For most applications, anyways).

As such, (this definition would be another sticking point) these less rigid methods should still be considered engineering, with engineering being a balancing of resources with outcomes, not fixation on mathematical models.

[1]: https://www.youtube.com/watch?v=RhdlBHHimeM

Re: Old box, dumb code, few thousand connections, no big deal

#25

Post by zaphar »

wondringaloud wrote: post #19
thenewwazoo wrote: post #7

But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…

I'm glad someone else sees this the way I do. What the blog writer did was tinker with something. They didn't engineer it. I was a mechanical engineer prior to switching to software. As a general rule, the things we do in software are very distant from engineering.

What the author did was build a prototype to demonstrate and explore what was possible. That is exactly what engineers do when exploring a problem.

Re: Old box, dumb code, few thousand connections, no big deal

#27

Post by pdonis »

rachelbythebay wrote: post #8
simfoo wrote: post #4

Honest question: why go through the hassle of multiplexing waiting in a single thread only to dispatch to a thread per client anyway? Simply using blocking IO for the clients in those threads should be much simpler right?

If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle, doing other housekeeping, or cleanly shutting down the whole thing in a timely fashion. When I ^C the server, it sends the same wake condvar-poke but it twiddles the flags so the worker shuts down instead.

Yes, using epoll with nonblocking I/O is better than blocking I/O on each worker thread. But that basically means that you are doing asynchronous programming--i.e., the exact same thing that the wonky Python/Gunicorn stack you described is doing! You're just doing it with better attention to important details.

Here, to me, is the key item:

The "listener" thread owns all of the file descriptors (listeners and clients both), and manages a single epoll set to watch over them.

This is exactly what any async server does: it centralizes all the file descriptor management and handling in one place, and only uses workers (whether they are threads or "green threads" or whatever) to read from/write to fd's that are marked as ready in the epoll set.

For your case, unless I'm misreading something, what the workers are doing in between the read/write is CPU intensive (or at least it's CPU work and not I/O work, even though it's not very "intensive" CPU work), so actual OS threads are a better choice for the workers since you can't rely on cooperative scheduling.

If what the workers were doing was I/O work (for example, sending a request to a remote database and waiting for a response), "green threads" would work fine (since their only real purpose would be to organize the I/O--the actual fd's are going to be managed by the central server that manages all the fd's and checks which ones are ready for read/write). And one definitely should not try to run "green threads" for the same server in multiple O/S threads (or worse still, multiple OS processes). For an I/O bound server, one shouldn't need to anyway.

Re: Old box, dumb code, few thousand connections, no big deal

#28

Post by pdonis »

cgh wrote: post #23
icedchai wrote: post #17

Earlier quoted context omitted.

Most select(2) based servers were single threaded, however. Not that this is necessarily a bad thing.

Yes, you are right, of course. I shamefully misread the article. It's closer to listen/accept/fork.

> It's closer to listen/accept/fork.

No, it isn't, it's doing the same thing as a select(2) server would do, except it's using epoll to avoid scaling issues when you have a lot of file descriptors in the polling set. The only difference is that the workers are doing something that requires CPU, not I/O, so OS threads are being used for them (a single threaded server would be fine if the workers were just doing more I/O, like sending a request to a remote database and waiting for a response). But the worker threads are not doing any I/O management at all; they read from or write to an fd only when the central server that is calling epoll tells them to. In the listen/accept/fork model, the central server forgets about an fd once it has passed it to a handler process, and the handler process using blocking I/O.

Re: Old box, dumb code, few thousand connections, no big deal

#29

Post by TheAdamAndChe »

ivalm wrote: post #22

What WSGI do people recommend for python? I've been using gunicorn but this made me think of alternatives. Quick google search found this benchmark [0], is it really that bjoern is much quicker? It seems all other WSGI are ~ equivalent. [0] - https://www.appdynamics.com/blog/engineering/a-performance-a...

I've used uwsgi a few times before. It is a pain to set up every time, but once it's configured, it works fine for the small, basic apps that I make.

Re: Old box, dumb code, few thousand connections, no big deal

#30

Post by Minor49er »

thenewwazoo wrote: post #7

But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…

I thought the author was referring to the kernel threading, not necessarily to the experiment that was put together
Post reply on HN