should try this in erlang/elixir. i’m gonna bet you it could handle hundred of thousands of connections on a beefy machine (an million of connections w/ optimizations)
Old box, dumb code, few thousand connections, no big deal
41–50 of 288 posts
Re: Old box, dumb code, few thousand connections, no big deal
#42But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…
I'm glad someone else sees this the way I do. What the blog writer did was tinker with something. They didn't engineer it. I was a mechanical engineer prior to switching to software. As a general rule, the things we do in software are very distant from engineering.
When saying "this is engineering" the article refers to all the engineering effort at the OS level that went into making this possible.
As the title says, this is dumb code. But it makes use of years of engineering effort to deliver a result which can get you very far without thinking about the low level.
Re: Old box, dumb code, few thousand connections, no big deal
#43Earlier quoted context omitted.
What the author did was build a prototype to demonstrate and explore what was possible. That is exactly what engineers do when exploring a problem.
When you hear "Engineer", you often think of calculus, statistics, formal testing, requirements gathering, documentation, repeatable results, etc... along with a fundamental understanding of the problem space and possible solutions. I think this is akin to NASA working on the Apollo program vs. someone in their garage attempting to build a go-cart for the first time. When you just slap things together and see if they…
Well I don't - maybe you do. I think more along the lines of "ingenuity" as described here: https://interestingengineering.com/the-origin-of-the-word-en...
Re: Old box, dumb code, few thousand connections, no big deal
#44Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit.
It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second.
Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. Or if they get a lot of value from libraries that are unique to the Python ecosystem. Or if they care more about quickly iterating on features than driving down server costs.
I agree that there's a point where it stops making sense, and there are plenty of engineers who don't recognize when they're past that point because they keep doubling down on sunk costs and things they're familiar with. But let's not be too quick to assume people are in that camp when we don't know all the tradeoffs they're facing.
Re: Old box, dumb code, few thousand connections, no big deal
#45should try this in erlang/elixir. i’m gonna bet you it could handle hundred of thousands of connections on a beefy machine (an million of connections w/ optimizations)
why tho? In real life if you're in need of handling millions of users per second, I bet you're already part of FAANG, at which point you simply open offices in each country and deploy local servers.
Re: Old box, dumb code, few thousand connections, no big deal
#46Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit. It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second. Even if they do need to serve to…
> Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages.
How hard is it to get up to speed on any other tech stack? ASP.NET Core is extremely fast and the learning curve is close to none, for example.
If someone was able to wrap his head around backend development with Python I'm pretty sure they have the mental fortitude to onboard a tech stack that doesn't suffer from major performance problems.
Re: Old box, dumb code, few thousand connections, no big deal
#47Earlier quoted context omitted.
Even in 2007 when I was starting to cut my teeth on larger web traffic there was a lot of discussion around serving 10k concurrent connections. I remember being blown away by a graph showing high throughput at 70k concurrent by a YAWS server, and started following Erlang as a result.
In 2004 we had eDonkey servers handling 1M concurrent connections. Code, changelog and a bit of history: https://lugdunum.shortypower.org/kiten.html
Oh, wow, I vaguely remember running across eDonkey at one point. I don't think I ever realized it could handle that kind of load. I was in a position to mostly stick with Apache for various non technical reason basically for long enough that eventually Apache got to the point that it could handle the traffic I needed to deal with especially with a CDN in front of it.
Re: Old box, dumb code, few thousand connections, no big deal
#48should try this in erlang/elixir. i’m gonna bet you it could handle hundred of thousands of connections on a beefy machine (an million of connections w/ optimizations)
why tho? In real life if you're in need of handling millions of users per second, I bet you're already part of FAANG, at which point you simply open offices in each country and deploy local servers.
nope. it’s about what’s possible.
Re: Old box, dumb code, few thousand connections, no big deal
#49Honest question: why go through the hassle of multiplexing waiting in a single thread only to dispatch to a thread per client anyway? Simply using blocking IO for the clients in those threads should be much simpler right?
If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle, doing other housekeeping, or cleanly shutting down the whole thing in a timely fashion. When I ^C the server, it sends the same wake condvar-poke but it twiddles the flags so the worker shuts down instead.
> If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle
Totally possible with another thread acting as watchdog timer and sending a signal which causes the read to return with EINTR which can then check a flag whether it should retry or abort. And that's for file IO. For socket IO you can just set it to non-blocking.
Re: Old box, dumb code, few thousand connections, no big deal
#50Rachel presumably wrote her server in a reasonable language like C++ (though I don't see a link to her source), but when I wrote httpdito⁰ ¹ ² I wrote it in assembly, and it can handle 2048 concurrent connections on similarly outdated hardware despite spawning an OS process per connection, more than one concurrent connection per byte of executable†. (It could handle more, but I had to set a limit somewhere.) It just serves files from the filesystem. It of course doesn't use epoll, but maybe it should — instead of Rachel's 50k requests per second, it can only handle about 20k or 30k on my old laptop. IIRC I wrote it in one night.
It might sound like I'm trying to steal her thunder, but mostly what I'm trying to say is she is right. Listen to her. Here is further evidence that she is right.
As I wrote in https://gitlab.com/kragen/derctuo/blob/master/vector-vm.md, single-threaded nonvectorized C wastes on the order of 97% of your computer's computational power, and typical interpreted languages like Python waste about 99.9% of it. There's a huge amount of potential that's going untapped.
I feel like with modern technologies like LuaJIT, LevelDB, ØMQ, FlatBuffers, ISPC, seL4, and of course modern Linux, we ought to be able to do a lot of things that we couldn't even imagine doing in 2005, because they would have been far too inefficient. But our imaginations are still too limited, and industry is not doing a very good job of imagining things.
—
⁰ http://canonical.org/~kragen/sw/dev3/server.s
¹ http://canonical.org/~kragen/sw/dev3/httpdito-readme
² https://news.ycombinator.com/item?id=6908064
† It's actually bloated up to 2060 bytes now because I added PDF and CSS content-types to it, but you can git clone the .git subdirectory and check out the older versions that were under 2000 bytes.