Old box, dumb code, few thousand connections, no big deal • Hacker News Board

Re: Old box, dumb code, few thousand connections, no big deal

#31

Post by dimatura » Thu, May 07, 2020, 10:57 PM UTC

ivalm wrote: post #22

What WSGI do people recommend for python? I've been using gunicorn but this made me think of alternatives. Quick google search found this benchmark [0], is it really that bjoern is much quicker? It seems all other WSGI are ~ equivalent. [0] - https://www.appdynamics.com/blog/engineering/a-performance-a...

I've mainly used gunicorn and uwsgi, without seeing any large differences. But my apps have fairly light network requirements.

It seems like the new trend for Python servers is ASGI (https://asgi.readthedocs.io/en/latest/), e.g. as in uvicorn (https://www.uvicorn.org/).

Re: Old box, dumb code, few thousand connections, no big deal

#32

Post by underdeserver » Thu, May 07, 2020, 10:57 PM UTC

pdonis wrote: post #27

rachelbythebay wrote: post #8

Earlier quoted context omitted.

If you get stuck in read(), you can't do neat things like waking up when it's time to kick a client for being idle, doing other housekeeping, or cleanly shutting down the whole thing in a timely fashion. When I ^C the server, it sends the same wake condvar-poke but it twiddles the flags so the worker shuts down instead.

Yes, using epoll with nonblocking I/O is better than blocking I/O on each worker thread. But that basically means that you are doing asynchronous programming--i.e., the exact same thing that the wonky Python/Gunicorn stack you described is doing! You're just doing it with better attention to important details. Here, to me, is the key item: The "listener" thread owns all of the file descriptors (listeners and clients…

Better attention to important details is what makes or breaks a library.

Re: Old box, dumb code, few thousand connections, no big deal

#33

Post by 205guy » Thu, May 07, 2020, 11:00 PM UTC

jpitz wrote: post #9

Would love to see the code.

Funny, when she revealed what the threads were doing, I wanted to search for Shakespeare in the output.

Re: Old box, dumb code, few thousand connections, no big deal

#34

Post by DizzyDoo » Thu, May 07, 2020, 11:05 PM UTC

ivalm wrote: post #22

What WSGI do people recommend for python? I've been using gunicorn but this made me think of alternatives. Quick google search found this benchmark [0], is it really that bjoern is much quicker? It seems all other WSGI are ~ equivalent. [0] - https://www.appdynamics.com/blog/engineering/a-performance-a...

I'm using Waitress in production: https://docs.pylonsproject.org/projects/waitress/en/stable/ It's main thing is that it's really simple, worth a look if you're using Django.

Re: Old box, dumb code, few thousand connections, no big deal

#35

Post by EdwardDiego » Thu, May 07, 2020, 11:06 PM UTC

ardit33 wrote: post #13

thenewwazoo wrote: post #7

But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…

What is stress load testing then? Most large companies do perform something to this.... Most large companies do some stress load testing first for any new system. Then they release to few percent of the users (less than 10%) and gradually increase to the rest of the userbase. I worked at Spotify, when Tidal was launched. They had failed to do proper capacity testing, and the service failed under the load the first we…

> It is remarkable how many tech companies have managed to stay mostly up with very few outages, given this whole pandemic situation, where everybody is online.

I must admit that it was very gratifying watching all our engineering decisions pay off when our system started seeing record high traffic every day and scaled effortlessly and with no outages - the traffic that came with lockdown is, even on our slower days, twice our planned for and tested for high water mark.

But it took us about 4 years of consistent effort in changing our organisational mindset all the way through, devs, testers, product managers, c-suite members, to get here. 4 years ago, we would've been waking everyone up and going without sleep for a couple of days trying to get something back online, then fixing the next system down the line that failed because of the load, then the next one, and then writing long post-mortems for our business team.

Re: Old box, dumb code, few thousand connections, no big deal

#36

Post by aeyes » Thu, May 07, 2020, 11:07 PM UTC

milesvp wrote: post #10

kccqzy wrote: post #3

The C10k problem was challenging around the turn of the century. I suppose it's now not. I wonder how much CPU would be saved using an event-based architecture.

Even in 2007 when I was starting to cut my teeth on larger web traffic there was a lot of discussion around serving 10k concurrent connections. I remember being blown away by a graph showing high throughput at 70k concurrent by a YAWS server, and started following Erlang as a result.

In 2004 we had eDonkey servers handling 1M concurrent connections.

Code, changelog and a bit of history: https://lugdunum.shortypower.org/kiten.html

Re: Old box, dumb code, few thousand connections, no big deal

#37

Post by Alupis » Thu, May 07, 2020, 11:08 PM UTC

zaphar wrote: post #25

wondringaloud wrote: post #19

Earlier quoted context omitted.

I'm glad someone else sees this the way I do. What the blog writer did was tinker with something. They didn't engineer it. I was a mechanical engineer prior to switching to software. As a general rule, the things we do in software are very distant from engineering.

What the author did was build a prototype to demonstrate and explore what was possible. That is exactly what engineers do when exploring a problem.

When you hear "Engineer", you often think of calculus, statistics, formal testing, requirements gathering, documentation, repeatable results, etc... along with a fundamental understanding of the problem space and possible solutions.

I think this is akin to NASA working on the Apollo program vs. someone in their garage attempting to build a go-cart for the first time.

When you just slap things together and see if they work - are you really engineering? Can you exactly repeat the process and achieve exactly the same result every time?

I think we often cross "research and development" with "engineering". Exploring a problem space and tinkering with concepts isn't engineering. Taking what you've learned, planning out and executing a solution to a precise set of requirements, and being able to repeat your steps and achieve those results again and again - is engineering.

Re: Old box, dumb code, few thousand connections, no big deal

#38

Post by throwaway894345 » Thu, May 07, 2020, 11:14 PM UTC

Unfortunately this approach precludes GIL languages like Python. If you're able to use a language/runtime that is amenable to multithreading, then using a thread per connection works fine for most use cases (and it's probably easier than using whatever async/await interface your language has).

Re: Old box, dumb code, few thousand connections, no big deal

#39

Post by th0ma5 » Thu, May 07, 2020, 11:17 PM UTC

throwaway894345 wrote: post #38

Unfortunately this approach precludes GIL languages like Python. If you're able to use a language/runtime that is amenable to multithreading, then using a thread per connection works fine for most use cases (and it's probably easier than using whatever async/await interface your language has).

True that the GIL prevents the specific expressed pattern you're talking about, but many have been able to do multiple threads and processes with Python itself and also things like gunicorn or uwsgi.

Re: Old box, dumb code, few thousand connections, no big deal

#40

Post by ivalm » Thu, May 07, 2020, 11:19 PM UTC

dimatura wrote: post #31

ivalm wrote: post #22

What WSGI do people recommend for python? I've been using gunicorn but this made me think of alternatives. Quick google search found this benchmark [0], is it really that bjoern is much quicker? It seems all other WSGI are ~ equivalent. [0] - https://www.appdynamics.com/blog/engineering/a-performance-a...

I've mainly used gunicorn and uwsgi, without seeing any large differences. But my apps have fairly light network requirements. It seems like the new trend for Python servers is ASGI ( https://asgi.readthedocs.io/en/latest/ ), e.g. as in uvicorn ( https://www.uvicorn.org/ ).

But from what I understand this replaces gevent, you still use gunicorn to manage workers?