Old box, dumb code, few thousand connections, no big deal • Hacker News Board

Re: Old box, dumb code, few thousand connections, no big deal

#121

Post by mapgrep » Fri, May 08, 2020, 3:07 AM UTC

xakahnx wrote: post #101

The general attitude here reminds a bit of the following post from the architect of the Varnish proxy. I think the attitude comes down to the fact that modern kernels and in general the foundations of network programming are pretty strong. We should trust them more. https://varnish-cache.org/docs/5.2/phk/notes.html

I would argue you can say the same about foundations of rdbms systems as well. People build similar elaborate caches around those, for example rails has a “russian doll” cache layer built in, not realizing how much time has gone into developing well tuned caches within the database system itself, which simply needs to be allocated sufficiently large ram.

Re: Old box, dumb code, few thousand connections, no big deal

#122

Post by umvi » Fri, May 08, 2020, 3:08 AM UTC

highfrequency wrote: post #90

kragen wrote: post #50

Rachel presumably wrote her server in a reasonable language like C++ (though I don't see a link to her source), but when I wrote httpdito⁰ ¹ ² I wrote it in assembly, and it can handle 2048 concurrent connections on similarly outdated hardware despite spawning an OS process per connection, more than one concurrent connection per byte of executable†. (It could handle more, but I had to set a limit somewhere.) It just…

> single-threaded nonvectorized C wastes on the order of 97% of your computer's computational power Can you elaborate on what this means exactly? For example, is there some reasonable C code that runs 33 times slower than some other ideal code? In what sense are we wasting 97% of our computer's computational power?

I think it's referring to simd instructions [0].

I'm still kind of a newb myself but from what I understand these are special CPU instructions that allow you execute the same instruction in parallel against multiple data points. This allows you to eke out a lot more performance. It's how simdjson[1] is able to outperform all other C++ json parsers.

[0] https://en.wikipedia.org/wiki/SIMD

[1] https://github.com/simdjson/simdjson

Re: Old box, dumb code, few thousand connections, no big deal

#123

Post by pmoleri » Fri, May 08, 2020, 3:11 AM UTC

rumanator wrote: post #46

bjt wrote: post #44

Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit. It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second. Even if they do need to serve to…

> Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. How hard is it to get up to speed on any other tech stack? ASP.NET Core is extremely fast and the learning curve is close to none, for example. If someone was able to wrap his head around backe…

Learning curve close to none? I like that many times there's "the way" of doing things, but when I have to do something slightly outside the recommended way, I feel that the effort required outweights all the benefits.

Re: Old box, dumb code, few thousand connections, no big deal

#124

Post by sytringy05 » Fri, May 08, 2020, 3:15 AM UTC

Compare this to the Enterprise platform I am dealing with on a current project, which has 4 x ec2 nodes with 8 CPUs and 32 Gb RAM.

I can DoS it with a single java client running 50 threads. If I use 100 the p95 shoots up to 30 - 40 seconds.

But the kicker is that no one (other than me) really cares. 50 concurrent threads is probably around the peak load it will get in prod, and various people involved think why bother trying to fix it?

It's driving me nuts.

Thanks for listening

Re: Old box, dumb code, few thousand connections, no big deal

#125

Post by 29athrowaway » Fri, May 08, 2020, 3:15 AM UTC

rumanator wrote: post #46

bjt wrote: post #44

Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit. It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second. Even if they do need to serve to…

> Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. How hard is it to get up to speed on any other tech stack? ASP.NET Core is extremely fast and the learning curve is close to none, for example. If someone was able to wrap his head around backe…

1) .NET APIs change very frequently.

2) C# is a very verbose language, that requires a lot of typing.

3) F#, the best language in .NET, is largely ignored by the .NET community.

Re: Old box, dumb code, few thousand connections, no big deal

#126

Post by samatman » Fri, May 08, 2020, 3:16 AM UTC

mdszy wrote: post #80

bjt wrote: post #44

Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit. It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second. Even if they do need to serve to…

Twitter ran on Rails until 10 million users.

Ah yes, the Fail Whale era.

Re: Old box, dumb code, few thousand connections, no big deal

#127

Post by peterwwillis » Fri, May 08, 2020, 3:21 AM UTC

The premise is that people don't understand their system, or software, and need to be told how it works. So you have an admittedly ignorant person telling other ignorant persons that there are possibilities out there. That's... not really useful.

What is useful is to spend 2 days researching massively concurrent network applications, and find the ones that were already written, and see how they evolved over time. For the most part, it's just understanding your operating system and network protocols. Once you learn how they all work, the answers come quickly. This problem has been solved many times.

But for the most part, none of you need to know how this stuff works. You can write the crappiest network server in the world, and there are still so many other components you can wrap around it to make it scale that you never even need to get close to network optimization. So you may have to run 10 instances of your app; who cares? We have virtually unlimited everything these days. The answer to a poorly performing app is "just throw more cloud at it".

Also, I feel the need to remind everyone that single app instance handling 100K+ concurrent connections is a terrible fucking idea. What happens when the app crashes? What happens when the hardware dies? What happens when you need to, like, upgrade/restart the app? Several million SYN packets in 100ms, and default kernel TIME_WAIT settings, do not make great bedfellows.

Re: Old box, dumb code, few thousand connections, no big deal

#128

Post by flyinglizard » Fri, May 08, 2020, 3:28 AM UTC

itwy wrote: post #116

flyinglizard wrote: post #108

Earlier quoted context omitted.

The only reason to use Python for anything more than few hundreds lines worth of utility is if you're working on a codebase that's already in Python, and even then it's debatable. There simply isn't an excuse for using Python for any infrastructure. It does nothing particularly well - or even right - other than very purpose-specific scripting. It can tie things together well enough. And your codebase becomes a liabil…

Any which language do you use for web apps if I may ask? Let me guess. Rust.

I primarily do all sorts of systems programming, but the few web backends I did were C# on Azure Functions and it was spectacular. I’m a very big proponent of Microsoft’s tooling and language development.

The difference in productivity between C# and Python is such that they might as well have been developed by different civilizations.

(I have occasionally used Python since about 2003 and C# since 2018)

Re: Old box, dumb code, few thousand connections, no big deal

#129

Post by freeone3000 » Fri, May 08, 2020, 3:36 AM UTC

flyinglizard wrote: post #109

stev_stev wrote: post #55

Earlier quoted context omitted.

This is where my head always goes as someone who does mostly Asp.Net Core. Why is it always between something like C++ or something like Python? Nowadays with middleware and endpoint routing, asp.net core can be almost as simple as flask (even if that’s not idiomatic or what the docs show you).

Or Azure Functions if you want to go the serverless route. Coupled with Visual Studio, you can build high quality, large scale apps at about the same effort of blinking a LED on Arduino.

Are we using the same Azure Functions? I spend more time messing with functions.json and determining binding types and waiting five minutes for error messages to come in than I ever did when I simply provisioned an app service and deployed code.

Re: Old box, dumb code, few thousand connections, no big deal

#130

Post by pmoleri » Fri, May 08, 2020, 3:36 AM UTC

throwaway894345 wrote: post #38

Unfortunately this approach precludes GIL languages like Python. If you're able to use a language/runtime that is amenable to multithreading, then using a thread per connection works fine for most use cases (and it's probably easier than using whatever async/await interface your language has).

> probably easier than using whatever async/await interface your language has

Is it? I haven't used threads directly in a while, but I remember dealing with sinchronization issues. Problems that just don't exist in single threaded node with async await.

I find the async Promise or Task to be a more useful abstraction than the thread. Although, you need threads or a task dispatcher with a pool of threads if you need to run cpu intensive stuff.