Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reining in the thundering herd: Getting to 80% CPU utilization with Django

blog.clubhouse.com

Post reply on HN
Print view View original

121–130 of 139 posts

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#121

Post by themarkers »

lmilcin wrote: post #44
smashed wrote: post #39

Earlier quoted context omitted.

To be honest the article does realize this, first blaming it on the poor hindsight from original developer (co-founder) and in the conclusion about maybe rewriting the whole thing. It seemed to be all about how to extract the most performance from the lemon they had to deal with. I found the linked reference really informative too: https://rachelbythebay.com/w/2020/03/07/costly/

I don't know Python or how complex their domain is but the number of workers suggests to me it is not that complex and their application spends most of its time switching contexts and in inefficient frameworks. Per my experience most applications that mostly serve documents from databases should be able to take on at least 10k requests per second on a single node. this is 600k requests per minute on one node, compare…

You are right if it’s a technical driving thing. But most are not that case.

CPU is much cheaper for scaling a business.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#122

Post by void_mint »

tored wrote: post #113
void_mint wrote: post #88

Earlier quoted context omitted.

I'm not really seeing what's silly about what you're linking. I'm seeing Java blowing the doors off of PHP, which matches my previous assertion.

Much better comparison for typical web load, what this discussion is about. https://www.techempower.com/benchmarks/#section=data-r20&hw=... Java wins as expected, but a typical setup with Spring versus the typical top PHP frameworks isn't blowing the doors off. Typical Python + Django is far behind, as someone pointed out. However what we can see in the diagrams is that ORM layers, regardless of language, are more ex…

In the link you provided PHP is 53% as fast as Spring. I would argue that’s an insane performance reduction.

If Python weren’t fast “enough”, would it be used to power so many successful web backends?

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#123

Post by tored »

void_mint wrote: post #122
tored wrote: post #113

Earlier quoted context omitted.

Much better comparison for typical web load, what this discussion is about. https://www.techempower.com/benchmarks/#section=data-r20&hw=... Java wins as expected, but a typical setup with Spring versus the typical top PHP frameworks isn't blowing the doors off. Typical Python + Django is far behind, as someone pointed out. However what we can see in the diagrams is that ORM layers, regardless of language, are more ex…

In the link you provided PHP is 53% as fast as Spring. I would argue that’s an insane performance reduction. If Python weren’t fast “enough”, would it be used to power so many successful web backends?

That is for spring-webflux, is that a typical spring set up for web today?

I haven’t coded spring for a few years now, but I was thinking about the traditional spring setup that most use and that is comparable.

You can of course use Python successfully, my argument is that it is easier with PHP, not that it is not possible with Python.

It is a similar argument compared with Java, it is easier with PHP than Java in a web context. Java has other benefit thats fits better for web services IMHO, general higher performance is one.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#124

Post by jstrong »

ram_rar wrote: post #105

> Python's model of running N separate processes for your app is not as unreasonable as people might have you believe! You can achieve reasonable results this way, with a little digging. I have been through this journey, we eventually migrated to Golang and it saved a ton of money and firefighting time. Unfortunately, python community hasnt been able to remove GIL, it has its benefits (especially for single threaded…

I read the article and could not believe that was their takeaway. sometimes people are determined to vindicate their technology choices, no matter what.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#125

Post by dilyevsky »

spullara wrote: post #28
dilyevsky wrote: post #12

Kinda funny they decided paying a ton of money to aws was ok but paying for nginx plus was not

The difference people see, as far as I can tell, is that AWS is charging you cost+ and pure software companies need to charge for value or die.

Maybe for barebones compute they’re cost+ but I don’t think that’s really true for other services. For example traffic should cost effectively zero to them but they charge a huge premium. Some other managed services also appear to use value based pricing

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#126

Post by toast0 »

thaumasiotes wrote: post #119
toast0 wrote: post #110

Earlier quoted context omitted.

> That... doesn't have much to do with the thundering herd problem. It also doesn't make much sense as a concept on its own merits -- say you come in to work and your inbox is full enough for three inboxes. Does that fact, in itself, mean that you decide you're done for the day? No, it just means you have a much longer queue to work through than usual. If my SLA is 24 hour response time, and the inbox is FIFO, and I…

OK. But that's not a problem of a thundering herd. It's a problem that you have more incoming work than you are theoretically able to handle even if you stay in continuous operation. Your problem is solved by adding the capacity to do more work. The thundering herd is solved by purposefully desynchronizing incoming work requests.

Oh, I agree it's not thundering herd, but it is a real problem. Especially if you start getting retries after the first requests timed out. Some sort of backoff with jitter to avoid synchronized retries helps, but what really helps is dropping or not accepting requests when the processing will not be timely. That's simple to say, but not always simple to do.

Adding capacity is also simple to say, but not always simple to do. And there can be a large difference between the capacity needed to handle a cold start at peak vs the capacity needed for peak under regular operations.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#127

Post by aeyes »

stuaxo wrote: post #117
aeyes wrote: post #83

Earlier quoted context omitted.

According to the article they have a monolithic Django application so this will have at least a couple of seconds start-up time. That is not a good match for Cloud Functions. Django also has in-memory caches, for example for templates which can be extremely slow (seconds) and CPU intensive to render. So you really don't want to have AWS or Google restart your application on AppEngine whenever they feel like it.

Silly question, but under nginx or apache do django instances persist or are they recreated for every new request?

The standard gunicorn configuration (and the one shown in the blog post) never restarts worker processes.

gunicorn has an option --max-requests to restart every X requests but unless you have unfixable memory leaks there is no reason to do this.

Nginx can't directly run WSGI applications, you can do it with Nginx Unit which also never restarts processes.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#128

Post by midrus »

zinodaur wrote: post #92
TekMol wrote: post #5

Performance is the only thing that is holding me back to consider Python for bigger web applications. Of the 3 main languages for web dev these days - Python, PHP and Javascript - I like Python the most. But it is scary how slow the default runtime, CPython, is. Compared to PHP and Javascript, it crawls like a snake. Pypy could be a solution as it seems to be about 6x faster on average. Is anybody here using Pypy for…

I feel like people let go of perf too easily. When using something like Golang, I have apps doing normal CRUD-ish queries at 10k QPS, on 32c/64g machines. For most web apps, 10k QPS is much more than they will ever see, and the fact that it is all done in a single process means you could do really cool things with in-memory datastructures. Instead, every single web app is written as a distributed system, when almost…

I could rephrase you comment as why would anyone use Go when I could just use assembler or C and keep all into a single node.

People don't use python because they want performance. People use python because of productivity, frameworks, libraries, documentation, resources and ecosystems. Most projects don't even need 10k qps, but instead most projects do need an ORM, a migrations system, authentication, sessions, etc. Python has bottle tested tools and frameworks for this.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#129

Post by _6pvr »

tored wrote: post #115
void_mint wrote: post #47

Earlier quoted context omitted.

Serverless is too overloaded a term to have any meaning. I'm not really seeing how Python or PHP "scales infinitely" in any way that C#, Java, C++ couldn't.

PHP is usually easier to scale because it just a matter of how many webservers. e.g. apache or nginx, you choose to deploy. This also possible with other platforms, but can be a bit trickier to get right. For large PHP setups it is usually the number of database connections that is the limiting factor, however that is why historically the replicated MySQL databases was such a good fit for PHP, thus only creating a li…

> For large PHP setups it is usually the number of database connections that is the limiting factor,

For pretty much every modern programming language, IO is the bottleneck over everything else.

To save you some time, there are practically no metrics in which I think PHP beats another programming language other than maturity, and even then, not really.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#130

Post by quickthrower2 »

latchkey wrote: post #94
Nextgrid wrote: post #80

Earlier quoted context omitted.

> you get to fire the devops person, which saves $150k+ a year. You are deluded or extremely short-sighted if you believe you can actually fire the devops guy. From my experience, the more you stray away from the conventional "dedicated server" paradigm the more you need a devops guy and you are in a very precarious position if you do fire him and something goes wrong.

You don't hire the devops person until you've scaled to the point that you need one. Additionally, your thought of having my company held hostage by a single devops person is terrifying. Now you need two of them, which is even more expensive. It is a great way to bootstrap a company by saving on a salary (or two) that can honestly be engineered out for a lot of SASS businesses. It worked super well for us... and call…

If you don’t have a devops person, then you end up with developers pitching in to fill that void. That’s OK and may be desirable but it is still a cost.
Post reply on HN