Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reining in the thundering herd: Getting to 80% CPU utilization with Django

blog.clubhouse.com

Post reply on HN
Print view View original

31–40 of 139 posts

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#31

Post by Spivak »

PaywallBuster wrote: post #18
latchkey wrote: post #4

If it is just a backend, why not port it over to one of the myriad of cloud autoscaling solutions that are out there? The opportunity cost of spending time figuring out why only 29 workers are receiving requests over adding new features that generate more revenue, seems like a quick decision. Personally, I just start off with that now in the first place, the development load isn't any greater and the solutions that a…

containers would've solved it one process per container, easy peasy

This doesn’t work so easily with architectures with process pools for workers. So now your app server needs to speak docker (or whatever control plane) to spawn new workers and deal with more complicated IPC. Also the startup time is brutal.

One process per container and multiprocessing is a huge lift most of the time. I’ve done it but it can be a mess because you don’t really have as much a handle on containers than subprocesses because you can only poke them at a distance through the control plane.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#33

Post by TekMol »

IshKebab wrote: post #29
TekMol wrote: post #5

Performance is the only thing that is holding me back to consider Python for bigger web applications. Of the 3 main languages for web dev these days - Python, PHP and Javascript - I like Python the most. But it is scary how slow the default runtime, CPython, is. Compared to PHP and Javascript, it crawls like a snake. Pypy could be a solution as it seems to be about 6x faster on average. Is anybody here using Pypy for…

Typescript is a nicer language than Python in many ways and it doesn't suffer from Python's crippling performance issues or dubious static typing situation. Plus you can run it in a browser so there's only one language to learn.

You cannot run TS in a browser.

You can compile it to JS or to Webassembly. But you can do that with every language.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#34

Post by dilyevsky »

Spivak wrote: post #22
dilyevsky wrote: post #12

Kinda funny they decided paying a ton of money to aws was ok but paying for nginx plus was not

I kinda get that honestly. It’s why I’ll spend $20 without even thinking for take out but not spend $2 for an app. It’s because the cost off the software is way way more than the money. It’s a commitment to actually use it and integrate it, deal with their sales team, talk to purchasing, handle licensing, and introducing friction to replacing it or using tools that don’t integrate well because “well we already pay fo…

Don’t you have to integrate cloud? This whole post is about having to put a bunch of workaround bc the cloud can’t scale apparently

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#36

Post by stu2010 »

Interesting to see this. It sounds like they're not on AWS, given that they mentioned that having 1000 instances for their production environment made them one of the bigger deployments on their hosting provider.

If not for the troubles they experienced with their hosting provider and managing deployments / cutting over traffic, it possibly could have been the cheaper option to just keep horizontally scaling vs putting in the time to investigate these issues. I'd also love to see some actual latency graphs, what's the P90 like at 25% CPU usage with a simple Gunicorn / gevent setup?

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#37

Post by ClumsyPilot »

Spivak wrote: post #22
dilyevsky wrote: post #12

Kinda funny they decided paying a ton of money to aws was ok but paying for nginx plus was not

I kinda get that honestly. It’s why I’ll spend $20 without even thinking for take out but not spend $2 for an app. It’s because the cost off the software is way way more than the money. It’s a commitment to actually use it and integrate it, deal with their sales team, talk to purchasing, handle licensing, and introducing friction to replacing it or using tools that don’t integrate well because “well we already pay fo…

"It’s why I’ll spend $20 without even thinking for take out but not spend $2 for an app."

I pay for apps, its not a healthy attotude

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#38

Post by latchkey »

lddemi wrote: post #20
latchkey wrote: post #4

If it is just a backend, why not port it over to one of the myriad of cloud autoscaling solutions that are out there? The opportunity cost of spending time figuring out why only 29 workers are receiving requests over adding new features that generate more revenue, seems like a quick decision. Personally, I just start off with that now in the first place, the development load isn't any greater and the solutions that a…

Author here. We do and did use autoscaling heavily but at a certain scale we just ran out of headroom on the smaller instance types we were using. Jumping to a much larger instance types meant that we will likely never run into those headroom issues again, plus solves other problems like faster spin up, better sidecar connection pooling and allows for a much higher hit rate on per instance caching.

You were autoscaling a single threaded process. You had 1000 connections coming in and scaling 1000 workers for those connections. Everything was filtered through gunicorn and nginx, which just adds additional latencies and complexity, for no real benefit.

What I'm talking about is just pointing at something like AppEngine, Cloud Functions, etc... (or whatever solution AWS has that is similar) and being done with it. I'm talking about not running your own infrastructure, at all. Let AWS and Google be your devops so that you can focus on building features.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#39

Post by smashed »

lmilcin wrote: post #32

1M requests per minute on 1000 web instances is not an achievement, it is a disaster. It is ridiculous people brag about it. Guys, if you have budget maybe I can help you up this by couple orders of magnitude.

To be honest the article does realize this, first blaming it on the poor hindsight from original developer (co-founder) and in the conclusion about maybe rewriting the whole thing.

It seemed to be all about how to extract the most performance from the lemon they had to deal with.

I found the linked reference really informative too: https://rachelbythebay.com/w/2020/03/07/costly/

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#40

Post by huffmsa »

waprin wrote: post #24
nsizx wrote: post #7

Earlier quoted context omitted.

So much this. Practically any other option is better than Python for web development if you're looking for performance.

Yet YouTube, Instagram, Pinterest, Reddit, Robinhood, DoorDash, and Lyft backend were originally primarily written in Python. What’s funny is that nobody can really deny Python is slow yet somehow the biggest websites in the world were written in it. More proof that Worse Is Better?

In the early stages:

Speed of development is far more important than optimizing CPU usage.

You can fake your way to fast responses with good caching, but there's not really many ways to fake having the best features.

Post reply on HN