Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reining in the thundering herd: Getting to 80% CPU utilization with Django

blog.clubhouse.com

Post reply on HN
Print view View original

71–80 of 139 posts

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#71

Post by chimen »

lmilcin wrote: post #60
taylorhughes wrote: post #55

Earlier quoted context omitted.

(CH employee here) The job of the cofounder is to create a thing that people want, which has nothing to do with performance. The first goal is capturing lightning in a bottle with social products. Performance doesn’t matter until the lightning is there, and 99%+ of the time you never have to worry about performance, because you don’t get the lightning. So, probably the correct choice is leveraging the tech stack that…

This is just silly excuse. The job of the cofounder is also to anticipate possible risks. And building your company on an astronomically inefficient technology sounds like a huge risk to me. Those 1000s of servers are probably a very significant cost with such small technical staff. Just by choosing the right technology for the problem, most of that cost could have been avoided. Django has nothing special in it that…

So Django is an "astronomically inefficient technology"? I would just stop if I were you.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#72

Post by luckycharms810 »

lmilcin wrote: post #32

1M requests per minute on 1000 web instances is not an achievement, it is a disaster. It is ridiculous people brag about it. Guys, if you have budget maybe I can help you up this by couple orders of magnitude.

This is a comically yet incredibly common engineering bad take. When you run a company there is only one question to answer, one north star - does it make money ?

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#73

Post by [deleted] »

stingraycharles wrote: post #19

Tangent, but I always had a different understanding of the “thundering herd” problem; that is, if a service is down for whatever reason, and it’s brought back online, it immediately grinds to a halt again because there are a bazillion requests waiting to be handled. And the solution to this problem is to slowly, rate-limited, bring the service back online, rather than letting the whole thundering herd go through the…

[deleted]

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#74

Post by vilified »

lmilcin wrote: post #60
taylorhughes wrote: post #55

Earlier quoted context omitted.

(CH employee here) The job of the cofounder is to create a thing that people want, which has nothing to do with performance. The first goal is capturing lightning in a bottle with social products. Performance doesn’t matter until the lightning is there, and 99%+ of the time you never have to worry about performance, because you don’t get the lightning. So, probably the correct choice is leveraging the tech stack that…

This is just silly excuse. The job of the cofounder is also to anticipate possible risks. And building your company on an astronomically inefficient technology sounds like a huge risk to me. Those 1000s of servers are probably a very significant cost with such small technical staff. Just by choosing the right technology for the problem, most of that cost could have been avoided. Django has nothing special in it that…

You sound just like the average sports fan commenting after a match about what x player should have done, shouldn't have done, blame it on decisions, style of the trainer, owner etc.. But you're just that.. a fan yapping about how they could do better.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#75

Post by yuliyp »

lmilcin wrote: post #32

1M requests per minute on 1000 web instances is not an achievement, it is a disaster. It is ridiculous people brag about it. Guys, if you have budget maybe I can help you up this by couple orders of magnitude.

Knowing nothing else, it's hard to know if this is good or not. It's 16 requests per second. Are those requests something like "Render a support article" or are they "Give the user a ranked feed of what they should see on their home screen"? Is most of the logic run by the web server or some combination of app servers / backend services behind it? What kind of hardware does the web server have?

All of those would affect the answer, and would preclude being able to guarantee "up this by couple orders of magnitude"

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#76

Post by luhn »

Unfortunately HAProxy doesn't buffer requests*, which is necessary for a production deployment of gunicorn. And for anybody using AWS, ALB doesn't buffer requests either. Because of this I'm actually running both HAProxy and nginx in front of my gunicorn instances—nginx in front for request buffering and HAProxy behind that for queuing.

If anybody is interested, I've packaged both as Docker containers:

HAProxy queuing/load shedding: https://hub.docker.com/r/luhn/spillway

nginx request buffering: https://hub.docker.com/r/luhn/gunicorn-proxy

* It does have an http_buffer_request option, but this only buffers the first 8kB (?) of the request.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#77

Post by ambicapter »

taylorhughes wrote: post #69
Ozzie_osman wrote: post #58

Earlier quoted context omitted.

Yea you are right. It could be a service being down and requests piling up, or a cache key expiring and many processes trying to regenerate the value at the same time, etc. I think the article just used this phrase to describe something else. (Great article otherwise).

Phrase borrowed from excellent uWSGI docs https://uwsgi-docs.readthedocs.io/en/latest/articles/Seriali...

Funny reading this comment after reading the article

> So many options meant plenty of levers to twist around, but the lack of clear documentation meant that we were frequently left guessing the true intention of a given flag.

And then reading your link, they complain >inside the docs< that the docs aren't complete. I have no idea what to believe anymore :D

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#78

Post by latchkey »

ddorian43 wrote: post #68
latchkey wrote: post #38

Earlier quoted context omitted.

You were autoscaling a single threaded process. You had 1000 connections coming in and scaling 1000 workers for those connections. Everything was filtered through gunicorn and nginx, which just adds additional latencies and complexity, for no real benefit. What I'm talking about is just pointing at something like AppEngine, Cloud Functions, etc... (or whatever solution AWS has that is similar) and being done with it.…

Now you just 5x their costs.

Not if you do it right.

a) you get to fire the devops person, which saves $150k+ a year.

b) you add appropriate caching layers in front of everything.

c) you spend time adding features, which generate revenue.

I've done all of this before at scale. This whole case study was written about work I did [1]. Two devs, 3 months to release, first year was $80m gross revenue on $500/month cloud bills. Infinite scalability, zero devops.

[1] https://cloud.google.com/customers/gearlaunch

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#80

Post by Nextgrid »

latchkey wrote: post #78
ddorian43 wrote: post #68

Earlier quoted context omitted.

Now you just 5x their costs.

Not if you do it right. a) you get to fire the devops person, which saves $150k+ a year. b) you add appropriate caching layers in front of everything. c) you spend time adding features, which generate revenue. I've done all of this before at scale. This whole case study was written about work I did [1]. Two devs, 3 months to release, first year was $80m gross revenue on $500/month cloud bills. Infinite scalability, z…

> you get to fire the devops person, which saves $150k+ a year.

You are deluded or extremely short-sighted if you believe you can actually fire the devops guy. From my experience, the more you stray away from the conventional "dedicated server" paradigm the more you need a devops guy and you are in a very precarious position if you do fire him and something goes wrong.

Post reply on HN