Reining in the thundering herd: Getting to 80% CPU utilization with Django
1–10 of 139 posts
Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django
#2ya, I pretty much agree with 3 suggestions at the end:
* use uWSGI (read the docs, so many options...)
* use HAProxy, so very very good
* scale python apps by using processes.
Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django
#3Famous last words, but I get the sense that the need to handle this sort of load on Clubhouse is plateauing and will decline from here. The app seems to have shed all the people that drew other people initially and lost its small, intimate feel and has turned into either crowded rooms where no one can say anything, or hyper specific rooms where no one has anything to say.
Good article though! I’ve dealt with these exact issues and they can be very frustrating.
Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django
#4If it is just a backend, why not port it over to one of the myriad of cloud autoscaling solutions that are out there?
The opportunity cost of spending time figuring out why only 29 workers are receiving requests over adding new features that generate more revenue, seems like a quick decision.
Personally, I just start off with that now in the first place, the development load isn't any greater and the solutions that are out there are quite good.
Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django
#5Performance is the only thing that is holding me back to consider Python for bigger web applications.
Of the 3 main languages for web dev these days - Python, PHP and Javascript - I like Python the most. But it is scary how slow the default runtime, CPython, is. Compared to PHP and Javascript, it crawls like a snake.
Pypy could be a solution as it seems to be about 6x faster on average.
Is anybody here using Pypy for Django?
Did Clubhouse document somewhere if they are using CPython or Pypy?
Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django
#6use PHP ;)
Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django
#7use PHP ;)
So much this. Practically any other option is better than Python for web development if you're looking for performance.
Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django
#8Aside: AWS only allows registering 1000 targets in a target group… i wonder if thats the limit they hit. If so, its documented.