Live data from Hacker News

Reining in the thundering herd: Getting to 80% CPU utilization with Django

blog.clubhouse.com

131–139 of 139 posts

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#131
post #127
post #117

Earlier quoted context omitted.

Silly question, but under nginx or apache do django instances persist or are they recreated for every new request?

The standard gunicorn configuration (and the one shown in the blog post) never restarts worker processes. gunicorn has an option --max-requests to restart every X requests but unless you have unfixable memory leaks there is no reason to do this. Nginx can't directly run WSGI applications, you can do it with Nginx Unit which also never restarts processes.

> unless you have unfixable memory leaks there is no reason to do this.

It's also useful to set this threshold to prevent long-lived connections to services/datastores not used by every request from accumulating and consuming resources on those services.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#132
post #23

Earlier quoted context omitted.

you now containers are just processes, right? This is what they did, but because they didn't need to schedule other jobs on the same machine, kubernetes or even docker would be overkill. In this case, simple VM orchestration seems like a fine solution.

Indeed, but you wouldn't be thinking about instance sizes, how many processes per instance and wondering if you're hitting kernel limits with all the issues coming up

You'd probably be worrying more about instance sizes if you ran a single executor per container; the memory overhead of your app would become a problem very quickly unless it's startup footprint was quite small.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#133
post #31

Earlier quoted context omitted.

containers would've solved it one process per container, easy peasy

This doesn’t work so easily with architectures with process pools for workers. So now your app server needs to speak docker (or whatever control plane) to spawn new workers and deal with more complicated IPC. Also the startup time is brutal. One process per container and multiprocessing is a huge lift most of the time. I’ve done it but it can be a mess because you don’t really have as much a handle on containers than…

> One process per container and multiprocessing

Do you mean multiprocessing inside the containers? Or are you managing multiprocessing child procs by forking into a container somehow? If the latter, I'd be really interested to learn how to do that; I didn't think it was possible, and it would be super useful for some of what I work on.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#134
post #16
post #13

Earlier quoted context omitted.

The top 5 web app programming languages by market share are PHP, Java, JS, Lua and Ruby.

...said a stranger on the internet without any sources to back up this claim.

source: https://www.wappalyzer.com/technologies/programming-language...

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#135
post #13
post #5

Performance is the only thing that is holding me back to consider Python for bigger web applications. Of the 3 main languages for web dev these days - Python, PHP and Javascript - I like Python the most. But it is scary how slow the default runtime, CPython, is. Compared to PHP and Javascript, it crawls like a snake. Pypy could be a solution as it seems to be about 6x faster on average. Is anybody here using Pypy for…

The top 5 web app programming languages by market share are PHP, Java, JS, Lua and Ruby.

lol at the downvotes. sorry your favorite language isn't on the list.

I specifically said "market share", not "best" or "favorite".

https://www.wappalyzer.com/technologies/programming-language...

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#136

Earlier quoted context omitted.

Indeed, but you wouldn't be thinking about instance sizes, how many processes per instance and wondering if you're hitting kernel limits with all the issues coming up

You'd probably be worrying more about instance sizes if you ran a single executor per container; the memory overhead of your app would become a problem very quickly unless it's startup footprint was quite small.

That's what they're doing now.

One app pool with one worker x number of cores

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#137
post #31

Earlier quoted context omitted.

containers would've solved it one process per container, easy peasy

This doesn’t work so easily with architectures with process pools for workers. So now your app server needs to speak docker (or whatever control plane) to spawn new workers and deal with more complicated IPC. Also the startup time is brutal. One process per container and multiprocessing is a huge lift most of the time. I’ve done it but it can be a mess because you don’t really have as much a handle on containers than…

That's what they're doing now.

One app pool with one worker x number of cores

Wrapping it around a container makes no difference

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#138

Earlier quoted context omitted.

You'd probably be worrying more about instance sizes if you ran a single executor per container; the memory overhead of your app would become a problem very quickly unless it's startup footprint was quite small.

That's what they're doing now. One app pool with one worker x number of cores

I assumed they're managing all those workers under one parent process which compiled their codebase on start. Perhaps that assumption was in error.

Re: Reining in the thundering herd: Getting to 80% CPU utilization with Django

#139
post #99

Earlier quoted context omitted.

Can Apache do request queuing?

Yes, under MPM, you have a listen backlog. If there's a spare worker available to process a request, they'll be picked off the backlog. https://httpd.apache.org/docs/2.4/mod/mpm_common.html#listen...

I don't think that's what I'm looking for, that's queuing at the front of the pipe but I need it queuing at the end of the pipe. Apache should be buffering and queuing lots of connections (with a timeout) and sending them single-file in gunicorn.

This lays out what I'm trying to achieve: https://aws.amazon.com/builders-library/using-load-shedding-...

Post reply on HN