Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Django 4.1

djangoproject.com

Post reply on HN
Print view View original

31–40 of 126 posts

Re: Django 4.1

#31

Post by JimDabell »

BozeWolf wrote: post #7
WhyNotHugo wrote: post #6

Earlier quoted context omitted.

Also, async views allow for better scaling. Say your view needs to make an API call to a third party service within a request-response cycle. Currently, this will block an entire worker/thread while doing a blocking request. With async, we can use async HTTP libraries and scale these WAY better.

Or just run 20 workers and let the OS handle async. OSes are pretty good at this. Async is pretty hard to reason about. Async is beneficial when you make multiple blocking (http) request to a resource at the same time and later fold that into one response. Parallel stuff. Or when you are forced to run a single thread. Or when you are memory bound. Reality is that most requests depend on previous requests quite often.…

> Or just run 20 workers and let the OS handle async.

That just means you can only handle 20 concurrent requests before performance drops off a cliff. If the third-party service you are talking to is slow, then you’ll just end up with 20 workers all waiting for the service to respond, while other requests pile up. With async, those workers could still be handling other requests. Adding more workers because you are blocking on i/o works for low traffic services, not for anything remotely busy.

Re: Django 4.1

#32

Post by Piezoid »

sirodoht wrote: post #2

I wonder how popular the async interface will become. I use Python and Django extensively but haven't touch any of the async operations.

I use the event loop to defer the (io heavy) post-processing of some requests without involving an off-process task runner like Celery. I use a custom implementation of this: https://django-simple-task.readthedocs.io/how-it-works.html It is way simpler when you don't need strong guaranties and checkpoints persistence.

Re: Django 4.1

#33

Post by sgt »

tgv wrote: post #20
WhatsName wrote: post #5

The most anticipated Django release for me personally. Async ORM access is a massive quality of life improvememt, same for async CBV. The later is also a blocker for Django Rest Framework going async. Sure most CRUD applications work perfectly fine using WSGI, but for anyone using other protocols like WS or MQTT in their app this is kind of a big deal.

Seriously: why use Django for a REST-like service? I inherited an application that (probably) evolved from form-based to REST with a JS frontend, and I fail to see the point. There's a model, a serializer, handlers to implement relations, and multiple views per data type . It's so much overhead, and everything has to be kept in sync.

I don't find it has a lot of overhead if you keep it simple. No need to use class based views for REST services, just add function-based views with the @api_view decorator.

You can then within minutes return JSON coming from your data model by also adding an additional decorator to make it render JSON output.

If returning data directly from managed ORM objects, you will need to serialize it, but you can easily build an API without doing that and just return a dict.

Re: Django 4.1

#34

Post by game_the0ry »

Am I bad SWE if I do not care for for async in django?

I am sure a a lot of django devs will like it, but if I need the performance that I would get from going async, I would not being using django as my web framework.

FWIW I like django / python for a lot of things, just not for performance.

Re: Django 4.1

#35

Post by redox99 »

BozeWolf wrote: post #7
WhyNotHugo wrote: post #6

Earlier quoted context omitted.

Also, async views allow for better scaling. Say your view needs to make an API call to a third party service within a request-response cycle. Currently, this will block an entire worker/thread while doing a blocking request. With async, we can use async HTTP libraries and scale these WAY better.

Or just run 20 workers and let the OS handle async. OSes are pretty good at this. Async is pretty hard to reason about. Async is beneficial when you make multiple blocking (http) request to a resource at the same time and later fold that into one response. Parallel stuff. Or when you are forced to run a single thread. Or when you are memory bound. Reality is that most requests depend on previous requests quite often.…

So now if the requests become slow because of the third party or whatever, and take 1 second each, you have a max performance of 20 rps.

With async instead you would still be able to handle thousands of rps.

Re: Django 4.1

#36

Post by nickjj »

VWWHFSfQ wrote: post #26
nickjj wrote: post #16

Earlier quoted context omitted.

> That seems to be using WSGI, not ASGI, though, so no natively `async` views..? One does not simply go async. Changing your entire app to go full async has serious implications, requires every library you use to support it and requires a completely different mode of thinking. The example project includes an ASGI file if you want to use an async app server like uvicorn with async views instead of gunicorn. That would…

> use an async app server like uvicorn with async views instead of gunicorn I believe gunicorn with the uvicorn worker class is the recommended production server. > For production deployments we recommend using gunicorn with the uvicorn worker class. https://www.uvicorn.org/#running-with-gunicorn

> I believe gunicorn with the uvicorn worker class is the recommended production server.

Yep for production that works. In development tho I don't think the gunicorn --reload flag works with that worker type.

There is an issue open for this at https://github.com/benoitc/gunicorn/issues/2339 and since I last tried about a year ago I guess they have an unofficial "workaround solution" in one of the comments but it involves making a custom class to support reloading but that requires copying in some code into your project and also using different worker types in dev vs prod.

I think this is a good example of why I don't default to uvicorn and async in the project with Django 4.1. The whole package of "async everything" isn't ready for prime time in my opinion. It will improve over time but it's a waiting game at this point. Early adapters can easily swap it in if they want.

Re: Django 4.1

#37

Post by BozeWolf »

chpmrc wrote: post #30
BozeWolf wrote: post #7

Earlier quoted context omitted.

Or just run 20 workers and let the OS handle async. OSes are pretty good at this. Async is pretty hard to reason about. Async is beneficial when you make multiple blocking (http) request to a resource at the same time and later fold that into one response. Parallel stuff. Or when you are forced to run a single thread. Or when you are memory bound. Reality is that most requests depend on previous requests quite often.…

Why would you block a process that could use CPU cycles to do something useful just to wait on I/O? With the right primitives async is not "hard" nor inconvenient. Maybe you're referring to concurrency?

No I am referring to async, really. Accidentally blocking operations like open() or psycopg or mongodb or some included library which (accidentally) uses any of that stuff. Basically any library requiring requests is not usable. Or worse: open().

It can be hard to determine if something is doing something blocking and if it is, it is hard to debug. Especially in python world which, unlike javascript, was not async from the start.

Re: Django 4.1

#38

Post by bastawhiz »

game_the0ry wrote: post #34

Am I bad SWE if I do not care for for async in django? I am sure a a lot of django devs will like it, but if I need the performance that I would get from going async, I would not being using django as my web framework. FWIW I like django / python for a lot of things, just not for performance.

Async doesn't make your code faster, though there are cases where it lets you handle more requests with a single server (especially if you have code that is async). But moreover, if you have libraries that use async code, it's really not ideal if the thing that runs those libraries isn't async. You don't have to like or want it, but it's there for the folks who want or need it.

Re: Django 4.1

#39

Post by BozeWolf »

JimDabell wrote: post #31
BozeWolf wrote: post #7

Earlier quoted context omitted.

Or just run 20 workers and let the OS handle async. OSes are pretty good at this. Async is pretty hard to reason about. Async is beneficial when you make multiple blocking (http) request to a resource at the same time and later fold that into one response. Parallel stuff. Or when you are forced to run a single thread. Or when you are memory bound. Reality is that most requests depend on previous requests quite often.…

> Or just run 20 workers and let the OS handle async. That just means you can only handle 20 concurrent requests before performance drops off a cliff. If the third-party service you are talking to is slow, then you’ll just end up with 20 workers all waiting for the service to respond, while other requests pile up. With async, those workers could still be handling other requests. Adding more workers because you are bl…

Agree! Up the number of workers to 40, fix your application in another way... or indeed go with async stuff! ;-)

I was just saying that async quite often is not of much help and just complicates things. Of course it has its use! Your case is a great example of converting a django view into an async view.

Re: Django 4.1

#40

Post by ralmidani »

game_the0ry wrote: post #34

Am I bad SWE if I do not care for for async in django? I am sure a a lot of django devs will like it, but if I need the performance that I would get from going async, I would not being using django as my web framework. FWIW I like django / python for a lot of things, just not for performance.

Enabling async means those already invested in Django can do more on one machine without a complete rewrite, and those considering it can adopt it with fewer reservations. There will always be more scalable platforms out there, but enabling async with Django helps it serve the “sweet spot” in terms of both ergonomics and scalability.
Post reply on HN