Write Fast Apps Using Async Python 3.6 and Redis • Hacker News Board

Re: Write Fast Apps Using Async Python 3.6 and Redis

#131

Post by zzzeek » Fri, Apr 28, 2017, 8:08 PM UTC

vittore wrote: post #122

zzzeek wrote: post #111

Earlier quoted context omitted.

depending on the latency of those database connections I've argued in the past that the overhead of adding asyncio context switching and boilerplate is more expensive than just hitting the two or three databases in serial (and if your web request is having to hit dozens of DB sources to serve one request, I think you've already lost the performance game :) ). When your one web request is contending with many other co…

Do you think it make more sense to do async backend when we are moving to real-time ( meaning websocket-based connections ) web apps?

I've maintained that async is better suited towards web services and lightweight databases like redis, and is not useful for relational databases. However, it's very hard to get async to make your code actually "faster", as opposed to just handling very high throughout with less resources. If people stop saying "faster!", I'll go away.

Re: Write Fast Apps Using Async Python 3.6 and Redis

#132

Post by manquer » Sat, Apr 29, 2017, 6:16 AM UTC

ma2rten wrote: post #83

zzzeek wrote: post #22

> we make heavy use of asyncio because it’s more performant more performant than....what exactly? If I need to load 1000 rows from a database and splash them on a webpage, will my response time go from the 300ms it takes without asyncio to something "more performant", like 50ms? Answer: no. async only gives you throughput, it has nothing to do with "faster" as far as the Python interpreter / GIL / anything like that.…

Will my response time go from the 300ms it takes without asyncio to something "more performant", like 50ms If you have to do 1000 queries it could, since could async will make it feasible to do them parallel. If it's a single query, maybe async would make it feasible to shard the database.

you usually see this pattern in ORMs with n+1 querys . If a single request requires 1000 db queries it is better to be optimising the query

Re: Write Fast Apps Using Async Python 3.6 and Redis

#133

Post by dismantlethesun » Sat, Apr 29, 2017, 12:38 PM UTC

cies wrote: post #117

dom0 wrote: post #116

Earlier quoted context omitted.

You can get 2-3 ms response time (sans network) with any of Django, Flask and Pyramid. Database queries tend to eat a lot, esp. if the queries are bad (long wait in the DBMS or post-filtering in Python/whichever); sometimes ORMs can eat a fair bit as well. But it's fairly rare to get that low, most pages for me (that I cared about) will take 10-30 ms. Using the correct tools and the right approach is fruitful as alwa…

> You can get 2-3 ms response time (sans network) with any of Django, Flask and Pyramid. Wow, never managed to do that. Maybe I have to try it again (last time checked on Django was some years ago).

Truth. The best I can get in Django is 30 ms.

Re: Write Fast Apps Using Async Python 3.6 and Redis

#134

Post by pdmccormick » Thu, May 04, 2017, 7:07 AM UTC

Waterluvian wrote: post #17

erikcw wrote: post #3

We've just recently started using Sanic[0] paired with Redis to great effect for a very high throughput web service. It also uses Python 3 asyncio/uvloop at its core. So far very happy with it. [0] https://github.com/channelcat/sanic

I have never found a good example of a Python web server that provides some mechanism for statefulness. Is it just fundamentally not possible to have shared state among requests handled by the threads of a process? Sanic's examples seem to be the same as Flask's: self-contained function calls attached to endpoints. I keep hitting a wall with Python when I want to do something like: 1. subscribe to a websocket connect…

Your request has stayed on my mind over the past few days, so I put this together for you: https://github.com/pdmccormick/sample-socketio-chat-app

Enjoy!