Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Write Fast Apps Using Async Python 3.6 and Redis

eng.paxos.com

Post reply on HN
Print view View original

31–40 of 134 posts

Re: Write Fast Apps Using Async Python 3.6 and Redis

#31

Post by bobbyi_settv »

Waterluvian wrote: post #17
erikcw wrote: post #3

We've just recently started using Sanic[0] paired with Redis to great effect for a very high throughput web service. It also uses Python 3 asyncio/uvloop at its core. So far very happy with it. [0] https://github.com/channelcat/sanic

I have never found a good example of a Python web server that provides some mechanism for statefulness. Is it just fundamentally not possible to have shared state among requests handled by the threads of a process? Sanic's examples seem to be the same as Flask's: self-contained function calls attached to endpoints. I keep hitting a wall with Python when I want to do something like: 1. subscribe to a websocket connect…

You normally use something like redis to store the state.

If you were going to share state in memory between threads, how would you handle the case where the second request goes to a different server or that the process has restarted? You'd need redis anyway, so you might as well just use it in all cases.

Re: Write Fast Apps Using Async Python 3.6 and Redis

#32

Post by GroSacASacs »

fritzy wrote: post #30

It seems strange that they would claim that Python's libuv based event loop is twice as fast as Node.js's libuv based event loop. There's some context missing to that statement or it's flat out false.

What does it even mean. The event loop is only used when there is nothing going on. Is it faster at doing nothing ?

Re: Write Fast Apps Using Async Python 3.6 and Redis

#33

Post by theptip »

sandGorgon wrote: post #4
erikcw wrote: post #3

We've just recently started using Sanic[0] paired with Redis to great effect for a very high throughput web service. It also uses Python 3 asyncio/uvloop at its core. So far very happy with it. [0] https://github.com/channelcat/sanic

There is also ApiStar - https://github.com/tomchristie/apistar It's built by Tom Christie - the original author of Django Rest Foundation. http://discuss.apistar.org/t/how-does-api-star-fit-in-with-d...

Interesting, hadn't seen that yet; thanks for sharing. Does look like it'll have some nice design concepts -- I've definitely come to view Django/DRF's strong coupling to the ORM as a hinderance to architectural flexibility/sanity as my application has grown.

Interestingly it eschews Swagger/OpenAPI in favour of JSON Schema, wonder how that'll pan out; I like the promise of codegen that swagger offers, but haven't found the generated clients to be particularly usable.

Re: Write Fast Apps Using Async Python 3.6 and Redis

#34

Post by Matthias247 »

mrks_ wrote: post #9
mixmastamyk wrote: post #7

Can anyone recommend a good book to get started on concurrency, with discussions of models, and a few implementations such as golang and python 3.5+? While I can write this kind of code, I don't feel like I completely understand some of the concepts.

I'm not far, but Seven Concurrency Models in Seven Weeks is pretty good and might fit what you're looking for. https://pragprog.com/book/pb7con/seven-concurrency-models-in...

I recommend the book very much. However it doesn't have a chapter the single-threaded concurrency handling (with eventloops, and futures/promises and sometimes even plain callbacks), which is currently en-vogue in lots of languages (JS, python asyncio, boost asio, etc). So this is something one should look up elsewhere.

Re: Write Fast Apps Using Async Python 3.6 and Redis

#35

Post by victorvation »

Matthias247 wrote: post #34
mrks_ wrote: post #9

Earlier quoted context omitted.

I'm not far, but Seven Concurrency Models in Seven Weeks is pretty good and might fit what you're looking for. https://pragprog.com/book/pb7con/seven-concurrency-models-in...

I recommend the book very much. However it doesn't have a chapter the single-threaded concurrency handling (with eventloops, and futures/promises and sometimes even plain callbacks), which is currently en-vogue in lots of languages (JS, python asyncio, boost asio, etc). So this is something one should look up elsewhere.

Do you have any reading material that you would recommend for single-threaded concurrency?

Re: Write Fast Apps Using Async Python 3.6 and Redis

#36

Post by tomchristie »

theptip wrote: post #33
sandGorgon wrote: post #4

Earlier quoted context omitted.

There is also ApiStar - https://github.com/tomchristie/apistar It's built by Tom Christie - the original author of Django Rest Foundation. http://discuss.apistar.org/t/how-does-api-star-fit-in-with-d...

Interesting, hadn't seen that yet; thanks for sharing. Does look like it'll have some nice design concepts -- I've definitely come to view Django/DRF's strong coupling to the ORM as a hinderance to architectural flexibility/sanity as my application has grown. Interestingly it eschews Swagger/OpenAPI in favour of JSON Schema, wonder how that'll pan out; I like the promise of codegen that swagger offers, but haven't fo…

One thing to clarify here. Swagger/OpenAPI use JSON Schema in order to describe parameters and response structures. The rest of the schema work will start to fall into place pretty quickly now that we've got the groundwork done. Swagger generation based on the annotations will be one of the features, but there'll be plenty more to get excited about too.

Re: Write Fast Apps Using Async Python 3.6 and Redis

#37

Post by theptip »

tomchristie wrote: post #36
theptip wrote: post #33

Earlier quoted context omitted.

Interesting, hadn't seen that yet; thanks for sharing. Does look like it'll have some nice design concepts -- I've definitely come to view Django/DRF's strong coupling to the ORM as a hinderance to architectural flexibility/sanity as my application has grown. Interestingly it eschews Swagger/OpenAPI in favour of JSON Schema, wonder how that'll pan out; I like the promise of codegen that swagger offers, but haven't fo…

One thing to clarify here. Swagger/OpenAPI use JSON Schema in order to describe parameters and response structures. The rest of the schema work will start to fall into place pretty quickly now that we've got the groundwork done. Swagger generation based on the annotations will be one of the features, but there'll be plenty more to get excited about too.

Isn't Swagger a subset of JSON Schema though? [1]

If APIStar happens to target the same subset, that's not a problem of course.

[1]: http://stackoverflow.com/a/32386131/37481

Re: Write Fast Apps Using Async Python 3.6 and Redis

#38

Post by StreamBright »

pekk wrote: post #29
StreamBright wrote: post #19

>>> The performance of uvloop-based asyncio is close to that of Go programs. I would prefer standard benchmarks for this. I hope they submit their framework to TechEnpower benchmarks. https://www.techempower.com/benchmarks/

Those benchmarks aren't any more standard than anything else.

Yes but you can see the most number of frameworks there running on the same hardware and same settings doing the same job. Also you can see the configuration how to achieve that.

Re: Write Fast Apps Using Async Python 3.6 and Redis

#39

Post by cristobal23 »

victorvation wrote: post #35
Matthias247 wrote: post #34

Earlier quoted context omitted.

I recommend the book very much. However it doesn't have a chapter the single-threaded concurrency handling (with eventloops, and futures/promises and sometimes even plain callbacks), which is currently en-vogue in lots of languages (JS, python asyncio, boost asio, etc). So this is something one should look up elsewhere.

Do you have any reading material that you would recommend for single-threaded concurrency?

http://krondo.com/in-which-we-begin-at-the-beginning/

Re: Write Fast Apps Using Async Python 3.6 and Redis

#40

Post by fafhrd91 »

erikcw wrote: post #3

We've just recently started using Sanic[0] paired with Redis to great effect for a very high throughput web service. It also uses Python 3 asyncio/uvloop at its core. So far very happy with it. [0] https://github.com/channelcat/sanic

problem with Sanic that it does not implement streaming properly. so it is very easy to kill any Sanic process, 10-20 seconds. again any.
Post reply on HN