Just in time, I've been needing to implement s3 upload asynchronously.
Python Asyncio
31–40 of 188 posts
Re: Python Asyncio
#32Python asyncio is pretty awful. The libraries are of extremely poor quality, and the slightest mistake can lead to blocking the event loop. After a few years of dealing with it I refuse to continue and am just using threads.
Re: Python Asyncio
#33Earlier quoted context omitted.
Ok. This is what has been a huge hangup for me. It really seems that if you're doing asyncio, you must do EVERYTHING async, it's like asyncio takes over (infects?) the entire program.
I feel the same way, it's a lot like `const` in C++. Basically it's enforcing usage for every upstream user to ensure there's no inefficiency (such as waiting for things like the network). JavaScript has an extremely nice "out" (I'm sure other languages do too): async function() { await async_thing() do_sync_stuff() } can be written function() { async_thing().then(do_sync_stuff) } which is quite intuitive and helps g…
problem is async doesn't enforce anything. You can still block from an async function.
Re: Python Asyncio
#34Not mentioned: The behavior is different between python 3.6 and 3.8. That was fun to debug.
Python is simply not the right language for asynchronous programming. Things that are easy in many other languages are hard and don't work as you'd expect. If you are in a python-only shop, brush up on your presentation skills and see if you can convince them to branch out.
Or write a shell script. You're better off with "./read_socket.sh &" than you are with any python library.
Re: Python Asyncio
#35Earlier quoted context omitted.
I feel the same way, it's a lot like `const` in C++. Basically it's enforcing usage for every upstream user to ensure there's no inefficiency (such as waiting for things like the network). JavaScript has an extremely nice "out" (I'm sure other languages do too): async function() { await async_thing() do_sync_stuff() } can be written function() { async_thing().then(do_sync_stuff) } which is quite intuitive and helps g…
problem is async doesn't enforce anything. You can still block from an async function.
what do you mean "block"? If you call an async function from a non-async context it doesn't block.
Re: Python Asyncio
#36async is not comparable to threads: - async is concurrency - threads are parallelism They are not exclusives, the best approach is to be multi-threaded while each of the threads uses concurrency to prevent blocking the cpu.
Threads are also for concurrency. In fact in python because of the GIL they are pretty much only for concurrency.
Re: Python Asyncio
#37After 2 years of using asyncio in production, I recommend to avoid it if you can. With async programming, you take the complexity of concurrent programming, which is way harder than you can imagine. Also nobody mentions this for some reason, but asyncio doesn't make your programs faster, in fact it makes everything 100x SLOWER (we measured it multiple times, compared the same thing to the sync version), but makes you…
[deleted]
Re: Python Asyncio
#38Earlier quoted context omitted.
Question though: asyncio is implemented as threads, which is where the GIL chokes, right?
asyncio is _not_ implemented as threads. It has features where it can wrap a sync function inside a thread in order to turn it into an async function, but if you just write "normal" async code, all your code is running in a single thread.
[deleted]
Re: Python Asyncio
#39Earlier quoted context omitted.
problem is async doesn't enforce anything. You can still block from an async function.
what do you mean "block"? If you call an async function from a non-async context it doesn't block.
Nothing prevents you from calling a blocking function (for example good old non-async socket functions) from an async function.
Re: Python Asyncio
#40Gevent is the only way I will do async in Python. Everything else ends up being a nightmare, and gevent is a lot more performant than I ever imagine it will be for a given situation.