Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python has had async for 10 years – why isn't it more popular?

tonybaloney.github.io

Post reply on HN
Print view View original

271–280 of 305 posts

Re: Python has had async for 10 years – why isn't it more popular?

#271

Post by jpc0 »

pkolaczk wrote: post #240
tptacek wrote: post #204

Earlier quoted context omitted.

No, Rust async is new-school colored-functions concurrency.

The parent comment is right. Rust async is simple state automata structs you can poll explicitly with no magic. Async/await is just some syntactic sugar on top of that, but you don’t have to use it. An obvious advantage of doing it that way is you don’t need any runtime/OS-level support. Eg your runtime doesn’t need to even have a concept of threads. It works on bare metal embedded. Another advantage is that it’s ful…

Have you actually tried to implement async in rust from the ground up.

It is nothing like what you just described

Re: Python has had async for 10 years – why isn't it more popular?

#272

Post by sfn42 »

throw-qqqqq wrote: post #66
lacker wrote: post #54

Earlier quoted context omitted.

I'm confused, I feel like the two of you are expressing opposite opinions. The comment you are responding to prefers green threads to be managed like goroutines, where the code looks synchronous, but really it's cooperative multitasking managed by the runtime, to explicit async/await. But then you criticize "code that looks synchronous but is really async". So you prefer the explicit "async" keywords? What exactly is…

First, I don’t mean to criticize anything or anyone. People value such things subjectively, but for me the async/sync split does no good. Goroutines feel like old-school, threaded code to me. I spawn a goroutine and interact with other “threads” through well defined IPC. I can’t tell if I’m spawning a green thread or a “real” system thread. C#’s async/await is different IMO and I prefer the other model. I think the a…

I always find it strange how people complain about features when the real problem is that they simply don't like how people use the feature.

Async in C# is awesome, and there's nothing stopping you from writing sync code where appropriate or using threads if you want proper multi threading. Async is primarily used to avoid blocking for non-cpu-bound work, like waiting for API/db/filesystem etc. If you use it everywhere then it's used everywhere, if you don't then it isn't. For a lot of apps it makes sense to use it a lot, like in web apis that do lots of db calls and such. This incurs some overhead but it has the benefit of avoiding blocked threads so that no threads sit idle waiting for I/O.

You can imagine in a web API receiving a large number of requests per second there's a lot of this waiting going on and if threads were idle waiting for responses you wouldn't be able to handle nearly as much throughout.

Re: Python has had async for 10 years – why isn't it more popular?

#273

Post by zelphirkalt »

TZubiri wrote: post #114
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

>most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). The problem is not python, it's a skill issue. First of all forking is not a workaround, it's the way multiprocessing works at the low level in Unix systems. Second of all, forking is multiprocessing, not multithrea…

Recently, I am working on a project that uses Threading (in Python) and so far have had zero issues with that. Neither did I have any issues before, when using multiprocessing.

What I did have issues with though, was async. For example pytest's async thingy is buggy for years with no fix in sight, so in one project I had to switch to manually making an event loop in that those tests.

But isn't the whole purpose of async, that it enabled concurrency, not parallelism, without the weight of a thread? I agree that in most cases it is not necessary to go there, but I can imagine systems with not so many resources, that benefit from such an approach when they do lots of io.

Re: Python has had async for 10 years – why isn't it more popular?

#274

Post by carlmr »

raxxorraxor wrote: post #258
throw-qqqqq wrote: post #66

Earlier quoted context omitted.

First, I don’t mean to criticize anything or anyone. People value such things subjectively, but for me the async/sync split does no good. Goroutines feel like old-school, threaded code to me. I spawn a goroutine and interact with other “threads” through well defined IPC. I can’t tell if I’m spawning a green thread or a “real” system thread. C#’s async/await is different IMO and I prefer the other model. I think the a…

> I think the async-concept gets overused (at my workplace at least). Problem is it that it self reinforces and before you look every little function is suddenly async. The irony is that it is used where you want to write in a synchronous style...

Yep, this is my biggest gripe with explicit async, all of a sudden a library that needn't be async forces me to use async (and in Rust forces me to use their async implementation), just because the author felt like async is a nice thing to try out.

Re: Python has had async for 10 years – why isn't it more popular?

#275

Post by zelphirkalt »

pbalau wrote: post #79
atomicnumber3 wrote: post #14

The author gets close to what I think the root problem is, but doesn't call it out. The truth is that in python, async was too little, too late. By the time it was introduced, most people who actually needed to do lots of io concurrently had their own workarounds (forking, etc) and people who didn't actually need it had found out how to get by without it (multiprocessing etc). Meanwhile, go showed us what good green…

You make a very good case for why python's async isn't more prevalent, but I think this is not painting the full image. Taking a general case, let's say a forum, in order to render a thread one needs to search for all posts from that thread, then get all the extra data needed for rendering and finally send the rendered output to the client. In the "regular" way of doing this, one will compose a query, that will filte…

When I read your comment I was thinking: "But then you would need to structure your db in such a way that ... ahh yes, they are getting to that ... but then what about actually rendering the results? Ah they are describing that here ..." so well done I think.

However, async tasks on a single core means potentially a lot of switching between those tasks. So async alone does not save the day here. It will have to be combined with true parallelism, to result in the speedup we want. Otherwise a single task rendering all the parts in sequence would be faster.

Also not, that it depends on where your db is. the process you describe implies at least 2 rounds of db communication. First one for the initial get forum thread query, then second one for all the async get forum replies requests. So if communication with the db takes a long time, you might as well lose what you gained, because you did 2 rounds of that communication.

So I guess it's not a trivial matter.

Re: Python has had async for 10 years – why isn't it more popular?

#276

Post by Philpax »

rich_sasha wrote: post #231

What the article and the comments don't seem to mention is also that the documentation is an outlier on the poor side. Most Python documentation is at least decent. asyncio hides a lot of the complexity behind a tutorial style "just do this" prose, only obliquely mentions the foot guns and gives little guidance on how to actually structure async code. IME writing an asyncio Python application is a bit like fixing a b…

I would disagree on the first paragraph, if only to say that the majority of Python stdlib documentation is written in that tutorial style, and I loathe it. It is always a chore to look something in the stdlib up, especially if you're used to the reference documentation for Rust/Go/Ruby/JavaScript.

Re: Python has had async for 10 years – why isn't it more popular?

#277

Post by rich_sasha »

Philpax wrote: post #276
rich_sasha wrote: post #231

What the article and the comments don't seem to mention is also that the documentation is an outlier on the poor side. Most Python documentation is at least decent. asyncio hides a lot of the complexity behind a tutorial style "just do this" prose, only obliquely mentions the foot guns and gives little guidance on how to actually structure async code. IME writing an asyncio Python application is a bit like fixing a b…

I would disagree on the first paragraph, if only to say that the majority of Python stdlib documentation is written in that tutorial style, and I loathe it. It is always a chore to look something in the stdlib up, especially if you're used to the reference documentation for Rust/Go/Ruby/JavaScript.

I think a lot of standard library have both. For example multiprocessing or logging. It's true, the tutorial is annoying, except perhaps on first reading, but at least the proper documentation is there. For asyncio the actual hard documentation bit is missing, incomplete or misleading, depending on where exactly you're looking.

Re: Python has had async for 10 years – why isn't it more popular?

#278

Post by tptacek »

pkolaczk wrote: post #240
tptacek wrote: post #204

Earlier quoted context omitted.

No, Rust async is new-school colored-functions concurrency.

The parent comment is right. Rust async is simple state automata structs you can poll explicitly with no magic. Async/await is just some syntactic sugar on top of that, but you don’t have to use it. An obvious advantage of doing it that way is you don’t need any runtime/OS-level support. Eg your runtime doesn’t need to even have a concept of threads. It works on bare metal embedded. Another advantage is that it’s ful…

I didn't say it was good or bad. I said it's new-school colored functions.

Re: Python has had async for 10 years – why isn't it more popular?

#279

Post by markandrewj »

ErikBjare wrote: post #255
markandrewj wrote: post #139

Earlier quoted context omitted.

I can tell you guys work with languages like Go, so this isn't true for yourselves, but I usually find it is developers that only ever work with synchronous code who find async complicated. Which isn't surprising, if you don't understand something it can seem complicated. My views is almost that people should learn how to write async code by default now. Regardless of the language. Writing modern applications basical…

I have no problem with async in JS or Rust, but async in Python is a very different beast, and like many people in this thread I do my best to avoid the fully loaded footgun altogether. Writing maintainable Python basically requires avoiding it, so I strongly disagree with "regardless of language".

Maybe, but I wouldn't go back to Python 2 without async. It has also improved over time in Python. I have also had success using async in Python. I do understand what the article talks about however. Understanding the difference between blocking and non-blocking code is also a concept relevant to Python. In Node it's one of the concepts you are first introduced to, because Node is single threaded by default. I also understand in Go and other languages there are different options.

https://nodejs.org/en/learn/asynchronous-work/overview-of-bl...

I will agree with what some is said a above, BEAM is pretty great. I have been using it recently through Elixir.

Post reply on HN