Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overhead of Python asyncio tasks

textual.textualize.io

Post reply on HN
Print view View original

71–80 of 88 posts

Re: Overhead of Python asyncio tasks

#71

Post by crabbone »

philote wrote: post #18
crabbone wrote: post #10

Reading this is like reading early Renaissance alchemist arguing about how much mercury they need to combine with how much silver to create gold... This is so far gone I don't even know where to begin... > It may be IO that gives AsyncIO its name, but Textual doesn't do any IO of its own. So why on Earth are you using AsyncIO? You don't need it, if that's true... > Those tasks are used to power message queues How are…

Message queues can be implemented in memory, so no network or disk I/O. In fact, it seems this project does use the built-in "queue" library: https://docs.python.org/3/library/queue.html

Lol. Waht I/O to disk? What are you talking about? Asyncio was never about disk I/O. It's about Internet socket I/O exclusively.

But, what do you think happens when you write (input) data to memory and then read (output) it from memory? I'll help you: it starts with "I" and ends with "O"!

Re: Overhead of Python asyncio tasks

#72

Post by crabbone »

louwrentius wrote: post #61
crabbone wrote: post #10

Reading this is like reading early Renaissance alchemist arguing about how much mercury they need to combine with how much silver to create gold... This is so far gone I don't even know where to begin... > It may be IO that gives AsyncIO its name, but Textual doesn't do any IO of its own. So why on Earth are you using AsyncIO? You don't need it, if that's true... > Those tasks are used to power message queues How are…

I knew you would be here in the comments. Crabbone won't leave any chance he gets to shit and dump on Python any HN post he can. Must be so frustrating, knowing that this silly language is one of the most populair programming languages in the world, despite its short comings, one of which is slower performance. This isn't about python or any language. The extreme negative tone and sense of superiority seems to point…

I don't care about Python performance.

But, yes, I'm very frustrated this trash is so popular. I never made this a secret :|

Python deserves negative tone, and if you think that popularity somehow makes it exempt from criticism, then you deserve the same.

Re: Overhead of Python asyncio tasks

#73

Post by vore »

crabbone wrote: post #70
vore wrote: post #13

Earlier quoted context omitted.

asyncio is really not all about I/O though, despite the name. You can just as easily use it for concurrency by interleaving tasks with each other. You can imagine multiple UI elements that all need to make progress, and using coroutines to have them cooperatively yield to each other and not just have one element block progress everywhere else. It's just using asyncio as a task scheduler, nothing more, nothing less. M…

[flagged]

I can't help but feel this is a low-effort troll, but I'll bite anyway.

> Where does this nonsense come from? Seriously? Who told you that?

Where is the I/O in this?

    async def task1():
        while True:
            print("a")
            await asyncio.sleep(0)


    async def task2():
        while True:
            print("b")
            await asyncio.sleep(0)


    async def main():
        t1 = asyncio.create_task(task1())
        t2 = asyncio.create_task(task2())
        await asyncio.gather(t1, t2)


    asyncio.run(main())
Fundamentally it's an event loop that allows you to register interest in I/O events. You can, of course, also just register no interest in any I/O events, where you still get scheduler features like timers and cooperative scheduling.

> This is wishful thinking. No. It doesn't work. Will not work, unless Python is purged of like 99% of it's current stuff and replaced with something else entirely. It's ridiculous to even consider this possibility applied to the concrete runtime you'd have to work with.

Where is the wishful thinking in the same code example I posted? You are free to control cooperation between your tasks as required.

> Come on man... you just put "cooperatively" and "will not block" in the same sentence. Of course it doesn't work like that. Will not work, especially in Python where you have to yield control explicitly. It's all toys. None of it works.

So? Just yield control explicitly? Just because the multitasking isn't preemptive doesn't mean you can't cooperatively make progress on multiple tasks.

> Well, that's stupid. Why would anyone want to do that? It simply won't work.

It works fine? The post is literally talking about how it works? Maybe you don't like the ergonomics but that's not a reason to be so dismissive of it.

In any case, just because you don't like it doesn't mean there is zero value in this, or that you can be such a massive piece of shit and respond like this.

Re: Overhead of Python asyncio tasks

#74

Post by morelisp »

selcuka wrote: post #67
tpmx wrote: post #9

Mostly a testament to how absurdly fast modern CPUs are despite Python itself being so slow. / Recent Python convert, in spite of the horrible general performance of the official implementation of the language. That sweet, sweet module library. Also, with Docker containers the deployment issues have been solved. It might be slow to execute but it's really efficient to develop with.

This has always been the case. Computing power doubles every few years, and it's cheap (probably less than 10% of your total project cost, unless you are doing highly specific stuff). It doesn't make sense to optimise for CPU cycles as the real bottleneck is usually developer efficiency, I/O, and UX.

Invest in Apple, their phone is going to be a big deal.

Re: Overhead of Python asyncio tasks

#75

Post by rjmill »

johndubchak wrote: post #4

After running that code on both a Windows SB3 and major souped up Lenovo running Ubuntu...I just feel inadequate.

Make sure you set your cpu frequency governor to "performance".

If your Linux machine is working an order of magnitude slower than you'd expect from the hardware, that's the first thing I'd check.

Re: Overhead of Python asyncio tasks

#76

Post by x-complexity »

jstx1 wrote: post #21

Async Python is still confusing af - when do I need it, what happens under the hood, does it actually help with performance, sometimes the GIL comes into play and sometimes it doesn't, why do we ever use threads at all if there's a GIL, why is it called async io if we can use it for anything. My mind is kind of scattered and people seems to be using a lot of async Python for some reason. Any good resources to clear t…

> Async Python is still confusing af - when do I need it,

It's needed when you're spending a lot of time waiting for an I/O request to complete (network/HTTP requests, disk reads/writes, database reads/writes)

> what happens under the hood,

https://tenthousandmeters.com/blog/python-behind-the-scenes-...

(Please read the entire blog post - It goes through the necessary concepts like generators, event loops, & coroutines)

> does it actually help with performance,

Refer to the first answer: You'll see improved performances if your workloads are mainly comprised of waiting for other stuff to complete. If you're compute-heavy, it'll be better to use the 'multiprocessing' library instead.

> sometimes the GIL comes into play and sometimes it doesn't,

The GIL comes into play when you have a lot of compute-heavy tasks: Otherwise, you'll rarely encounter it.

It's only when you have that many compute-heavy tasks that you start to use the 'multiprocessing' library.

> why do we ever use threads at all if there's a GIL,

Threads exist because it was there before asyncio & event loops came into Python.

> why is it called asyncio if we can use it for anything.

Its name came from PEP 3156, proposing the asyncio library back in 2012.

https://peps.python.org/pep-3156/

As for why, asynchronous I/O stands in contrast to synchronous I/O, where the program/thread had to wait for the I/O request to complete before it can do anything else. Making tasks asynchronous allows it to do other stuff while it waits for a task's request to complete, increasing CPU & I/O utilization.

> My mind is kind of scattered and people seems to be using a lot of async Python for some reason. Any good resources to clear things up?

Highly recommend this video from mcoding: It's fairly simple & goes through a sample implementation.

https://www.youtube.com/watch?v=ftmdDlwMwwQ

Also, this article:

https://realpython.com/async-io-python/

Re: Overhead of Python asyncio tasks

#78

Post by biorach »

crabbone wrote: post #71
philote wrote: post #18

Earlier quoted context omitted.

Message queues can be implemented in memory, so no network or disk I/O. In fact, it seems this project does use the built-in "queue" library: https://docs.python.org/3/library/queue.html

Lol. Waht I/O to disk? What are you talking about? Asyncio was never about disk I/O. It's about Internet socket I/O exclusively . But, what do you think happens when you write (input) data to memory and then read (output) it from memory? I'll help you: it starts with "I" and ends with "O"!

> Asyncio was never about disk I/O. It's about Internet socket I/O exclusively.

Game over. You have no clue what you're talking about.

Re: Overhead of Python asyncio tasks

#79

Post by geysersam »

TickleSteve wrote: post #32
schmichael wrote: post #29

Earlier quoted context omitted.

Your method of estimating instructions includes printing multiple lines which context switch to the kernel to perform IO. I'm not sure how an "instruction count" metric is useful anyway. edit: I'm not actually sure you are counting the context switch, but I still don't think estimating instruction count that way is particularly useful.

Its useful to show how much waste there is in a solution. Having an operation that can be executing 250,000 times per second on a modern processor is extremely slow ... not fast.

It's irrelevant that it's slow if it's only a small part of total runtime.

Re: Overhead of Python asyncio tasks

#80

Post by louwrentius »

crabbone wrote: post #72
louwrentius wrote: post #61

Earlier quoted context omitted.

I knew you would be here in the comments. Crabbone won't leave any chance he gets to shit and dump on Python any HN post he can. Must be so frustrating, knowing that this silly language is one of the most populair programming languages in the world, despite its short comings, one of which is slower performance. This isn't about python or any language. The extreme negative tone and sense of superiority seems to point…

I don't care about Python performance. But, yes, I'm very frustrated this trash is so popular. I never made this a secret :| Python deserves negative tone, and if you think that popularity somehow makes it exempt from criticism, then you deserve the same.

But this isn't about "something being popular makes it exempt from criticism".

You don't really criticise. You just exaggerate and emo-dump all over python and the people who use it. Your stance seems to be that python needs to be destroyed and dumped and replaced by some other language. Your stance is that people who use python are idiots.

That kind of 'criticism' is so totally useless it's kind of seriously pathological. Your 'critisism' says more about you (due to the extremely antagonising tone) and not much about Python really.

Quite sad when you think about it.

Post reply on HN