Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first year of free-threaded Python

labs.quansight.org

Post reply on HN
Print view View original

71–80 of 302 posts

Re: The first year of free-threaded Python

#71

Post by delusional »

bayindirh wrote: post #42
HDThoreaun wrote: post #36

Earlier quoted context omitted.

> But a simple solution to address your fears: simply don't use threads. You'll be fine. Im not worried about new code. Im worried about stuff written 15 years ago by a monkey who had no idea how threads work and just read something on stack overflow that said to use threading. This code will likely break when run post-GIL. I suspect there is actually quite a bit of it.

Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave. Older code will break, but they break all the time. A language changes how something behaves in a new revision, s…

> Software rots

No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots.

The only reason we need to change software is that we trade that off against something else. Instructions are reworked, because chasing the universal Turing machine takes a few sacrifices. If all software has to run on the same hardware, those two artifacts have to have a dialogue about what they need from each other.

If we didnt want the universal machine to do anything new. If we had a valuable product. We could just keep making the machine that executes that product. It never rots.

Re: The first year of free-threaded Python

#72

Post by Flamentono2 »

falcor84 wrote: post #62
pjmlp wrote: post #50

Earlier quoted context omitted.

Yes, because I am quite certain someone without anything better to do would correct me on that. For me Facebook will always be Facebook, and Twitter will always be Twitter.

> Yes, because I am quite certain someone without anything better to do would correct me on that. Well, you sure managed to avoid that by setting up camp on that hill. Kudos on so much time saved. > For me Facebook will always be Facebook, and Twitter will always be Twitter. Well, for me the product will always be "Thefacebook", but that's since I haven't used it since. But I do respect that there's a company running…

With money which destroied our society

Re: The first year of free-threaded Python

#73

Post by kstrauser »

biorach wrote: post #59
rich_sasha wrote: post #46

Earlier quoted context omitted.

Ah that's very, very sad. I guess they have embraced and extended, there's only one thing left to do.

At this stage the cliched and clueless comments about embrace/extend/extinguish are tiresome and inevitable whenever Microsoft is mentioned. A few decades ago MS did indeed have a playbook which they used to undermine open standards. Laying off some members of the Python team bears no resemblence whatsoever to that. At worst it will delay the improvement of free-threaded Python. That's all. Your comment is lazy and u…

cough Bullshit cough

* VSCode got popular and they started preventing forks from installing its extensions.

* They extended the Free Source pyright language server into the proprietary pylance. They don’t even sell it. It’s just there to make the FOSS version less useful.

* They bought GitHub and started rate limiting it to unlogged in visitors.

Every time Microsoft touches a thing, they end up locking it down. They can’t help it. It’s their nature. And if you’re the frog carrying that scorpion across the pond and it stings you, well, you can only blame it so much. You knew this when they offered the deal.

Every time. It hasn’t changed substantially since they declared that Linux is cancer, except to be more subtle in their attacks.

Re: The first year of free-threaded Python

#74

Post by robertlagrant »

biorach wrote: post #67
wisty wrote: post #66

Earlier quoted context omitted.

It sounds like an oblique reference to that time they temporarily suspended one of the of the most valuable members of the community, apparently for having the audacity to suggest that their powers to suspend members of the community seemed a little arbitrary and open to abuse.

Well they could just say that instead of wasting people's time with oblique references

Saying "This stinks of BS" is going to mean you have little standing to criticise other people for wasting time.

Re: The first year of free-threaded Python

#75

Post by OskarS »

immibis wrote: post #22
miohtama wrote: post #6

Earlier quoted context omitted.

GIL or no-GIL concerns only people who want to run multicore workloads. If you are not already spending time threading or multiprocessing your code there is practically no change. Most race condition issues which you need to think are there regardless of GIL.

With the GIL, multithreaded Python gives concurrent I/O without worrying about data structure concurrency (unless you do I/O in the middle of it) - it's a lot like async in this way - data structure manipulation is atomic between "await" expressions (except in the "await" is implicit and you might have written one without realizing in which case you have a bug). Meanwhile you still get to use threads to handle severa…

That doesn't match with my understanding of free-threaded Python. The GIL is being replaced with fine-grained locking on the objects themselves, so sharing data-structures between threads is still going to work just fine. If you're talking about concurrency issues like this causing out-of-bounds errors:

    if len(my_list) > 5:
        print(my_list[5])
(i.e. because a different thread can pop from the list in-between the check and the print), that could just as easily happen today. The GIL makes sure that only one python interpreter runs at once, but it's entirely possible that the GIL is released and switches to a different thread after the check but before the print, so there's no extra thread-safety issue in free-threaded mode.

The problems (as I understand it, happy to be corrected), are mostly two-fold: performance and ecosystem. Using fine-grained locking is potentially much less efficient than using the GIL in the single-threaded case (you have to take and release many more locks, and reference count updates have to be atomic), and many, many C extensions are written under the assumption that the GIL exists.

Re: The first year of free-threaded Python

#76

Post by kstrauser »

delusional wrote: post #71
bayindirh wrote: post #42

Earlier quoted context omitted.

Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave. Older code will break, but they break all the time. A language changes how something behaves in a new revision, s…

> Software rots No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots. The only reason we need to change software is that we trade that off against something else. Instr…

That’s not what the phrase implies. If you have a C program from 1982, you can still compile it on a 1982 operating system and toolchain and it’ll work just as before.

But if you tried to compile it on today’s libc, making today’s syscalls… good luck with that.

Software “rots” in the sense that it has to be updated to run on today’s systems. They’re a moving target. You can still run HyperCard on an emulator, but good luck running it unmodded on a Mac you buy today.

Re: The first year of free-threaded Python

#77

Post by kccqzy »

fulafel wrote: post #69
miohtama wrote: post #6

Earlier quoted context omitted.

GIL or no-GIL concerns only people who want to run multicore workloads. If you are not already spending time threading or multiprocessing your code there is practically no change. Most race condition issues which you need to think are there regardless of GIL.

A lot of Python usage is leveraging libraries with parallel kernels inside written in other languages. A subset of those is bottlenecked on Python side speed. A sub-subset of those are people who want to try no-GIL to address the bottleneck. But if non-GIL becomes pervasive, it could mean Python becomes less safe for the "just parallel kernels" users.

Yes sure. Thought experiment: what happens when these parallel kernels suddenly need to call back in to Python? Let's say you have a multithreaded sorting library. If you are sorting numbers then fine nothing changes. But if you are sorting objects you need to use a single thread because you need to call PyObject_RichCompare. These new parallel kernels will then try to call PyObject_RichCompare from multiple threads.

Re: The first year of free-threaded Python

#78

Post by dahcryn »

delusional wrote: post #71
bayindirh wrote: post #42

Earlier quoted context omitted.

Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave. Older code will break, but they break all the time. A language changes how something behaves in a new revision, s…

> Software rots No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots. The only reason we need to change software is that we trade that off against something else. Instr…

yes it does.

If software is implicitly built on wrong understanding, or undefined behaviour, I consider it rotting when it starts to fall apart as those undefined behaviours get defined. We do not need to sacrifice a stable future because of a few 15 year old programs. Let the people who care about the value that those programs bring, manage the update cycle and fix it.

Re: The first year of free-threaded Python

#79

Post by odiroot »

AlexanderDhoore wrote: post #2

Am I the only one who sort of fears the day when Python loses the GIL? I don't think Python developers know what they’re asking for. I don't really trust complex multithreaded code in any language. Python, with its dynamic nature, I trust least of all.

It's called job security. We'll be rewriting decades of code that's broken by that transition.

Re: The first year of free-threaded Python

#80

Post by eblume »

delusional wrote: post #71
bayindirh wrote: post #42

Earlier quoted context omitted.

Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave. Older code will break, but they break all the time. A language changes how something behaves in a new revision, s…

> Software rots No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots. The only reason we need to change software is that we trade that off against something else. Instr…

Software is written with a context, and the context degrades. It must be renewed. It rots, sorry.
Post reply on HN