Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first year of free-threaded Python

labs.quansight.org

Post reply on HN
Print view View original

221–230 of 302 posts

Re: The first year of free-threaded Python

#221

Post by zahlman »

HDThoreaun wrote: post #36
jillesvangurp wrote: post #23

Earlier quoted context omitted.

You are not the only one who is afraid of changes and a bit change resistant. I think the issue here is that the reasons for this fear are not very rational. And also the interest of the wider community is to deal with technical debt. And the GIL is pure technical debt. Defensible 30 years ago, a bit awkward 20 years ago, and downright annoying and embarrassing now that world + dog does all their AI data processing w…

> But a simple solution to address your fears: simply don't use threads. You'll be fine. Im not worried about new code. Im worried about stuff written 15 years ago by a monkey who had no idea how threads work and just read something on stack overflow that said to use threading. This code will likely break when run post-GIL. I suspect there is actually quite a bit of it.

>Im worried about stuff written 15 years ago

Please don't - it isn't relevant.

15 years ago, new Python code was still dominantly for 2.x. Even code written back then with an eye towards 3.x compatibility (or, more realistically, lazily run through `2to3` or `six`) will have quite little chance of running acceptably on 3.14 regardless. There have been considerable removals from the standard library, `async` is no longer a valid identifier name (you laugh, but that broke Tensorflow once). The attitude taken towards """strings""" in a lot of 2.x code results in constructs that can be automatically made into valid syntax that appears to preserve the original intent, but which are not at all automatically fixed.

Also, the modern expectation is of a lock-step release cadence. CPython only supports up to the last 5 versions, released annually; and whenever anyone publishes a new version of a package, generally they'll see no point in supporting unsupported Python versions. Nor is anyone who released a package in the 3.8 era going to patch it if it breaks in 3.14 - because support for 3.14 was never advertised anyway. In fact, in most cases, support for 3.9 wasn't originally advertised, and you can't update the metadata for an existing package upload (you have to make a new one, even if it's just a "post-release") even if you test it and it does work.

Practically speaking, pure-Python packages usually do work in the next version, and in the next several versions, perhaps beyond the support window. But you can really never predict what's going to break. You can only offer a new version when you find out that it's going to break - and a lot of developers are going to just roll that fix into the feature development they were doing anyway, because life's too short to backport everything for everyone. (If there's no longer active development and only maintenance, well, good luck to everyone involved.)

If 5 years isn't long enough for your purposes, practically speaking you need to maintain an environment with an outdated interpreter, and find a third party (RedHat seems to be a popular choice here) to maintain it.

Re: The first year of free-threaded Python

#222

Post by sgarland »

ogrisel wrote: post #53
sgarland wrote: post #48

> Instead, many reach for multiprocessing, but spawning processes is expensive Agreed. > and communicating across processes often requires making expensive copies of data SharedMemory [0] exists. Never understood why this isn’t used more frequently. There’s even a ShareableList which does exactly what it sounds like, and is awesome. [0]: https://docs.python.org/3/library/multiprocessing.shared_mem...

You cannot share arbitrarily structured objects in the `ShareableList`, only atomic scalars and bytes / strings. If you want to share structured Python objects between instances, you have to pay the cost of `pickle.dump/pickle.dump` (CPU overhead for interprocess communication) + the memory cost of replicated objects in the processes.

So don’t do that? Send data to workers as primitives, and have a separate process that reads the results and serializes it into whatever form you want.

Re: The first year of free-threaded Python

#223

Post by charleshn »

crackez wrote: post #190
charleshn wrote: post #168

Earlier quoted context omitted.

> Spawning processes generally takes much less than 1 ms on Unix It depends on whether one uses clone, fork, posix_spawn etc. Fork can take a while depending on the size of the address space, number of VMAs etc.

Fork on Linux should use copy-on-write vmpages now, so if you fork inside python it should be cheap. If you launch a new Python process from let's say the shell, and it's already in the buffer cache, then you should only have to pay the startup CPU cost of the interpreter, since the IO should be satisfied from buffer cache...

> Fork on Linux should use copy-on-write vmpages now, so if you fork inside python it should be cheap.

No, that's exactly the point I'm making, copying PTEs is not cheap on a large address space, woth many VMAs.

You can run a simple python script allocating a large list and see how it affects fork time.

Re: The first year of free-threaded Python

#224

Post by kstrauser »

ori_b wrote: post #204
chubot wrote: post #112

Earlier quoted context omitted.

Spawning processes generally takes much less than 1 ms on Unix Spawning a PYTHON interpreter process might take 30 ms to 300 ms before you get to main(), depending on the number of imports It's 1 to 2 orders of magnitude difference, so it's worth being precise This is a fallacy with say CGI. A CGI in C, Rust, or Go works perfectly well. e.g. sqlite.org runs with a process PER REQUEST - https://news.ycombinator.com/it…

As another example: I run https://shithub.us with shell scripts, serving a terabyte or so of data monthly (mostly due to AI crawlers that I can't be arsed to block). I'm launching between 15 and 3000 processes per request. While Plan 9 is about 10x faster at spawning processes than Linux, it's telling that 3000 C processes launching in a shell is about as fast as one python interpreter.

The interpreter itself is pretty quick:

  ᐅ time echo "print('hi'); exit()" | python
  hi
  
  ________________________________________________________
  Executed in   21.48 millis    fish           external
     usr time   16.35 millis  146.00 micros   16.20 millis
     sys time    4.49 millis  593.00 micros    3.89 millis

Re: The first year of free-threaded Python

#225

Post by zahlman »

bayindirh wrote: post #42
HDThoreaun wrote: post #36

Earlier quoted context omitted.

> But a simple solution to address your fears: simply don't use threads. You'll be fine. Im not worried about new code. Im worried about stuff written 15 years ago by a monkey who had no idea how threads work and just read something on stack overflow that said to use threading. This code will likely break when run post-GIL. I suspect there is actually quite a bit of it.

Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave. Older code will break, but they break all the time. A language changes how something behaves in a new revision, s…

> A language changes how something behaves in a new revision, suddenly 20 year old bedrock tools are getting massively patched to accommodate both new and old behavior.

In my estimation, the only "20 year old bedrock tools" in Python are in the standard library - which currently holds itself free to deprecate entire modules in any minor version, and remove them two minor versions later - note that this is a pseudo-calver created by a coincidentally annual release cadence. (A bunch of stuff that old was taken out recently, but it can't really be considered "bedrock" - see https://peps.python.org/pep-0594/).

Unless you include NumPy's predecessors when dating it (https://en.wikipedia.org/wiki/NumPy#History). And the latest versions of NumPy don't even support Python 3.9 which is still not EOL.

Requests turns 15 next February (https://pypi.org/project/requests/#history).

Pip isn't 20 years old yet (https://pypi.org/project/pip/#history) even counting the version 0.1 "pyinstall" prototype (not shown).

Setuptools (which generally supports only the Python versions supported by CPython, hasn't supported Python 2.x since version 45 and is currently on version 80) only appears to go back to 2006, although I can't find release dates for versions before what's on PyPI (their own changelog goes back to 0.3a1, but without dates).

Re: The first year of free-threaded Python

#226

Post by kstrauser »

zahlman wrote: post #209
morningsam wrote: post #201

Earlier quoted context omitted.

>Spawning a PYTHON interpreter process might take 30 ms to 300 ms Which is why, at least on Linux, Python's multiprocessing doesn't do that but fork()s the interpreter, which takes low-single-digit ms as well.

Even when the 'spawn' strategy is used (default on Windows, and can be chosen explicitly on Linux), the overhead can largely be avoided. (Why choose it on Linux? Apparently forking can cause problems if you also use threads.) Python imports can be deferred (`import` is a statement , not a compiler or pre-processor directive), and child processes (regardless of the creation strategy) name the main module as `__mp_main…

I really wish Python had a way to annotate things you don't care about cleaning up. I don't know what the API would look like, but I imagine something like:

  l = list(cleanup=False)
  for i in range(1_000_000_000): l.append(i)
telling the runtime that we don't need to individually GC each of those tiny objects and just let the OS's process model free the whole thing at once.

Sure, close TCP connections before you kill the whole thing. I couldn't care less about most objects, though.

Re: The first year of free-threaded Python

#227

Post by ptx »

dotancohen wrote: post #218
ptx wrote: post #171

Earlier quoted context omitted.

Although keep in mind that the callback will be "called in a thread belonging to the process" (say the docs), presumably some thread that is not the UI thread. So the callback needs to post an event to the UI thread's event queue, where it can be picked up by the UI thread's event loop and only then perform the UI updates. I don't know how that's done in Pyside, though. I couldn't find a clear example. You might have…

Thank you. Perhaps I should trigger the transcription thread from the UI thread, then? It is a UI button that initiates it after all.

The tricky part is coming back onto the UI thread when the background work finishes. Your transcription thread has to somehow trigger the UI work to be done on the UI thread.

It seems the way to do it in Qt is with signals and slots, emitting a signal from your QThread and binding it to a slot in the UI thread, making sure to specify a "queued connection" [1]. There's also a lower-level postEvent method [2] but people disagree [3] on whether that's OK to call from a regular Python thread or has to be called from a QThread.

So I would try doing it with Qt's thread classes, not with concurrent.futures.

[1] https://doc.qt.io/qt-5/threads-synchronizing.html#high-level...

[2] https://doc.qt.io/qt-6/qcoreapplication.html#postEvent

[3] https://www.mail-archive.com/pyqt@riverbankcomputing.com/msg...

Re: The first year of free-threaded Python

#228

Post by zahlman »

delusional wrote: post #71
bayindirh wrote: post #42

Earlier quoted context omitted.

Software rots, software tools evolve. When Intel released performance primitives libraries which required recompilation to analyze multi-threaded libraries, we were amazed. Now, these tools are built into processors as performance counters and we have way more advanced tools to analyze how systems behave. Older code will break, but they break all the time. A language changes how something behaves in a new revision, s…

> Software rots No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots. The only reason we need to change software is that we trade that off against something else. Instr…

>execute the same binary

Only if you statically compile or don't upgrade your dependencies. Or don't allow your dependencies to innovate.

Re: The first year of free-threaded Python

#229

Post by zahlman »

kstrauser wrote: post #76
delusional wrote: post #71

Earlier quoted context omitted.

> Software rots No it does not. I hate that analogy so much because it leads to such bad behavior. Software is a digital artifact that can does not degrade. With the right attitude, you'd be able to execute the same binary on new machines for as long as you desired. That is not true of organic matter that actually rots. The only reason we need to change software is that we trade that off against something else. Instr…

That’s not what the phrase implies. If you have a C program from 1982, you can still compile it on a 1982 operating system and toolchain and it’ll work just as before. But if you tried to compile it on today’s libc, making today’s syscalls… good luck with that. Software “rots” in the sense that it has to be updated to run on today’s systems. They’re a moving target. You can still run HyperCard on an emulator, but goo…

> You can still run HyperCard on an emulator, but good luck running it unmodded on a Mac you buy today.

I grew up with HyperCard, so I had a moment of sadness here.

Re: The first year of free-threaded Python

#230

Post by zahlman »

rocqua wrote: post #134
quectophoton wrote: post #7

Earlier quoted context omitted.

I don't want to add more to your fears, but also remember that LLMs have been trained on decades worth of Python code that assumes the presence of the GIL.

This could, indeed, be quite catastrophic. I wonder if companies will start adding this to their system prompts.

Suppose they do. How is the LLM supposed to build a model of what will or won't break without a GIL purely from a textual analysis?

Especially when they've already been force-fed with ungodly amounts of buggy threaded code that has been mistakenly advertised as bug-free simply because nobody managed to catch the problem with a fuzzer yet (and which is more likely to expose its faults in a no-GIL environment, even though it's still fundamentally broken with a GIL)?

Post reply on HN