Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.14 garbage collection rigamarole

theconsensus.dev

Post reply on HN
Print view View original

61–70 of 85 posts

Re: Python 3.14 garbage collection rigamarole

#61

Post by zelphirkalt »

_old_dude_ wrote: post #31

The usual GC tradeoff is between memory and CPU performance. If you set the memory max high, the GC will run less often, you get less pause time. So I do not understand why it's a surprise that minimizing the pause time requires more memory. Is it because there is no knob to set either the max pause time or the max memory ?

When GC runs less often, doesn't that rather increase the amount of things needed to be cleaned up _when_ GC finally runs? So actually the GC time _at that point_ should be longer, not shorter, while in total summed up, it should be shorter. Running GC often means cleaning up less stuff for every GC run, so the GC time for each run should be shorter.

Do I have any misconceptions?

Re: Python 3.14 garbage collection rigamarole

#62

Post by nosioptar »

zelphirkalt wrote: post #61
_old_dude_ wrote: post #31

The usual GC tradeoff is between memory and CPU performance. If you set the memory max high, the GC will run less often, you get less pause time. So I do not understand why it's a surprise that minimizing the pause time requires more memory. Is it because there is no knob to set either the max pause time or the max memory ?

When GC runs less often, doesn't that rather increase the amount of things needed to be cleaned up _when_ GC finally runs? So actually the GC time _at that point_ should be longer, not shorter, while in total summed up, it should be shorter. Running GC often means cleaning up less stuff for every GC run, so the GC time for each run should be shorter. Do I have any misconceptions?

I think you're right, based on screwing around with Flash's GC a couple decades ago. (Iirc, you could only really control it with the debug player.)

Re: Python 3.14 garbage collection rigamarole

#64

Post by jeremycarter »

tannhaeuser wrote: post #33

Do people use python for new projects apart from ML stuff which hasn't moved to all-native yet? My experience with Python is a really bad one for professional work: it's chaotic and slow, and has by far the worst versioning and packaging story of any mainstream language, yet its proponents keep praising it in denial. I guess Python is an ok target for agentic coding, but my god do look Claude's commit messages preten…

I find the cult of python is hard to shift from some people. They just can't see it.

Re: Python 3.14 garbage collection rigamarole

#65

Post by ameliaquining »

carra wrote: post #58

I see in the front page Pyodide 314.0 and now also Python 3.14. Is it a coincidence or is there some meaning to these "pi versions" in Python?

In Pyodide's new versioning scheme, the major version is the concatenated major and minor versions of the corresponding Python release. Pyodide 314 is based on Python 3.14 (which is just the fifteenth minor version of Python 3, counting from zero, and just happens to match the common approximation of pi).

Re: Python 3.14 garbage collection rigamarole

#66

Post by scott_w »

vlovich123 wrote: post #59
scott_w wrote: post #18

Earlier quoted context omitted.

Anyone who’s worked in incident response will tell you why you’re wrong. Tweaking the GC while the system was functionally broken is the worst time to do it. Correct incident response is revert first, figure out how to fix it later.

The difference being this is not a live system and thus incident response is very different. Applying best practices from incident response to development of a language is simply incorrect

You're simply incorrect here. The GC was in versions 3.14.0 to 3.14.4. See https://docs.python.org/3/whatsnew/3.14.html#whatsnew314-inc...

On what planet is the currently released version of any software "not a live system?"

Re: Python 3.14 garbage collection rigamarole

#67

Post by zahlman »

moron4hire wrote: post #55
zahlman wrote: post #27

Earlier quoted context omitted.

> Why couldn’t the core team and the core-adjacent PyPA make a tool as liked as uv Incredibly large backward-compatibility burdens [0], internal politics [1] and general institutional dysfunction (no footnote; if you're familiar then you need no explanation, and if you aren't then the less said the better). Nothing to do with Python as a language. Most of the important pieces to get uv-like performance are algorithmi…

Any claims that Python has a huge backwards compat mission go right out the window when you consider Python 3. 3 was a perfect chance to fix all of the major problems with Python, problems other languages have solved so there isn't even a need to invent things from scratch. They didn't and that's why the community is still split on adoption.

I have no idea what you're talking about. 3.x objectively did fix the major problems and was not particularly inventive in its approach. The fact that they didn't fix more things all at once — and the fact that they still received massive complaints about supposedly fixing too much — is the origin of that backwards compatibility mission. Actually talking to the devs for any significant length of time makes it clear how much lasting trauma was caused by the change, and how much that cemented pro-backwards-compatibility views. And the community has not been remotely "split on adoption" for years; hardly anyone publishes packages advertised as 2.x compatible any more (e.g. urllib3 gave up almost two years ago). But to the extent that there have been any holdouts at all, it's been people who want ancient systems to work, not people who think that not enough changed to merit upgrading.

Re: Python 3.14 garbage collection rigamarole

#68

Post by vlovich123 »

scott_w wrote: post #66
vlovich123 wrote: post #59

Earlier quoted context omitted.

The difference being this is not a live system and thus incident response is very different. Applying best practices from incident response to development of a language is simply incorrect

You're simply incorrect here. The GC was in versions 3.14.0 to 3.14.4. See https://docs.python.org/3/whatsnew/3.14.html#whatsnew314-inc... On what planet is the currently released version of any software "not a live system?"

Do you live on a planet where the Python language maintainers are deploying Python into your servers or managing them? Do you live on a planet where a new version of Python being released gets instantly and automatically deployed into your systems without testing and validation?

You are responsible for that, not them. And if Python 3.13 is fine for you and you report a performance regression for 3.14, you can still stay on 3.13. And as you say, it was introduced in a new release. What happens when the other side goes and says “3.14.5 regressed on the GC pause times and my p95 web server latencies went up. Please revert”? At least one side can make the case “performance was changed on a major release of Python boundary” while the other is changing the performance on a minor release boundary. It’s an arbitrary decision that speaks to the politics of the organization and less about a well reasoned technical plan.

Re: Python 3.14 garbage collection rigamarole

#69

Post by elfly »

nomel wrote: post #12
vlovich123 wrote: post #7

I suspect 3.14.4 could have been tweaked slightly to address the issue without a revert - they could have prioritized checking the liveliness of objects sorted by size. I’m pretty sure that would fix the max RSS issue without needing a revert and the people unhappy with 3.14 could keep using 3.13 or switch to 3.14 and simply inject explicit calls to gc.gc(). Figuring out how to measure the size of an object can be tr…

> I suspect 3.14.4 could have been tweaked slightly to address the issue without a revert I'm sure all the people that have been working on this for years would be interested in your small tweak, that they didn't think of, and would happily accept the PR!

Maybe they could have included an option to switch between GC implementations, like Java had done before.

Maybe in all those years they could have thought of that.

Re: Python 3.14 garbage collection rigamarole

#70

Post by fulafel »

zelphirkalt wrote: post #61
_old_dude_ wrote: post #31

The usual GC tradeoff is between memory and CPU performance. If you set the memory max high, the GC will run less often, you get less pause time. So I do not understand why it's a surprise that minimizing the pause time requires more memory. Is it because there is no knob to set either the max pause time or the max memory ?

When GC runs less often, doesn't that rather increase the amount of things needed to be cleaned up _when_ GC finally runs? So actually the GC time _at that point_ should be longer, not shorter, while in total summed up, it should be shorter. Running GC often means cleaning up less stuff for every GC run, so the GC time for each run should be shorter. Do I have any misconceptions?

Depends on the GC. For some the amount of work is related just to live objects.
Post reply on HN