Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverting the incremental GC in Python 3.14 and 3.15

discuss.python.org

Post reply on HN
Print view View original

81–90 of 134 posts

Re: Reverting the incremental GC in Python 3.14 and 3.15

#81

Post by pron »

winrid wrote: post #75
pron wrote: post #28

Earlier quoted context omitted.

Java lost almost all those knobs a while ago (I mean they're there, but you're better off relying on the defaults). The modern GCs have one or at most two knobs remaining, and even that will become unnecessary next year. As to predictablity, you get maximal pause time of well under 1ms for heaps up to 16TB.

The max pause time thing is a meme :) I have gotten multi second pause times with ZGC. It depends on what hardware you run it on.

The new generational ZGC? I'm sceptical.

Re: Reverting the incremental GC in Python 3.14 and 3.15

#82

Post by pron »

refulgentis wrote: post #77
pron wrote: post #28

Earlier quoted context omitted.

Java lost almost all those knobs a while ago (I mean they're there, but you're better off relying on the defaults). The modern GCs have one or at most two knobs remaining, and even that will become unnecessary next year. As to predictablity, you get maximal pause time of well under 1ms for heaps up to 16TB.

Next year? Do tell

https://openjdk.org/jeps/8377305

Re: Reverting the incremental GC in Python 3.14 and 3.15

#83

Post by murderfs »

refulgentis wrote: post #79
coldtea wrote: post #71

Earlier quoted context omitted.

> As far as I know, java has 7 GC implementations, none of which are perfect, all of which have drawbacks Compared to Python's, all of them are beyond perfect. And 99.9% of the time you don't even need to use anything but the default.

> Compared to Python's, all of them are beyond perfect. I somehow understand the situation less after reading this. Is Python's GC bad , or are there cyclic reference issues? Is it possible to detect cyclic references perfectly? What does beyond perfect mean? If we have 7 and 0.1% of the time you need one of the 6 that is non-default, how do we choose? Is the understated version of "Compared to Python's, all of them…

> Is Python's GC bad, or are there cyclic reference issues?

Unless you're being pedantic and including reference counting without cycle detection as GC, if your GC has cyclic reference issues, your GC is bad.

> Is it possible to detect cyclic references perfectly?

Yes? That's the entire point of tracing GC. You have some set of root objects that you start with (globals, objects on thread stacks, etc.) and then you mark every object that's reachable from them. Anything that's not reachable is garbage, even if there are cycles within them.

Re: Reverting the incremental GC in Python 3.14 and 3.15

#84

Post by athoscouto »

nas wrote: post #73
athoscouto wrote: post #23

We've been impacted by this. I migrated our services to Python 3.14 so we could attach profilers during runtime. A couple of services looked like they had a memory leak. Memory was continuously increasing over time. Thanks to Python 3.14, we were able to use memray to understand what was going on. Those services were recreating HTTP clients (aiohttp) for every inbound request, and memory allocated by the downstream S…

If you are using "httpx", it's likely caused by a reference cycle. I made a PR to fix it but the maintainers haven't applied it. :-( https://github.com/encode/httpx/pull/3733 The reference cycle httpx creates is kind of a worst-case scenario for the incremental GC issue. Both the generational (3.13 and older) and incremental GC are triggered by the net new "container" objects (objects that have references to others,…

It was httpx indeed. i had aiohttp in mind because we ended up replacing that particular client with it

Re: Reverting the incremental GC in Python 3.14 and 3.15

#85

Post by remexre »

winrid wrote: post #75
pron wrote: post #28

Earlier quoted context omitted.

Java lost almost all those knobs a while ago (I mean they're there, but you're better off relying on the defaults). The modern GCs have one or at most two knobs remaining, and even that will become unnecessary next year. As to predictablity, you get maximal pause time of well under 1ms for heaps up to 16TB.

The max pause time thing is a meme :) I have gotten multi second pause times with ZGC. It depends on what hardware you run it on.

Have a reproducer?

Re: Reverting the incremental GC in Python 3.14 and 3.15

#86

Post by pjmlp »

jaapz wrote: post #65
pjmlp wrote: post #62

Earlier quoted context omitted.

The only thing Go has going for it was getting lucky with Docker and co, and UNIX/Plan 9/Inferno pedigree. Now we're stuck with it in anything CNCF related.

I like my programming language flame wars just as much as the next guy but Go is a really easy language to get started with, while also being very fast. It's not just luck

> The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt.

-- Rob Pike

Re: Reverting the incremental GC in Python 3.14 and 3.15

#87

Post by pjmlp »

stackskipton wrote: post #69
onlyrealcuzzo wrote: post #60

Earlier quoted context omitted.

Well, .NET is just not in the same class as Go and Rust. Go is, essentially, nearly perfect at what it does - even if the language itself leaves much to be desired and would ideally be much safer. Microsoft should up their game. They have a few research languages in development. They've always been great with languages. Hopefully, they rise to the occassion.

What? If you are talking web development, .Net is just about the same as Go. It's 100% Java OOP type writing but result is same, very performant API server. Sure, Rust is completely different beast with different target system.

Java 1.5 kind of thing, with plenty error handling boilerplate, errors as strings, and SCM urls straight in the code...

Re: Reverting the incremental GC in Python 3.14 and 3.15

#89

Post by JimDabell »

nas wrote: post #73
athoscouto wrote: post #23

We've been impacted by this. I migrated our services to Python 3.14 so we could attach profilers during runtime. A couple of services looked like they had a memory leak. Memory was continuously increasing over time. Thanks to Python 3.14, we were able to use memray to understand what was going on. Those services were recreating HTTP clients (aiohttp) for every inbound request, and memory allocated by the downstream S…

If you are using "httpx", it's likely caused by a reference cycle. I made a PR to fix it but the maintainers haven't applied it. :-( https://github.com/encode/httpx/pull/3733 The reference cycle httpx creates is kind of a worst-case scenario for the incremental GC issue. Both the generational (3.13 and older) and incremental GC are triggered by the net new "container" objects (objects that have references to others,…

> I made a PR to fix it but the maintainers haven't applied it. :-( https://github.com/encode/httpx/pull/3733

Unfortunately, you may be the wrong gender to contribute to Encode repositories like httpx:

> I've closed off access to issues and discussions.

> I don't want to continue allowing an online environment with such an absurdly skewed gender representation. I find it intensely unwelcoming, and it's not reflective of the type of working environments I value.

— https://github.com/encode/httpx/discussions/3784

Discussed on Hacker News here: https://news.ycombinator.com/item?id=47193563

A fork discussed here: https://news.ycombinator.com/item?id=47514603

Re: Reverting the incremental GC in Python 3.14 and 3.15

#90

Post by wiseowise »

__loam wrote: post #38
mywittyname wrote: post #36

Earlier quoted context omitted.

Why are you miffed about it? I legitimately hate reading golang with passion and find python to be pretty intuitive, outside of the odd ambitious list comprehensions. I worked in a golang shop for several years, so it's not just an familiarity situation either. We are just different. That's not something to be mad about.

[flagged]

> Python is a garbage language. Dynamic types are a disaster

Python has gradual type system.

Post reply on HN