Earlier quoted context omitted.
No we should write one of the many modern programming languages that handle certain projects way better, including kotlin, go, or Java. The only things python is best in class at are scripting and as a harness for high performance c++ or fortran.
What about the projects that python handles better?
Reverting the incremental GC in Python 3.14 and 3.15
121–130 of 134 posts
Re: Reverting the incremental GC in Python 3.14 and 3.15
#122Re: Reverting the incremental GC in Python 3.14 and 3.15
#123Earlier quoted context omitted.
The max pause time thing is a meme :) I have gotten multi second pause times with ZGC. It depends on what hardware you run it on.
The new generational ZGC? I'm sceptical.
Re: Reverting the incremental GC in Python 3.14 and 3.15
#124Earlier quoted context omitted.
Are you not confusing GC (freeing memory) with the memory allocator ? Memory allocator: tcmalloc, jemalloc, they are concerned with fetching (and releasing) pages of memory from the OS and allocating objects for the program GC is only responsible for saying to the memory allocator "this object is no longer used" (please stay focused on java)
> GC is only responsible for saying to the memory allocator "this object is no longer used" This is simply not true. Not only are Java's GCs responsible for allocation (which they do simply by bumping a pointer, similar to stack allocation), unlike Python's refcounting collector or Go's nonmoving tracing collector, they have no free operation of any kind and never free objects. Moving collectors don't even know when…
Re: Reverting the incremental GC in Python 3.14 and 3.15
#125Earlier quoted context omitted.
We've been chasing down similar aiohttp client creation issues (liked to ...aiobotocore usage) for months now. It's annoying that somehow talking to S3 etc requires so much churn. We have been trying to cache session objects and the like but clearly are still missing something. Chasing this down has also made me realize how little Python libs use `weakref`, and just will build up so many circular references. The othe…
It's really fascinating to read this, since I've encountered similar memory issues in other languages (ruby, go, etc.). Debugging these issues is a pain. Is there a way to make all this much easier to debug and to prevent memory issues in the first place? Is the abstraction level not quite right?
So for example
class Request
class Session
request.session exists, and the session is "part" of the request. but session.request often exists as a facility. That's a reference cycle which prevents the request (and anything it's pointed at!) from being deallocated at the end of a request.
But in this case, you could easily do something like:
session._request = weakref.ref(request) # on session creation
and then have session.request call session._request() (and maybe assert session._request() is not None if you want to be certain). If you're confident that the session is a "child" of the request, and that you would _never_ have a hold of the session after the request is done, this is a cheap trick that makes session.request cost a little bit more but not much.
I think most Python libraries just don't do memory perf analyses here, and also "believe" in the garbage collector. When GC runs, both request and session will get deallocated, after all! But the long term effects of everyone relying on the GC are that GC is expensive when it doesn't need to be, and when looking through memory you just have more stuff to dig through
Re: Reverting the incremental GC in Python 3.14 and 3.15
#126Re: Reverting the incremental GC in Python 3.14 and 3.15
#127Earlier quoted context omitted.
Why are people still building systems on top of a language that continually undergoes fundamental changes nearly 40 years after release? Is this not the strongest indication that this language is not well designed, it is unstable, and encounters many issues that flat out don't exist in other high level languages?
What language that is actually used 40 years after release isn't undergoing big, fundamental changes? Java? Nope, you're getting a fundamental change in Valhalla C++? Nope, new language edition every few years with fundamental changes C? C23 has a number of fairly fundamental changes, expect more in the next language revision I think your sense of causality is backwards here. These languages are getting fundamental c…
That's fine, but that's clearly not what I'm talking about.
Languages like F#, Elixir, etc. don't undergo fundamental changes. Yes, every language evolves. But for Python, we're talking about grafting literally fundamental stuff on top of a language not designed for any of these things.
For example, if someone went and redesigned Python to solve its warts, you'd basically end up with F#.
Re: Reverting the incremental GC in Python 3.14 and 3.15
#128Earlier quoted context omitted.
I like my programming language flame wars just as much as the next guy but Go is a really easy language to get started with, while also being very fast. It's not just luck
> The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt. -- Rob Pike
Re: Reverting the incremental GC in Python 3.14 and 3.15
#129Earlier quoted context omitted.
The new generational ZGC? I'm sceptical.
So if I run it on a Pentium 3 you're telling me I can't get a long pause time? "1ms" means nothing
Re: Reverting the incremental GC in Python 3.14 and 3.15
#130Earlier quoted context omitted.
> GC is only responsible for saying to the memory allocator "this object is no longer used" This is simply not true. Not only are Java's GCs responsible for allocation (which they do simply by bumping a pointer, similar to stack allocation), unlike Python's refcounting collector or Go's nonmoving tracing collector, they have no free operation of any kind and never free objects. Moving collectors don't even know when…
That's not true, shenadoah does decommit and will continue to in the new generational version.