Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The first year of free-threaded Python

labs.quansight.org

Post reply on HN
Print view View original

291–300 of 302 posts

Re: The first year of free-threaded Python

#291

Post by westurner »

kstrauser wrote: post #241
Izkata wrote: post #237

Earlier quoted context omitted.

There's already a global: import gc gc.disable() So I imagine putting more in there to remove objects from the tracking.

That can go a long way, so long as you remember to manually GC the handful of things you do care about.

Is there a good way to add __del__() methods or to wrap Context Manager __enter__()/__exit__() methods around objects that never needed them because of the gc?

Hadn't seen this:

  import gc
  gc.disable()
Cython has __dealloc__() instead of __del__()?

Re: The first year of free-threaded Python

#292

Post by oconnor663 »

celeritascelery wrote: post #98
rfoo wrote: post #26

Earlier quoted context omitted.

> Does free-threaded Python provide the same guarantees Mostly. Some of the "can be pre-empted on the boundary between any two bytecode instructions" bugs are really hard to hit without free-threading, though. And without free-threading people don't use as much threading stuff. So by nature it exposes more bugs. Now, my rants: > have any other effects on multi-threaded Python code It stops people from using multi-pro…

> That's the trade-off. Personally I think a single digit percentage slow-down of single-threaded code worth it. Maybe. I would expect that 99% of python code going forward will still be single threaded. You just don’t need that extra complexity for most code. So I would expect that python code as a whole will have worse performance, even though a handful of applications will get faster.

Sure but of those 99%, how many are performance-sensitive, CPU-bound (in Python not in C) applications? It's clearly some, not saying it's an easy tradeoff, but I assume the large majority of Python programs out there won't notice a slowdown.

Re: The first year of free-threaded Python

#293

Post by westurner »

westurner wrote: post #291
kstrauser wrote: post #241

Earlier quoted context omitted.

That can go a long way, so long as you remember to manually GC the handful of things you do care about.

Is there a good way to add __del__() methods or to wrap Context Manager __enter__()/__exit__() methods around objects that never needed them because of the gc? Hadn't seen this: import gc gc.disable() Cython has __dealloc__() instead of __del__()?

Also, there's a recent proposal to add explicit resource management to JS: "JavaScript's New Superpower: Explicit Resource Management" https://news.ycombinator.com/item?id=44012227

Re: The first year of free-threaded Python

#294

Post by igouy »

smilliken wrote: post #289
igouy wrote: post #176

Earlier quoted context omitted.

We say the context has breaking changes. We say the context is not backwards compatible.

Can you see how this comes off as a pedantic difference? If I ran a program 10 years ago and it worked, then run it today and it doesn't work, we say the program is broken and needs to be updated. We don't say the world around it is broken and needs to revert back to its original state.

We do revert back to a previous context if that seems practical: revert back to a previous compiler or library version.

Re: The first year of free-threaded Python

#295

Post by seabrookmx »

dontlaugh wrote: post #276
seabrookmx wrote: post #243

Earlier quoted context omitted.

While it certainly has its rough edges, I'm a big asyncio user. So I'll be over here happily writing concurrent python that's single threaded, ie. pretending my Python is nodejs. For the web/network workloads most of us write, I'd highly recommend this.

Asyncio being able to use thread pools would reduce memory usage at the very least.

How so? As opposed to running multiple processes you mean?

Re: The first year of free-threaded Python

#296

Post by dontlaugh »

seabrookmx wrote: post #295
dontlaugh wrote: post #276

Earlier quoted context omitted.

Asyncio being able to use thread pools would reduce memory usage at the very least.

How so? As opposed to running multiple processes you mean?

Exactly, you could do like other runtimes and run a single process that can saturate all cores.

You wouldn’t be duplicating the interpreter, your code, config, etc.

Re: The first year of free-threaded Python

#297

Post by Flamentono2 »

falcor84 wrote: post #85
biorach wrote: post #68

Earlier quoted context omitted.

> Well, you sure managed to avoid that by setting up camp on that hill. Kudos on so much time saved. Why are you picking a fight about this?

I think I'm taking it personally because I had previously changed my name and had people repeatedly call me by my old name just to annoy/hurt me. Obviously I know that companies aren't people and don't have feelings, but I can't understand why you would intentionally avoid using their chosen name, even when it's more effort to you.

Oh in this particular case i do this on purpose too.

To make it very clear that people don't forget were the Money is coming from.

Its Facebook, its Facebook money

Re: The first year of free-threaded Python

#298

Post by dotancohen »

ptx wrote: post #227
dotancohen wrote: post #218

Earlier quoted context omitted.

Thank you. Perhaps I should trigger the transcription thread from the UI thread, then? It is a UI button that initiates it after all.

The tricky part is coming back onto the UI thread when the background work finishes. Your transcription thread has to somehow trigger the UI work to be done on the UI thread. It seems the way to do it in Qt is with signals and slots, emitting a signal from your QThread and binding it to a slot in the UI thread, making sure to specify a "queued connection" [1]. There's also a lower-level postEvent method [2] but peopl…

Terrific, thank you. You've put me on the right track.

Re: The first year of free-threaded Python

#299

Post by duped »

kragen wrote: post #250
duped wrote: post #236

Earlier quoted context omitted.

Tbh if you're optimizing python code you've already lost

On a 64-core machine, Python code that uses all the cores will be modestly faster than single-threaded C, even if all the inner loops are in Python. If you can move the inner loops to C, for example with Numpy, you can do much better still. (Python is still harder to get right than something like C or OCaml, of course, especially for larger programs, but often the smaller amount of code and quicker feedback loop can…

I strongly doubt this claim. Python is more than 64x slower than C without synchronization overhead in most numeric tasks, with synchronization overhead on those processes it should be much worse.

Python is so much slower than any native or JIT compiled language that it begets things like numpy in the first place.

Re: The first year of free-threaded Python

#300

Post by tinix »

sgarland wrote: post #220
tinix wrote: post #150

Earlier quoted context omitted.

shared memory only works on dedicated hardware. if you're running in something like AWS fargate, there is no shared memory. have to use the network and file system which adds a lot of latency, way more than spawning a process. copying processes through fork is a whole different problem. green threads and an actor model will get you much further in my experience.

Well don’t use Fargate, there’s your problem. Run programs on actual servers, not magical serverless bullshit.

> Well don’t use Fargate, there’s your problem. Run programs on actual servers, not magical serverless bullshit.

That kind of absolutism misses the point of why serverless architectures like Fargate exist. It might feel satisfying, but it closes the door on understanding why stateless and ephemeral workloads exist in the first place.

I get the frustration, but dismissing a production architecture outright ignores the constraints and trade-offs that lead teams to adopt it in the first place. It's worth asking: if so many teams are using this shit in production, at scale, with real stakes, what do they know that might be missing from my current mental model?

Serverless, like any abstraction, isn't magic. It's a tool with defined trade-offs, and resource/process isolation is one of them. If you're running containerized workloads at scale, optimizing for organizational velocity, security boundaries, and multi-tenant isolation, these constraints aren't bullshit, they're literally design parameters and intentional boundaries.

It's easy to throw shade from a distance, but the operational reality of running modern systems, especially in regulated or high-scale environments, looks very different from a home lab or startup sandbox.

Post reply on HN