Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State of Python 3.13 performance: Free-threading

codspeed.io

Post reply on HN
Print view View original

121–130 of 193 posts

Re: State of Python 3.13 performance: Free-threading

#121

Post by dagw »

salomonk_mur wrote: post #102
andai wrote: post #24

Earlier quoted context omitted.

There was a discussion the other day about how Python devs apparently don't care enough for backwards compatibility. I pointed out that I've often gotten Python 2 code running on Python 3 by just changing print to print(). But then a few hours later, I tried running a very small project I wrote last year and it turned out that a bunch of my dependencies had changed their APIs. I've had similar (and much worse) experi…

What APIs were broken? They couldn't be in the standard library. If the dependency was in external modules and you didn't have pinned versions, then it is to be expected (in almost any active language) that some APIs will break.

They couldn't be in the standard library.

Why not? Python does make breaking changes to the standard library when going from 3.X to 3.X+1 quite regularly.

Re: State of Python 3.13 performance: Free-threading

#122

Post by the5avage »

nas wrote: post #112
the5avage wrote: post #111

Earlier quoted context omitted.

Is this implemented with lockless programming? Is it a reason for the performance drop in single thread code? Does it eliminate the need for a GC pause completely?

You should probably just read the PEP, which explains these things: https://peps.python.org/pep-0703/#reference-counting If by GC you mean the cyclic GC, free-threaded Python currently stops all threads while the cyclic GC is running.

Thank you:)

Re: State of Python 3.13 performance: Free-threading

#123

Post by jghag »

notatallshaw wrote: post #70
refdl wrote: post #25

[flagged]

You seem to be conflating problems and different groups of people that aren't directly related. To clarify some different groups: * The faster-cpython project, headed by Guido and his team at Microsoft, is continuing fine, most of the low hanging fruit was accomplished by 3.11, further improvements were moved to medium term goals and they were delayed further by the free threaded project which broke assumptions that…

Thanks for the party line, backed by the usual flaggers. This is how Python maintains its market share and popularity. Google though does not seem too impressed by Python any longer. Others will follow.

Re: State of Python 3.13 performance: Free-threading

#124

Post by rfoo »

willseth wrote: post #98
rfoo wrote: post #40

Earlier quoted context omitted.

> Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible Scientific computing community have a bunch of code calling numpy or whatever stuff. They are pretty fast because, well, numpy isn't written in Python. However, there is a scalability issue: they can only drive so many threads (not 1, but not many) in a process due to GIL. Okay,…

This is misleading. Most of the compute intensive work in Numpy releases the GIL, and you can use traditional multithreading. That is the case for many other compute intensive compiled extensions as well.

No. Like the siblings said, say, you have a program which spends 10% time in Python code between these numpy calls. The code is still not scalable, because you can run at most 10 such threads in a Python process before you hit the hard limit imposed by GIL.

There is no need to eliminate the 10% or make it 5% or whatever, people happily pay 10% overhead for convenience, but being limited to 10 threads is a showstopper.

Re: State of Python 3.13 performance: Free-threading

#125

Post by epistasis »

WillDaSilva wrote: post #118
lmm wrote: post #113

Earlier quoted context omitted.

> I'm not sure it is discouraged so much as just not what people did in Python-land for a long time. It's obviously the right thing to do, it's totally doable, it's just inertia and habit that might mean it isn't done. Pinning obviously the wrong thing, it only works if everyone does it and if everyone does it then making changes becomes very hard. The right thing is to have deterministic dependency resolution so tha…

When they suggest you pin your dependencies, they don't just mean your direct dependencies, but rather all transitive dependencies. You can take this further by having a lock file that account for different Python versions, operating systems, and CPI architectures – for instance , by using UV or Poetry – but a simple `pip freeze` is often sufficient.

That works for your project, but then nobody can include you as a library without conflicts.

But having that lock file will allow somebody to reconstruct your particular moment in time in the future. Its just that those lock files do not exist for 99.9% of Python projects in time.

Re: State of Python 3.13 performance: Free-threading

#126

Post by cdchn »

andai wrote: post #60
adamc wrote: post #34

Earlier quoted context omitted.

It's hard to comment on this without knowing more about the dependencies and when/how they changed their APIs. I would say if it was a major version change, that isn't too shocking. For a minor version change, it should be. Stuff that is actually included with Python tends to be more stable than random Pypi packages, though. NPM packages also sometimes change. That's the world.

Yeah, I guess I should have done a pip freeze to specify the versions in the requirements file. I wasn't thinking ahead. Turns out one dependency had 3 major releases in the span of a year! (Which basically confirms what I was saying, though I don't know how typical that is.)

3rd party package maintainers usually don't do as good a job of maintaining backwards compatibility or doing it right as do the core library maintainers, thats why you were able to upgrade from 2 to 3 by changing print to print() but then sometimes dependencies you install with pip break for inexplicable reasons.

Re: State of Python 3.13 performance: Free-threading

#127

Post by whyever »

elashri wrote: post #99
Timon3 wrote: post #58

Earlier quoted context omitted.

The big difference is that npm will automatically (since 2017) save a version range to the project metadata, and will automatically create this metadata file if it doesn't exist. Same for other package managers in the Node world. I just installed Python 3.13 with pip 24.2, created a venv and installed a package - and nothing, no file was created and nothing was saved. Even if I touch requirements.txt and pyproject.to…

> Of course there are other package managers for Python that do this better I think if you are comparing with what NPM does then you would have to say that native pip can do that too. It is just one command `pip freeze > requirements.txt` It does include everything in the venv (or in you environment in general) but if you stick to only add required things (one venv for each project) then you will get requirements.txt…

I don't think this is the same. Does it also cover transitive dependencies?

Re: State of Python 3.13 performance: Free-threading

#128

Post by jessekv »

__mharrison__ wrote: post #109
Timon3 wrote: post #58

Earlier quoted context omitted.

The big difference is that npm will automatically (since 2017) save a version range to the project metadata, and will automatically create this metadata file if it doesn't exist. Same for other package managers in the Node world. I just installed Python 3.13 with pip 24.2, created a venv and installed a package - and nothing, no file was created and nothing was saved. Even if I touch requirements.txt and pyproject.to…

I recommend to start using UV. It is very fast and tracks the libraries you are using. After years of venv/pip, I'm not going back (unless a client requires it).

Another nice thing about uv is it can install python itself in the venv.

So no need to mess around with brew/deadsnakes and multiple global python versions on your dev system.

This is actually an improvement over the node/nvm approach.

Re: State of Python 3.13 performance: Free-threading

#129

Post by graemep »

eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

I do use Python and I am not that bothered about speed.

Very little of what I use it for has performance bottlenecks in the Python. Its the database or the network or IO or whatever.

On the few occasions when it does I can rewrite critical bits of code.

I definitely care more about backward compatibility than I do about performance.

It feels like Python development is being driven by the needs of one particular group (people who use ML heavily, possibly because they have deep pockets) and I wonder whether this, and a few other things will make it less attractive a language for me and others.

Re: State of Python 3.13 performance: Free-threading

#130

Post by elashri »

whyever wrote: post #127
elashri wrote: post #99

Earlier quoted context omitted.

> Of course there are other package managers for Python that do this better I think if you are comparing with what NPM does then you would have to say that native pip can do that too. It is just one command `pip freeze > requirements.txt` It does include everything in the venv (or in you environment in general) but if you stick to only add required things (one venv for each project) then you will get requirements.txt…

I don't think this is the same. Does it also cover transitive dependencies?

Sorry if what I said about NPM is not accurate. But in reality if you are pinning the dependencies (all of them actual get pinned) then when pip is installing it will grab the correct version of the transitive dependency (both packages are pinned)

So I am not sure when this will become a problem.

Post reply on HN