Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State of Python 3.13 performance: Free-threading

codspeed.io

Post reply on HN
Print view View original

131–140 of 193 posts

Re: State of Python 3.13 performance: Free-threading

#131

Post by winrid »

graemep wrote: post #129
eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

I do use Python and I am not that bothered about speed. Very little of what I use it for has performance bottlenecks in the Python. Its the database or the network or IO or whatever. On the few occasions when it does I can rewrite critical bits of code. I definitely care more about backward compatibility than I do about performance. It feels like Python development is being driven by the needs of one particular group…

Your DB is probably faster than you think. I rewrote an API in Python to Java and it is around 6x faster with just same dumb N+1 queries, and the new API also includes all the frontend calculations that Python wasn't doing before.

Re: State of Python 3.13 performance: Free-threading

#132

Post by kwertzzz »

andai wrote: post #24
eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

There was a discussion the other day about how Python devs apparently don't care enough for backwards compatibility. I pointed out that I've often gotten Python 2 code running on Python 3 by just changing print to print(). But then a few hours later, I tried running a very small project I wrote last year and it turned out that a bunch of my dependencies had changed their APIs. I've had similar (and much worse) experi…

Sadly, several python projects do not use semantic versioning, for example xarray [0] and dask. Numpy can make backward incompatible changes after a warning for two releases[1]. In general, the python packaging docs do not really read as an endorsement of semantic versioning [2]:

> A majority of Python projects use a scheme that resembles semantic versioning. However, most projects, especially larger ones, do not strictly adhere to semantic versioning, since many changes are technically breaking changes but affect only a small fraction of users...

[0] https://github.com/pydata/xarray/issues/6176

[1] https://numpy.org/doc/stable/dev/depending_on_numpy.html

[2] https://packaging.python.org/en/latest/discussions/versionin...

Re: State of Python 3.13 performance: Free-threading

#133

Post by DeathArrow »

eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

>Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "real" semantics.

Have you heard of Mojo? It is a very performant superset of Python. https://www.modular.com/mojo

Re: State of Python 3.13 performance: Free-threading

#134

Post by pjmlp »

eigenspace wrote: post #30
jillesvangurp wrote: post #29

Earlier quoted context omitted.

Why does python have to be slow? Improvements over the last few releases have made it quite a bit faster. So that kind of counters that a bit. Apparently it didn't need to be quite as slow all along. Other languages can be fast. So, why not python? I think with the GIL some people are overreacting: most python code is single threaded because of the GIL. So removing it doesn't actually break anything. The GIL was just…

> Why does python have to be slow? Because the language's semantics promise that a bunch of insane stuff can happen at any time during the running of a program, including but not limited to the fields of classes changing at any time. Furthermore, they promise that their integers are aribtrary precision which are fundamentally slower to do operations with than fixed precision machine integers, etc. The list of stuff l…

SELF, Smalltalk, Common Lisp are even crazier and have good performance, that has always been a common excuse, which is being proven false as proper resources are now being addressed into CPython.

And that while ignoring all the engineering that has gone into PyPy, largely ignored by the community.

Re: State of Python 3.13 performance: Free-threading

#135

Post by dagw »

epistasis wrote: post #125
WillDaSilva wrote: post #118

Earlier quoted context omitted.

When they suggest you pin your dependencies, they don't just mean your direct dependencies, but rather all transitive dependencies. You can take this further by having a lock file that account for different Python versions, operating systems, and CPI architectures – for instance , by using UV or Poetry – but a simple `pip freeze` is often sufficient.

That works for your project, but then nobody can include you as a library without conflicts. But having that lock file will allow somebody to reconstruct your particular moment in time in the future. Its just that those lock files do not exist for 99.9% of Python projects in time.

That works for your project, but then nobody can include you as a library without conflicts.

I think this is the core to much misunderstandings and arguments around this question. Some people are writing code that only they will run, on a python they've installed, on hardware they control. Others are writing code that has to work on lots of different versions of python, on lots of different hardware, and when being run in all kinds of strange scenarios. These two groups have quite different needs and don't always understand each other and the problems they face.

Re: State of Python 3.13 performance: Free-threading

#136

Post by devjab »

epistasis wrote: post #94
devjab wrote: post #85

Earlier quoted context omitted.

I’m curious as to which packages you are unable to find older versions for. You mention snakemake, but that doesn’t seem to have any sort of issues. https://pypi.org/project/snakemake/#history

It's not about finding old packages, it's about not finding the magical compatible set of package versions. Pip is nice in that you can install packages individually to get around some version conflicts. But with conda and npm and CRAN I have always found my stuck without being able to install dependencies after 15 minutes of mucking. Its rare that somebody has left the equivalent of the output of a `pip freeze` arou…

It’s a little hard for me to talk about Python setups which don’t use Poetry as that is basically the standard around here. I would argue that not controlling your packages regardless of the package manager you use is very poor practice.

How can you reasonably expect to work with any tech that breaks itself by not controlling its dependencies? You’re absolutely correct that this is probably more likely to be an issue with Python, but that’s the thing with freedom. It requires more of you.

Re: State of Python 3.13 performance: Free-threading

#137

Post by otabdeveloper4 »

eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

Parallelism != speed.

If you have 128 cores you can compromise on being a bit slow. You can't compromise on being single-threaded though.

Re: State of Python 3.13 performance: Free-threading

#139

Post by Timon3 »

elashri wrote: post #99
Timon3 wrote: post #58

Earlier quoted context omitted.

The big difference is that npm will automatically (since 2017) save a version range to the project metadata, and will automatically create this metadata file if it doesn't exist. Same for other package managers in the Node world. I just installed Python 3.13 with pip 24.2, created a venv and installed a package - and nothing, no file was created and nothing was saved. Even if I touch requirements.txt and pyproject.to…

> Of course there are other package managers for Python that do this better I think if you are comparing with what NPM does then you would have to say that native pip can do that too. It is just one command `pip freeze > requirements.txt` It does include everything in the venv (or in you environment in general) but if you stick to only add required things (one venv for each project) then you will get requirements.txt…

Sure, you can manually do that. But my point is that pip doesn't do this automatically, and that is what makes so many Python projects essentially unusable without investing massive amounts of time into debugging. Good defaults and good standards matter a lot.

Re: State of Python 3.13 performance: Free-threading

#140

Post by Netcob »

eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

I was experimenting with Python for some personal data-based project. My goal was to play around with some statistics functions, maybe train some models and so on. Unfortunately most of my data needs a lot of preprocessing before it will be of any use to numpy&co. Mostly just picking out the right objects from a stream of compressed JSON data and turning them into time series. All of that is easy to do in Python, which is what I'd expect for the language used this much in data science (or so I've heard).

But then I do have a lot of data, there's a lot of trial&error there for me so I'm not doing these tasks only once, and I would have appreciated a speedup of up to 16x. I don't know about "high performance", but that's the difference between a short coffee break and going to lunch.

And if I was a working on an actual data-oriented workstation, I would be using only one of possibly 100+ cores.

That just seems silly to me.

Post reply on HN