Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State of Python 3.13 performance: Free-threading

codspeed.io

Post reply on HN
Print view View original

151–160 of 193 posts

Re: State of Python 3.13 performance: Free-threading

#151

Post by zelphirkalt »

BerislavLopac wrote: post #144
klysm wrote: post #31

Earlier quoted context omitted.

So pin your deps? Language backwards compatibility and an API from some random package changing are completely distinct.

Yes and no. There are different types of dependencies, and there are different rules for them, but here's an overview of the best practices: 1. For applications, scripts and services (i.e. "executable code"), during development, pin your direct dependencies; ideally to the current major or minor version, depending how much you trust their their authors to follow SemVer. Also make sure you regularly update and retest…

In a poetry lock file transitive dependencies are automatically locked and thereby pinned. It will ensure, that you get the same thing each time, or get an error about things not matching hashsums, when something suspicious is going on, that would be worth raising an issue on a repo, if none exists.

Re: State of Python 3.13 performance: Free-threading

#152

Post by BerislavLopac »

zelphirkalt wrote: post #151
BerislavLopac wrote: post #144

Earlier quoted context omitted.

Yes and no. There are different types of dependencies, and there are different rules for them, but here's an overview of the best practices: 1. For applications, scripts and services (i.e. "executable code"), during development, pin your direct dependencies; ideally to the current major or minor version, depending how much you trust their their authors to follow SemVer. Also make sure you regularly update and retest…

In a poetry lock file transitive dependencies are automatically locked and thereby pinned. It will ensure, that you get the same thing each time, or get an error about things not matching hashsums, when something suspicious is going on, that would be worth raising an issue on a repo, if none exists.

> In a poetry lock file transitive dependencies are automatically locked and thereby pinned

That is true for all formats of lock files, by definition.

Re: State of Python 3.13 performance: Free-threading

#153

Post by elashri »

Timon3 wrote: post #139
elashri wrote: post #99

Earlier quoted context omitted.

> Of course there are other package managers for Python that do this better I think if you are comparing with what NPM does then you would have to say that native pip can do that too. It is just one command `pip freeze > requirements.txt` It does include everything in the venv (or in you environment in general) but if you stick to only add required things (one venv for each project) then you will get requirements.txt…

Sure, you can manually do that. But my point is that pip doesn't do this automatically, and that is what makes so many Python projects essentially unusable without investing massive amounts of time into debugging. Good defaults and good standards matter a lot.

> without investing massive amounts of time into debugging

Again even if you are going to spend sometime to learn something that will have better tool for doing that like uv and poetry package managers. This is no massive amount of time. And eveb pip freeze is just one standard command and will give you portable environment be everything will be pinned in your environment. You just don't want to do everything with system global environment which is a common sense not a huge ask.

So I am not sure what is the massive amount of debugging needes for that.

Re: State of Python 3.13 performance: Free-threading

#154

Post by epistasis »

zelphirkalt wrote: post #149
epistasis wrote: post #125

Earlier quoted context omitted.

That works for your project, but then nobody can include you as a library without conflicts. But having that lock file will allow somebody to reconstruct your particular moment in time in the future. Its just that those lock files do not exist for 99.9% of Python projects in time.

A lib can still lock its dependencies and have version ranges declared at the same time. The lock file is an artifact than is used to reproducibly build the lib, while the version ranges are used to see, whether some other project can use the lib. It is only a matter of tooling. Locking ones dependencies remains the right thing to do, even for a lib.

This is of course the right answer. But unfortunately it has only recently become supported by packaging tooling, and is extremely uncommon to encounter in the wild.

Re: State of Python 3.13 performance: Free-threading

#155

Post by roelschroeven »

d0mine wrote: post #79
wormlord wrote: post #12

Earlier quoted context omitted.

> On the other hand, a lot of these changes to try and speed up the base language are going to be highly disruptive. E.g. disabling the GIL will break tonnes of code, lots of compilation projects involve changes to the ABI, etc. Kind of related, the other day I was cursing like a sailor because I was having issues with some code I wrote that uses StrEnum not working with older versions of Python, and wondering why I…

At least bugfix versions could have kept Enum behavior the same. Postponing breaking changes until the next minor version. Some Enum features work differently (incompatible) in Python 3.11.x versions.

> Some Enum features work differently (incompatible) in Python 3.11.x versions.

I know that Python 3.11 added some things, like StrEnum; those obviously won't work on older Python versions. But I'm not aware of things that work in a certain Python 3 version but don't work in newer ones. You're even talking about incompatibilities between different 3.11.x versions? Can you give some more detail on that?

Re: State of Python 3.13 performance: Free-threading

#156

Post by Timon3 »

elashri wrote: post #153
Timon3 wrote: post #139

Earlier quoted context omitted.

Sure, you can manually do that. But my point is that pip doesn't do this automatically, and that is what makes so many Python projects essentially unusable without investing massive amounts of time into debugging. Good defaults and good standards matter a lot.

> without investing massive amounts of time into debugging Again even if you are going to spend sometime to learn something that will have better tool for doing that like uv and poetry package managers. This is no massive amount of time. And eveb pip freeze is just one standard command and will give you portable environment be everything will be pinned in your environment. You just don't want to do everything with sy…

I am specifically talking about the scenario where I, someone experienced with Python, am trying to use a project from someone who is not experienced in Python. There is nothing I can change since the project was already created some time ago. Most of the time, the only way for me to get the project running is to spend a lot of time debugging version issues - to find combinations of dependencies that are installable, and then to see if they work in the application context, and then to see if any bugs are part of the application or the dependency.

You might explain that away by asking why I'd want to run these projects, but a large percentage of data science projects I've tried to run end up in this scenario, and it simply doesn't happen with npm. A command existing is not a valid replacement for good defaults, since it literally affects my ability to run these projects.

Re: State of Python 3.13 performance: Free-threading

#157

Post by LtWorf »

andai wrote: post #24
eigenspace wrote: post #3

I don't really have a dog in this race as I don't use Python much, but this sort of thing always seemed to be of questionable utility to me. Python is never really going to be 'fast' no matter what is done to it because its semantics make most important optimizations impossible, so high performance "python" is actually going to always rely on restricted subsets of the language that don't actually match language's "re…

There was a discussion the other day about how Python devs apparently don't care enough for backwards compatibility. I pointed out that I've often gotten Python 2 code running on Python 3 by just changing print to print(). But then a few hours later, I tried running a very small project I wrote last year and it turned out that a bunch of my dependencies had changed their APIs. I've had similar (and much worse) experi…

I'd drop libraries that do like that.

Re: State of Python 3.13 performance: Free-threading

#158

Post by LtWorf »

lmm wrote: post #113
vosper wrote: post #67

Earlier quoted context omitted.

> Pinning deps is discouraged by years of Python practice. I'm not sure it is discouraged so much as just not what people did in Python-land for a long time. It's obviously the right thing to do, it's totally doable, it's just inertia and habit that might mean it isn't done.

> I'm not sure it is discouraged so much as just not what people did in Python-land for a long time. It's obviously the right thing to do, it's totally doable, it's just inertia and habit that might mean it isn't done. Pinning obviously the wrong thing, it only works if everyone does it and if everyone does it then making changes becomes very hard. The right thing is to have deterministic dependency resolution so tha…

For some reason the "secure" thing to do is considered to be to pin everything and then continuously bump everything to latest, to get the security fixes.

At which point one might directly not pin, but that's "insecure" (https://scorecard.dev/)

Re: State of Python 3.13 performance: Free-threading

#159

Post by LtWorf »

zelphirkalt wrote: post #149
epistasis wrote: post #125

Earlier quoted context omitted.

That works for your project, but then nobody can include you as a library without conflicts. But having that lock file will allow somebody to reconstruct your particular moment in time in the future. Its just that those lock files do not exist for 99.9% of Python projects in time.

A lib can still lock its dependencies and have version ranges declared at the same time. The lock file is an artifact than is used to reproducibly build the lib, while the version ranges are used to see, whether some other project can use the lib. It is only a matter of tooling. Locking ones dependencies remains the right thing to do, even for a lib.

If you include a range you have to test with everything in the range.

Re: State of Python 3.13 performance: Free-threading

#160

Post by LtWorf »

salomonk_mur wrote: post #102
andai wrote: post #24

Earlier quoted context omitted.

There was a discussion the other day about how Python devs apparently don't care enough for backwards compatibility. I pointed out that I've often gotten Python 2 code running on Python 3 by just changing print to print(). But then a few hours later, I tried running a very small project I wrote last year and it turned out that a bunch of my dependencies had changed their APIs. I've had similar (and much worse) experi…

What APIs were broken? They couldn't be in the standard library. If the dependency was in external modules and you didn't have pinned versions, then it is to be expected (in almost any active language) that some APIs will break.

Python drops modules from the standard library all the time these days. It's a pain in the ass.

Now even asyncore is gone -_-' Have fun rewriting all the older async applications!

Post reply on HN