Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uv overtakes pip in CI

wagtail.org

Post reply on HN
Print view View original

171–180 of 184 posts

Re: Uv overtakes pip in CI

#171

Post by simonw »

wishitwerentso wrote: post #169
simonw wrote: post #22

Earlier quoted context omitted.

It's mostly about age. Python has been around for 35 years now. The first version of a Python package directory was the cheeseshop (Monthy Python reference) in 2003. The earliest version of a pip-like tool was "easy_install" which - I kid you not - worked by scraping the HTML listing page of the cheeseshop and downloading zip files linked from that! More recent languages like Node.js and Rust and Go all got to create…

35 years is misleading. Python existed, yes, but was very different. e.g. Pandas was released in 2008. Most use packages much more recent than that. 35 years ago Perl was faster than Python and had deep adoption (through 2007? or so)

How is it misleading?

The question is why Python packaging has such a complicated history. The age of the language is entirely relevant to that - the reason Go and Rust have it so good here is that they are much younger, coming out after may of the initial packaging lessons had been learned elsewhere.

Re: Uv overtakes pip in CI

#172

Post by notatallshaw »

simonw wrote: post #22
torben-friis wrote: post #10

As an outsider to python, I never got how a language who got popular for being simple, elegant and readable could end up with perhaps the most complex tooling situation (dependencies, envs, etc). Any time I glance at the community there seems to be a new way of doing things. What caused python to go through these issues? Is there any fundamental design flaw ?

It's mostly about age. Python has been around for 35 years now. The first version of a Python package directory was the cheeseshop (Monthy Python reference) in 2003. The earliest version of a pip-like tool was "easy_install" which - I kid you not - worked by scraping the HTML listing page of the cheeseshop and downloading zip files linked from that! More recent languages like Node.js and Rust and Go all got to create…

> There is one part of Python that I consider a design flaw when it comes to packaging: the sys.modules global dictionary means it's not at all easy in Python to install two versions of the same package at the same time. This makes it really tricky if you have dependency A and dependency B both of which themselves require different versions of dependency C.

But it solves the problem that if A and B both depend on C the user can pass an object from A to B that was created by C without worrying about it breaking.

In less abstract terms, let's say numpy one day changed it's internal representation of an array, so if one version of numpy read an array of a different version of numpy it would crash or worse read it but misinterpret it. Now if I have one data science library produces numpy arrays and another visualization library that takes numpy arrays, I can be confident that only one version of numpy is installed and the visualization library isn't going to misinterpret the data from the data because it is using a different version of numpy.

This stability of installed versions have allowed entire ecosystems build around core dependencies in a way that would be tricky without that. I would therefore not consider it a design flaw.

Re: Uv overtakes pip in CI

#173

Post by simonw »

notatallshaw wrote: post #172
simonw wrote: post #22

Earlier quoted context omitted.

It's mostly about age. Python has been around for 35 years now. The first version of a Python package directory was the cheeseshop (Monthy Python reference) in 2003. The earliest version of a pip-like tool was "easy_install" which - I kid you not - worked by scraping the HTML listing page of the cheeseshop and downloading zip files linked from that! More recent languages like Node.js and Rust and Go all got to create…

> There is one part of Python that I consider a design flaw when it comes to packaging: the sys.modules global dictionary means it's not at all easy in Python to install two versions of the same package at the same time. This makes it really tricky if you have dependency A and dependency B both of which themselves require different versions of dependency C. But it solves the problem that if A and B both depend on C t…

I wouldn't mind a codebase where numpy objects created by dependency B can't be shared directly with dependency A without me first running some kind of conversion function on them - I'd take that over "sorry you want to use dependency A and dependency B in this project, you're just out of luck".

Re: Uv overtakes pip in CI

#174

Post by notatallshaw »

simonw wrote: post #173
notatallshaw wrote: post #172

Earlier quoted context omitted.

> There is one part of Python that I consider a design flaw when it comes to packaging: the sys.modules global dictionary means it's not at all easy in Python to install two versions of the same package at the same time. This makes it really tricky if you have dependency A and dependency B both of which themselves require different versions of dependency C. But it solves the problem that if A and B both depend on C t…

I wouldn't mind a codebase where numpy objects created by dependency B can't be shared directly with dependency A without me first running some kind of conversion function on them - I'd take that over "sorry you want to use dependency A and dependency B in this project, you're just out of luck".

> I wouldn't mind a codebase where numpy objects created by dependency B can't be shared directly with dependency A without me first running some kind of conversion function on them

Given there's no compiler to enforce this check, and Python is dynamic language, I don't see how you implement that without some complicated object provenance feature, making every single object larger and every use of that object (calling with it, calling it, assigning it to an attribute, assigning an attribute to it) impose an expensive runtime check.

But maybe I'm missing something obvious.

Re: Uv overtakes pip in CI

#175

Post by wishitwerentso »

simonw wrote: post #171
wishitwerentso wrote: post #169

Earlier quoted context omitted.

35 years is misleading. Python existed, yes, but was very different. e.g. Pandas was released in 2008. Most use packages much more recent than that. 35 years ago Perl was faster than Python and had deep adoption (through 2007? or so)

How is it misleading? The question is why Python packaging has such a complicated history. The age of the language is entirely relevant to that - the reason Go and Rust have it so good here is that they are much younger, coming out after may of the initial packaging lessons had been learned elsewhere.

it is misleading because I was around 35 years ago, and very few people were using Python. Python did not become very popular until web frameworks and pandas became a thing in python.

Re: Uv overtakes pip in CI

#176

Post by wishitwerentso »

wishitwerentso wrote: post #175
simonw wrote: post #171

Earlier quoted context omitted.

How is it misleading? The question is why Python packaging has such a complicated history. The age of the language is entirely relevant to that - the reason Go and Rust have it so good here is that they are much younger, coming out after may of the initial packaging lessons had been learned elsewhere.

it is misleading because I was around 35 years ago, and very few people were using Python. Python did not become very popular until web frameworks and pandas became a thing in python.

if you doubt this, ask any llm "what was the first year where Python users surpassed the number of Perl users?"

Re: Uv overtakes pip in CI

#177

Post by simonw »

wishitwerentso wrote: post #175
simonw wrote: post #171

Earlier quoted context omitted.

How is it misleading? The question is why Python packaging has such a complicated history. The age of the language is entirely relevant to that - the reason Go and Rust have it so good here is that they are much younger, coming out after may of the initial packaging lessons had been learned elsewhere.

it is misleading because I was around 35 years ago, and very few people were using Python. Python did not become very popular until web frameworks and pandas became a thing in python.

I still don't understand why that makes what I wrote "misleading". I never said Python was popular 35 years, I just said that the age of the language was relevant to understanding why the packaging history is complex.

Re: Uv overtakes pip in CI

#178

Post by mixmastamyk »

0xpgm wrote: post #135

Still waiting to see how the VC-funded company behind Uv will make money. Before that, I wouldn't want to be too dependent on it.

Uv and ruff are near feature complete and open source. It's very likely they'll survive in one way or the other and are already better than the tools they're meant to replace.

Re: Uv overtakes pip in CI

#179

Post by simonw »

notatallshaw wrote: post #174
simonw wrote: post #173

Earlier quoted context omitted.

I wouldn't mind a codebase where numpy objects created by dependency B can't be shared directly with dependency A without me first running some kind of conversion function on them - I'd take that over "sorry you want to use dependency A and dependency B in this project, you're just out of luck".

> I wouldn't mind a codebase where numpy objects created by dependency B can't be shared directly with dependency A without me first running some kind of conversion function on them Given there's no compiler to enforce this check, and Python is dynamic language, I don't see how you implement that without some complicated object provenance feature, making every single object larger and every use of that object (callin…

You let people make the mistake and have the library throw an exception if they do that, not through type checking but just through something eventually calling a method that doesn't exist.

Re: Uv overtakes pip in CI

#180

Post by notatallshaw »

simonw wrote: post #179
notatallshaw wrote: post #174

Earlier quoted context omitted.

> I wouldn't mind a codebase where numpy objects created by dependency B can't be shared directly with dependency A without me first running some kind of conversion function on them Given there's no compiler to enforce this check, and Python is dynamic language, I don't see how you implement that without some complicated object provenance feature, making every single object larger and every use of that object (callin…

You let people make the mistake and have the library throw an exception if they do that, not through type checking but just through something eventually calling a method that doesn't exist.

> You let people make the mistake and have the library throw an exception if they do that, not through type checking but just through something eventually calling a method that doesn't exist.

Exceptions or crashes would be annoying, but yes, are manageable, although try telling that to new users of the language that their code doesn't work because they didn't understand the transitive dependency tree of their install and it automatically vendored different versions of a library for different dependencies, and how did they not know that from some random exception occurring in a dependency.

But as I explain in my example, the real problem is that one version of the library reads the data in a different layout from the other, so instead you end of with subtle data errors. Now your code is working but your getting the wrong output, good luck debugging that.

Post reply on HN