Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PYX: The next step in Python packaging

astral.sh

Post reply on HN
Print view View original

411–420 of 479 posts

Re: PYX: The next step in Python packaging

#411

Post by zanie »

alisonatwork wrote: post #281
zanie wrote: post #262

Earlier quoted context omitted.

Please open an issue with some details about the memory usage. We're happy to investigate and feedback on how it's working in production is always helpful. (I work on uv)

Last time I looked into this I found this unresolved issue, which is pretty much the same thing: https://github.com/astral-sh/uv/issues/7004 We run on-prem k8s and do the pip install stage in a 2CPU/4GB Gitlab runner, which feels like it should be sufficient for the uv:python3.12-bookworm image. We have about 100 deps that aside from numpy/pandas/pyarrow are pretty lightweight. No GPU stuff. I tried 2CPU/8GB runners…

Thanks that's helpful.

Did you try reducing the concurrency limit?

Re: PYX: The next step in Python packaging

#412

Post by sevensor »

monster_truck wrote: post #10

I've been burned too many times by embracing open source products like this. We've been fed promises like these before. They will inevitably get acquired. Years of documentation, issues, and pull requests will be deleted with little-to-no notice. An exclusively commercial replacement will materialize from the new company that is inexplicably missing the features you relied on in the first place.

It’s great that astral are doing what they do, but it’s important to hedge against their success. We must maintain ecosystem fragmentation, or Astral becomes a choke point where a bad actor can seize the entire Python community and extract rent from it. Like Anaconda but successful. So keep using pylint, keep using mypy, keep using PyPI. Not exclusively, but as a deterrent against this kind of capture.

Re: PYX: The next step in Python packaging

#413

Post by zahlman »

imtringued wrote: post #364
notatallshaw wrote: post #92

Earlier quoted context omitted.

> However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original). FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. I'm not saying here that pip doesn't have…

>FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved. No. This something people tell each other, that it's a lack of resources, but in reality almost all OSS projects with long standing flaws don't have a resources problem. They have a prioritization problem, wh…

> They have a prioritization problem

This is what forks are for. Alternately, money would probably help. Otherwise it's a bit rude to impose your priorities on others.

Perhaps some larger projects lose track of what they intended to prioritize. This is also a resources problem. There's nobody available to do management (and probably also not a social structure internally that would allow for it).

> because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance.

The developers will always be in a better position than end users to assess the long-term maintenance impact.

Re: PYX: The next step in Python packaging

#414

Post by zahlman »

BrenBarn wrote: post #255
zahlman wrote: post #74

Earlier quoted context omitted.

Pyx represents the server side, not the client side. The analogue in the pre-existing Python world is PyPI. Many ideas are being added to recent versions of pip that are at least inspired by what uv has done — and many things are possible in uv specifically because of community-wide standards development that also benefits pip. However, pip has some really gnarly internal infrastructure that prevents it from taking a…

> For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. Conda has been able to do this for years.

Good to know. "which in turn are not all original" may have been an understatement. I don't know a lot of these details, because my interest in these things is more recent, and not so much driven by the complex multi-language projects.

Re: PYX: The next step in Python packaging

#415

Post by ljm »

hexo wrote: post #393
rmonvfer wrote: post #356

To be honest, this was just a matter of time. As a long time Python developer, I just can’t wrap my head around the lack of something like this. GitHub was going to get hosted packages for Python but never did because it “didn’t align with their strategy objectives and a reallocation of resources” [1] (or some other similar corpospeak) Astral is a great company and I think we can’t question what they’ve achieved and…

to be honest. ill never use uv. python ecosystem tools should be in python.

This is quite a disappointing self-limitation given the improvements uv brings to the table. You’re missing out on some good stuff.

Re: PYX: The next step in Python packaging

#417

Post by cgravill »

charliermarsh wrote: post #54
m4r71n wrote: post #36

What does GPU-aware mean in terms of a registry? Will `uv` inspect my local GPU spec and decide what the best set of packages would be to pull from Pyx? Since this is a private, paid-for registry aimed at corporate clients, will there be an option to expose those registries externally as a public instance, but paid for by the company? That is, can I as a vendor pay for a Pyx registry for my own set of packages, and t…

> Will `uv` inspect my local GPU spec and decide what the best set of packages would be to pull from Pyx? We actually support this basic idea today, even without pyx. You can run (e.g.) `uv pip install --torch-backend=auto torch` to automatically install a version of PyTorch based on your machine's GPU from the PyTorch index. pyx takes that idea and pushes it further. Instead of "just" supporting PyTorch, the registr…

Is there an intention to bring the auto backend selection to the non-pip interface? I know we can configure this like you show https://docs.astral.sh/uv/guides/integration/pytorch/ but we have folks on different accelerators on Linux and remembering ‘uv sync --extra cu128’ at the right time is fragile so we just make cpu folks have the CUDA overhead too currently.

(As always, big fans of Astral’s tools. We should get on with trying pyx more seriously)

Re: PYX: The next step in Python packaging

#418

Post by rockmeamedee »

CloseChoice wrote: post #370

Does anyone have insights how this compares to anaconda's approach? To me both seem very similar, ux conda, pyx conda-forge. Sure, astral's products are remarkable and widely loved, but I would like to understand if there's a USP beyond that?

Yeah, uv is cool and a step above conda, but that business model doesn’t look very profitable…

They did say they want their thing to have understanding of the code, so maybe they’ll sell semgrep-like features and SBOM/compliance on top. Semgrep is ok popular, but if it maybe bundled into something else (like the package registry itself) that might get enough people over the line to buy it.

Private registries and “supply chain security” tools individually aren’t the hottest market, but maybe together the bundle could provide enough value. Let’s see how it goes.

Re: PYX: The next step in Python packaging

#419

Post by jamespo »

pepoluan wrote: post #362
stackskipton wrote: post #200

Earlier quoted context omitted.

They have no need to, current repos show everything is under MIT/Apache. They could close the source at any time and not worry about CLA. >bigger picture here is that Astral wants to build services as a product What services? pyx? Looks nice but I doubt my boss is going to pay for it. More likely they just say "Whatever, package is in PyPi, use that." UV, Ruff, Ty. Again, maybe they can get some data/quant firm who R…

It ultimately depends on a company's risk appetite. Some companies specify that packages need to be hosted on their internal repository/artifactory, and for these companies, pyx might be a good tool.

Exactly, allowing employees to install pypi packages from the internet isn't different to allowing them to install any software from the internet.

Re: PYX: The next step in Python packaging

#420

Post by nilamo »

woodruffw wrote: post #275
fragmede wrote: post #264

Earlier quoted context omitted.

Is code to detect when the user is not in that subset and say that it doesn't work really really hard for some non-obvious reason? If the default case for the vast majority of users doesn't work, it doesn't seem like printing a more useful error message to them should be that hard.

> it doesn't seem like printing a more useful error message to them should be that hard. I think the existing error message is useful: $ pip search foo ERROR: XMLRPC request failed [code: -32500] RuntimeError: PyPI no longer supports 'pip search' (or XML-RPC search). Please use https://pypi.org/search (via a browser) instead. See https://warehouse.pypa.io/api-reference/xml-rpc.html#deprecated-methods for more informa…

That is a nicer message than I remember. I don't have an issue with that. It let's you know it doesn't work, that it will never work, and provides links to an alternative and more information.

It either used to do fewer of those, or my memory is Swiss cheese, but I remember needing to search for the why's and alternatives.

Post reply on HN