I have no connection to astral, and while we dabble in uv (which has been great) we mostly use conda because of CUDA linking etc, but an alternative is at the least welcome and at best an improvement, so I don’t get the degree of cynicism or even hostility on this thread. If you don’t want to use pyx stick with pypi or write your own damn python packaging server.
PYX: The next step in Python packaging
451–460 of 479 posts
Re: PYX: The next step in Python packaging
#452All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.
Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…
We actually built a platform that eliminates all these steps, you can now reproduce GitHub repos with 0 manual config in 60% cases. check for more info on https://x.com/KeploreAI
We just launched it and waiting for first users to be astonished :), let me know if you have any questions
Re: PYX: The next step in Python packaging
#453All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.
Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…
[deleted]
Re: PYX: The next step in Python packaging
#454[dead]
Re: PYX: The next step in Python packaging
#455Earlier quoted context omitted.
React native is just an example, the point is that the npm registry has no issue distributing binaries. Sass, Prisma, native DB drivers, or any other project using node-gyp or Node's NAPI are valid examples.
Pypi doesn't have any issue distributing binaries either these days? The issue is (a) diversity of CPU/GPU microarchitectures inc. CUDA (b) diversity of OS-es. I've never had to compile a native Node library on a POWER9 GPU cluster, but I have had to compile Python ones.
And the JS world has ESM modules, CommonJS, various polyfills for compatibility, and other quirks that need to be accommodated by the package manager. With native code compilation affected by these settings.
Sorry, but Python is really not that unique.
Re: PYX: The next step in Python packaging
#456To be honest, this was just a matter of time. As a long time Python developer, I just can’t wrap my head around the lack of something like this. GitHub was going to get hosted packages for Python but never did because it “didn’t align with their strategy objectives and a reallocation of resources” [1] (or some other similar corpospeak) Astral is a great company and I think we can’t question what they’ve achieved and…
Anaconda solved the same problem ~10+ years ago already.
HAHAHAH don't even get me started on how bad anaconda is.
On how slow the installer + interpreter, how they avoided being a part of the usual pip workflow, bloated environment, cross platform inconsistencies, extremely slow dependency resolution, etc etc etc...
Re: PYX: The next step in Python packaging
#457Earlier quoted context omitted.
> I think there's a pretty long track record of financially incentivized companies being willing to take that pain. MongoDB's AGPL transition comes to mind. MongoDB had a CLA from the start, didn't it? > Astral's tools don't require contributors to sign a CLA. That's a pretty vital difference!
>> Astral's tools don't require contributors to sign a CLA. > That's a pretty vital difference Not really when the license is MIT/Apache. They can create a closed source fork and continue development there.
Ah yeah, whoops. Either a CLA or permissive license is enough on its own to facilitate such a license change.
Re: PYX: The next step in Python packaging
#458Earlier quoted context omitted.
I understand this is meant as caricature, but for doing local development tools like mise or asdf are really something I've never looked back from. For containers it's either versioned Docker image or compile yourself.
The problem for me: a non-python developer, is that I just don't know what to do, ever, to run an existing script or program. It seems every project out there uses a different package manager, a different version of python, a different config file to set all of that up. Most of the time, I just have a random .py file somewhere. Sometimes it's a full project that I can look at and find out what package manager it's us…
Uv solves this (with some new standards). ./script.py will now install the python version, create a venv, and install dependencies (very quickly) if they don’t exist already.
#!/usr/bin/env -S uv run --script # /// script # requires-python = ">=3.12" # dependencies = [ # "ffmpeg-normalize", # ] # ///
Re: PYX: The next step in Python packaging
#459What are the reasons that Python can't implement the same sort of module/packaging system as NodeJS? That seems to work well enough. Executing a Python script in the same directory as some sort of project.json file that contains all the complicated dependency details would be a pretty good solution to me. But I'm probably missing a whole bunch of details. (Feel free to educate me). In general I really dislike the cur…
It can. That's what uv is. Put an '#!/usr/bin/env -S uv run python' shebang in your script, add a `pyproject.toml` with all of your deps, and you're done.
Again, you're creating a new session with special environmental variables. It's nice that it uses pyproject.toml (well, except for the toml part), but it still needs uv.
I was suggesting that the Python executable have this sort of functionality built in, with no need for a new session.
Re: PYX: The next step in Python packaging
#460Earlier quoted context omitted.
NPM has always been commercial (rather than managed by a foundation), and it was nominally acquired by GitHub rather than Microsoft, so at some level as long as GitHub is not causing issues (noting the recent GitHub changes should maybe also imply some consideration of problems for NPM), NPM is "safe". Astral on the other hand has basically been rewrites in Rust of existing community-based open source tools, for whic…
> as long as GitHub is not causing issues Astral are not causing issues though. Why does “as long as Astral is not causing issues” not apply? > Anaconda/ContinuumIO was also treated with similar suspicion to Astral I haven’t observed this. I have seen condo talked about a fair amount but any issues have always revolved around it being awkward to use. But practically every discussion here or on Reddit about Astral has…
Sorry, the quotes around safe were supposed to imply GitHub is not that safe in my opinion, but it's possibly why other people aren't concerned about NPM (also, being for a different programming language and community may help).
Anaconda/ContinuumIO (the company) was absolutely treated with suspicion, see e.g. https://www.mail-archive.com/numpy-discussion%40scipy.org/ms... (and you'll find many such threads around that time on mailing lists/forums that were where the scientific python community was), and while the sky didn't fall in, their history hasn't been spotless. In many ways Astral is the more "webby" version of Anaconda/ContinuumIO, and so assuming Astral will behave (and evolve) in a similar way to Anaconda/ContinuumIO seems to me at least to be a rational thing to do?