Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PYX: The next step in Python packaging

astral.sh

Post reply on HN
Print view View original

331–340 of 479 posts

Re: PYX: The next step in Python packaging

#331

Post by thrown-0825 »

int_19h wrote: post #195
nixpulvis wrote: post #169

Can I ask a dumb question. Why does Ruby (for example) not have this problem, but python still can't ship a standard solution which isn't constantly changing and rolled up in some corporate offering?

Ruby is mostly used in web dev, where most if not all of your dependencies tend to be pure Ruby. Python is used heavily for DS/ML/AI, which is exactly the area where native code packages are necessary and prevalent. Worse yet is that those packages often involve GPU code, and things like CUDA bring their own complications. If you're writing web apps in Python, dependencies haven't really been a problem for a long tim…

Ruby libs have lots of c bindings.

The reason python is a mess is because it became the default for academics and data scientists who have no real interest in code quality or maintainability and the rot just spread from there.

Re: PYX: The next step in Python packaging

#333

Post by mavamaarten »

ilvez wrote: post #278
Eduard wrote: post #224

Earlier quoted context omitted.

I started with "sudo apt install python" a long time ago and this installed python2. This was during the decades-long transition from python2 to python3, so half the programs didn't work so I installed python3 via "sudo apt install python3". Of course now I had to switch between python2 and python3 depending on the program I wanted to run, that's why Debian/Ubuntu had "sudo update-alternatives --config python" for ma…

I understand this is meant as caricature, but for doing local development tools like mise or asdf are really something I've never looked back from. For containers it's either versioned Docker image or compile yourself.

The problem for me: a non-python developer, is that I just don't know what to do, ever, to run an existing script or program.

It seems every project out there uses a different package manager, a different version of python, a different config file to set all of that up.

Most of the time, I just have a random .py file somewhere. Sometimes it's a full project that I can look at and find out what package manager it's using. Sometimes it has instructions, most of the time not. _That's_ the situation I struggle with.

Do I just run ./script.py? python script.py? python3 script.py? python3.12 script.py? When inevitably I miss some dependencies, do I just pip install? python pip install? pipx install?

As a developer I'm sure that you just set it up and forget about it. And once things work, they probably keep working for you. But man, it really reflects negatively upon Python itself for me. I don't hate the language, but I sure hate the experience.

Re: PYX: The next step in Python packaging

#334

Post by arkh »

bastawhiz wrote: post #204
runningmike wrote: post #103

All python packaging challenges are solved. Lesson learned is that there is not a single solution for all problems. getting more strings attached with VC funded companies and leaning on their infrastructure is a high risk for any FOSS community.

Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…

Coming from the php ecosystem this kind of package manager problems feels crazy.

Maybe Python and js people should just use composer too.

Re: PYX: The next step in Python packaging

#336

Post by pjmlp »

How many Python packaging solutions do we need?

Thankfully I only use it for OS and application scripting and keep myself to the standard library and application exposed APIs as much as possible, even if there are marvelous alternatives outside the standard library.

Re: PYX: The next step in Python packaging

#337

Post by pietroppeter »

simonw wrote: post #14

This is effectively what Charlie said they were going to build last September when quizzed about their intended business model on Mastodon: https://hachyderm.io/@charliermarsh/113103564055291456

And this fact effectively builds trust in the vision and in execution.

Re: PYX: The next step in Python packaging

#338

Post by pletnes »

Eduard wrote: post #224
bastawhiz wrote: post #204

Earlier quoted context omitted.

Well I started with pip because it's what I was told to use. But it was slow and had footguns. And then I started using virtualenv, but that only solved part of the problem. So I switched to conda, which sometimes worked but wrecked my shell profile and often leads to things mysteriously using the wrong version of a package. So someone told me to use pipenv, which was great until it was abandoned and picked up by som…

I started with "sudo apt install python" a long time ago and this installed python2. This was during the decades-long transition from python2 to python3, so half the programs didn't work so I installed python3 via "sudo apt install python3". Of course now I had to switch between python2 and python3 depending on the program I wanted to run, that's why Debian/Ubuntu had "sudo update-alternatives --config python" for ma…

I think I have a similar experience in some ways, but building python from source should work on linux in my experience. On a debian ish system I’d expect apt installing build essentials and the libraries you need and you should be good. I’ve done it with some pain on red hat-ish distros, which have tended to ship with python versions older than I’ve experience with. (I guess it’s better these days..?)

Re: PYX: The next step in Python packaging

#339

Post by pletnes »

mavamaarten wrote: post #333
ilvez wrote: post #278

Earlier quoted context omitted.

I understand this is meant as caricature, but for doing local development tools like mise or asdf are really something I've never looked back from. For containers it's either versioned Docker image or compile yourself.

The problem for me: a non-python developer, is that I just don't know what to do, ever, to run an existing script or program. It seems every project out there uses a different package manager, a different version of python, a different config file to set all of that up. Most of the time, I just have a random .py file somewhere. Sometimes it's a full project that I can look at and find out what package manager it's us…

I believe what is missing is a way of distributing apps. You face similar issues if you get the C++ source of a random program - there are quite a few build systems in use! However, the compiled program can often just be zipped and shipped, somehow.

Re: PYX: The next step in Python packaging

#340

Post by m_kos »

int_19h wrote: post #191
m_kos wrote: post #40

> Why is it so hard to install PyTorch, or CUDA, or libraries like FlashAttention or DeepSpeed that build against PyTorch and CUDA? This is so true! On Windows (and WSL) it is also exacerbated by some packages requiring the use of compilers bundled with outdated Visual Studio versions, some of which are only available by manually crafting download paths. I can't wait for a better dev experience.

Given that WSL is pretty much just Linux, I don't see what relevance Visual Studio compiler versions have to it. WSL binaries are always built using Linux toolchains. At the same time, even on Windows, libc has been stable since Win10 - that's 10 years now. Which is to say, any binary compiled by VC++ 2015 or later is C-ABI-compatible with any other such binary. The only reasons why someone might need a specific comp…

If you have to use, e.g., CUDA Toolkit 11.8, then you need a specific version of VS and its build tools for CUDA's VS integration to work. I don't know why exactly that is and I wish I didn't have to deal with it.
Post reply on HN