Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pypi.org is running a survey on the state of Python packaging

pypi.org

Post reply on HN
Print view View original

131–140 of 193 posts

Re: Pypi.org is running a survey on the state of Python packaging

#131

Post by jogjayr »

Kwpolska wrote: post #20
zbentley wrote: post #2

I imagine many of you have feedback that could be useful to folks making decisions about the future of Python packaging, a common subject of complaint in many discussions here. Remember not to just complain, but to offer specific problems/solutions--i.e. avoid statements like "virtualenvs suck, why can't it be like NPM?" and prefer instead feedback like "the difference between Python interpreter version and what virt…

"virtualenvs suck, why can't it be like NPM?" is a specific problem and a specific solution. The problem being having to manage venvs (which have many gotchas and pitfalls, and no standarization), and the solution is to replace those with packages being installed into the project folder with standardized and well-known tools.

I did a proof-of-concept for this a few years ago: https://github.com/jogjayr/pykg

It literally uses npm, the NPM registry, and node_modules for Python dependency management.

Re: Pypi.org is running a survey on the state of Python packaging

#132

Post by shireboy »

timtom39 wrote: post #37
shireboy wrote: post #30

I’ve been tinkering with stable diffusion lately and this has been a rude introduction to python. Coming from .net (nuget) and JavaScript (npm), it’s baffling that there isn’t an established solution for python. It looks to me like people are trying, but different libraries use different techniques. To a newcomer this is confusing.

ML/AI is the deep end of python dependencies. Lots of hardware specific requirements (e.g. CUDA, cuDNN, AVX2 TensorFlow binaries, etc). A typical python web application is a lot simpler.

To be fair, this is a big part of the problem- I’m doing this on a m1 mac, PLUS am new to python. Fortunately there are some good guides. Still, this is a solvable problem in other package managers.

Re: Pypi.org is running a survey on the state of Python packaging

#133

Post by deepsquirrelnet »

greymalik wrote: post #121
smeagull wrote: post #113

Earlier quoted context omitted.

I think some of the pain has been Apple's fault. Requiring conda for some official packages means you always have two competing ecosystems on one machine - which is just asking for pain.

What packages are you referring to? I’ve been doing professional Python dev on a Mac for the past three years and have never had a reason to use conda, so I’m curious what I’m missing.

Conda environments are the most isolated from the host os, outside of using docker or a vm. They’re also the most heavyweight. As a package manager, you can install a lot of non-python stuff. It’s more similar to apt or yum than pip. You can even do things like run bash on windows with m2, which I usually find preferable to wsl.

Re: Pypi.org is running a survey on the state of Python packaging

#134

Post by aeyes »

greymalik wrote: post #121
smeagull wrote: post #113

Earlier quoted context omitted.

I think some of the pain has been Apple's fault. Requiring conda for some official packages means you always have two competing ecosystems on one machine - which is just asking for pain.

What packages are you referring to? I’ve been doing professional Python dev on a Mac for the past three years and have never had a reason to use conda, so I’m curious what I’m missing.

pyarrow for example, if you don't have a wheel then good luck building one without using conda. It's not impossible but the developers don't really support it.

Official dependency and build tooling is not properly geared for C extensions. You'll be looking at compiler errors to figure out what dependencies you are missing.

Re: Pypi.org is running a survey on the state of Python packaging

#135

Post by tibbon »

thrown_22 wrote: post #125
IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

Installing Python only applications is trivial. What you're complaining about is all the missing code in other languages which isn't controlled by Python and depends on the OS to provide. This is why we created Linux distributions in the first place. It is not the place of every language to reinvent the wheel - poorly.

Isn’t wheel one of pythons many packaging schemes (intended to replace eggs?)?

> There should be one– and preferably only one –obvious way to do it.

Oh no…

Re: Pypi.org is running a survey on the state of Python packaging

#136

Post by pabs3 »

dalke wrote: post #57
orf wrote: post #41

For improvements I commented: Remove setup.py files and mandate wheels. This is the root cause of a lot of the evil in the ecosystem. Next on the list would be pypi namespaces, but there are good reasons why that is very hard. The mission statement they are proposing, “a packaging ecosystem for all”, completely misses the mark. How about a “packaging ecosystem that works” first? I spent a bunch of time recently fixin…

> Remove setup.py files and mandate wheels What alternative is there for me? My package has a combination of hand-built C extensions and Cython extensions, as well as a code generation step during compilation. These are handled through a subclass of setuptools.command.build_ext.build_ext. Furthermore, I have compile-time options to enable/disable certain configuration options, like enabling/disabling support for Open…

> not add the AVX2 compiler flag

It is a better idea to do instruction selection at runtime in the code that currently uses AVX2. I recently wrote some docs for Debian contributors about the different ways to achieve this:

https://wiki.debian.org/InstructionSelection

Re: Pypi.org is running a survey on the state of Python packaging

#137

Post by dec0dedab0de »

YetAnotherNick wrote: post #8

I wish there is some package manager in middle of conda and pip. Conda is too strict and often get stuck in SAT solving. pip doesn't even ask when reinstalling a version currently being used. Edit: Typo: reinstalling a version of package currently being used

Sounds like you want pipenv, but that might be too close to conda

Re: Pypi.org is running a survey on the state of Python packaging

#138

Post by tibbon »

> There should be one– and preferably only one –obvious way to do it.

I think the answer to Python’s packaging woes has been in plain sight all along. There are so many competing package managers and systems that it is hard to keep track of them all. Of course; this is just what happens in open source, but the problem seems uniquely bad for Python. In Ruby, Swift, Rust or Golang I don’t encounter there being quite the multitude of options for each.

Work on cross comparability and roadmap to merge them all down to eventually one obvious way to install packages.

Lost an hour of time trying to get Stable Diffusion working on an M1 Mac yesterday

Re: Pypi.org is running a survey on the state of Python packaging

#139

Post by kapitanjakc »

IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

What you feel with python is what I feel with JS

managing npm and package.json which has dependency issues.

With python it always has been pip install package and we are done.

Although I do share your pain with versioning. I once spent a week debugging an issue only to find out that there's a fixed version available.

Post reply on HN