Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pypi.org is running a survey on the state of Python packaging

pypi.org

Post reply on HN
Print view View original

111–120 of 193 posts

Re: Pypi.org is running a survey on the state of Python packaging

#111

Post by LukeShu »

orf wrote: post #49
korijn wrote: post #48

Earlier quoted context omitted.

...and ensure _all_ package metadata required to perform dependency resolution can be retrieved through an API (in other words without downloading wheels).

Yeah, that’s sort of what I meant by my suggestion. Requirements that can only be resolved by downloading and executing code is a huge burden on tooling

If the package is available as a wheel, you don't need to execute code to see what the requirements are; you just need to parse the "METADATA" file. However, the only way to get the METADATA for a wheel (using PyPA standard APIs, anyway) is to download the whole wheel.

For comparison, pacman (the Arch Linux package manager) packages have fairly similar ".PKGINFO" file in them; but in order to support resolving dependencies without downloading the packages, the server's repository index includes not just a listing of the (name, version) tuple for each package, it also includes each package's full .PKGINFO.

Enhancing the PyPA "Simple repository API" to allow fetching the METADATA independently of the wheel would be a relatively simple enhancement that would make a big difference.

----

As I was writing this comment, I discovered that PyPA did this; they adopted PEP 658 in March of this year! https://github.com/pypa/packaging.python.org/commit/1ebb57b7...

Re: Pypi.org is running a survey on the state of Python packaging

#112

Post by klkqw »

siproprio wrote: post #77

State of the art python packaging must include support for common use cases such as conda+machine learning. It’s incredible how even Julia’s Pkg.jl supports better python packaging in combination with conda than the official python packaging tools. This is very clearly a question of the culture of the core python developers (such as brett cannon) who seem to think the machine learning people with their compilers and…

The Twitter hero faction of the Python core "developers" (many of whom have not done much actual work for a decade) has always been pro-web and anti-C-extensions.

They have been peddling the dream of speeding up Python by neglecting the scientific ecosystem for a long time without having either the expertise or a clear plan how to achieve that.

All that while the primarily useful part of Python is its C extension capabilities. The web does not need Python, query/response applications are handled just fine and better by functional languages.

Re: Pypi.org is running a survey on the state of Python packaging

#113

Post by smeagull »

IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

I think some of the pain has been Apple's fault. Requiring conda for some official packages means you always have two competing ecosystems on one machine - which is just asking for pain.

Re: Pypi.org is running a survey on the state of Python packaging

#114

Post by smeagull »

bno1 wrote: post #105
IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

I wish pip had some package deduplication implemented. Even some basic local environments have >100MB of dependencies. ML environments go into the gigabytes range from what I remember.

Cargo allows you to share a single package directory. Having a single site-packages and the ability to import a particular version (or latest by default) would solve this.

Re: Pypi.org is running a survey on the state of Python packaging

#115

Post by smeagull »

YetAnotherNick wrote: post #8

I wish there is some package manager in middle of conda and pip. Conda is too strict and often get stuck in SAT solving. pip doesn't even ask when reinstalling a version currently being used. Edit: Typo: reinstalling a version of package currently being used

Conda sucks because it always wants things its own way, and introduces yet another site-packages and environment I need to track. Absolutely hate Apple for requiring it for their metal and M1 support.

Re: Pypi.org is running a survey on the state of Python packaging

#117

Post by user8139471 »

bzxcvbn wrote: post #78
d0mine wrote: post #66

Earlier quoted context omitted.

Isn't PyPI pronounced like: pie (food/π) + pea + eye (). that is different from the french "pipi": pea + pea.

Sure. Now tell that to a bunch of immature college juniors. If you read the letters "pypi" in French, it sounds exactly like "pipi". And wait until you learn what "bit" sounds like in French.

What does "bit" sound like in French?

Re: Pypi.org is running a survey on the state of Python packaging

#118

Post by Asraelite »

smeagull wrote: post #114
bno1 wrote: post #105

Earlier quoted context omitted.

I wish pip had some package deduplication implemented. Even some basic local environments have >100MB of dependencies. ML environments go into the gigabytes range from what I remember.

Cargo allows you to share a single package directory. Having a single site-packages and the ability to import a particular version (or latest by default) would solve this.

This is also nice because you can clean up your build files all in one go. Every now and again I delete my cargo target directory and get 5GB back.

Re: Pypi.org is running a survey on the state of Python packaging

#120

Post by x3n0ph3n3 »

orf wrote: post #41

For improvements I commented: Remove setup.py files and mandate wheels. This is the root cause of a lot of the evil in the ecosystem. Next on the list would be pypi namespaces, but there are good reasons why that is very hard. The mission statement they are proposing, “a packaging ecosystem for all”, completely misses the mark. How about a “packaging ecosystem that works” first? I spent a bunch of time recently fixin…

I suggested "one packaging system to rule them all." The fragmentation in this space is frustrating.
Post reply on HN