Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pypi.org is running a survey on the state of Python packaging

pypi.org

Post reply on HN
Print view View original

181–190 of 193 posts

Re: Pypi.org is running a survey on the state of Python packaging

#181

Post by kapitanjakc »

moring wrote: post #152
kapitanjakc wrote: post #139

Earlier quoted context omitted.

What you feel with python is what I feel with JS managing npm and package.json which has dependency issues. With python it always has been pip install package and we are done. Although I do share your pain with versioning. I once spent a week debugging an issue only to find out that there's a fixed version available.

> With python it always has been pip install package and we are done. The worst I encountered was a somewhat lengthy manual to build a project (cannot remember what it was), and not only did you have to manually install all required dependencies -- npm automates this by having an npm-readable list in package.json -- but after running all those commands, the last note said something like "oh BTW, whenever we wrote "pi…

Ah the classic boogaloo of them finding it was pip3 and adding it tocs in the last...

I agree with that, its pretty painful

Re: Pypi.org is running a survey on the state of Python packaging

#182

Post by korijn »

orf wrote: post #92
korijn wrote: post #86

Earlier quoted context omitted.

Yeah. Well, mandating wheels and getting rid of setup.py at least avoids having to run scripts, and indeed enables the next step which would be indexing all the metadata and exposing it through an API. I just thought it wouldn't necessarily be obvious to all readers of your comment.

Just to be clear, package metadata already is sort of available through the pypi json api. I've got the entire set of all package metadata here: https://github.com/orf/pypi-data $ gzcat release_data/c/d/cdklabs.cdk-hyperledger-fabric-network.json.gz | jq '. | to_entries | .[].value.info.requires_dist' | head [ "typeguard (~=2.13.3)", "publication (>=0.0.3)", "jsii ( =1.63.2)", "constructs ( =10.0.5)", "aws-cdk-lib (…

I'm aware! The issue is the mixed content (wheels, and... not wheels) on pypi. If the data is incomplete, it's useless in the sense that you're never going to be able to guarantee good results.

Re: Pypi.org is running a survey on the state of Python packaging

#183

Post by gwenzek »

tux3 wrote: post #85
ziml77 wrote: post #52

Earlier quoted context omitted.

I tend to just give up on a package if it requires a C toolchain to install. Even if I do end up getting things set up in a way that the library's build script is happy with, I'll be inflicting pain on anyone else who then tries to work with my code.

It feels so suboptimal to need the C toolchain to do things, but having no solid way to depend on it as a non-C library (especially annoying in Rust, which insists on building everything from source and never installing libraries globally). I make a tool/library that requires the C toolchain at runtime. That's even worse than build time, I need end users to have things like lld, objdump, ranlib, etc installed anywher…

I feel Zig could help here. The binaries ship with LLVM statically linked. You could rely on them to provide binaries for a variety of architecture / OS, and use it to compile code on the target machine. I'll probably explore this at some point for Pip.

Re: Pypi.org is running a survey on the state of Python packaging

#184

Post by insane_dreamer »

5d8767c68926 wrote: post #60
YetAnotherNick wrote: post #8

I wish there is some package manager in middle of conda and pip. Conda is too strict and often get stuck in SAT solving. pip doesn't even ask when reinstalling a version currently being used. Edit: Typo: reinstalling a version of package currently being used

My ask would be to get rid of the need for conda all together. Conda obviously offers a lot of value in sharing hairy compiled packages, but it does not play well with anything else. None of available the tooling really works with both conda and pip. It fragments the already lousy packaging story.

other than an occasional pkg conflict, we've found conda and pip to work pretty well together (defining pip dependencies in the conda env) -- across a lot of different python envs

Re: Pypi.org is running a survey on the state of Python packaging

#185

Post by mdmglr »

ciupicri wrote: post #75
mdmglr wrote: post #17

My wishlist: We need a way to configure an ordered list of indexes pip searches for packages. —extra-index-url or using a proxy index is not the solution. Also namespaces and not based on a domain. So for example: pip install apache:parquet Also some logic either in the pip client or index server to minimize typosquatting Also pip should adopt a lock file similar to npm/yarn. Instead of requirements.txt And also “pip…

> apache:parquet How are you going to name the file storing the wheel for that package? Using ":" on Windows is going to be problematic.

That’s right. Then some other delimiter.

Re: Pypi.org is running a survey on the state of Python packaging

#186

Post by mdmglr »

tempest_ wrote: post #63
mdmglr wrote: post #17

My wishlist: We need a way to configure an ordered list of indexes pip searches for packages. —extra-index-url or using a proxy index is not the solution. Also namespaces and not based on a domain. So for example: pip install apache:parquet Also some logic either in the pip client or index server to minimize typosquatting Also pip should adopt a lock file similar to npm/yarn. Instead of requirements.txt And also “pip…

If you are using poetry you can add something to the pyproject.toml to handle the indexes, though I am not sure if they are ordered or not [[tool.poetry.source]] name = "my-pypi" url = " https://my-pypi-index.wherever " secondary = true

Thanks. I’ll look into this.

Re: Pypi.org is running a survey on the state of Python packaging

#187

Post by BerislavLopac »

pletnes wrote: post #178
BerislavLopac wrote: post #160

Earlier quoted context omitted.

> It’s only the last 5 years, give or take, that you can install C-compiled packages with pip Give, quite a bit. The egg format was introduced in 2004 [0], and even the newer wheel, that replaced it, is turning 10 later this month. [0] https://packaging.python.org/en/latest/discussions/wheel-vs-...

9 out of the top 360 packages still do not offer wheels! https://pythonwheels.com/

Not having a wheel is not really an issue for Python-only packages.

Re: Pypi.org is running a survey on the state of Python packaging

#188

Post by isitmadeofglass »

rumblings wrote: post #153
IceHegel wrote: post #101

If I see some JS, Go, or Rust code online I know I can probably get it running on my machine in less than 5 min. Most of the time, it's a ‘git clone’ and a 'yarn' | 'go install' | 'cargo run', and it just works. With python, it feels like half the time I don't even have the right version of python installed, or it’s somehow not on the right path. And once I actually get to installing dependencies, there are often ver…

After working with Java for over a year professionally, I really appreciate its dependency management system(Maven or Gradle). Whereas with Python it is always a mess(Poetry looks promising though).

> Whereas with Python it is always a mess

I’m curious because this gets repeated a lot by many people. What specific messes do you get into?

I’m asking because my experience with python these days is always just doing “python -m venv .venv” activating it and then using “pip install -r requirements.txt”

To update dependencies I do “pip install —-upgrade dep” run tests and then go “pip freeze > requirements.txt” this never fails me. Though sometimes updating dependencies does make tests fail of cause, but that’s the fault of the individual packages not the packaging system. Even so I’d say even that is rare for me these days.

I know the might only work for 95% of the workflows out there, but I’m very curious as to what specific messes the last 5% end up struggling with and what makes people like you feel that it’s always a mess and not just “sometimes it gets messy” etc.

Re: Pypi.org is running a survey on the state of Python packaging

#189

Post by isitmadeofglass »

tibbon wrote: post #135
thrown_22 wrote: post #125

Earlier quoted context omitted.

Installing Python only applications is trivial. What you're complaining about is all the missing code in other languages which isn't controlled by Python and depends on the OS to provide. This is why we created Linux distributions in the first place. It is not the place of every language to reinvent the wheel - poorly.

Isn’t wheel one of pythons many packaging schemes (intended to replace eggs?)? > There should be one– and preferably only one –obvious way to do it. Oh no…

> There should be one– and preferably only one –obvious way to do it.

Does not mean

> There should only ever be one way to do each thing.

But somehow extremely many people read it like that.

Re: Pypi.org is running a survey on the state of Python packaging

#190

Post by pletnes »

BerislavLopac wrote: post #187
pletnes wrote: post #178

Earlier quoted context omitted.

9 out of the top 360 packages still do not offer wheels! https://pythonwheels.com/

Not having a wheel is not really an issue for Python-only packages.

You’re running install-time code if you don’t, so I disagree. The wheel format is better suited even for pure python distribution. You can read more details on that site.
Post reply on HN