11–20 of 45 posts
Since using GNU Guix though, I'm so glad it doesn't have a dependency resolver as part of building or installing packages! It's so much better for it, no slow or unpredictable resolving, you know what it's going to do.
I think this is one reason why I've never used pip for managing Python software, I've only ever used Debian, and then Guix.
Are there other languages that have dependencies decided at installtime?
Did I get it right, the problem is basically that the dependency info is turing complete and decided "at installtime", thus the dependency graph cannot be computed without installing many versions of all dependencies? (I do not use python, so I don't know much about how pip works) Are there other languages that have dependencies decided at installtime?
It's NP-complete in general I think, definitely the case in Haskell-land. I think OCaml sets an explicit timeout for dependency resolution?
> the dependency graph cannot be computed without installing many versions of all dependencies?
Does PyPi not have an index or something? (with statically known package bounds?)
Did I get it right, the problem is basically that the dependency info is turing complete and decided "at installtime", thus the dependency graph cannot be computed without installing many versions of all dependencies? (I do not use python, so I don't know much about how pip works) Are there other languages that have dependencies decided at installtime?
They need to store the version requirements metadata outside the packages. Having to download the entire wheel just to see whether it is compatible is ridiculous. This could all be computed by downloading an index file, then performing the resolution. They made a half-assed attempt when pypa/pip added the "data-requires-python" tag, but that covers python interpreter version only, it needed to have been done for all…
The TL;DR is that it probably requires a PEP.
FD: I've been working on adjacent features in PyPI.
Earlier quoted context omitted.
I think that will be proven before python fixes their package management.
At least they are trying to fix it, vs other languages where there are literally no paths forward. IMO python has one of the more mature packaging ecosystems out there at this point. It is so incredibly easy.
Ouch, what languages are you using where this is true? For example, JS / NPM / Yarn are absolutely blowing pip out of the water. I guess I can imagine Java or C++ users having your perspective though
I used to think package managers and dependency resolvers were inseperable, I even did a little bit of work on the APT resolver. Since using GNU Guix though, I'm so glad it doesn't have a dependency resolver as part of building or installing packages! It's so much better for it, no slow or unpredictable resolving, you know what it's going to do. I think this is one reason why I've never used pip for managing Python s…
Did I get it right, the problem is basically that the dependency info is turing complete and decided "at installtime", thus the dependency graph cannot be computed without installing many versions of all dependencies? (I do not use python, so I don't know much about how pip works) Are there other languages that have dependencies decided at installtime?
setup(
install_requires=[random.choice(["urllib3", "requests"])]
)
So that leads to this situation, from another comment:
"And as an example, [the botocore package, which has releases nearly daily] depends on python-dateutil>=2.1,Sounds like there's a long term plan that could fix this situation. Binary wheels already have the needed metadata in a way that could be exposed by PyPI via a fast "fetch dependency constraints for all versions" API, but isn't yet. And for source dists there's a very new plan ( https://www.python.org/dev/peps/pep-0643/ ) to let them indicate that they don't modify install_requires at runtime, so their deps could also be exposed via API, but the ecosystem will have to catch up with that.
I dunno what pip does in the meantime, though!
(I'm a python dev but haven't followed this beyond skimming the bug, so I hope I'm getting this right.)
They need to store the version requirements metadata outside the packages. Having to download the entire wheel just to see whether it is compatible is ridiculous. This could all be computed by downloading an index file, then performing the resolution. They made a half-assed attempt when pypa/pip added the "data-requires-python" tag, but that covers python interpreter version only, it needed to have been done for all…
They need to store the version requirements metadata outside the packages. Having to download the entire wheel just to see whether it is compatible is ridiculous. This could all be computed by downloading an index file, then performing the resolution. They made a half-assed attempt when pypa/pip added the "data-requires-python" tag, but that covers python interpreter version only, it needed to have been done for all…