21–30 of 45 posts
Earlier quoted context omitted.
I think that will be proven before python fixes their package management.
At least they are trying to fix it, vs other languages where there are literally no paths forward. IMO python has one of the more mature packaging ecosystems out there at this point. It is so incredibly easy.
I used to think package managers and dependency resolvers were inseperable, I even did a little bit of work on the APT resolver. Since using GNU Guix though, I'm so glad it doesn't have a dependency resolver as part of building or installing packages! It's so much better for it, no slow or unpredictable resolving, you know what it's going to do. I think this is one reason why I've never used pip for managing Python s…
Wait, how does guix avoid a dependency resolver? It still has dependencies for packages... are they just hardcoding exact dependencies (including versions) and relying on the "can install multiple versions of a package" property to make that work? That seems inefficient, though I guess maybe they need that for the "this hash means this exact binary" outcome?
Guix package definitions are truely code though, so if you want to generate packages on the fly by using a dependency resolver, you can totally write some code to make that happen.
With respect to inefficiency, what do you mean? It's quite time efficient when building and installing to not have to attempt to resolve dependencies.
Perhaps something like mamba [1] could speed up the resolver. That is if the resolver is actually achieving anything at all. [1] https://github.com/mamba-org/mamba
The pip section in a env file is just a list of arguments passed through to the pip install command. Prior to pip 20.3 we had to add `--use-feature=2020-resolver` to get an install that resolved for our teams that used mamba.
They need to store the version requirements metadata outside the packages. Having to download the entire wheel just to see whether it is compatible is ridiculous. This could all be computed by downloading an index file, then performing the resolution. They made a half-assed attempt when pypa/pip added the "data-requires-python" tag, but that covers python interpreter version only, it needed to have been done for all…
I wish different language/platform communities were better at learning from each other (and this does go in all directions). In the field of software these days, we don't do much learning from prior art. It's just too hard to keep up with it all.
Heck, I recall this happening with rubygems -- the introductio of the dependencies API to efficiently get the minimum data needing for resolving dependencies before downloading actual packages, and then several iterations on it -- and I can't find any actual documentary evidence of it to share right now. I don't know how anyone WOULD use it as prior art; the source code for a fairly complex project in a language you aren't familiar with isn't going to work, even if you knew to go look for it, which why would you.
Earlier quoted context omitted.
Wait, how does guix avoid a dependency resolver? It still has dependencies for packages... are they just hardcoding exact dependencies (including versions) and relying on the "can install multiple versions of a package" property to make that work? That seems inefficient, though I guess maybe they need that for the "this hash means this exact binary" outcome?
Guix has packages, and packages have inputs (like dependencies), and you're right in that normally package definitions specify the exact dependencies it the code (they're hardcoded). Guix package definitions are truely code though, so if you want to generate packages on the fly by using a dependency resolver, you can totally write some code to make that happen. With respect to inefficiency, what do you mean? It's qui…
It's disastrous for security patches, only highly inconvenient for things like performance improvement releases. But this is why we have dependency resolving, right? What am I missing?
Perhaps something like mamba [1] could speed up the resolver. That is if the resolver is actually achieving anything at all. [1] https://github.com/mamba-org/mamba
My understanding is mamba, like conda, just call pip. So it likely wouldn't make a difference. The pip section in a env file is just a list of arguments passed through to the pip install command. Prior to pip 20.3 we had to add `--use-feature=2020-resolver` to get an install that resolved for our teams that used mamba.
Earlier quoted context omitted.
At least they are trying to fix it, vs other languages where there are literally no paths forward. IMO python has one of the more mature packaging ecosystems out there at this point. It is so incredibly easy.
>IMO Python has one of the more mature packaging ecosystems out there Ouch, what languages are you using where this is true? For example, JS / NPM / Yarn are absolutely blowing pip out of the water. I guess I can imagine Java or C++ users having your perspective though
There are a lot of smart people working on these things in each ecosystem, and when you think some package manager is far surperior over others in every way, you are more than often simply wrong. Or saying it the other way (and paraphrasing a commenter from another thread), the only package manager you think is good is from the ecosystem you are not deeply familiar with.
Earlier quoted context omitted.
My understanding is mamba, like conda, just call pip. So it likely wouldn't make a difference. The pip section in a env file is just a list of arguments passed through to the pip install command. Prior to pip 20.3 we had to add `--use-feature=2020-resolver` to get an install that resolved for our teams that used mamba.
No, conda is not calling pip
Conda installs conda packages and conda uses pip to install pip packages. However a pip package can be converted to a conda package, and then in that case the dependency will be installed by conda and not pip.
They need to store the version requirements metadata outside the packages. Having to download the entire wheel just to see whether it is compatible is ridiculous. This could all be computed by downloading an index file, then performing the resolution. They made a half-assed attempt when pypa/pip added the "data-requires-python" tag, but that covers python interpreter version only, it needed to have been done for all…