New Pip resolver takes a long time to complete

#1

Post by josmek » Thu, Dec 03, 2020, 8:35 PM UTC

New Pip resolver takes a long time to complete
github.com

Re: New Pip resolver takes a long time to complete

#2

Post by mst » Thu, Dec 03, 2020, 9:06 PM UTC

I built a resolver like this for perl/CPAN at once point.

Then realised it would have this problem a few days before I'd planned to release, and didn't ship it.

I am feeling very lucky right now.

Re: New Pip resolver takes a long time to complete

#3

Post by rdlfo » Thu, Dec 03, 2020, 9:08 PM UTC

Perhaps something like mamba [1] could speed up the resolver. That is if the resolver is actually achieving anything at all.

[1] https://github.com/mamba-org/mamba

Re: New Pip resolver takes a long time to complete

#4

Post by riedel » Thu, Dec 03, 2020, 9:11 PM UTC

Maybe by fixing the solver they can finally prove P=NP ;)

Re: New Pip resolver takes a long time to complete

#5

Post by oehtXRwMkIs » Thu, Dec 03, 2020, 9:15 PM UTC

riedel wrote: post #4

Maybe by fixing the solver they can finally prove P=NP ;)

I think that will be proven before python fixes their package management.

Re: New Pip resolver takes a long time to complete

#6

Post by noitpmeder » Thu, Dec 03, 2020, 9:17 PM UTC

oehtXRwMkIs wrote: post #5

riedel wrote: post #4

Maybe by fixing the solver they can finally prove P=NP ;)

I think that will be proven before python fixes their package management.

At least they are trying to fix it, vs other languages where there are literally no paths forward. IMO python has one of the more mature packaging ecosystems out there at this point. It is so incredibly easy.

Re: New Pip resolver takes a long time to complete

#7

Post by FloatArtifact » Thu, Dec 03, 2020, 9:20 PM UTC

noitpmeder wrote: post #6

oehtXRwMkIs wrote: post #5

Earlier quoted context omitted.

I think that will be proven before python fixes their package management.

At least they are trying to fix it, vs other languages where there are literally no paths forward. IMO python has one of the more mature packaging ecosystems out there at this point. It is so incredibly easy.

Distribution still is a pain unless you're going through pip.

Re: New Pip resolver takes a long time to complete

#8

Post by pydry » Thu, Dec 03, 2020, 9:24 PM UTC

FloatArtifact wrote: post #7

noitpmeder wrote: post #6

Earlier quoted context omitted.

At least they are trying to fix it, vs other languages where there are literally no paths forward. IMO python has one of the more mature packaging ecosystems out there at this point. It is so incredibly easy.

Distribution still is a pain unless you're going through pip.

who doesn't use pip?

Re: New Pip resolver takes a long time to complete

#9

Post by sjburt » Thu, Dec 03, 2020, 9:26 PM UTC

They need to store the version requirements metadata outside the packages. Having to download the entire wheel just to see whether it is compatible is ridiculous. This could all be computed by downloading an index file, then performing the resolution.

They made a half-assed attempt when pypa/pip added the "data-requires-python" tag, but that covers python interpreter version only, it needed to have been done for all dependencies.

What's especially irksome is that Debian and RPM both solved this 20+ years ago and python has refused to learn any lessons.

Re: New Pip resolver takes a long time to complete

#10

Post by dang » Thu, Dec 03, 2020, 9:27 PM UTC

Recent and related: https://news.ycombinator.com/item?id=25253236