we need one, only one build system, not many. Feels like atm python is a bit shattered around like c++
Using JS or rust seems easier with the unified build system.
Python: Please stop screwing over Linux distros
21–30 of 384 posts
Re: Python: Please stop screwing over Linux distros
#22And stop putting everything on the planet into a god damn virtual environment.
What don't you like about virtual environments?
As far as I can tell, they're a strict requirement, if you ever work on more than one project and want to isolate the dependencies (because you need different versions of the same package, because you want to ensure you've tracked all deps, whatever).
Is it that you don't isolate dependencies between your python projects at all, or is there some other solution you prefer?
Re: Python: Please stop screwing over Linux distros
#23> pin their dependencies to 10 versions and 6 vulnerabilities ago
That is the real problem IMHO. Most python users like to pin dependencies so that "their program don't break", and that's also the reason why so much effort is put in what they call "correct" dependency resolution resulting in the creation of new python package managers that all do "more correct" dependency resolution and make programs "break less" and at the same time require less efforts from the maintainers to actually maintain their code by doing dependency upgrades. And then one day you want to upgrade a dependency and you realize you're 10 releases behind on 10 dependencies, and what was supposed to be a quick maintenance task is now 100 maintenance task.
If you don't pin, your program will break one day or another, a user might open an issue with the traceback, or, your program will break directly in CI where you will see the traceback. Upgrade it, or contribute to the dependency, but just go ahead and fix it, instead of being defensive and trying to have dependency resolution that "doesn't break". At the same time you'll be adopting the actual practice of "Continuous Integration", of your dependencies, which has a better cost/benefit ratio.
I always avoid pinning dependencies, I try to make pip just install the latest version of everything, I am willing to contribute upgrade fixes to any Python package I use, but some dependencies do pin which breaks my own aggressive Continuous Integration practice, heck, I'd even need an option for pip to ignore version resolution at all so that I can make all my contributions to upgrade everything I use. I even remember when I had CI test matrix with all combinations of versions of everything, I don't do that anymore, I just support the latest of everything, we can always have the latest python with containers anyway so that's not even a blocker anymore. If you're not a "techbro using containers", it'd be fine too because you should then be able to make your distro packages at any point in time and expect all of them to work together, minus the delta of the handful of upgrades that are pending to release here and there.
Re: Python: Please stop screwing over Linux distros
#24I am not one to install python packages using my distro's package manager, but I totally agree with the sentiment that we need a more standard build/dependency management system in python. I like poetry, and I think most people are heading that way, but it doesn't seem to play super nice with pyenv (which is a critical tool) a lot of the time, and I think that a first party endorsement of the "one true build system"…
I’m not a Python dev. I do need to occasionally run things written in Python. I made the mistake of trying to get pip + Conda + pyenv (or whatever) to install a fairly simple tool. I have no idea how the dev got their setup working, but it was totally and utterly unreproducable, even after they sat down on my computer for several hours. In that amount of time, I could have probably rewritten it in PHP (that’s actuall…
I think this actually gets to the crux of the matter. The existing python dependency management tools (especially the new shiny ones like poetry) are very much designed by python devs, for python devs. What you're describing is a totally different use case, which is running released software in a sensible way.
The distro package managers are probably the best place for that, but bridging the gap between them and the python ecosystem is an obvious challenge.
Re: Python: Please stop screwing over Linux distros
#25Earlier quoted context omitted.
Virtual environments are a good way to have different packages for different applications running on different versions of Python. My usual rule is you don't mess with the system-provided Python environments for specific applications you are working with. I would even suggest dropping support for many packages at the distro level unless they are required by other non-python packages (the same way Django and Twisted a…
> don't mess with the system-provided Python environments for specific applications you are working with Right, but then just use pip install --user instead of a virtualenv. Actually --user is the default now when running pip install from non-root user, so, just pip install as a user will work and not mess with the system packages, just don't do sudo pip install.
I've run into problems when running pip install --user. Someone installs a package in the user environment, and that works. But later you'll install a package in a shared virtualenv (because a Python program needs to run as a service or by another users or whatever) and pip doesn't install it because it's already on the user environment. However, other users don't have access to that environment and they will have an import error that you won't see from your user.
Not to mention that installing every dependency in the same environment is a recipe for both disaster with version conflicts and bloat when you don't really know which packages belong to which applications.
Re: Python: Please stop screwing over Linux distros
#26Interesting take. I use Python daily and virtualenv is really good for our use cases (running CI/CD, testing installation and running in production in a container). I am not experiencing too many problems with pip + venv. We also maintain our own libraries and even those were relatively easy to set up. Mypy and yapf also helps to maintain a style and type correctness (do not pass in None accidentally).
Re: Python: Please stop screwing over Linux distros
#27I am not one to install python packages using my distro's package manager, but I totally agree with the sentiment that we need a more standard build/dependency management system in python. I like poetry, and I think most people are heading that way, but it doesn't seem to play super nice with pyenv (which is a critical tool) a lot of the time, and I think that a first party endorsement of the "one true build system"…
I’m not a Python dev. I do need to occasionally run things written in Python. I made the mistake of trying to get pip + Conda + pyenv (or whatever) to install a fairly simple tool. I have no idea how the dev got their setup working, but it was totally and utterly unreproducable, even after they sat down on my computer for several hours. In that amount of time, I could have probably rewritten it in PHP (that’s actuall…
This is especially frustrating to read because one of the main selling points of Conda is reproducibility. In data science teams, I've found it indispensable for making sure people can all run each others' project code.
So for anyone reading this in the future: don't try to use Pyenv to install Conda. Pyenv tries to set up shims for every binary in the Conda env, which will likely break your PATH.
Pyenv supports installing Conda because Anaconda used to be "just" a Python distribution.
They can otherwise coexist without trouble on the same system.
Re: Python: Please stop screwing over Linux distros
#28This is my decision flow and I rarely have an issue:
Are you the end user of the Python code?
Yes -> Is available in your distro?
Yes->Use package manager
No->Use pip install in user mode
No -> Create virtualenv with the Python version you want (including pypy!) and do your pip thing there
Re: Python: Please stop screwing over Linux distros
#29I am not one to install python packages using my distro's package manager, but I totally agree with the sentiment that we need a more standard build/dependency management system in python. I like poetry, and I think most people are heading that way, but it doesn't seem to play super nice with pyenv (which is a critical tool) a lot of the time, and I think that a first party endorsement of the "one true build system"…
I’m not a Python dev. I do need to occasionally run things written in Python. I made the mistake of trying to get pip + Conda + pyenv (or whatever) to install a fairly simple tool. I have no idea how the dev got their setup working, but it was totally and utterly unreproducable, even after they sat down on my computer for several hours. In that amount of time, I could have probably rewritten it in PHP (that’s actuall…
Both Conda and pyenv are for managing (their own) Python installations to run different Python versions in parallel, it doesn't really compute to try and use both at the same time.
Re: Python: Please stop screwing over Linux distros
#30It seems a bit weird to criticise lots of people for trying to solve the dependency problem. I know lots of people hold up npm as the standard. Npm came out around 10 years ago, Node around 12. Both of which had the benefit of hindsight at how the problem had developed for other languages. Python came out 30 years ago and pip came out 20 years later. Of course there is a lot of stuff left behind. What do people suggest, a breaking change, maybe? Python 4?