I wish there was a way to lock the global python so you couldn't install packages to it by accident
You can do this in UNIX systems by dropping a line like this in your ~/.bashrc
. /home/matt/envs/menv/bin/activate
Use `Python -m Pip`
21–30 of 141 posts
Re: Use `Python -m Pip`
#22Earlier quoted context omitted.
I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…
As someone two months into their first python job - the ecosystem is solid for ML / data science, but there's a lot of places where it's painfully lacking. I hate the most popular ORM (sqlalchemy) and alembic has a lot of footguns: for instance, if you autogenerate a migration where you change a table name, it will try to drop the old table and create a new one. In web dev a lot of the OSS community has moved on to m…
Much prefer the Django ORM, which is a pain to use if you are not building web apps
Re: Use `Python -m Pip`
#23Pyenv[1] solves the multiple versions of python problem in my experience. You can install the version of python you want and then can set that version globally, per directory, and then use that versions pip or take it further and use a virtualenv/poetry shell. 1: https://github.com/pyenv/pyenv
This!
I use a combination pyenv, direnv and poetry for my projects.
Re: Use `Python -m Pip`
#24Python is a fun language but the ecosystem around it is horrible. It's a shame. I just want to pip install like I would a package manager
I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…
To be honest, I don't see how you can give this advice. A couple of the most popular package management systems for Python are very obviously deficient (and sigh, these are usually the ones I get stuck working with, due to outside constraints).
A classic example is version pinning. Rust has Cargo.lock, Ruby has Gemfile.lock. Python? It depends on which one of the multitude of options you pick. But at least a couple of the most popular ones basically don't do this (pip) or do this in a hacky, ugly way that makes you want to tear your hair out (Conda). As a result, at least in the projects I work on, people tend to skip this.
I hope it should be obvious what bad things can happen if you don't pin your dependencies, but for the uninitiated: I'm talking about things like projects breaking inexplicably after 6 months because you rebuilt some Docker container and something or other got upgraded and is now incompatible.
Beyond the technical issues, there's a human engineering problem of getting everyone on the same page about the right processes, and Python makes that vastly harder by not having a One True Solution that everyone just uses.
Re: Use `Python -m Pip`
#25Earlier quoted context omitted.
I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…
As someone two months into their first python job - the ecosystem is solid for ML / data science, but there's a lot of places where it's painfully lacking. I hate the most popular ORM (sqlalchemy) and alembic has a lot of footguns: for instance, if you autogenerate a migration where you change a table name, it will try to drop the old table and create a new one. In web dev a lot of the OSS community has moved on to m…
agreed on the lockfile. we use pip-tools to achieve this, which gives the `pip-compile` command. a Makefile is tying together the different development and build workflows we have, such that it can be executed both locally as well as in our CICD build server. complexity is manageable, but it's roughly comparable to a compiled language if you want locked-down reproducible builds (which I recommend for anything production grade).
Re: Use `Python -m Pip`
#26Python is a fun language but the ecosystem around it is horrible. It's a shame. I just want to pip install like I would a package manager
I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…
The reason for the conflicting advice is because people keep making new package managers that still suck overall. Like they fix one major package manager problem and somehow neglect the other 60% of problems.
So you end up with all these different Python package managers that are all good at individual different things but no single package manager that is good at all of the things.
I swear that is a major problem with most libraries or tools (and products overall actually) that people make. It’s like people fix their pet peeve but forget that the whole picture matters way more.
Re: Use `Python -m Pip`
#27Earlier quoted context omitted.
I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…
To be honest, I don't see how you can give this advice. A couple of the most popular package management systems for Python are very obviously deficient (and sigh, these are usually the ones I get stuck working with, due to outside constraints). A classic example is version pinning. Rust has Cargo.lock, Ruby has Gemfile.lock. Python? It depends on which one of the multitude of options you pick. But at least a couple o…
"pip freeze" generates a versioned list of packages to install in the same format as requirements.txt -- in this example below i've called it versions.txt. I have a bash wrapper "bin/venv-create" which essentially does this
python3 -m venv .venv/
if [[ -f versions.txt ]] && [[ versions.txt -nt requirements.txt ]]; then
pip install --requirement versions.txt
else
pip install --requirement requirements.txt
pip freeze > versions.txt
fi
I also have a bin/venv-python wrapper which sets PYTHONPATH, PYTHONDONTWRITEBYTCODE before chain calling .venv/bin/python3 with the arguments, and this is how pip above is called. (again, omitted above for clarity.)
This won't cover everyone's usage scenario, but it works for me. YMMV.
https://iam.georgecox.com/2021/09/25/python-3-venv/ explains the details.
Re: Use `Python -m Pip`
#28I wish there was a way to lock the global python so you couldn't install packages to it by accident
I have:
$ cat ~/.pip/pip.conf
[global]
require-virtualenv = true
Re: Use `Python -m Pip`
#29What is the best way for an author of a Python package to develop and test on multiple versions of Python (e.g. 3.6, 3.7, 3.8, 3.9, 3.10), and be able to switch easily among the various Python versions? Every time I try to research this, I get lost in the chaos of Python packaging and environments.
Currently I do a `pip3 install -e .`, which uses the default Python provided by the OS. Then I hope that my continuous integration matrix catches any problems in the other Python versions.
Re: Use `Python -m Pip`
#30Python is a fun language but the ecosystem around it is horrible. It's a shame. I just want to pip install like I would a package manager
I think you mean the package management rather than the ecosystem. For all its failings, the python ecosystem (for machine learning / data science in particular) is unmatched, which is why python is so popular. As a "professional" python user (who got there in a roundabout way), I'd say the biggest problem with package management is all the conflicting advice and different ways to accomplish the same thing (there are…
Python's package management is to package managers as C++ is to programming languages: everyone who uses it understands about 10% of it, and no two people understand the same 10%.