131–140 of 229 posts
Earlier quoted context omitted.
`pip freeze > requirements.txt`
This assumes semantic versioning and does not actually pin dependencies to the hash of their bytes, like a lockfile does.
https://github.com/jazzband/pip-tools https://gist.github.com/hynek/5e85706cee589a204251b333595853...
https://gist.github.com/hynek/5e85706cee589a204251b333595853...
update-deps:
pip-compile --upgrade --generate-hashes --output-file requirements/main.txt requirements/main.in
pip-compile --upgrade --generate-hashes --output-file requirements/dev.txt requirements/dev.in
init:
pip install --editable .
pip install --upgrade -r requirements/main.txt -r requirements/dev.txt
rm -rf .tox
update: update-deps init
.PHONY: update-deps init updateEarlier quoted context omitted.
I don't understand the pyproject.toml hate. pyproject.toml exists specifically so you can specify your build system. Without it, you are basically forced to use setuptools/distutils as is currently the case. Hence, pyproject.toml and setup.cfg aren't at all in conflict.
- there is nothing you can do with pyproject.toml that you can't with setup.cfg. E.G: you are not forced to use setuptools to use setup.cfg. Any tool supporting pyproject.toml could support setup.cfg as easily, since it's a documented plain text format. It's a political decision. - there are things you can't do with pyproject.toml you can with setup.cfg. E.G: you can't use pyproject.toml with legacy tools, or with ju…
Earlier quoted context omitted.
And it isn't only Javascript that manages to have a single tool for this. Clojure has a single tool. Java has a single tool. Rust has a single tool. This is a solved problem across a variety of popular and mainstream programming languages. I don't mean to suggest that the problem isn't complicated, but this isn't a problem that doesn't have a wealth of previously written solutions to look at for inspiration.
What's the "one tool" for Java? Is it Maven? Ivy? sbt? Gradle? You may say some of those are "build tools" rather than dependency tools, but I don't see how it's different than what we are discussing in Python. My Java app declares its dependencies in a build.sbt file using Scala syntax and has them cached in an Ivy directory. Yours declares them in a pom file using XML syntax and has them cached in a Maven directory…
Most shops I know use one of maven or gradle. I can’t think of any other serious contenders in Java (if you still use ant: please consider alternatives).
Pipenv is a really interesting development for Python, and I'm glad that someone was working to improve dependency locking for Python. However, Kenneth abused his position with PyPA (and quickly bumped a what is a beta product to version 18) to imply Pipenv was more stable, more supported and more official than it really was. And worse still, for anyone saying "but ts open source, you get what you pay for", Kenneth a…
Sometimes, improvements don't happen in a straight line. There's been a lot of work on Pipenv over the the last 6 months, predominantly by Dan Ryan and Tzu-Ping Chung, and it's getting stronger and stronger with each release. If you've gone back to using pip I'd encourage you to give Pipenv another try. Introducing a lockfile is a big step forward for Python dependency management, and the team working on Pipenv are c…
Huh? I'm not really familiar with the state of dependency management for Python/dynamic languages but... there's much more out there beyond just lockfiles. I'm a bit appalled Python is so far behind.
The root problem is that just having a system like pip to manage your Python dependencies is woefully insufficient, because it installs dependencies globally, you might need different versions in different projects, and you might be trying to link in some native code, so that behavior will change based on what you already have installed or what your OS is. Also, to make matters worse, you also have to manage your dependency on Python itself, since there are multiple mutually incompatible versions.
So you can use virtualenv to handle the problem of isolating one Python project from another, pyenv to handle the problem of different Python projects using different versions of Python. Then you need a system like pipenv to tie them all together. Except pipenv, itself, is in Python, so there's a bootstrapping issue with using a tool written in Python that has a dependency on a Python interpreter to indirectly manage your dependency on a Python interpreter. Sometimes if you do something ridiculous like "using a Mac where you haven't specifically installed the right version of Python via pyenv or Homebrew yet", things will break in confusing ways.
One way around this whole mess is to put everything in a Docker container, so you get to stipulate that everything runs on a particular Linux distro with a particular set of dependencies from the ground up and there's no possibility of anything else at all on your or any other machine ever polluting that dependency chain and even if you're on a Mac it'll just install dependencies and run code inside of a VM. The isolation you're trying to accomplish with virtualenv or even pyenv, whether by hand or using a tool like pipenv, is pretty much just a poor man's Docker container anyway. But this adds complexity of its own while still punting the real work off to a tool like Pip.
Pipenv is a really interesting development for Python, and I'm glad that someone was working to improve dependency locking for Python. However, Kenneth abused his position with PyPA (and quickly bumped a what is a beta product to version 18) to imply Pipenv was more stable, more supported and more official than it really was. And worse still, for anyone saying "but ts open source, you get what you pay for", Kenneth a…
Sometimes, improvements don't happen in a straight line. There's been a lot of work on Pipenv over the the last 6 months, predominantly by Dan Ryan and Tzu-Ping Chung, and it's getting stronger and stronger with each release. If you've gone back to using pip I'd encourage you to give Pipenv another try. Introducing a lockfile is a big step forward for Python dependency management, and the team working on Pipenv are c…
Earlier quoted context omitted.
Poetry is one of the few things that gives me hope about the mess of python packaging. It is also great to see the author is very responsive. My only concern is the lack of integrated "toolchain" management (what versxon of python to use, something like rustup) that is cross platform.
I agree that providing toolchains is very important. The only non-system package manager that provides Python and its own toolchains - for Linux and macOS presently - which are used to compile every C, C++ and Fortran package, including Python itself is conda and the Anaconda Distribution. Not doing this leads to static linking and that's inefficient and insecure. Disclaimer: I work for Anaconda Inc.
Nix[0] is also perfectly usable without NixOS, and provides all of that, but has far more non-Python libraries and applications packaged. It's also not constantly trying to sell you an enterprise version...
Meanwhile, I've recently started using the setup.py file and its install_requires and extra_required fields for production requirements and dev requirements, respectively. I don't even use requirements.txt as thanks to setup.py I only have to install the folder itself. Am I weird? Am I not supposed to do that? Is it bad practice?
Earlier quoted context omitted.
And it isn't only Javascript that manages to have a single tool for this. Clojure has a single tool. Java has a single tool. Rust has a single tool. This is a solved problem across a variety of popular and mainstream programming languages. I don't mean to suggest that the problem isn't complicated, but this isn't a problem that doesn't have a wealth of previously written solutions to look at for inspiration.
What's the "one tool" for Java? Is it Maven? Ivy? sbt? Gradle? You may say some of those are "build tools" rather than dependency tools, but I don't see how it's different than what we are discussing in Python. My Java app declares its dependencies in a build.sbt file using Scala syntax and has them cached in an Ivy directory. Yours declares them in a pom file using XML syntax and has them cached in a Maven directory…
I get why Python needs lockfiles, but goddamn, that need is a symptom of the mess of managing Python dependencies.
There's still a long way to go - I use Airflow for ETL management, and I'm using pipenv to manage it - except Pipenv can't create a lockfile because a dependency of a dependency of Airflow requires in flask-wtf In a Maven pom.xml I can easily exclude or override the conflicting dependency manually if needed, but I can't in a Pipfile.