Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What the Heck Is Pyproject.toml?

snarky.ca

Post reply on HN
Print view View original

111–120 of 199 posts

Re: What the Heck Is Pyproject.toml?

#111

Post by takluyver »

pottertheotter wrote: post #46
talentedcoin wrote: post #40

I don’t want to be too negative, and I realize this isn’t an option for everyone ... but seriously, just use conda if you can. Judging from comments here and in the past, we all know packaging is one of Python’s Achilles’ heels. I spent a couple years bouncing around with pyenv/pipenv, then poetry, then back. I lovingly followed all the flamewars here and on github. Ultimately, I realize now, I laboured under the con…

I've had the opposite experience. I used conda for several years, primarily for data science, but regularly ran into problems with packages not working, updates for packages bring severely delayed, and issues with conda itself. Switched to pyenv+virtualenv 18 months ago and haven't had a single issue. I believe both of our stories. I just always wonder how this happens that two people have such opposite experiences.

Just to note: this kind of difference is part of why it's so difficult to improve packaging. There are lots of different tools, and they all work well for some people in some situations. There's no realistic prospect of persuading them all to use a single tool - but the fragmentation is a big part of what makes packaging so daunting.

Re: What the Heck Is Pyproject.toml?

#112

Post by takluyver »

twic wrote: post #108
toyg wrote: post #53

Earlier quoted context omitted.

Ruby serves one ecosystem (web), barely. Python serves several huge ones: web, datascience, 3d graphics, sysadmin... all with different needs, different conventions, different “standard” tools to do this and that. The community is orders of magnitude bigger than ruby by now, and anything that requires coordination is much harder to accomplish than in a lot of other ecosystems. Python is now on par with ecosystems lik…

> Python serves several huge ones: web, datascience, 3d graphics, sysadmin... all with different needs, different conventions, different “standard” tools to do this and that. Them having different conventions and standard tools is the failure, not a constraint. What are the different needs that these ecosystems have which mean they couldn't use a common package manager? I don't believe there are any.

For instance, in the scientific computing world, compiled extensions are hugely important, whereas they're relatively rare in web development.

There's probably no logical reason why a single common package manager for these domains is impossible. But they have different priorities, and people write tools that solve their own problems. Who is going to decide which package managers are unnecessary, and convince all of the users who are happy with them to switch to something else?

Re: What the Heck Is Pyproject.toml?

#113

Post by yamrzou »

I have found this blog post[1] quite informative regarding Python projects packaging. It describes well how all the tools (pyenv, poetry, black, flake8, isort, pre-commit, pytest, coverage, tox) fit together.

Poetry seems the go-to tool for dependencies management and it centralizes all your package configurations alongside its dependencies in pyproject.toml.

[1] https://medium.com/georgian-impact-blog/python-tooling-makes...

Re: What the Heck Is Pyproject.toml?

#114

Post by williamdclt »

BiteCode_dev wrote: post #97
earthboundkid wrote: post #50

Earlier quoted context omitted.

I am an experienced Python programmer. I often have to collaborate with people who are not. There is no greater hell than talking them through installing a working Python environment. It’s impossible. Meanwhile, the pip project is full of developers who tell you that your problem is not a problem and you’re discouraging people by pointing out that they’re blame shifting: https://github.com/pypa/pip/issues/4995 I hone…

> There is no greater hell than talking them through installing a working Python environment. It’s impossible. I give a lot of Python professional trainings, and I get to do that regularly, in very diverse situations. It's indeed full of gotchas. Since it's not going to be solved quickly, meanwhile, here is what works if you need to help people setuping python: 1 - Install Python correctly The first version Python do…

That's a pretty solid guide! It's even useful for me that writes Python (non-exclusively) professionally.

My company settled on pipenv, and we regret it. It's slow and buggy and has crazy behaviour. It's not bad enough that it's worth updating all projects (or rather, there's no good enough alternative) but I regret about once a week having advocated for it.

Note: I think you made a mistake, you say that you can rename or move a venv but then you say you can't.

Re: What the Heck Is Pyproject.toml?

#115

Post by globular-toast »

Pandabob wrote: post #106
earthboundkid wrote: post #50

Earlier quoted context omitted.

I am an experienced Python programmer. I often have to collaborate with people who are not. There is no greater hell than talking them through installing a working Python environment. It’s impossible. Meanwhile, the pip project is full of developers who tell you that your problem is not a problem and you’re discouraging people by pointing out that they’re blame shifting: https://github.com/pypa/pip/issues/4995 I hone…

As a comment to your first paragraph, I've found pyenv combined with its virtualenv plugin to be a decently consistent experience.

I've never understood the need for pyenv-virtualenv when virtualenvwrapper exists. To add more confusion there is also pyenv-virtualenvwrapper.

Re: What the Heck Is Pyproject.toml?

#116

Post by globular-toast »

kelnos wrote: post #77
jshen wrote: post #45

Earlier quoted context omitted.

How do other people replicate your venv? How does production replicate your venv?

You pin all your direct dependencies in requirements.txt, run "pip freeze >requirements-frozen.txt" to get a list of the versions of all dependencies (direct and transitive), and then you ship that. Replicating is just "virtualenv --python=python3 venv && . ./venv/bin/activate && pip install -r requirements-frozen.txt". Am I missing something here? Yes, to be absolutely sure, you need to have the same versions of pyt…

> You pin all your direct dependencies in requirements.txt, run "pip freeze >requirements-frozen.txt" to get a list of the versions of all dependencies (direct and transitive), and then you ship that.

What?! Seriously? That is not how it's supposed to work. Requirements.txt is already meant to be the frozen dependencies (like the output of pip freeze). Why change that? I much prefer how pip-tools does it with a new file, requirements.in that contains the unpinned direct dependencies. I thought poetry did the same thing but with pyproject.toml?

Re: What the Heck Is Pyproject.toml?

#117

Post by appleflaxen »

BiteCode_dev wrote: post #97
earthboundkid wrote: post #50

Earlier quoted context omitted.

I am an experienced Python programmer. I often have to collaborate with people who are not. There is no greater hell than talking them through installing a working Python environment. It’s impossible. Meanwhile, the pip project is full of developers who tell you that your problem is not a problem and you’re discouraging people by pointing out that they’re blame shifting: https://github.com/pypa/pip/issues/4995 I hone…

> There is no greater hell than talking them through installing a working Python environment. It’s impossible. I give a lot of Python professional trainings, and I get to do that regularly, in very diverse situations. It's indeed full of gotchas. Since it's not going to be solved quickly, meanwhile, here is what works if you need to help people setuping python: 1 - Install Python correctly The first version Python do…

this is an awesome summary; I use python occasionally, and have run into several of the issues you flag in your post, but would probably do it all over again next time I need to put something in production. So thank you for taking the time to summarize things so thoughtfully.

Re: What the Heck Is Pyproject.toml?

#118

Post by [deleted] »

kelnos wrote: post #77
jshen wrote: post #45

Earlier quoted context omitted.

How do other people replicate your venv? How does production replicate your venv?

You pin all your direct dependencies in requirements.txt, run "pip freeze >requirements-frozen.txt" to get a list of the versions of all dependencies (direct and transitive), and then you ship that. Replicating is just "virtualenv --python=python3 venv && . ./venv/bin/activate && pip install -r requirements-frozen.txt". Am I missing something here? Yes, to be absolutely sure, you need to have the same versions of pyt…

[deleted]

Re: What the Heck Is Pyproject.toml?

#119

Post by viraptor »

jshen wrote: post #82
kelnos wrote: post #77

Earlier quoted context omitted.

You pin all your direct dependencies in requirements.txt, run "pip freeze >requirements-frozen.txt" to get a list of the versions of all dependencies (direct and transitive), and then you ship that. Replicating is just "virtualenv --python=python3 venv && . ./venv/bin/activate && pip install -r requirements-frozen.txt". Am I missing something here? Yes, to be absolutely sure, you need to have the same versions of pyt…

This is worse than how it’s done on most modern languages.

Different commands, but it's pretty much the same in similar languages: Ruby uses bundler configuration instead of virtualenv and gemfiles lock instead of requirements.txt. JS uses npm/yarn and lockfie. Go uses go.mod and does a lot of assumptions instead of virtualenv. (these days, it was worse)

Modern languages do pretty much the same, just have a nicer wrapper for it.

Re: What the Heck Is Pyproject.toml?

#120

Post by nicoburns »

earthboundkid wrote: post #51
jshen wrote: post #44

Earlier quoted context omitted.

This is BS. most other languages, with the same constraints, have done far better. Take Ruby as one example.

Yes, Python’s packaging is partly bad because it’s based on C extensions, which are intrinsically bad, but it’s also just plain bad compared to its peers, like Ruby and NPM.

NPM also has support for C extensions, and they mostly work reliably (the only problems I've had are forgetting to `npm rebuild` when switching between node versions, or when switching between native mac builds and linux-in-docker builds.
Post reply on HN