Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What the Heck Is Pyproject.toml?

snarky.ca

Post reply on HN
Print view View original

31–40 of 199 posts

Re: What the Heck Is Pyproject.toml?

#31

Post by kevin_thibedeau »

pdonis wrote: post #3
hprotagonist wrote: post #2

The thing i have yet to reliably figure out is this (i live in the science part of python): without a setup.py, how do I: from Cython.Build import cythonize import numpy as np import versioneer setup(..., version=versioneer.get_version(), cmdclass=versioneer.get_cmdclass(), ext_modules=cythonize(extensions), include_dirs=[np.get_include()], ...) I'm 100% on board with not having executable configuration files, except…

All of those setup options are now setup.cfg options: https://docs.python.org/3/distutils/configfile.html At least, that's supposed to be the case; I am not 100% certain that it always is. However, even so, AFAIK setup.cfg does not directly support using third-party libraries to compute the setup options, as your setup.py does. The only workaround I'm aware of for that if you want to get rid of setup.py is to have a…

There are lot that of options that setup.cfg doesn't support. "use_2to3" is one.

Re: What the Heck Is Pyproject.toml?

#32

Post by skrtskrt »

xzel wrote: post #17
j88439h84 wrote: post #12

Earlier quoted context omitted.

Poetry is a modern packaging system for Python. It's really good. I hope everyone converges on it. http://python-poetry.org/

I just started up a project using this. It’s pretty good so far. The documentation on common tasks is a little bit lacking. VS Code integration is sort of broken in a few random ways (for example if I try and do a find all function definition it just hangs). I also wish there was a npm style script section in the definitions. I do think it really just works though and I think it’s a great solution compared to require…

I echo the sentiments about documentation. It works great but they really need to verbosely document every command option and usage. They seem like they’re trying so hard to keep the website clean and minimal, but docs are not the place for that.

Re: What the Heck Is Pyproject.toml?

#33

Post by brettcannon »

fermienrico wrote: post #4

Python desperately needs a better packaging solution than pip from Python Software Foundation. Stop all new features until this - the biggest pain point for Python across all expertise levels from newbies to experienced Python programmers - is officially solved. No pipenv, no poetry, no conda, etc from third party devs who sometimes get tired of the pressure [1]. It is immense. Combine all this virtualenv stuff with…

Do note that Python the language is developed by a separate group as Python the packaging ecosystem. So even when Guido was a BDFL he completely stayed out of the packaging situation.

And there is a packaging BDFL, but there's a severe lack of time and effort in the form of volunteers to tackle a lot of the projects, big or small. Plus rolling these things out takes years and it takes even longer to gain traction as people are often reluctant to change their build tools once things work.

As an example of timelines, take what my blog post covers. PEP 518 and the concept of pyproject.toml was conceived in 2016, four years ago. Getting PEP 517 finalized and it all worked into pip didn't happen until pip 19 in January 2019, over a year ago (https://pip.pypa.io/en/stable/news/#id203). And a year later there's still enough of a need to educate folks that I wrote this blog post to help get the word out.

IOW the timelines are huge and the pay-off is way down the line. And that's if you manage to be motivated enough to do the work. Burn-out is a huge problem in the packaging world. I mean reading that you think "it is a fucking mess and it's unacceptable" doesn't motivate me and others to keep trying to improve things if that's how us volunteers will be treated for years to come while we try to rectify the situation. Add on to the fact that we aren't getting paid for this and it makes the idea of pulling out my laptop tonight to work on some packaging stuff feel rather pointless.

So please be understanding that people are working on things as best they can and that overt negativity about it all will work against you getting what you want.

Re: What the Heck Is Pyproject.toml?

#34

Post by brettcannon »

no_wizard wrote: post #30
brettcannon wrote: post #23

Earlier quoted context omitted.

> I can't see a world where the Python Software Foundation can't get support for this, I think they have the contacts at the big companies to get it funded. You might be surprised at how hard it is to get money donated for this sort of thing. Large companies often have their own build infrastructure and thus build things for themselves from scratch, to the point of building their own build toolchains (e.g. look at Go…

First of all, as an aside, thank you for all the work you do on Python! I actually got started pretty early on in my professional career with Python (3.5+ too, which I am thankful for in hindsight). I was using VS Code and the Python Extension before MS even acquired its developer (is Don still there? I hope he is. Such a genuinely down to earth guy). Without the work you guys do I don't know if I would have taken th…

> First of all, as an aside, thank you for all the work you do on Python!

Welcome!

> is Don still there?

Yep, working on the data science features of the extension.

> on Talk Python I remember specifically Michael and some others talking about the massive influx of money into Python and its ecosystem in the last year and half or so.

As I said there have been donations for PyPI and pip just got its big donation. The CZI donation was also much bigger than pip and covered a lot of the science ecosystem.

> Has the PSF ever considered having an actual business? Like say, consulting and/or having developers work on features that businesses pay to have implemented?

There isn't a need for the PSF to have a business subsidiary to accomplish that as people can donate money for such things if they truly wanted to. It's also really hard to manage due to tax laws in the US around non-profits and such. Plus there would be start-up capital costs, etc. just like any other business. I just don't think the risk of trying to get it set up would lead to any benefit when people have always had the option to donate directly to see if something could be done.

If people want to help out financially they can go to https://donate.pypi.org for packaging donations and https://www.python.org/psf/donations/python-dev/ for the language and CPython. And if someone wanted to fund a specific thing I'm sure the PSF is happy to discuss such an idea to see how feasible it would be.

Re: What the Heck Is Pyproject.toml?

#35

Post by brettcannon »

throwaway894345 wrote: post #29
brettcannon wrote: post #23

Earlier quoted context omitted.

> I can't see a world where the Python Software Foundation can't get support for this, I think they have the contacts at the big companies to get it funded. You might be surprised at how hard it is to get money donated for this sort of thing. Large companies often have their own build infrastructure and thus build things for themselves from scratch, to the point of building their own build toolchains (e.g. look at Go…

It would probably go a long way if they advertised a roadmap of things they’d like to work on given donations. Maybe they have this out there and I just haven’t seen it, but I imagine it would be easy enough for me to persuade my small employer to throw a few thousand USD their way each year until some sane packaging solution is developed.

There's a rough list at https://wiki.python.org/psf/Fundable%20Packaging%20Improveme.... There's no official roadmap as getting a group of volunteers to agree to a list of priorities would be near impossible. :)

If there's a specific pain point your employer would like to see fixed and is willing to donate money to help resolve that pain point then please go to https://donate.pypi.org/.

Re: What the Heck Is Pyproject.toml?

#36

Post by symmitchry »

Waterluvian wrote: post #26

Is toml even supported in the standard library? This seems just bizarre. Python's really overtaking node for massive churn. I feel like an old guy when peers say, "don't use virtual environment use pipenv." then someone says no no don't even use that, use poetry. Etc. I think about 50% of why I enjoy, in my soul, about Rust when I'm hacking away and personal goofs is that the dependency and project management system…

Python also doesn't support yaml in the standard library, interestingly.

Re: What the Heck Is Pyproject.toml?

#37

Post by no_wizard »

brettcannon wrote: post #34
no_wizard wrote: post #30

Earlier quoted context omitted.

First of all, as an aside, thank you for all the work you do on Python! I actually got started pretty early on in my professional career with Python (3.5+ too, which I am thankful for in hindsight). I was using VS Code and the Python Extension before MS even acquired its developer (is Don still there? I hope he is. Such a genuinely down to earth guy). Without the work you guys do I don't know if I would have taken th…

> First of all, as an aside, thank you for all the work you do on Python! Welcome! > is Don still there? Yep, working on the data science features of the extension. > on Talk Python I remember specifically Michael and some others talking about the massive influx of money into Python and its ecosystem in the last year and half or so. As I said there have been donations for PyPI and pip just got its big donation. The C…

That makes sense, on its face, as to why they don't have a business arm, at least currently.

One thing I would suggest, on its face, is that any time someone has to reach you (as in, the foundation, not you specifically), that introduces friction, even the smallest amount of it, can leave lost donations on the table. If there was a simple page even, that said something like

Do you have a feature you need implemented in Python? Are there features you would like to fast track? For a simple donation, we can prioritize the work you need to grow your business

This with a nice little contact form, would eliminate that friction, rather than being in a situation where I might be reaching out to the wrong wing of the foundation, or not feeling comfortable sending a general inquiry etc.

you could also put a feature page that tracks how much money needed to implement x too, to solicit donations. I've seen this work in other communities. I think Django has gotten funding using this technique before.

Just a thought, I know you guys are doing everything you can and then some, and Python is unique in that its one of the few language communities of its size without any formal corporate benefactor.

Re: What the Heck Is Pyproject.toml?

#38

Post by btashton »

fermienrico wrote: post #4

Python desperately needs a better packaging solution than pip from Python Software Foundation. Stop all new features until this - the biggest pain point for Python across all expertise levels from newbies to experienced Python programmers - is officially solved. No pipenv, no poetry, no conda, etc from third party devs who sometimes get tired of the pressure [1]. It is immense. Combine all this virtualenv stuff with…

How is go the poster child for clean dependency management? Every release since they started doing modules the defaults change and break some workflow. Add on top of that it activity pushed vendoring forever.

I'm not super happy with package management under python but at least pip does not break every release.

Re: What the Heck Is Pyproject.toml?

#39

Post by fermienrico »

brettcannon wrote: post #33
fermienrico wrote: post #4

Python desperately needs a better packaging solution than pip from Python Software Foundation. Stop all new features until this - the biggest pain point for Python across all expertise levels from newbies to experienced Python programmers - is officially solved. No pipenv, no poetry, no conda, etc from third party devs who sometimes get tired of the pressure [1]. It is immense. Combine all this virtualenv stuff with…

Do note that Python the language is developed by a separate group as Python the packaging ecosystem. So even when Guido was a BDFL he completely stayed out of the packaging situation. And there is a packaging BDFL, but there's a severe lack of time and effort in the form of volunteers to tackle a lot of the projects, big or small. Plus rolling these things out takes years and it takes even longer to gain traction as…

Definitely, it is always good to keep in the back of our minds that FOSS is developed by people in their free time and on a volunteering basis. No one owes us anything. No one should feel entitled. I am sorry if my complains come out as such. Python runs the world and it is because of the contributions of many amazing people.

Re: What the Heck Is Pyproject.toml?

#40

Post by talentedcoin »

I don’t want to be too negative, and I realize this isn’t an option for everyone ... but seriously, just use conda if you can.

Judging from comments here and in the past, we all know packaging is one of Python’s Achilles’ heels. I spent a couple years bouncing around with pyenv/pipenv, then poetry, then back. I lovingly followed all the flamewars here and on github.

Ultimately, I realize now, I laboured under the conceit that it was desirable to have a “native” solution that did not depend on a suite like conda. I now think I was totally wrong.

I’m so much happier and more productive now that I’ve switched. For any data science use case conda is hands-down better, and for everything else it’s still more useable than any of the alternatives. I hate to say this but it’s true. The only thing I can think of that was harder to work with a conda setup was the keychain in macOS, and I’m not even sure now that that’s an issue anymore.

conda really does just work.

Post reply on HN