Live data from Hacker News

Thoughts on the Python packaging ecosystem

pradyunsg.me

111–120 of 121 posts

Re: Thoughts on the Python packaging ecosystem

#111

Earlier quoted context omitted.

I’d prefer a crazy aunt to a creepy uncle… but maybe let’s leave family out of it altogether.

Parent is referring to a long-running (but somewhat obscure at this point unfortunately) joke https://peps.python.org/pep-0401/

I know. I was trying to say "fuck the patriarchy" with a bit more color.

Re: Thoughts on the Python packaging ecosystem

#112
post #54

Earlier quoted context omitted.

You can do this now by creating a Python virtual environment[0]. Then you can package your project with a requirements file and some instructions on its use. I use Python VENV often, and it works really well. [0] https://realpython.com/python-virtual-environments-a-primer/

Apparently you missed the point. local needs to be the default. Not some magic incantation In node cd projectFoo # do stuff, node uses packages local to projectFoo cd ../projectBar # do stuff, node uses packages local to projectBar There is no "activation", the default is it just works. Installing packages local to the project is also the default.

This is the important difference. Scripting languages should default to examining the current directory and then its parent directory etc. to find the resources they need. Python doesn't have this default and probably can't change at this point.

Re: Thoughts on the Python packaging ecosystem

#113
post #5

Packaging and Nvidia are pretty much the main reasons python has degraded in status for the past 7 years. It started with the added and unnecessary confusion from conda (and silly things like pythonxy), but has really escalated through extra stupidities like poetry. Much of the features that may be enticing for these extra packages should have pushed the developers to improve the standard tools in python (pip). The o…

Poetry certainly made our life much easier and was a productivity improvement. It the the tool that brought us back from Conda because it was able to solve environments than nothing in Pip land was able to do.

The problem is that poetry/conda/etc chose to make their own thing to begin with, when they should've just improved the standard language tool with the features they wanted. That's one key difference between shit languages (such as R) and languages that have good tooling (e.g. Rust) - the community works on the core together more rather than each person doing their own thing and confusing everyone in the community by not having a standard way to do anything. Every package management developer for python should have just added features to the standard package manager instead of doing their own thing.

Re: Thoughts on the Python packaging ecosystem

#114

Earlier quoted context omitted.

These days I don’t think I’m being too brash in saying that anyone dealing with Python 2 to Python 3 transition stuff is so deep in the realm of enterprise abandonware that it’s not worth paying it much attention. Not necessarily including you here! But as someone that went through the transition, I’m certainly not going to deny how much of a shit show it was. And I get a feeling that the two situations were/are caus…

It is, sadly, not always Enterprise abandonware. The industry I am in is extremely conservative about its upgrades and changes. Between that, our custom software interfacing with the 800 pound gorilla for our market, and a few other things, we're behind. Now, this software depends on a very specific version of ArcGIS. Desktop, not Pro. Not just version 10. Not just version 10.2. But version 10.2.1. If you go look at…

I guessed you worked in GIS before you mentioned ArcGIS. Our old software was tied with QGIS and I worked hard to de-couple. Tying a Python env to these whales is insane.

I think a Cython extension or a second process that uses ArcGIS, with a simple interface between them, might be a better choice for the future? Never needed that though.

Re: Thoughts on the Python packaging ecosystem

#115

Earlier quoted context omitted.

Python's demise would be it's undeserving fate to be the world's best "glue" language. The language needs to stand on it's own to be competitive in the long run. Pure Python modules are much easier to install and keep updated.

It will be a VERY deserving fate, since the community is always busy fighting amongst themselves and the maintainers are conservative to the point of being unreasonable. Maybe such a community deserves obscurity. Python has real problems that need addressing, and that was true 10 years ago as well, packaging included. Pretending this isn't the case makes more serious techies just laugh at Python, and it's fully deser…

Also a good language for non-professional programmers. When people ask me which language they should learn as a first language, I recommend Python. Not because it's a great language; but because it has a massive ecosystem that allows beginners to build fun things with minimal effort.

The greatest strength of Python, I think, is that it allows cross-disciplinary collaboration, because it's easy for non-programmers to learn.

Which makes the lack of a functional easy-to-use package manager all the more frustrating.

Re: Thoughts on the Python packaging ecosystem

#116

Earlier quoted context omitted.

It will be a VERY deserving fate, since the community is always busy fighting amongst themselves and the maintainers are conservative to the point of being unreasonable. Maybe such a community deserves obscurity. Python has real problems that need addressing, and that was true 10 years ago as well, packaging included. Pretending this isn't the case makes more serious techies just laugh at Python, and it's fully deser…

Also a good language for non-professional programmers. When people ask me which language they should learn as a first language, I recommend Python. Not because it's a great language; but because it has a massive ecosystem that allows beginners to build fun things with minimal effort. The greatest strength of Python, I think, is that it allows cross-disciplinary collaboration, because it's easy for non-programmers to…

You're not wrong but it's a double-edged sword: people's first language leaves a deep impression and people often can't un-write (from their brains) the wrong programming practices they learned from Python.

I've seen people openly admit they'd learn their second language much faster if they didn't try to constantly compare it with Python.

As usual, hindsight is 20/20, and nobody is telling you these things beforehand. And those who are lucky enough to have wisdom shared with them are usually quick to dismiss it and not listen to it.

Re: Thoughts on the Python packaging ecosystem

#117
post #74
post #72

Earlier quoted context omitted.

Pip alone is not enough. Python makes backwards incompatible changes to the language all the time, even in dot releases. This means that you can't just install everything into some shared directories since some code would need version X of Python and some would need version Y. So you need virtualenv or something like that. But virtualenv isn't enough either because most actually useful Python code calls into C librar…

I don't understand your paragraph about C libraries. With wheels (which pip can install) there is no problem packing, and linking to, compiled C libraries.

It's often not feasible for legal or regulatory reasons to install bundled versions of C libraries. Like most enterprise software wants to use the system version of openssl because otherwise the vendor has to scramble to update for each openssl security issue.

Even when it is possible to install bundled libraries, someone has to do the work to set up Python packages that bundle the native libraries. This work is done for some of the most important pacakges like TensorFlow, but probably not for some more esoteric library you want to use. I also suspect that people seriously using TensorFlow probably want careful control over which version they are using, rather than letting a random Python package manage that.

Basically, interacting with native libraries is a hard problem to fully solve for any language. A lot of other languages like Java have struggled with this as well. But at least in Java you seldom use native dependencies. In Python you use them all the time (arguably, serving as glue code for crusty old FORTRAN and C libraries is where Python shines.)

Python's stubborn determination not to standardize on anything for packaging (we have easy_install, conda, pip, pipenv, poetry, and who knows how many others) as well as its insistence on breaking its own compatibility certainly don't help either. And Python is more hostile than most languages to integrating with the system package manager (RPM, deb, etc.) because of how it sprays its files around the filesystem.

Re: Thoughts on the Python packaging ecosystem

#118

Earlier quoted context omitted.

Also a good language for non-professional programmers. When people ask me which language they should learn as a first language, I recommend Python. Not because it's a great language; but because it has a massive ecosystem that allows beginners to build fun things with minimal effort. The greatest strength of Python, I think, is that it allows cross-disciplinary collaboration, because it's easy for non-programmers to…

You're not wrong but it's a double-edged sword: people's first language leaves a deep impression and people often can't un-write (from their brains) the wrong programming practices they learned from Python. I've seen people openly admit they'd learn their second language much faster if they didn't try to constantly compare it with Python. As usual, hindsight is 20/20, and nobody is telling you these things beforehand…

But isn’t this true no matter the first- and second-language combo? I started out writing C and this definitely influenced how I learned and wrote python code. My colleagues that started out with ruby, or java, also program in distinct ways that show their “accent”, so to say.

Re: Thoughts on the Python packaging ecosystem

#119

Earlier quoted context omitted.

Also a good language for non-professional programmers. When people ask me which language they should learn as a first language, I recommend Python. Not because it's a great language; but because it has a massive ecosystem that allows beginners to build fun things with minimal effort. The greatest strength of Python, I think, is that it allows cross-disciplinary collaboration, because it's easy for non-programmers to…

You're not wrong but it's a double-edged sword: people's first language leaves a deep impression and people often can't un-write (from their brains) the wrong programming practices they learned from Python. I've seen people openly admit they'd learn their second language much faster if they didn't try to constantly compare it with Python. As usual, hindsight is 20/20, and nobody is telling you these things beforehand…

[deleted]

Re: Thoughts on the Python packaging ecosystem

#120

Earlier quoted context omitted.

You're not wrong but it's a double-edged sword: people's first language leaves a deep impression and people often can't un-write (from their brains) the wrong programming practices they learned from Python. I've seen people openly admit they'd learn their second language much faster if they didn't try to constantly compare it with Python. As usual, hindsight is 20/20, and nobody is telling you these things beforehand…

But isn’t this true no matter the first- and second-language combo? I started out writing C and this definitely influenced how I learned and wrote python code. My colleagues that started out with ruby, or java, also program in distinct ways that show their “accent”, so to say.

You are saying "this is not true" while agreeing with me -- confusing. :)
Post reply on HN