Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python's New Package Landscape

andrewsforge.com

Post reply on HN
Print view View original

61–70 of 174 posts

Re: Python's New Package Landscape

#61

Post by toyg »

dlitvakb wrote: post #4

I have been using purely setuptools for all of our open source Python libraries at Contentful, but have found that lately I've been getting deprecation warnings from PyPI not to use `setup.py upload` anymore. What should the alternative be now? Edit: I'm reading about twine right now, but I cannot begin to comprehend why it's not bundled directly if this is what they are intending for us to use to upload packages.

Anything PyPI-related has recently gone into the (terrible) habit of recommending very recent (and often half-baked) tools that live entirely outside of stdlib. It seems pretty silly to me, considering Python core developers made significant efforts to bundle and support pip and virtualenv (venv) in the stdlib precisely to avoid having a lot of de-facto essential libraries outside the core distribution.

If the problem is that stdlib cannot move as fast as PyPI-related development requires, maybe that should be fixed, rather than trying to bypass all quality checks and then relying on obscure shared knowledge to navigate the ecosystem. Maybe there should be a system where specific network-sensitive stdlib modules could be updated faster than the rest.

Re: Python's New Package Landscape

#62

Post by vxNsr »

patagonia wrote: post #60

Imagine I’m just getting started with Python, and I see this article. I think to myself, “Awesome, a primer!” Then I start reading (these comments)... mayyybe I should try Julia... or anything else, at least while I’m still getting started.

Kinda exactly what went through my mind, I've dabbled in python, but only to use other ppl's projects, which sometimes required setting up pip, etc. So I was looking forward to reading about how it'd been simplified :(

Re: Python's New Package Landscape

#63

Post by maw »

scrollaway wrote: post #10

Published May 11th, 2018. But it's interesting it's popping up again. It's a good explanation of the landscape as of 2018, though Pipenv has since gone in a weird direction. There's a lot of recommendations for it, but I sometimes get the feeling people don't understand what they're recommending, such as replacing some things that work (setup.cfg) by things that don't do the same thing (Pipfile). Man, the Python pack…

> The JS ecosystem manages to have two tools, both of which can do all of this. I don't know how we keep messing up when we have good prior work to look at.

Agreed. You know you really blew it when even PHP does it better, and composer is unquestionably better than anything we've got in Python.

Re: Python's New Package Landscape

#64

Post by Keats »

mlthoughts2018 wrote: post #27
Keats wrote: post #18

Earlier quoted context omitted.

I tried a few of the package management in Python recently ( https://www.vincentprouillet.com/blog/overview-package-manag... ) and had the same conclusion with Pipenv. It is way too slow and frankly the UX is not that great either.

I’m disappointed your post did not cover using conda. As the pipenv drama has rolled on, I’ve moved from viewing conda merely as the best user experience in Python environment & package management to instead viewing it as the only serious option for professional scientific computuing work (and quite possibly any professional Python work at all).

I should probably write another post once I've tried conda a bit more. I've used it very recently for some numpy/pytorch environment and it was quite nice.

Re: Python's New Package Landscape

#65

Post by sirn »

rhizome31 wrote: post #17
prepend wrote: post #6

Earlier quoted context omitted.

Am I weird that I use anaconda instead of virtualenvs? I guess it’s overkill if you aren’t using the other conda features.

It seems that Anaconda is under-appreciated outside of pydata circles. Before using it I had no idea that it could manage virtual environments, dependencies and different versions of python. The fact that it's not a community-driven project might be one of the reasons. Meanwhile, in a galaxy far away, people are also using buildout.

I have been revisiting buildout recently and I wish there's something that merge ease of use of pipenv with buildout concept. Perhaps something similar to Nix throw in the mix, but more specific to a Python project. I heard that I can do this with Conda, but I never tried.

Being able to define and install external dependencies (e.g. ImageMagick, libsodium, etc.) from a configuration file local to a project is something I missed the most, especially when I'm working on several projects at once.

Re: Python's New Package Landscape

#66

Post by mfe5003 »

Insanity wrote: post #49
prepend wrote: post #6

Earlier quoted context omitted.

Am I weird that I use anaconda instead of virtualenvs? I guess it’s overkill if you aren’t using the other conda features.

I tend to only use anaconda for "data science work" and not for my small side projects. I "feel" like it is overkill to use anaconda for things unrelated to 'data science' and the likes, but I'm not sure why I feel that way. It kind of makes sense to use for other projects as well since you don't need to import all the things conda offers.

When I have used it, and I have to for a certain project, it is incredibly slow to resolve depenendcies. Enough so that I go for a walk or do something else for 15 minutes while it thinks about whatever it's doing.

Re: Python's New Package Landscape

#67

Post by zitterbewegung »

patagonia wrote: post #60

Imagine I’m just getting started with Python, and I see this article. I think to myself, “Awesome, a primer!” Then I start reading (these comments)... mayyybe I should try Julia... or anything else, at least while I’m still getting started.

This isn’t a primer but more like a survey of packaging options.

Using pip and virtualenv is usually fine or using pipenv .

Re: Python's New Package Landscape

#68

Post by zitterbewegung »

whitehouse3 wrote: post #3

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

I thought I was the only one to not get pipenv to work. I tried it out after it was released and it was buggy. Every time I used it afterwards I would get a weird edge case that would make me go back to pip and virtualenv.

Re: Python's New Package Landscape

#69

Post by another-cuppa »

whitehouse3 wrote: post #3

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

Pipenv is hopelessly slow. It's a shame. Remember when git first came out and it changed the way we worked because it was so quick to commit now? (I fully expect that most git users here don't remember that, actually). There is no going back. I will not use slow tools. My tools need to be at the very least as fast as me.

Re: Python's New Package Landscape

#70

Post by di »

dlitvakb wrote: post #4

I have been using purely setuptools for all of our open source Python libraries at Contentful, but have found that lately I've been getting deprecation warnings from PyPI not to use `setup.py upload` anymore. What should the alternative be now? Edit: I'm reading about twine right now, but I cannot begin to comprehend why it's not bundled directly if this is what they are intending for us to use to upload packages.

Hello, I'm the person who deprecated `setup.py upload`. The warnings should be telling you that `twine` is the preferred tool for uploading.

The reason for this is that right now, that command comes from `distutils`, which is part of the standard library. There is a huge disadvantage to bundling this functionality with your Python distribution, namely that it can only get upgraded when you upgrade your Python distribution. A lot of folks are still running versions of Python from several years ago, which is fine, but it means that they are missing out on anything new that's been added in the meantime.

For example, earlier this year, we released a new package metadata version which allows people to specify their package descriptions with Markdown. This required a new metadata field, which old versions of `distutils` know nothing about.

Upgrading `distutils` to support it would require that these changes go though the long process of making it into a Python release, and even then they would only be available to folks using the latest release.

Moving this functionality from `distutils` to a tool like `twine` means that new features can be made available nearly immediately (just have to make a release to PyPI) and that they're available to users on any Python distribution (just have to upgrade from PyPI).

The `distutils` standard library module comes from a time when we didn't have PyPI and thus, didn't have a better way to distribute this code to users. We have PyPI now though, so bundling `distutils` with Python is becoming less and less useful.

Post reply on HN