Live data from Hacker News

Python's New Package Landscape

andrewsforge.com

41–50 of 174 posts

Re: Python's New Package Landscape

#41
Interestingly this URL gets blocked by my work's security thing. Never saw that before.

edit: I requested an exemption but corp IT staff came back and said there's definitely been malware identified on that site. So... be careful with your clicks.

edit2: Well who knows where the malware alert is coming from, might be an ad or something.

Re: Python's New Package Landscape

#42

Published May 11th, 2018. But it's interesting it's popping up again. It's a good explanation of the landscape as of 2018, though Pipenv has since gone in a weird direction. There's a lot of recommendations for it, but I sometimes get the feeling people don't understand what they're recommending, such as replacing some things that work (setup.cfg) by things that don't do the same thing (Pipfile). Man, the Python pack…

[deleted]

Re: Python's New Package Landscape

#44
post #12

No mention of containers? I didn't write Python code since a while now, but it would have been nice to have a comparison with container technologies, which weren't available at the creation time of pypi and pip. Containers solve both the problems of the article : isolation and repeatability, for any language. Are virtual env tools still needed in the container era?

Containers would seem to solve the isolation part of the problem, but dependency management is not something containers can deal with effectively.

Big monolithic Python projects will face dependency issues for sure. However, softwares structured into simpler, smaller components, using the right languages for the right tasks, will probably have simpler dependencies for each modules.

That's what Go and tools like Bazel allows for : static builds, which forces to modularize the project into smaller independent components.

In case of static builds, the protocol between components is the C ABI, or an RPC protocol, but it could be a mesh of microservices too.

What is currently happening with the explosions of tools with Python is the result (take it with a grain of salt, only my opinion) of people only working with Python and not exploring enough outside of it

Re: Python's New Package Landscape

#45
post #37

The biggest problem I have with python packaging tools is how do I start using them. I'd rather not install all of them in my global site-packages. Do I need to create a virtualenv just to get a tool to manage my virtualenv's? I have seen poetry is working on their bootstrapping story. I could not get their current solution to work on Ubuntu. Maybe what they are developing towards will work. https://github.com/sdispa…

pipenv --three && pipenv shell && pipenv install and you're set

Re: Python's New Package Landscape

#46
post #29
post #16

Earlier quoted context omitted.

It is an overkill for pure-python packages or packages with simple C extensions. Conda was developed specifically to handle non-python dependencies, which would be difficult to build in setup.py. Also, a conda package is not a replacement for a distutils/setuptools package. When building a conda package, one still calls setup.py. So every python conda package has to be a distutils/setuptools package anyway.

Thanks for caveat. Nonetheless, anaconda makes my life so much easier when working with python libraries. If anybody got any other reasons to be careful of it, I'm interested!

If you need to work with cutting-edge python tools (e.g.from GitHub) it’s often easier to use virtualenv to control the versions you need installed.

Re: Python's New Package Landscape

#47

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

Can you elaborate on why using a requirements.txt file is "manual hell"?

I rely on it for pretty much everything and I didn't run into game breaking problems.

Re: Python's New Package Landscape

#48
post #6

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

Am I weird that I use anaconda instead of virtualenvs? I guess it’s overkill if you aren’t using the other conda features.

It's worth mentioning that the package manager component of anaconda is released as a separate (small) install: miniconda[1]. It includes only Python and the package manager, and not all of the 700+ packages installed as part of a full anaconda installation.

With its ability to install Python and non-python packages (including binaries), conda is my go-to for managing project environments and dependencies. Between the bioconda[2] and conda-forge[3] channels, it meets the needs of many on the computational side of the biological sciences. Being able to describe a full execution environment with a yaml file is huge win for replicable science.

1. https://conda.io/miniconda.html https://conda.io/docs/user-guide/install/index.html

2. http://bioconda.github.io/

3. https://conda-forge.org/

Re: Python's New Package Landscape

#49
post #6

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

Am I weird that I use anaconda instead of virtualenvs? I guess it’s overkill if you aren’t using the other conda features.

I tend to only use anaconda for "data science work" and not for my small side projects.

I "feel" like it is overkill to use anaconda for things unrelated to 'data science' and the likes, but I'm not sure why I feel that way.

It kind of makes sense to use for other projects as well since you don't need to import all the things conda offers.

Re: Python's New Package Landscape

#50

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

> Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies)

I've never seen anything like that on a number of fairly large apps – a minute or two, at most. Are some of those dependencies extremely large or self-hosted somewhere other than PyPI?

Post reply on HN