Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python's New Package Landscape

andrewsforge.com

Post reply on HN
Print view View original

51–60 of 174 posts

Re: Python's New Package Landscape

#51

Post by gtycomb »

blattimwind wrote: post #38
epage wrote: post #37

The biggest problem I have with python packaging tools is how do I start using them. I'd rather not install all of them in my global site-packages. Do I need to create a virtualenv just to get a tool to manage my virtualenv's? I have seen poetry is working on their bootstrapping story. I could not get their current solution to work on Ubuntu. Maybe what they are developing towards will work. https://github.com/sdispa…

pip install --user

I stumbled on this (--user flag) only the other day and it simplified things immensely. Too much information, various ways of doing things explained in bits and pieces over the decades makes it look confusing.

Often there is a very simple way, even in Python packaging and deploying. In my situation the easiest way began along these lines --

1. Install python3 for the local user from the source distribution (make sure you have compilers etc that the configure check lists out)

2. After compiling the sources and finishing with 'make install', make Python available in your local search path

3. And use pip with this magical --user flag as needed. No virtual env, conda, etc etc.

4. Leave HOMEPATH etc alone as this conflicts with the setup of the admin's system wide installs (when you su)

Things can go smoothly with pip alone.

Re: Python's New Package Landscape

#52

Post by dagw »

flatfilefan wrote: post #46
131012 wrote: post #29

Earlier quoted context omitted.

Thanks for caveat. Nonetheless, anaconda makes my life so much easier when working with python libraries. If anybody got any other reasons to be careful of it, I'm interested!

If you need to work with cutting-edge python tools (e.g.from GitHub) it’s often easier to use virtualenv to control the versions you need installed.

conda environments support pip and arbitrary pip commands. So if you use pip to for example install a specific version of a library directly from github that information will be stored in your conda environment and be reproduced every time you recreate your environments.

Re: Python's New Package Landscape

#53

Post by azag0 »

Insanity wrote: post #49
prepend wrote: post #6

Earlier quoted context omitted.

Am I weird that I use anaconda instead of virtualenvs? I guess it’s overkill if you aren’t using the other conda features.

I tend to only use anaconda for "data science work" and not for my small side projects. I "feel" like it is overkill to use anaconda for things unrelated to 'data science' and the likes, but I'm not sure why I feel that way. It kind of makes sense to use for other projects as well since you don't need to import all the things conda offers.

For one, as a package developer, publishing a source distribution of a package on PyPI is almost trivial. Publishing on Anaconda Cloud requires you to build the binary packages on all the OS's that you want to support (and for all Python versions you want to support) which most people delegate to some CI. So there is a whole new level of complexity involved.

Re: Python's New Package Landscape

#54

Post by acdha »

scrollaway wrote: post #10

Published May 11th, 2018. But it's interesting it's popping up again. It's a good explanation of the landscape as of 2018, though Pipenv has since gone in a weird direction. There's a lot of recommendations for it, but I sometimes get the feeling people don't understand what they're recommending, such as replacing some things that work (setup.cfg) by things that don't do the same thing (Pipfile). Man, the Python pack…

> - "Deploying a Project", local dev or production (pipenv. Well, if it's configured with a pipfile, otherwise virtualenv, and who knows where your dependencies are, maybe requirements.txt. Pipenv does create a virtualenv anyway, so you can use that. Anyway you should be in docker, probably. Make sure you have pip installed systemwide. Yes I know it comes with python, but some distributions remove it from Python. Stop asking why, it's simple. What do you mean this uses Python 3.6 but there's only Python 3.5 available on Debian? Wait, no, don't install pyenv, that's not a good idea! COME BACK!)

This makes the situation sound a lot more complex than it actually is by conflating separate layers: the system distribution issue is exactly the same for both Python and JS (if Debian ships an old v8 you either need to install a new one, perhaps using Docker to make that easy and isolated). Similarly, the question of whether you install the app using pip or pipenv is a different layer from whether you're using Docker or not, just as Docker is unrelated to the question of whether you use npm or yarn.

For a new project in 2018, you can simply say “Use pipenv. Deploy in Docker, using pipenv.” and it works as well as the JS world. People sometimes choose to make their projects too complicated or to manage things at the wrong level but that's a social problem which is hard to solve with tooling.

Re: Python's New Package Landscape

#55

Post by sambe »

Whenever talk in Python-world goes towards packaging, I feel like I have been transported to Javascript-world: it's never clear to me what concrete problems are being solved by the new tools/libraries.

This article seems well-written and well-intentioned. Despite reading it, I don't know why I would not have loose dependencies in setup.py and concrete, pinned dependencies in requirements.txt. It's never felt hard to manage or to sync up - the hard part is wading through all the different tools and recommendations.

Re: Python's New Package Landscape

#58

Post by dagw »

acdha wrote: post #54
scrollaway wrote: post #10

Published May 11th, 2018. But it's interesting it's popping up again. It's a good explanation of the landscape as of 2018, though Pipenv has since gone in a weird direction. There's a lot of recommendations for it, but I sometimes get the feeling people don't understand what they're recommending, such as replacing some things that work (setup.cfg) by things that don't do the same thing (Pipfile). Man, the Python pack…

> - "Deploying a Project", local dev or production (pipenv. Well, if it's configured with a pipfile, otherwise virtualenv, and who knows where your dependencies are, maybe requirements.txt. Pipenv does create a virtualenv anyway, so you can use that. Anyway you should be in docker, probably. Make sure you have pip installed systemwide. Yes I know it comes with python, but some distributions remove it from Python. Sto…

Use pipenv. Deploy in Docker, using pipenv

As a developer I definitely see the advantage of this approach and realize that it trivially solves some very hard problems (and causes a whole bunch of different problems if you're running Windows...). As an end user I'm not super thrilled about the prospect each tool I use coming in its own docker container and needing to spin up 20 different containers each time I want to do anything.

Re: Python's New Package Landscape

#59

Post by erichurkman »

acdha wrote: post #54
scrollaway wrote: post #10

Published May 11th, 2018. But it's interesting it's popping up again. It's a good explanation of the landscape as of 2018, though Pipenv has since gone in a weird direction. There's a lot of recommendations for it, but I sometimes get the feeling people don't understand what they're recommending, such as replacing some things that work (setup.cfg) by things that don't do the same thing (Pipfile). Man, the Python pack…

> - "Deploying a Project", local dev or production (pipenv. Well, if it's configured with a pipfile, otherwise virtualenv, and who knows where your dependencies are, maybe requirements.txt. Pipenv does create a virtualenv anyway, so you can use that. Anyway you should be in docker, probably. Make sure you have pip installed systemwide. Yes I know it comes with python, but some distributions remove it from Python. Sto…

One difference: large swaths of Python developers grew up using the system-provided version of Python.

Most Node developers grew up with

    curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash
or

    brew install node
or using one of the dozen other ways to install node. Distinct versions and per-project packages were the norm from day one. That was not true with Python.

Re: Python's New Package Landscape

#60

Post by patagonia »

Imagine I’m just getting started with Python, and I see this article. I think to myself, “Awesome, a primer!”

Then I start reading (these comments)... mayyybe I should try Julia... or anything else, at least while I’m still getting started.

Post reply on HN