Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python's New Package Landscape

andrewsforge.com

Post reply on HN
Print view View original

91–100 of 174 posts

Re: Python's New Package Landscape

#91

Post by aprdm »

patagonia wrote: post #76
zitterbewegung wrote: post #67

Earlier quoted context omitted.

This isn’t a primer but more like a survey of packaging options. Using pip and virtualenv is usually fine or using pipenv .

“Usually” is great until it’s not. I don’t want to get invested in a languages only to discover issues down the line. So, if something as basic as packaging is potentially problematic and there are other options available, I might look elsewhere before rolling the dice that this problem is not too problematic. Perhaps “Primer” was the wrong word, but I believe the sentiment is valid. Reading the comments, there is si…

pip and virtualenv has been solving the problem and is the standard for ever.

other people tried different approaches, like, pipenv and is just that, a separate project trying to solve the same problem.

I don't know why / who said that pipenv is the official recommended way, if it is it should not be and I hope it is not.

Re: Python's New Package Landscape

#92

Post by y4mi »

another-cuppa wrote: post #69
whitehouse3 wrote: post #3

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

Pipenv is hopelessly slow. It's a shame. Remember when git first came out and it changed the way we worked because it was so quick to commit now? (I fully expect that most git users here don't remember that, actually). There is no going back. I will not use slow tools. My tools need to be at the very least as fast as me.

my current project is at 16 dependencies atm and ... its really not as bad as you make it sound.

    pipenv lock  5.65s user 0.29s system 77% cpu 7.639 total
i think 7.6 seconds is fine for an operation that you'd rarely do

it would probably take ages at work though. just opening a WSL terminal takes several seconds there, which is predictably instantaneous (<100ms) on fedora linux at home

Re: Python's New Package Landscape

#93

Post by aprdm »

antpls wrote: post #44
bovermyer wrote: post #33

Earlier quoted context omitted.

Containers would seem to solve the isolation part of the problem, but dependency management is not something containers can deal with effectively.

Big monolithic Python projects will face dependency issues for sure. However, softwares structured into simpler, smaller components, using the right languages for the right tasks, will probably have simpler dependencies for each modules. That's what Go and tools like Bazel allows for : static builds, which forces to modularize the project into smaller independent components. In case of static builds, the protocol bet…

Can you give practical examples of what you are mentioning? Like, how to achieve ""static builds"" with python? Got me interested!

Re: Python's New Package Landscape

#94

Post by cik »

We just went through this cycle - ultimately we build packages (debs) and dockers, for deployment within VMs. Our build process - depending on the component pushes the deb to repos, or uses the deb in the docker.

After trying to replace pip with Pipenv, we had to stop. The dependency resolution time for 20 declared dependencies (that in turn pull down > 100 components) takes well over 5 minutes. With poetry - it takes less than 33 seconds on a clean system. The times are consistent for both Ubuntu 16.04 and Mac OS X.

Our only goal is to get to the point we're now in - tracking dependencies, and separate dev requirements (like ipython and pdbpp) from our other requirements. Poetry made it fast, simple, and made me an addict.

Over two days, I moved our entire codebase and every single (active) personal project I had to poetry. I don't regret it :)

Re: Python's New Package Landscape

#95

Post by xapata »

patagonia wrote: post #60

Imagine I’m just getting started with Python, and I see this article. I think to myself, “Awesome, a primer!” Then I start reading (these comments)... mayyybe I should try Julia... or anything else, at least while I’m still getting started.

https://www.anaconda.com/download/

Re: Python's New Package Landscape

#96

Post by xapata »

No mention of Anaconda?! How strange. I recommend using `conda` instead of virtualenv, and instead of pip where possible.

A Python project does not only depend on Python modules, but non-Python modules as well. Beyond Python, conda helps manage your other dependencies, like your database. I use Miniconda instead of Anaconda, to avoid the initial mega-download.

Re: Python's New Package Landscape

#97

Post by wiremine »

iainmerrick wrote: post #79
wiremine wrote: post #71

Earlier quoted context omitted.

> Pipenv is hopelessly slow. Interesting, this has never been a problem for me. I've built some large tools and while it isn't fast, it's always completed in a few minutes.

A few minutes??! That sounds very slow.

To be clear: with few deps it's very fast for me, it's just lager projects with LOTS of non-trivial deps where it can slow up.

Re: Python's New Package Landscape

#98

Post by wiremine »

another-cuppa wrote: post #72
wiremine wrote: post #71

Earlier quoted context omitted.

> Pipenv is hopelessly slow. Interesting, this has never been a problem for me. I've built some large tools and while it isn't fast, it's always completed in a few minutes.

What OS?

Mid-2015 MacBook Pro running the newest OS

Re: Python's New Package Landscape

#99

Post by xapata »

azag0 wrote: post #53
Insanity wrote: post #49

Earlier quoted context omitted.

I tend to only use anaconda for "data science work" and not for my small side projects. I "feel" like it is overkill to use anaconda for things unrelated to 'data science' and the likes, but I'm not sure why I feel that way. It kind of makes sense to use for other projects as well since you don't need to import all the things conda offers.

For one, as a package developer, publishing a source distribution of a package on PyPI is almost trivial. Publishing on Anaconda Cloud requires you to build the binary packages on all the OS's that you want to support (and for all Python versions you want to support) which most people delegate to some CI. So there is a whole new level of complexity involved.

You can use pip-install from within a conda environment. Conda isolates better than virtualenv does, IMHO.

Re: Python's New Package Landscape

#100

Post by SJetKaran »

y4mi wrote: post #92
another-cuppa wrote: post #69

Earlier quoted context omitted.

Pipenv is hopelessly slow. It's a shame. Remember when git first came out and it changed the way we worked because it was so quick to commit now? (I fully expect that most git users here don't remember that, actually). There is no going back. I will not use slow tools. My tools need to be at the very least as fast as me.

my current project is at 16 dependencies atm and ... its really not as bad as you make it sound. pipenv lock 5.65s user 0.29s system 77% cpu 7.639 total i think 7.6 seconds is fine for an operation that you'd rarely do it would probably take ages at work though. just opening a WSL terminal takes several seconds there, which is predictably instantaneous (<100ms) on fedora linux at home

SSD vs HDD may be?
Post reply on HN