Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python's New Package Landscape

andrewsforge.com

Post reply on HN
Print view View original

131–140 of 174 posts

Re: Python's New Package Landscape

#131

Post by jonafato »

zmmmmm wrote: post #124
patagonia wrote: post #60

Imagine I’m just getting started with Python, and I see this article. I think to myself, “Awesome, a primer!” Then I start reading (these comments)... mayyybe I should try Julia... or anything else, at least while I’m still getting started.

Wait until someone from the python community recites to you "there should be one and only one good way to do something" ....!

I think plenty of people in the Python community will earnestly say that while acknowledging that there isn't universal agreement on what that one good way is. It's an ideal to strive for, not a statement of fact.

Re: Python's New Package Landscape

#132

Post by jabwork »

philosopherlawr wrote: post #112
andybak wrote: post #43

In case this scares any new users, I've used nothing more than pip and virtualenv for several years with no issues of note.

There are lots of errors when it comes to reproducing the build on other machines. Pip install -r requirements.txt does not guarantee that you will install the same version of packages on a new machine, and in fact, you will typically not.

I've never had problems with this when requirements.txt contains package versions

Have you, or are you not using explicit versions supplied by eg pip freeze?

Re: Python's New Package Landscape

#133

Post by xapata »

aprdm wrote: post #93
antpls wrote: post #44

Earlier quoted context omitted.

Big monolithic Python projects will face dependency issues for sure. However, softwares structured into simpler, smaller components, using the right languages for the right tasks, will probably have simpler dependencies for each modules. That's what Go and tools like Bazel allows for : static builds, which forces to modularize the project into smaller independent components. In case of static builds, the protocol bet…

Can you give practical examples of what you are mentioning? Like, how to achieve ""static builds"" with python? Got me interested!

Dropbox does this for their Windows client, I think.

Re: Python's New Package Landscape

#134

Post by [deleted] »

di wrote: post #120
1wd wrote: post #86

Earlier quoted context omitted.

Why not bundle twine like pip? In fact, why not merge the twine functionality into pip?

> Why not bundle twine like pip? The `pip` package is not actually bundled with your Python distribution, instead the standard library has `ensurepip` which provides a means of bootstrapping a `pip` installation without `pip` itself. See [0]. > In fact, why not merge the twine functionality into pip? This has been considered and still might happen, see [1], specifically the comment at [2]. [0] https://docs.python.org…

[deleted]

Re: Python's New Package Landscape

#135

Post by 1wd »

di wrote: post #120
1wd wrote: post #86

Earlier quoted context omitted.

Why not bundle twine like pip? In fact, why not merge the twine functionality into pip?

> Why not bundle twine like pip? The `pip` package is not actually bundled with your Python distribution, instead the standard library has `ensurepip` which provides a means of bootstrapping a `pip` installation without `pip` itself. See [0]. > In fact, why not merge the twine functionality into pip? This has been considered and still might happen, see [1], specifically the comment at [2]. [0] https://docs.python.org…

> The `pip` package is not actually bundled with your Python distribution

It is bundled, as mentioned in the link [0] you posted: "pip is an independent project with its own release cycle, and the latest available stable version is bundled with maintenance and feature releases of the CPython reference interpreter."

> the standard library has `ensurepip`

Ensurepip is for Python distributions, which are supposed to do use it automatically to provide the bundled pip. See [3]: "Ensurepip is the mechanism that Python uses to bundle pip with Python." Basically it's the installer of the bundled pip. At least that's how I understand it.

> This has been considered and still might happen, see [1]

Note that while the users there all basically say the same thing (twine should be merged into pip as "pip publish") the (two out of three) PyPA devs say it "would be a major mistake" and they are "against adding pip publish". (Before starting offtopic rants against poetry...) I somehow doubt this will improve soon.

[3] https://mail.python.org/mm3/archives/list/distutils-sig@pyth...

Re: Python's New Package Landscape

#137

Post by Drdrdrq »

guitarbill wrote: post #121
kankroc wrote: post #47

Earlier quoted context omitted.

Can you elaborate on why using a requirements.txt file is "manual hell"? I rely on it for pretty much everything and I didn't run into game breaking problems.

because if you pin stuff in requirements.txt, they either never get updated, or you have to go through, check which ones have updated, and manually edit the requirements.txt. the combination of Pipfile and Pipfile.lock were designed to solve this in a much better way (briefly: understanding standard deps vs development deps, and using the Pipfile.lock file for exact pinning/deployment pinning, vs general compatibilit…

That is not my experience. Until recently, I used to pin versions in requirements.txt, then from time to time I removed the pinned versions, reinstalled everything, tested and added new versions to requirements.txt. Most of the work was testing for incompatibilities, but no package manager will help you there.

Recently I switched to pipenv because zappa insists on having virtualenv (as app dev I never had any need for it - but it seems my case is an exception, as I almost never work on multiple apps in parallel). Pipenv does make version management a bit easier, but it wasn't difficult (for me) to begin with.

From talking with other developers I know my view is somewhat unorthodox, but I haven't encountered the problems they describe, or the pain hasn't been that big for me to embrace all the issues that come with virtualenvs.

Re: Python's New Package Landscape

#138

Post by Drdrdrq »

Bogdanp wrote: post #19
scrollaway wrote: post #15

Earlier quoted context omitted.

> Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies) Do you have scipy/numpy/keras or cython somewhere in the deps? pipenv lock is slow, but not 20-30 mins slow unless there's a very very large download and/or a long compilation somewhere in there.

It takes about 2 minutes (feels like 5!) on my 2016 MBP to install 102 dependencies. Doing that in Docker takes about 1.5x the time. I haven't seen it take 20-30 minutes, but 2-3 minutes is still obscenely slow in my view.

There is no reason for Docker to be slower, it must be some kind of configuration issue. Containers are basically just processes and there is virtually no difference in execution times.

Re: Python's New Package Landscape

#139

Post by qwerty456127 »

Sadly there are things you can't always install reliably from the Python repository. E.g. you may have to install things like scipy and keras from the OS or a 3-rd party (like brew or conda) repository as pip install would fail in the build process.

Re: Python's New Package Landscape

#140

Post by qwerty456127 »

kankroc wrote: post #47
whitehouse3 wrote: post #3

While pipenv has garnered a lot of attention and praise for ease of use, it falls over whenever I integrate it with any serious work. Pipenv lock can take 20-30 minutes on a small flask app (~18 dependencies). And it often mixes up virtualenvs, enabling the wrong one with seemingly no remedy. I see the problems on Windows, MacOS and Ubuntu. 2018 is not the year of pipenv, for me. I'm sticking with regular virtualenvs…

Can you elaborate on why using a requirements.txt file is "manual hell"? I rely on it for pretty much everything and I didn't run into game breaking problems.

The only problem I know with requirements.txt is that many people would require particular versions there while later versions work perfectly fine. Every time I clone someone's Python project to work with I have to manually replace all the =s with >=s to avoid downloading obsolete versions of the dependencies and have never encountered a problem.

Anyway, for me the most annoying thing about the Python projects architecture (and about the whole Python perhaps) is that you can't split a module into multiple files so you have to either import everything manually all over the project trying to avoid circular imports or just put everything in a single huge source file - I usually choose the latter and I hate it. The way namespaces and scopes work in C# feels just so much better.

Post reply on HN