How to make a Python package in 2021 • Hacker News Board

Re: How to make a Python package in 2021

#131

Post by CraigJPerry » Thu, Apr 08, 2021, 9:07 AM UTC

TylerE wrote: post #34

staticassertion wrote: post #23

Earlier quoted context omitted.

TBH as someone trying to use Python professionally it is extremely frustrating that basic things with regards to package management are something you have to iterate towards, as opposed to just being obvious and default.

One thing that has become clear to me, from playing around a bit with go, rust, and nim, is that it is astonishingly better when the language has exactly one set of tools, that everyone uses. Even if that set of tools is kinda crappy (Glances at go), it's just so nice to not have all the bikeshedding and just get on with it.

I’m not familiar enough with Go but at first glance the go mod stuff seemed pretty decent.

E.g. more flexible than Cargo in that you could have a large codebase with two different versions of a dependency isolated to their compile units allowing you to gradually adopt a breaking change dependency in a large code base. I was kinda bowled over with that feature (the isolation part is key).

For python I’m finding poetry much more ergonomic than pipenv, it’s not just the speed difference it’s the convenience of one tool which aligns with what you’re saying although the existence of poetry doesn’t delete historical aberrations in Python’s long history.

Re: How to make a Python package in 2021

#132

Post by CraigJPerry » Thu, Apr 08, 2021, 9:10 AM UTC

zomglings wrote: post #30

staticassertion wrote: post #23

Earlier quoted context omitted.

TBH as someone trying to use Python professionally it is extremely frustrating that basic things with regards to package management are something you have to iterate towards, as opposed to just being obvious and default.

I sympathize. It is unfortunate that the python community never settled around a tool like leiningen for clojure or cargo for rust or npm for node. What we saw with npm was the entire community iterating towards a feature set and everyone reaping the benefits automatically with npm updates. package-lock.json is a good example of this.

Yarn / npm are still in competition today i think?

The js tooling is particularly immature, e.g. fake packages along with typo squatting is rife. Compare with boring old maven where that’s not been an issue in >10 years at this point.

Re: How to make a Python package in 2021

#133

Post by cixinlui » Thu, Apr 08, 2021, 9:10 AM UTC

zomglings wrote: post #30

staticassertion wrote: post #23

Earlier quoted context omitted.

TBH as someone trying to use Python professionally it is extremely frustrating that basic things with regards to package management are something you have to iterate towards, as opposed to just being obvious and default.

I sympathize. It is unfortunate that the python community never settled around a tool like leiningen for clojure or cargo for rust or npm for node. What we saw with npm was the entire community iterating towards a feature set and everyone reaping the benefits automatically with npm updates. package-lock.json is a good example of this.

I know Python much better than I do rust or node, but I think the Python design decision was decent here: you can use different tools, but all of them should put the configuration in pyproject.toml. That file has fields which are universal and others which can depend on the exact tooling used. So build tools, repos and so on can get the info they need and at least potentially, do the right thing for code packages with different tools.

Re: How to make a Python package in 2021

#134

Post by toyg » Thu, Apr 08, 2021, 9:11 AM UTC

divbzero wrote: post #85

jolux wrote: post #77

Earlier quoted context omitted.

I realize this is controversial but from reading the docs I really thought Pipenv was the official solution. Took me a while to realize this wasn't the case.

I went through the same progression, thinking pipenv was the official solution before deciding it wasn’t. Then, just now, I realized that pipenv [1] is currently owned by the Python Packaging Authority (PyPA) who also owns pip [2] and virtualenv [3]. I don’t know the right answer but this illustrates the confusion of not coalescing around an official solution. [1]: https://github.com/pypa/pipenv [2]: https://github.c…

What happened was that Kenneth Reitz socially-engineered his way into the PyPA to get his tool blessed. The community lashed out (since the tool had obvious shortcomings and a somewhat dubious development process) and recommendations were softened. Eventually the PyPA had to take over pipenv when Reitz had other issues, and they are now forever burdened with what is a bit of a dud.

Re: How to make a Python package in 2021

#135

Post by cixinlui » Thu, Apr 08, 2021, 9:12 AM UTC

goerz wrote: post #67

staticassertion wrote: post #61

Earlier quoted context omitted.

Well, Python did set up a standard - pyproject.toml. And it's what poetry uses. It's just, uh, "light".

It's also not ready yet, missing critical features like editable installs. Right now, you still need a shim setup.py. Until pyproject.toml can actually replace setupy.py, I see little incentive to start using it: It's just one more file to add. The one exception is if the package actually has build requirements, e.g. for Cython modules.

I don't recognize this. I use poetry all the time, without setup.py and the local installs are editable. I have published half a dozen packages which don't have a setup.py and they all work fine.

Re: How to make a Python package in 2021

#136

Post by cixinlui » Thu, Apr 08, 2021, 9:14 AM UTC

guggle wrote: post #54

neolog wrote: post #3

Just use Poetry [1]. It's popular and works well. [1] https://python-poetry.org/

That's a 72MB download, and yet another way to fragment the ecosystem. Not something I'd get just to make a package when a default Python setup has recommended tools and everything I need to make and/or install packages.

72MB? On a development machine? Are you on dialup?

Poetry doesn't fragment the ecosystem. Unlike setuptools it uses pyproject.toml, which can be read by other tools, and is the correct way of storing package configuration.

A package built using Poetry is installable without Poetry in the exact same way as one built using setuptools.

Re: How to make a Python package in 2021

#137

Post by cixinlui » Thu, Apr 08, 2021, 9:21 AM UTC

ThePhysicist wrote: post #106

phodge wrote: post #104

Earlier quoted context omitted.

I don't see how your version is easier than the sequence of commands in the first few steps of the article, which is basically `pip install flit; flit init; flit publish`. Flit is just as easy to install as twine, but you save yourself the hassle of having to write a setup.py.

Maybe I'm too old-fashioned then. But I like that you don't have any dependencies when using distutils/setuputils with a `setup.py` file, so if you don't distribute your code you're already done. I'm also not a fan of tools that are just wrappers around other tools.

How does setuptools not count as a dependency?

If you've never run into setuptools compatibility problems, you've either been much luckier than me, or you haven't done much with Python packages.

Vanilla ubuntu used to come with a version of setuptools which didn't work for installing many recent packages.

Re: How to make a Python package in 2021

#138

Post by toyg » Thu, Apr 08, 2021, 9:21 AM UTC

BerislavLopac wrote: post #127

zomglings wrote: post #15

Earlier quoted context omitted.

Iteratively. You don't need to solve all those problems at once. Version pinning can be done in setup.py using the same syntax you would see in a requirements.txt file. You should be very conservative when pinning versions in a library, though. You can lean on your ci tool (eg. Github actions) to handle testing, hash checking, credential management, etc. But I recommend all of this start as a bunch of locally runnabl…

> You should be very conservative when pinning versions in a library, though No; you should be very conservative when pinning versions in an application, not in a library. Check this article for the explanation: https://caremad.io/posts/2013/07/setup-vs-requirement/

Anybody familiar with the history of requests knows this is bad advice.

Re: How to make a Python package in 2021

#139

Post by feu » Thu, Apr 08, 2021, 9:26 AM UTC

blondin wrote: post #46

staticassertion wrote: post #23

Earlier quoted context omitted.

TBH as someone trying to use Python professionally it is extremely frustrating that basic things with regards to package management are something you have to iterate towards, as opposed to just being obvious and default.

yea, it is. but it's a sane thing to do. recommending poetry for a beginner is a bad idea. (nothing against that package). python is a mature, old, software system. three times older than go or rust. way older than zig. these modern languages have learned from the field as a whole and implemented tools that people are taking for granted nowadays. as with any mature software system, people & companies have established…

Recommending the standard set of modules is the opposite of "going simple." Poetry removes a lot of the complexity and user-unfriendliness inherent in the previous set of standard modules. For any Python beginner coming from another popular language Poetry is likely going to be very similar to the dependency and package management in the language they're coming from. Python's standard set of modules sticks out like a sore thumb when compared to the tools in other popular languages.

Re: How to make a Python package in 2021

#140

Post by fctorial » Thu, Apr 08, 2021, 9:35 AM UTC

zomglings wrote: post #8

The number of extra tools used in this article boggles my mind. Are you writing a simple library? Create a setup.py. Copy paste an existing setup.py and modify it to suit your purposes. Now you have a working, pip installable python package. Want to publish to PyPI? Use twine. It's standard and it's simple. You don't need complicated tooling for simple projects.

For all the hate nodejs gets, it solved the software packaging problem. That is the sole reason npm ecosystem is so big.

I have to say that it probably isn't a fare comparison because python is much older than nodejs. Python package management might very well have been state of the art in 1995.