Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to make a Python package in 2021

antonz.org

Post reply on HN
Print view View original

91–100 of 215 posts

Re: How to make a Python package in 2021

#91

Post by stinos »

icy wrote: post #42

Honestly, I don't even bother "packaging" Python tools anymore. Just put it in all in a git repo, and pip can install using pip install git+https://myg.it/repo.git

Also automatcially handles the issue of public vs private etc. Note this [1] recommends using zip files for speed, especially for larger repositories:

    pip install https://github.com/django/django/archive/master.zip

    pip install https://github.com/django/django/archive/stable/1.7.x.zip
[1] https://stackoverflow.com/questions/20101834/pip-install-fro...

Re: How to make a Python package in 2021

#92

Post by zo1 »

goerz wrote: post #67
staticassertion wrote: post #61

Earlier quoted context omitted.

Well, Python did set up a standard - pyproject.toml. And it's what poetry uses. It's just, uh, "light".

It's also not ready yet, missing critical features like editable installs. Right now, you still need a shim setup.py. Until pyproject.toml can actually replace setupy.py, I see little incentive to start using it: It's just one more file to add. The one exception is if the package actually has build requirements, e.g. for Cython modules.

The only reason we use pyproject.toml is because of the stupid black formatter that refuses to support setup.cfg, which every other python tool under the sun supports.

Re: How to make a Python package in 2021

#93

Post by dagenix »

knome wrote: post #69
dagenix wrote: post #59

Earlier quoted context omitted.

> Version pinning can be done in setup.py using the same syntax you would see in a requirements.txt file The problem with this approach is that it doesn't handle transitive dependencies well. Say you depend on version 1.4.6 of a particular library. And then that library depends on version >= 2 of some other library. When you install your package, you know that you'll get version 1.4.6 of the first library but have no…

Seems like a solid argument for a switch to use go's minimal version selection the version selected during a build is the one with the most minimal version that satisfies all other constraints. this means if you have libA that needs dep>=1.1 and libB that needs dep>=1.3, you get dep=1.3 even if dep1.9 is out. your build never changes because of a new version release, as long as they release with proper semantic versi…

Its at least consistent - which, IMO, is better than just getting a random version. I do, however, think it's a bit unfortunate that it prevents picking up bug / security fixes in transitive dependencies.

Imagine that you depend on library A of a particular version which itself depends on library B. With minimal version selection, as long you don't bump you dependency on library A (or some other library that depends on library B), you'll continue to get that same version of library B. But, then library B releases a critical security fix. With minimal version selection, there isn't a great way to pick up that fix. You can _hope_ that library A releases a new version that requires the fix - but that may or may not happen and could take a while. Or, you could add an explicit dependency on the new version of library B - which is unfortunate, since, your main package doesn't depend on library B directly.

Lock files solve this problem. You can depend on whatever version of library A that you need and lock the transitive dependencies. And once library B releases its fix, you can update your lock file without having up bump the version of library A.

Tools like poetry providing the features to automate this workflow.

Re: How to make a Python package in 2021

#96

Post by 1337shadow »

neolog wrote: post #11
zomglings wrote: post #8

The number of extra tools used in this article boggles my mind. Are you writing a simple library? Create a setup.py. Copy paste an existing setup.py and modify it to suit your purposes. Now you have a working, pip installable python package. Want to publish to PyPI? Use twine. It's standard and it's simple. You don't need complicated tooling for simple projects.

How do you handle version pinning? hash checking? CI? testing on multiple platforms? multiple python versions? deployment? credential management? package data? version bumps? Sure, experts know how to do all these things because they spent many days learning them, but I'd rather outsource to a tool.

I usually don't pin, I'd rather deal with upstream BC breaks as they are published instead of accumulating tech debt. I call this "continuously integrating upstream code", because Continuous Integration is a practice, not a tool.

Re: How to make a Python package in 2021

#98

Post by ThePhysicist »

The author mixes different things like linting and testing into the packaging process, which (IMHO) are not really part of making a package. The process is really much easier than this article makes it seem:

- Write a simple setup.py file.

- Generate a source or binary release by e.g. running "python setup.py sdist"

- You're done!

Adding a setup.py file is already enough to make your library pip-installable, so you could argue that this is a package already. The files generated by "setup.py" can also be pip-imported, so they are also packages. Now you might want to upload your package to a repository, for which there are different tools available. The simplest one being twine. Again, you just install it and run "twine upload -r dist/*" and your packages get uploaded to PyPi (it will ask for a username and password). So why complicate things?

Re: How to make a Python package in 2021

#99

Post by 1337shadow »

jimmaswell wrote: post #39
neolog wrote: post #25

Earlier quoted context omitted.

It's fragmented, but it doesn't need to be error prone if people use the good tools instead of the old low-level tools.

There would be no room for error if we just put the libraries in with the project as files instead of adding all these extra steps. Nobody seems to like this simple, bulletproof method anymore for some reason though.

That's exactly what a package manager does

Re: How to make a Python package in 2021

#100

Post by tomwojcik »

anaganisk wrote: post #70
guggle wrote: post #54

Earlier quoted context omitted.

That's a 72MB download, and yet another way to fragment the ecosystem. Not something I'd get just to make a package when a default Python setup has recommended tools and everything I need to make and/or install packages.

Default way is pip freeze which I feel is too verbose, is tracking every dependency of django worth it?

Yes. Also, check pip-tools by jazzband.
Post reply on HN