Live data from Hacker News

How to create a Python package in 2022

mathspp.com

131–140 of 153 posts

Re: How to create a Python package in 2022

#131

PyPi is adding support for GitHub OIDC for publishing packages soon, so there will be no need to generate API keys - you can just grant your GitHub Actions permissions to publish to PyPi. https://github.com/pypi/warehouse/issues/10619

Surely you mean OAuth2? I really hope you mean OAuth2.

Re: How to create a Python package in 2022

#132
post #80
post #20

Hey, original author here. Thanks a lot for sharing this! Also, can't believe everyone let me get away with not writing about documentation! I'll see to it that it gets done and added to the article.

This is really nice. The only thing I'm missing here is a simple way to bump versions. Any ideas on how to do that? For Node, it's quite simple and even built into npm. Also the version is only part of the package.json file. For Python you probably have your version somewhere in __init__.py, and I always end up writing ugly bash scripts that modify multiple places with sed.

I never really understood this. Sure, changing the version in one place would be better and I would do that if it was possible with the current Python tools. But I have seen insane setups to achieve this, parsing __init__.py with regex from setup.py, using third-party dependencies that do only that, sometimes scripts with hundreds of lines included in the distribution to support it.

Is changing the number 2 places really that big of a deal?

You should have a release checklist anyway, with steps like sending an announcement email or tweet etc. How much time does this really save, at the cost of so much complexity?

Re: How to create a Python package in 2022

#133
post #59

Earlier quoted context omitted.

Poetry has path dependencies and the develop flag to enable the same editable functionality https://python-poetry.org/docs/dependency-specification/#pat...

Thanks, I think that the thing this still lacks compared to `pip install -e` is that the develop dependency is then only visible inside other packages configured with poetry. So it is not useful for example if you want to locally install a package you make to be accessed through cli or any other python script not using poetry.

I'm not sure what you are talking about. It will be installed in the virtual environment just the same. There is no mechanism for it to "only be visible inside other packages configured with poetry".

Re: How to create a Python package in 2022

#134
post #46

This is a fantastic resource. Are there any similar resources for setting up internal packages that you don't intend to publish publicly? I can think of a number of situations where I would have benifitted from it, but the process of configuring a package to publish, hosting it and then pulling it when necessary is a mystery to me.

You can mostly just put wheels and source dists on any HTTP server with directory listing enabled and have it work (with --extra-index-url). Use a subdirectory per package.

https://packaging.python.org/en/latest/guides/hosting-your-o...

Re: How to create a Python package in 2022

#135
post #112

Earlier quoted context omitted.

> pip freeze doesn't pin transitive dependencies AFAIK it lists all installed packages and hence pins all dependencies.

Yes, unfortunately all dependencies are frozen at the same level, so it becomes really hard to distinguish between what your actual dependencies, and sub-dependencies, and sub-sub-dependencies are.

Use pip install -c constraints.txt.

Re: How to create a Python package in 2022

#136
post #66

Earlier quoted context omitted.

Our CI pipelines invoke pre-commit. That way it's trivial to run the exact same tools locally as would be run in CI. Running the tools locally is basically about tightening the development loop. Many of the commonly used tools (e.g. black, isort etc.) actually make the changes to the files so you'll never even commit failing versions. Do you really want to push changes to some remote CI system only to be told it's fa…

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? Yes, that's what CI/CD is for. > Pre-commit is completely optional for each developer. No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time…

Does it matter if it’s run twice? Usually it’s so fast you hardly even notice it.

If your pre-commit takes more than 3 seconds to run it’s set up incorrectly IMO, and should belong to a more manually (and CI of course) invoked test suite instead.

Re: How to create a Python package in 2022

#137
post #132
post #80

Earlier quoted context omitted.

This is really nice. The only thing I'm missing here is a simple way to bump versions. Any ideas on how to do that? For Node, it's quite simple and even built into npm. Also the version is only part of the package.json file. For Python you probably have your version somewhere in __init__.py, and I always end up writing ugly bash scripts that modify multiple places with sed.

I never really understood this. Sure, changing the version in one place would be better and I would do that if it was possible with the current Python tools. But I have seen insane setups to achieve this, parsing __init__.py with regex from setup.py, using third-party dependencies that do only that, sometimes scripts with hundreds of lines included in the distribution to support it. Is changing the number 2 places re…

Your program needs to be able to output its version (e.g. with a --version CLI option), and with setup.py I could at least parse __init__.py. The pyproject.toml file doesn't do that, so suddenly I have to maintain two version numbers.

I maintain half a dozen small Python packages. I don't do emails, tweets, etc. I just want to create releases easily when there's a bug fix. It not only saves time to automate this step (I can use the same script to release each package), it also means you can't forget things. Before I had a script, I always forgot to push the tag, or run the changelog, etc.

Re: How to create a Python package in 2022

#138
post #70

Earlier quoted context omitted.

1. Because it defeats the purpose of pre-commit. 2. pre-commit install installs the hook by default. 3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

> 1. Because it defeats the purpose of pre-commit. No, it doesn't. Pre-commits are just a DX bonus so you won't have to wait for the CI/CD server. 2. pre-commit install installs the hook by default. Am I missing something? If you run "$ pre-commit install" of course it will install the hooks. Do you mean that poetry install, setup.py themselves are making the magic of installing the hooks themselves? If so, it's inde…

Filling the CI/CD pipeline with tiny commits to fix linting when it should be done before commit or push is why pre-commit exists. It's not BS, clogging the job queue shits in everyone else.

Of course I don't know GPs specific context so they may have other details, but generally it's

Re: How to create a Python package in 2022

#139
post #57

I wish people would just forget about pre-commit, this thing is especially useless in a setting where a CI/CD pipeline exists. It's not that hard to write a simple Makefile or shellscript to run linters on push. Pre-commit is one of the most annoying tools that have come into existence in recent years that everyone seems to be cargo-culting. It doesn't play well with editors since in order to find the actual binary p…

It's convenient to run the lint step faster/sooner than at CI/CD time. Depending on your setup, the separate linter deps handled by pre-commit can be more convenient than hassle both locally and in your CI/CD pipeline (re the makefile script you mention).

Having done it both ways several times I lean pre-commit for now

Re: How to create a Python package in 2022

#140
post #87

What a great article. We start with learning that we absolutely need this Poetry thing because… it's what everyone else uses. It's refreshing to see author who can skip usual badly argued justifications and just plain admin that he does not know shit and is just following rest of the herd. Then we continue by "solving" depependencies by usual way of ignoring them and just freezing whatever happens to be present. Then…

Jeez, I skimmed the article, and saw what I assume to be a comprehensive but basic primer on modern packaging, like you say. But I also inferred that the author is probably a newer programmer, with only a few years of experience. He's learning about tools and best practices in an accessible language and having fun sharing knowledge through his blog.

The sentiment behind your comments is shared, but I don't see the need to sarcastically rant about it and rail all the suggestions OP made.

If anything, I'm surprised someone with more experience didn't see the post for what it is, and attacking someone's post like this just shows immaturity when you could have easily taken those opinions and formed a constructive argument or given good advice.

Post reply on HN