Live data from Hacker News

How to create a Python package in 2022

mathspp.com

81–90 of 153 posts

Re: How to create a Python package in 2022

#81
post #70

Earlier quoted context omitted.

1. Because it defeats the purpose of pre-commit. 2. pre-commit install installs the hook by default. 3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

Tell me you use Emacs without telling me you use Emacs.

Guilty

Re: How to create a Python package in 2022

#82

We've been fine-ish with classic setup.py/setup.cfg + gha for publishing to pypi. But as we do OSS data science (gpu graph ai + viz), where conda is typical nowadays... ... Have to admit: We recently ended up contracting conda packaging out because it was nowhere near clear enough to make sense for our core team to untangle. Would love to see a similar tutorial on a github flow packaging & publishing to conda. Still…

I found conda-forge community quite helpful here [1]. They make feedstock repositories based on templates that cover a lot of automation. Their bots pickup updated packages in pypi and automatically file merge requests, run tests and even merge updates if tests pass successfully. Basically you only need to maintain your recipe here and there when your dependencies change. 1. https://conda-forge.org/docs/user/introduc…

Yep, that was one of the better docs!

The contractor still took a couple weeks to figure out and get up. I assumed it'd be one evening for initial bulk as we already had setup.cfg etc, but after searching conda tutorials.. not surprised. Our ~final meta file is pretty simple, so no idea why the docs are so indirect.

Re: How to create a Python package in 2022

#83
post #70

Earlier quoted context omitted.

1. Because it defeats the purpose of pre-commit. 2. pre-commit install installs the hook by default. 3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

Tell me you use Emacs without telling me you use Emacs.

Not sure why, though? I've used emacs for 15 years and pre-commit for probably 5 years at this point.

Re: How to create a Python package in 2022

#84
post #79

Earlier quoted context omitted.

> Yes, that's what CI/CD is for. Not really. CI/CD is a methodology where a team immediately integrates new changes into a trunk/release. The CI/CD pipeline is there to give the team the confidence to merge your stuff. When I see people constantly pushing breaking code into a CI pipeline I see an incredible amount of wasted time and shared computing resource. Especially if it's some trivial formatting check that you…

> The CI/CD pipeline is there to give the team the confidence to merge your stuff. That's why linting is a part of every CI stage. Linters check your code for bugs. > Have your team installed hooks that take a long time to run? Yes, they are called tests.

Wait, you're not supposed to run tests with pre-commit. I can see why that would be frustrating. If that's the case, your team is doing it wrong.

Re: How to create a Python package in 2022

#85
The title should be: How to create a "Python DISTRIBUTION package".

The term "python package" means something entirely different (or at the very least is ambiguous in a pypi/distribution context).

To add to the confusion, creating a totally normal, runnable python package in a manner that makes it completely self-contained such that it can be "distributed" in a standalone manner, while still being a totally normal boring python package, is also totally possible (if not preferred, in my view).

(shameless plug: https://github.com/tpapastylianou/self-contained-runnable-py... )

Re: How to create a Python package in 2022

#86
post #80
post #20

Hey, original author here. Thanks a lot for sharing this! Also, can't believe everyone let me get away with not writing about documentation! I'll see to it that it gets done and added to the article.

This is really nice. The only thing I'm missing here is a simple way to bump versions. Any ideas on how to do that? For Node, it's quite simple and even built into npm. Also the version is only part of the package.json file. For Python you probably have your version somewhere in __init__.py, and I always end up writing ugly bash scripts that modify multiple places with sed.

Try commitizen.

Re: How to create a Python package in 2022

#87
What a great article.

We start with learning that we absolutely need this Poetry thing because… it's what everyone else uses. It's refreshing to see author who can skip usual badly argued justifications and just plain admin that he does not know shit and is just following rest of the herd. Then we continue by "solving" depependencies by usual way of ignoring them and just freezing whatever happens to be present.

Then there is inevitable firing up of virtualenv, because that's just what you have to do when dealing with messed up dependencies.

Next one is new to me. Apparently, one does not just set up git hooks nowadays but use separate tool with declarative config. Because if you ever happen upon something not covered by the Tool, that would mean you are no longer part of the herd.

Then we push our stuff straight to pypi, because of course our stuff can't possibly have any dependencies outside of python herd ecosystem. It's not like we knew our dependencies anyway.

Then comes the fun part, pulling in tox, because when you have special tool to handle dependencies, what you just need is another tool with different environment and dependency model.

Code quality section I will just skip over, seeing what pass for code quality these days makes me too sad. What follows is setup of several proprietary projects that modern opensource seemingly can't exist without. What is more interresting is "tyding up" by moving code from git root to subdir. Now, this is of course perfectly sensible thing to, but I wonder why is it called 'src'? Maybe some herd memeber saw compiled language somewhere and picked it up without understanding difference between compiled binary and source code?

Now don't take this as if I have problem with the article content in itself. No, as a primer to modern python packaging it's great. It's not authors fault that his work is so comprehensive it lays out bare all the idiosyncrasies, herd mentality, cargocultism and general laziness of python ecosystem these days. Or is it?

Re: How to create a Python package in 2022

#89
post #55

My only gripe with poetry is that when I tried it out last year there was no equivalent for `pip install -e` which can be used to install other python package dependancies that you are simultaneously developing. I found that feature useful enough to stick with setuptools and setup.py instead.

[deleted]

Re: How to create a Python package in 2022

#90
post #87

What a great article. We start with learning that we absolutely need this Poetry thing because… it's what everyone else uses. It's refreshing to see author who can skip usual badly argued justifications and just plain admin that he does not know shit and is just following rest of the herd. Then we continue by "solving" depependencies by usual way of ignoring them and just freezing whatever happens to be present. Then…

the irony here is python packaging has sucked forever, and this is just another example of it. "Do more with less" has never entered the average python developers mind.

you'd think herd mentality might help it but it only creates more packaging solutions.

Now days, I've stopped using python outside of tiny scripts and I will never touch it for a large project.

Post reply on HN