Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to create a Python package in 2022

mathspp.com

Post reply on HN
Print view View original

71–80 of 153 posts

Re: How to create a Python package in 2022

#71

Post by wyuenho »

chucksmash wrote: post #67
wyuenho wrote: post #66

Earlier quoted context omitted.

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? Yes, that's what CI/CD is for. > Pre-commit is completely optional for each developer. No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time…

rm ./.git/hooks/pre-commit Or just do not install the hook in the first place. Choice is still the developer's unless your team is doing something else here that limits your ability to delete/rename files locally.

That does not alleviate the problem with difficulty having your editor to use the correct version and settings for the linters and formatters. This approach also does not address the issue of having 2 separate sets of configuration when you are trying address problem number 1 above.

Re: How to create a Python package in 2022

#72

Post by IshKebab »

bvrmn wrote: post #61
diekhans wrote: post #23

Lot of good info and saved away! However, it drinks the code coverage cool-aid that started like 30 years ago when code coverage tools emerged. Management types said "high test code coverage == high quality"; lets bean count that!! A great way to achieve high code coverage is to have less than robust code that does not check for crazy error cases that are really hard to reproduce in test cases. Code coverage is a too…

I guess you have no much experience with python and how easy to get 100% coverage in it.

He's not saying it's difficult to get 100% coverage. He's saying that 100% is a bit suss because you probably wouldn't achieve 100% coverage unless that is your goal (even if Python does make it easy) and that's the wrong goal.

Re: How to create a Python package in 2022

#73

Post by [deleted] »

emacs28 wrote: post #55

My only gripe with poetry is that when I tried it out last year there was no equivalent for `pip install -e` which can be used to install other python package dependancies that you are simultaneously developing. I found that feature useful enough to stick with setuptools and setup.py instead.

[deleted]

Re: How to create a Python package in 2022

#74

Post by doliveira »

wyuenho wrote: post #70
doliveira wrote: post #68

Earlier quoted context omitted.

How is it mandatory? You can just not install the hooks

1. Because it defeats the purpose of pre-commit. 2. pre-commit install installs the hook by default. 3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

> 1. Because it defeats the purpose of pre-commit.

No, it doesn't. Pre-commits are just a DX bonus so you won't have to wait for the CI/CD server.

2. pre-commit install installs the hook by default.

Am I missing something? If you run "$ pre-commit install" of course it will install the hooks. Do you mean that poetry install, setup.py themselves are making the magic of installing the hooks themselves? If so, it's indeed a bad practice.

3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

I don't get why anyone would care, unless you're polluting the CI/CD history with tons of tiny commits all day along just to run linting.

Re: How to create a Python package in 2022

#75

Post by fnord123 »

wyuenho wrote: post #70
doliveira wrote: post #68

Earlier quoted context omitted.

How is it mandatory? You can just not install the hooks

1. Because it defeats the purpose of pre-commit. 2. pre-commit install installs the hook by default. 3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

Tell me you use Emacs without telling me you use Emacs.

Re: How to create a Python package in 2022

#76

Post by doliveira »

zx14 wrote: post #24
uranusjr wrote: post #19

Earlier quoted context omitted.

Just fyi, both npm and yarn are “third party” tools. Cargo is closer to the Rust core, and you can find tools in a similar position in Python as well (e.g. setuptools, hatch). Packaging tools being managed by a different group than people working on the “core language” is actually the norm, since those are very different topics and only very few brilliant people care about both of them at the same time. Python people…

Instead of being insecure about criticisms to the Python ecosystem and calling my disappointment "hatred" I'd rather we focused on solutions. Just because this mess happens in some other languages doesn't mean it's the right thing. Having a very fragmented community is not a good thing for a beginner. Also, npm is far more of a de facto choice than poetry, which is still better than the state python finds itself in.

It doesn't even happen in the example of Node. Npm is even bundled together with NodeJS itself, and yarn is fully compatible with package.json.

Re: How to create a Python package in 2022

#77

Post by globular-toast »

wyuenho wrote: post #66
globular-toast wrote: post #62

Earlier quoted context omitted.

Our CI pipelines invoke pre-commit. That way it's trivial to run the exact same tools locally as would be run in CI. Running the tools locally is basically about tightening the development loop. Many of the commonly used tools (e.g. black, isort etc.) actually make the changes to the files so you'll never even commit failing versions. Do you really want to push changes to some remote CI system only to be told it's fa…

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? Yes, that's what CI/CD is for. > Pre-commit is completely optional for each developer. No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time…

> Yes, that's what CI/CD is for.

Not really. CI/CD is a methodology where a team immediately integrates new changes into a trunk/release. The CI/CD pipeline is there to give the team the confidence to merge your stuff. When I see people constantly pushing breaking code into a CI pipeline I see an incredible amount of wasted time and shared computing resource. Especially if it's some trivial formatting check that you should have already done locally. You do what works for you, but tools like pre-commit were invented to save time and effort and they work well.

> I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time sure the code I commit will pass all these linter checks.

Then what is the problem? Have your team installed hooks that take a long time to run? Even on larger codebases pre-commit adds a negligible amount of time to each commit, unless perhaps you've touched every file in the codebase or something. Honestly your gripe with pre-commit seems mostly irrational.

Re: How to create a Python package in 2022

#78

Post by wyuenho »

doliveira wrote: post #69
wyuenho wrote: post #57

I wish people would just forget about pre-commit, this thing is especially useless in a setting where a CI/CD pipeline exists. It's not that hard to write a simple Makefile or shellscript to run linters on push. Pre-commit is one of the most annoying tools that have come into existence in recent years that everyone seems to be cargo-culting. It doesn't play well with editors since in order to find the actual binary p…

Pre-commit is optional, you can just not install them into .git/ ... Although I'd indeed prefer just before push, or just make display warnings One thing that's really annoying these days are CI/CD that can't be replicated locally, generating quite annoying delays in the development. Jenkins seems particularly problematic in this regard: the steps get encoded in some cryptic pet Jenkins server, and then you have to w…

> One thing that's really annoying these days are CI/CD that can't be replicated locally

pre-commit just runs your linter, formatter and tests. Surely you can fully replicate this step locally. Just run make lint both locally and on CI.

Anything else that's hard to replicate has to do with the distributed systems that you are probably working on because these systems are all probably proprietary stuff that live on the cloud.

> Although I'd indeed prefer just before push, or just make display warnings

Wholehearted agree. This is a compromise I can live with.

Re: How to create a Python package in 2022

#79

Post by wyuenho »

globular-toast wrote: post #77
wyuenho wrote: post #66

Earlier quoted context omitted.

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? Yes, that's what CI/CD is for. > Pre-commit is completely optional for each developer. No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time…

> Yes, that's what CI/CD is for. Not really. CI/CD is a methodology where a team immediately integrates new changes into a trunk/release. The CI/CD pipeline is there to give the team the confidence to merge your stuff. When I see people constantly pushing breaking code into a CI pipeline I see an incredible amount of wasted time and shared computing resource. Especially if it's some trivial formatting check that you…

> The CI/CD pipeline is there to give the team the confidence to merge your stuff.

That's why linting is a part of every CI stage. Linters check your code for bugs.

> Have your team installed hooks that take a long time to run?

Yes, they are called tests.

Re: How to create a Python package in 2022

#80

Post by slhck »

RojerGS wrote: post #20

Hey, original author here. Thanks a lot for sharing this! Also, can't believe everyone let me get away with not writing about documentation! I'll see to it that it gets done and added to the article.

This is really nice. The only thing I'm missing here is a simple way to bump versions. Any ideas on how to do that?

For Node, it's quite simple and even built into npm. Also the version is only part of the package.json file. For Python you probably have your version somewhere in __init__.py, and I always end up writing ugly bash scripts that modify multiple places with sed.

Post reply on HN