Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to create a Python package in 2022

mathspp.com

Post reply on HN
Print view View original

61–70 of 153 posts

Re: How to create a Python package in 2022

#61

Post by bvrmn »

diekhans wrote: post #23

Lot of good info and saved away! However, it drinks the code coverage cool-aid that started like 30 years ago when code coverage tools emerged. Management types said "high test code coverage == high quality"; lets bean count that!! A great way to achieve high code coverage is to have less than robust code that does not check for crazy error cases that are really hard to reproduce in test cases. Code coverage is a too…

I guess you have no much experience with python and how easy to get 100% coverage in it.

Re: How to create a Python package in 2022

#62

Post by globular-toast »

wyuenho wrote: post #57

I wish people would just forget about pre-commit, this thing is especially useless in a setting where a CI/CD pipeline exists. It's not that hard to write a simple Makefile or shellscript to run linters on push. Pre-commit is one of the most annoying tools that have come into existence in recent years that everyone seems to be cargo-culting. It doesn't play well with editors since in order to find the actual binary p…

Our CI pipelines invoke pre-commit. That way it's trivial to run the exact same tools locally as would be run in CI.

Running the tools locally is basically about tightening the development loop. Many of the commonly used tools (e.g. black, isort etc.) actually make the changes to the files so you'll never even commit failing versions. Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? There's nothing at all stopping you from doing that. Pre-commit is completely optional for each developer. I would just recommend it for sanity reasons.

Re: How to create a Python package in 2022

#63

Post by bvrmn »

dannyboland wrote: post #41
zmmmmm wrote: post #38

I really dislike the Python convention of plonking source code effectively in the root directory. It means literally any random thing in there can get picked up if you put your package in the PYTHONPATH. Is there a reason the Python world did not standardise on putting source code in a "src" directory like every other language?

Yeah, I adopted the “src” dir layout after reading this post some time back: https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-str... It describes some of the outdated motivations for the other layouts commonly seen with python, as well as the many benefits of the “src” layout.

Post doesn't make sense. It clearly states that you only be sure that package is working by actually installing it into clean virtualenv and testing it there. `src` or any other layout doesn't matter.

Re: How to create a Python package in 2022

#64

Post by frumiousirc »

RojerGS wrote: post #20

Hey, original author here. Thanks a lot for sharing this! Also, can't believe everyone let me get away with not writing about documentation! I'll see to it that it gets done and added to the article.

A very nice post. Consider adding a prominent RSS/atom feed for your blog. My lack of finding one means I won't easily catch any future posts.

Re: How to create a Python package in 2022

#65

Post by talideon »

RojerGS wrote: post #25
woodruffw wrote: post #6

This is really nicely written; kudos to the author for compiling a great deal of information in a readable format. If I can be forgiven one nitpick: Poetry does not use a PEP 518-style[1] build configuration by default, which means that its use of `pyproject.toml` is slightly out of pace with the rest of the Python packaging ecosystem. That isn't to say that it isn't excellent, because it is! But you the standards ha…

Your nitpick is forgiven! Thanks a lot for this information, I was not aware of this... However, I took a look at PEP 518 and failed to understand what was wrong with Poetry's default configuration. Can you help me out?

It's actually PEP 621 (https://peps.python.org/pep-0621/) that the OP meant to refer to.

IIRC, there's work ongoing in Poetry to allow it to support PEP 621.

Re: How to create a Python package in 2022

#66

Post by wyuenho »

globular-toast wrote: post #62
wyuenho wrote: post #57

I wish people would just forget about pre-commit, this thing is especially useless in a setting where a CI/CD pipeline exists. It's not that hard to write a simple Makefile or shellscript to run linters on push. Pre-commit is one of the most annoying tools that have come into existence in recent years that everyone seems to be cargo-culting. It doesn't play well with editors since in order to find the actual binary p…

Our CI pipelines invoke pre-commit. That way it's trivial to run the exact same tools locally as would be run in CI. Running the tools locally is basically about tightening the development loop. Many of the commonly used tools (e.g. black, isort etc.) actually make the changes to the files so you'll never even commit failing versions. Do you really want to push changes to some remote CI system only to be told it's fa…

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check?

Yes, that's what CI/CD is for.

> Pre-commit is completely optional for each developer.

No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time sure the code I commit will pass all these linter checks. I can use git commit --no-verify to bypass pre-commit, but then again, what's the point of using pre-commit in the first place if you need to bypass it? There is absolutely no point of linting twice locally and thrice in total just to hit the first stage of your deployment.

Re: How to create a Python package in 2022

#67

Post by chucksmash »

wyuenho wrote: post #66
globular-toast wrote: post #62

Earlier quoted context omitted.

Our CI pipelines invoke pre-commit. That way it's trivial to run the exact same tools locally as would be run in CI. Running the tools locally is basically about tightening the development loop. Many of the commonly used tools (e.g. black, isort etc.) actually make the changes to the files so you'll never even commit failing versions. Do you really want to push changes to some remote CI system only to be told it's fa…

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? Yes, that's what CI/CD is for. > Pre-commit is completely optional for each developer. No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time…

  rm ./.git/hooks/pre-commit
Or just do not install the hook in the first place. Choice is still the developer's unless your team is doing something else here that limits your ability to delete/rename files locally.

Re: How to create a Python package in 2022

#68

Post by doliveira »

wyuenho wrote: post #66
globular-toast wrote: post #62

Earlier quoted context omitted.

Our CI pipelines invoke pre-commit. That way it's trivial to run the exact same tools locally as would be run in CI. Running the tools locally is basically about tightening the development loop. Many of the commonly used tools (e.g. black, isort etc.) actually make the changes to the files so you'll never even commit failing versions. Do you really want to push changes to some remote CI system only to be told it's fa…

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? Yes, that's what CI/CD is for. > Pre-commit is completely optional for each developer. No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time…

How is it mandatory? You can just not install the hooks

Re: How to create a Python package in 2022

#69

Post by doliveira »

wyuenho wrote: post #57

I wish people would just forget about pre-commit, this thing is especially useless in a setting where a CI/CD pipeline exists. It's not that hard to write a simple Makefile or shellscript to run linters on push. Pre-commit is one of the most annoying tools that have come into existence in recent years that everyone seems to be cargo-culting. It doesn't play well with editors since in order to find the actual binary p…

Pre-commit is optional, you can just not install them into .git/ ... Although I'd indeed prefer just before push, or just make display warnings

One thing that's really annoying these days are CI/CD that can't be replicated locally, generating quite annoying delays in the development. Jenkins seems particularly problematic in this regard: the steps get encoded in some cryptic pet Jenkins server, and then you have to wait minutes until an agent picks it up and reaches the step you actually care about. Other tools are a little quicker, but still...

So, I think at the very least pre-commit hooks help with this "over-reliance" on the CI/CD server. It's so much better DX when you can run parts of the pipeline instantaneously.

Re: How to create a Python package in 2022

#70

Post by wyuenho »

doliveira wrote: post #68
wyuenho wrote: post #66

Earlier quoted context omitted.

> Do you really want to push changes to some remote CI system only to be told it's failed some boring QA check? Yes, that's what CI/CD is for. > Pre-commit is completely optional for each developer. No it is not, it's installed at your git pre-commit hook, and it gets run every fucking time I commit, even if I've set up my editor correctly to auto-format and lint everything at I develop, meaning 99.9999% of the time…

How is it mandatory? You can just not install the hooks

1. Because it defeats the purpose of pre-commit.

2. pre-commit install installs the hook by default.

3. If you happen to fail linting a few times a year, there's always an anal coworker telling your boss in a 1-on-1 you are not following some BS ways of working.

Post reply on HN