Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boring Python: Code quality

b-list.org

Post reply on HN
Print view View original

61–70 of 232 posts

Re: Boring Python: Code quality

#61

Post by ReflectedImage »

bombolo wrote: post #56
ReflectedImage wrote: post #41

Earlier quoted context omitted.

You should never be using static typing with a scripting language like Python or Ruby. Dynamically typed code is 1/3rd the size of statically typed code, that means that one developer who is using dynamic typing is equivalent to 3 developers using statically typed code via MyPy. Since the code is 1/3rd of the size it contains 1/3rd of the bugs. This is confirmed by all the studies that have been done on the topic. If…

> Dynamically typed code is 1/3rd the size of statically typed code, This is absolutely not true. > Since the code is 1/3rd of the size it contains 1/3rd of the bugs. That is made up and contrary to all empirical evidence I've ever collected. I'd be curious if you have a source, but I doubt it.

Anyone with experience of writing both dynamic typed and statically typed can tell you that.

Infact, you could just try it out for yourself.

But here is your internet source for this blatantly obvious fact: https://games.greggman.com/game/dynamic-typing-static-typing...

Re: Boring Python: Code quality

#62

Post by biorach »

bombolo wrote: post #48
kuu wrote: post #9

Earlier quoted context omitted.

> Black formats things differently depending on the version. Then add black as part of your environment with an specific version...

I'm sure you will get many contributions to your project if you refuse people with the wrong distribution from contributing.

I think by now it's a reasonable requirement for contributors to use a virtualenv when working on a project

Re: Boring Python: Code quality

#63

Post by throwawaaarrgh »

bombolo wrote: post #7

> I recommend using two tools together: Black and isort. Black formats things differently depending on the version. So a project with 2 developers, one running arch and one running ubuntu, will get formatted back and forth. isort's completely random… For example the latest version I tried decided to alphabetically sort all the imports, regardless if they are part of standard library or 3rd party. This is a big change…

> So a project with 2 developers, one running arch and one running ubuntu, will get formatted back and forth.

Any team of developers who aren't using the exact same environment are going to run into conflicts.

At the very least, there must be a CI job that runs quality gates in a single environment in a PR and refuses to merge until the code is correct. The simplest way is to just fail the build if the job results in modified code, which leaves it to the dev to "get things right". Or you could have the job do the rewriting for simplicity. Just assuming the devs did things the right way before shipping their code is literally problems waiting to happen.

To avoid CI being a bottleneck, the devs should be developing using the same environment as the CI qualify gates (or just running them locally before pushing) with the same environment. The two simple ways to do this are a Docker image or a VM. People who hate that ("kids today and their Docker! get off my lawn!!") could theoretically use pyenv or poetry to install exact versions of all the Python stuff, but different system deps would still lead to problems.

Re: Boring Python: Code quality

#64

Post by mau »

One thing that is underestimated is keep the tools version in sync between your app dev dependencies and pre-commit. This also includes plugins for specific tools (for instance flake8). A solution would be to define the hooks in pre-commit to run the tools inside your venv.

About typings: I agree the eco-system is not mature enough, especially for some frameworks such as Django, but the effort is still valuable and in many cases the static analysis provided by mypy is more useful than not using it at all. So I would suggest to try do your best to make it work.

Re: Boring Python: Code quality

#65

Post by bombolo »

ReflectedImage wrote: post #61
bombolo wrote: post #56

Earlier quoted context omitted.

> Dynamically typed code is 1/3rd the size of statically typed code, This is absolutely not true. > Since the code is 1/3rd of the size it contains 1/3rd of the bugs. That is made up and contrary to all empirical evidence I've ever collected. I'd be curious if you have a source, but I doubt it.

Anyone with experience of writing both dynamic typed and statically typed can tell you that. Infact, you could just try it out for yourself. But here is your internet source for this blatantly obvious fact: https://games.greggman.com/game/dynamic-typing-static-typing...

I do have such experience and I really can't tell that. Which is why I wondered if anyone else was in fact saying that.

> But here is your internet source for this blatantly obvious fact: https://games.greggman.com/game/dynamic-typing-static-typing...

Ah no I meant a proper peer reviewed source. The claim that untyped code has fewer bugs is completely bonkers, so I was quite sure that no such source existed.

Why do you think microsoft, google and facebook are all in the business of typechecking python? If typechecking would actually introduce bugs, it'd be better not doing it right?

Using github for statistics is flawed. There are millions of 10 line js libraries. Yes it's easy to not make type mistakes in 10 lines. I suppose that type errors increase more than linearly with size.

Re: Boring Python: Code quality

#66

Post by nine_k »

dustrider wrote: post #44
c120 wrote: post #40

Not sure if I like the recommendation to not let Black change your code and just give out errors. I absolutely let Black change code and see the value in Black that it does that so the devs do not have to spend time on manually formatting code. Black shouldn't break anything (and hasn't broken anything for me in the years I used it) but in the unlikely case it does it, there's still pytests/unittests after that that…

As I understood it, it was to not let black do the formatting during CI builds. In local dev you’d let it reformat. Even while it won’t break anything you want CI to be your safety net, flagging a local setup as being wrong is more valuable than magically autocorrecting it.

We have Black as a pre-commit hook; works fine, even if it disagrees with your IDE a little bit sometimes.

CI/CD has no business changing your code; it builds stuff using it, exactly as if commit such-and-such.

Re: Boring Python: Code quality

#67

Post by bombolo »

throwawaaarrgh wrote: post #63
bombolo wrote: post #7

> I recommend using two tools together: Black and isort. Black formats things differently depending on the version. So a project with 2 developers, one running arch and one running ubuntu, will get formatted back and forth. isort's completely random… For example the latest version I tried decided to alphabetically sort all the imports, regardless if they are part of standard library or 3rd party. This is a big change…

> So a project with 2 developers, one running arch and one running ubuntu, will get formatted back and forth. Any team of developers who aren't using the exact same environment are going to run into conflicts. At the very least, there must be a CI job that runs quality gates in a single environment in a PR and refuses to merge until the code is correct. The simplest way is to just fail the build if the job results in…

> Any team of developers who aren't using the exact same environment are going to run into conflicts.

You've never done any open source development I guess?

Do you think all the kernel developers run the same distribution, the same IDE, the same compiler version? LOL.

Same applies for most open source projects.

Re: Boring Python: Code quality

#68

Post by bombolo »

adammarples wrote: post #30
wodenokoto wrote: post #27

Earlier quoted context omitted.

That would only make it more likely that two developers would be using two different versions of Black. The further you get away from the project folder the more likely each developer is to have a different environment.

Just put a versioned black into pre-commit yaml and put that in your source and forget about it

So now we have one more (useless) build requirement for developers?

Re: Boring Python: Code quality

#69

Post by bombolo »

jsmeaton wrote: post #59
bombolo wrote: post #49

Earlier quoted context omitted.

It doesn't… people use a million different distributions. Forcing everyone to use a single version of black means that people will just not bother with your project. The authors of black just don't understand that it'd be ok to introduce new rules to format new syntax, but it isn't ok to just change how previous things work.

This is mostly nonsense and FUD. We have virtualenvironments, requirements files, setup.py with extra_requires that can all be used to manage versions without relying on the particular packages installed on an OS. Most people contributing to open source would be familiar with at least some of these methods and if they are not it’s a good opportunity for learning. And if they are not, then maintainers can pull, run bl…

> Having a tool that dictates formatting is a lot less oppressive to new developers than 100 comments nitpicking style choices.

Yes, it would work very well if said tool didn't change its mind every 6 months, generating huge commits at every bump

> Most people contributing to open source would be familiar with at least some of these methods and if they are not it’s a good opportunity for learning.

You seem unfamiliar with the fact that other people aren't necessarily clones of yourself and might not behave like you.

> CI prevents poorly formatted code from entering main.

If you run black on CI… which of course I don't since every time they make a new version I'd have the CI start failing.

And no pinning is not a "solution"… it's at best a workaround for badly written software.

> The actual changes between black versions of late have been minor at best. You’re making a mountain out of a molehill.

If you have 10 lines of code, I guess your diff can't be more than 10 lines. If you have more than 10 lines…

Re: Boring Python: Code quality

#70

Post by Lutger »

dustrider wrote: post #44
c120 wrote: post #40

Not sure if I like the recommendation to not let Black change your code and just give out errors. I absolutely let Black change code and see the value in Black that it does that so the devs do not have to spend time on manually formatting code. Black shouldn't break anything (and hasn't broken anything for me in the years I used it) but in the unlikely case it does it, there's still pytests/unittests after that that…

As I understood it, it was to not let black do the formatting during CI builds. In local dev you’d let it reformat. Even while it won’t break anything you want CI to be your safety net, flagging a local setup as being wrong is more valuable than magically autocorrecting it.

This is the way. We have format-on-save in our editor, works like a charm. Sometimes the CI still catches sometimes, but generally its very low friction.
Post reply on HN