Boring Python: Code quality • Hacker News Board

Re: Boring Python: Code quality

#151

Post by 411111111111111 » Tue, Dec 20, 2022, 6:18 PM UTC

nine_k wrote: post #136

411111111111111 wrote: post #72

Earlier quoted context omitted.

> CI/CD has no business changing your code; it builds stuff using it, exactly as if commit such-and-such. That going too far unless you define code to be a subset of the files checked into the repository and simply define any file that's touched in an automated manner to be not code There are a lot of useful automations that can be part of the CI/CD pipeline, such as increasing a version number, generating a changelo…

It's not about not trusting automated generation of certain artifacts. Stamping version numbers may be a fair idea. (A changelog entry needs human review and approval, in my book.) It's mostly about the flow of data and control: source files, beside some known auto-generated files / single lines, are the source , and whatever is generated is downstream from them, not altering them. It's like a React app: data flows t…

Its completely fine if you wish to treat your repository like that, but that doesn't make this treatment the only viable and correct way.

There are a lot of people which include their helm package as part of their project repository and even more that generate the changelog from standardized git messages like conventional commits and still wish to persist them in a changelog.md inside of the repository to make completely banal examples spelled out to the letter. It works well and lets you scale these things pretty well inside of a corporation with a lot of teams.

Its completely fine to keep all that out of your automated pipeline, triggering only after these things have occured... but that's not the only viable choice a developer can make and you're very much talking from of an extremely limited perspective if thats your point of view.

Re: Boring Python: Code quality

#152

Post by nomel » Tue, Dec 20, 2022, 6:22 PM UTC

liendolucas wrote: post #150

jerrygenser wrote: post #84

Even since the start of python typing, it was recommended to use a more generic type like Iterable instead of List. The author claims that List is too specific -- this seems like a straw man argument against typing that doesn't acknowledge python's own advice. Also, mypy has gotten really good in recent years and I can vouch that on projects that have typing I catch bugs much much sooner. Previously I would only catc…

> The other thing typing does is allow for refactoring code. No. What allows you confident refactoring code are automated tests. I honestly can't understand why people are so obsessed about types, especially in languages like Python or Javascript.

> I honestly can't understand why people are so obsessed about types

It's a very powerful sanity check that lets me write correct code faster, avoiding stupid bugs that the unit tests will also, eventually, find.

And, to me, reading the code is much much nicer. Types provide additional context to what's going on, at first glance, so I don't have to try to guess what something is, based on its name:

    results: list[SomeAPIResult] = some_api.get_results()

is much easier to grock.

Re: Boring Python: Code quality

#153

Post by craftkiller » Tue, Dec 20, 2022, 6:24 PM UTC

hbrn wrote: post #144

jerrygenser wrote: post #84

Even since the start of python typing, it was recommended to use a more generic type like Iterable instead of List. The author claims that List is too specific -- this seems like a straw man argument against typing that doesn't acknowledge python's own advice. Also, mypy has gotten really good in recent years and I can vouch that on projects that have typing I catch bugs much much sooner. Previously I would only catc…

Iterable is an import away, while list is already at my fingers. There's zero harm in using list in private interfaces: I know I'm the only one passing the value, I know it is always a list. As an argument type, Iterable is compatible with list, so it's benefits are minimal (with rare exceptions). Lists are easier to inspect in a debugging session. Iterable can be useful as return type, because it limits the interfac…

> Iterable is an import away, while list is already at my fingers.

`list` might be but `List` isn't. Are you not defining the type of the contents of the list?

Re: Boring Python: Code quality

#154

Post by burntsushi » Tue, Dec 20, 2022, 6:26 PM UTC

sseagull wrote: post #142

burntsushi wrote: post #131

Earlier quoted context omitted.

I haven't heard anyone complain about how Rust programs are packaged. Take your passive aggressive bullshit somewhere else.

How Linux package managers handle these newer languages with their own package managers (including rust) is an ongoing pain point. Here’s an article from 2017 about it, and I don’t know if things have improved: https://lwn.net/Articles/712318/

I didn't say it wasn't a pain point or that there weren't challenges. I said I don't hear about people complain about how Rust programs are packaged. Not that the packaging of Rust programs (among others) itself doesn't present interesting challenges depending on the policies of a particular distro. Archlinux, for example, has far fewer problems than Debian because of differences in policy.

The poster I was responding to was literally posting false information. I'm correcting it. This doesn't need to turn into a huge long sprawling discussion about packaging Rust programs. The main point that I was making is that lock files do not prevent Rust programs from being packaged. bombolo then went off on their own little tangents spouting nonsense without bothering to acknowledge their mistake.

Re: Boring Python: Code quality

#155

Post by TrickardRixx » Tue, Dec 20, 2022, 6:31 PM UTC

ReflectedImage wrote: post #41

anderskaseorg wrote: post #33

> For example, you basically never care whether something is exactly of type list, you care about things like whether you can iterate over it or index into it. This is an odd complaint. typing.Sequence[T] has been there since the first iteration of typing (3.5), for exactly that use case, along with many related collection types. https://docs.python.org/3/library/typing.html mypy isn’t perfect, but it’s sure better t…

You should never be using static typing with a scripting language like Python or Ruby. Dynamically typed code is 1/3rd the size of statically typed code, that means that one developer who is using dynamic typing is equivalent to 3 developers using statically typed code via MyPy. Since the code is 1/3rd of the size it contains 1/3rd of the bugs. This is confirmed by all the studies that have been done on the topic. If…

Please see Raymond Hettinger's keynote on efficiently handling bugs[0]. He makes the case that static type checking is a boon for Python except for in specific programs that make extensive use of covariant and/or contravariant types.

[0] https://www.youtube.com/watch?v=ARKbfWk4Xyw

Re: Boring Python: Code quality

#156

Post by hbrn » Tue, Dec 20, 2022, 6:34 PM UTC

craftkiller wrote: post #153

hbrn wrote: post #144

Earlier quoted context omitted.

Iterable is an import away, while list is already at my fingers. There's zero harm in using list in private interfaces: I know I'm the only one passing the value, I know it is always a list. As an argument type, Iterable is compatible with list, so it's benefits are minimal (with rare exceptions). Lists are easier to inspect in a debugging session. Iterable can be useful as return type, because it limits the interfac…

> Iterable is an import away, while list is already at my fingers. `list` might be but `List` isn't. Are you not defining the type of the contents of the list?

typing.List is deprecated.

https://docs.python.org/3/library/typing.html#typing.List

> Deprecated since version 3.9: builtins.list now supports subscripting ([]). See PEP 585 and Generic Alias Type.

Re: Boring Python: Code quality

#157

Post by dragonwriter » Tue, Dec 20, 2022, 6:42 PM UTC

hbrn wrote: post #144

jerrygenser wrote: post #84

Even since the start of python typing, it was recommended to use a more generic type like Iterable instead of List. The author claims that List is too specific -- this seems like a straw man argument against typing that doesn't acknowledge python's own advice. Also, mypy has gotten really good in recent years and I can vouch that on projects that have typing I catch bugs much much sooner. Previously I would only catc…

Iterable is an import away, while list is already at my fingers. There's zero harm in using list in private interfaces: I know I'm the only one passing the value, I know it is always a list. As an argument type, Iterable is compatible with list, so it's benefits are minimal (with rare exceptions). Lists are easier to inspect in a debugging session. Iterable can be useful as return type, because it limits the interfac…

> As an argument type, Iterable is compatible with list, so it’s benefits are minimal (with rare exceptions).

Iterable is not compatible with list, but list is compatible with iterable. As the more general type, Iterable is better as an argument type unless you have a reason to force consumers to use lists. Even in private interfaces, I tend to prefer it, because I often end up wanting to pass something constructed on the fly, and creating an extra list for that rather than using a genexp just seems wasteful.

Re: Boring Python: Code quality

#158

Post by dragonwriter » Tue, Dec 20, 2022, 6:46 PM UTC

liendolucas wrote: post #150

jerrygenser wrote: post #84

Even since the start of python typing, it was recommended to use a more generic type like Iterable instead of List. The author claims that List is too specific -- this seems like a straw man argument against typing that doesn't acknowledge python's own advice. Also, mypy has gotten really good in recent years and I can vouch that on projects that have typing I catch bugs much much sooner. Previously I would only catc…

> The other thing typing does is allow for refactoring code. No. What allows you confident refactoring code are automated tests. I honestly can't understand why people are so obsessed about types, especially in languages like Python or Javascript.

> What allows you confident refactoring code are automated tests.

Typing facilitates automated testing; e.g., hypothesis can infer test strategies for type-annotated code.

Re: Boring Python: Code quality

#159

Post by eesmith » Tue, Dec 20, 2022, 7:05 PM UTC

bombolo wrote: post #54

eesmith wrote: post #23

Earlier quoted context omitted.

It could, certainly. But 1) you don't have one black commit for every non-black commit, do you? Because the general best practice is to do like kuu suggested and have a specific black version as part of the development environment, with a pre-commit hook to ensure no random formatting gets introduced. 2) assuming 500 commits in your bisection, that's, what, about 9 compilations you'll need to do, so it will take you…

As I said in other comments, if you try to force contributors to reproduce exactly your local setup, you will be left with no contributors. Which is why you set up a CI to run the tests… because people will most likely not. As for build times, it was an extreme example. But even an extra step taking 5 extra minutes is very annoying to me…

> if you try to force contributors to reproduce exactly your local setup, you will be left with no contributors. ...

That's not been my experience. To the contrary, having a requirements.txt means your contributors are more likely to have a working environment, as when your package depends on package X but the contributor has a 5-year-old buggy version of X, doesn't realize it, and it causes your program to do the wrong thing.

In any case, your argument only makes sense if no one on the project uses black or other code formatter. Even if you alone use it, odds are good that most of your collaborator's commits will need to be reformatted.

> .. an extra step taking 5 extra minutes ...

How do black reformatting changes cause an extra 5 minutes? What Python code base with only a couple of contributors and no need for a requirements.txt takes 5+ minutes to byte-compile and package the Python code, and why?

Adding 5 minutes to you build means your bisections are taking at least an hour, so it seems like focusing on black changes is the wrong place to look.

Re: Boring Python: Code quality

#160

Post by timothycrosley » Tue, Dec 20, 2022, 7:06 PM UTC

anderskaseorg wrote: post #12

If you aren’t happy with Flake8, Pylint, and isort (or maybe if you are!), I recommend checking out Ruff: https://github.com/charliermarsh/ruff It’s literally 100 times faster, with comparable coverage to Flake8 plus dozens of plugins, automatic fixes, and very active development.

FWIW, I wrote isort, but am seriously considering migrating my projects to use Ruff. Long term I think the design is just better over the variety of tools we use within the Python ecosystem today. The fact we have a plethora of projects that are meant to run per a commit with each one reparsing the AST independently, and often using a different approach to do so, just feels untenable long term to me.