Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Which Python type checker should I use?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 83 posts

Re: Ask HN: Which Python type checker should I use?

#71

Post by mixmastamyk »

IshKebab wrote: post #70
mixmastamyk wrote: post #63

Earlier quoted context omitted.

Pyflakes will catch most of those kinds of bugs. Fast option if for some reason you can’t or don’t have time to type.

No it won't. It won't even catch trivial errors like `import os; os.sdfsdfsdf()`. Perhaps you haven't used static typing so aren't familiar with the kinds of bugs it can prevent?

Doctor, it hurts when I punch myself in the face. I need something to prevent that.

I’m talking about realistic errors, not theoretical ones that would crash at first run. You have one test, right? Run the code once before deploying? I thought so. In which case, pyflakes and a few tests find a great majority (not all) of errors devs actually have.

Re: Ask HN: Which Python type checker should I use?

#72

Post by yen223 »

damezumari wrote: post #28

I would not personally use any of them. Typed python in previous workplace was pretty much shitshow, and if starting from scratch instead of untyped codebase it might be worth it, maybe, for some specific cases, but in general combination of bad library typing and bad type checking tools leads to horrible user experience. So going forward my personal projects continue to be ducktyped, and if I care about types, I use…

I agree.

Type-checking in general is good. However, Python's type-checking mechanisms are bad. The syntax for adding types is very obviously bolted on, they miss basic type errors all the time, and third-party libraries often miss type annotations anyway.

Python is fundamentally a duck-typed language, and that's how you should use it. If type-checking is important to you, use a language that is designed for it.

Re: Ask HN: Which Python type checker should I use?

#73

Post by roland35 »

yen223 wrote: post #72
damezumari wrote: post #28

I would not personally use any of them. Typed python in previous workplace was pretty much shitshow, and if starting from scratch instead of untyped codebase it might be worth it, maybe, for some specific cases, but in general combination of bad library typing and bad type checking tools leads to horrible user experience. So going forward my personal projects continue to be ducktyped, and if I care about types, I use…

I agree. Type-checking in general is good. However, Python's type-checking mechanisms are bad. The syntax for adding types is very obviously bolted on, they miss basic type errors all the time, and third-party libraries often miss type annotations anyway. Python is fundamentally a duck-typed language, and that's how you should use it. If type-checking is important to you, use a language that is designed for it.

Eh it's either Python or C++ with ROS, and Python is much easier to work with. Adding type checks to method arguments isn't much different than doing it in Rust, and makes life much easier with anything larger a tiny program.

If there is anything complicated, you can either make a quick generic type (TypeVar) or even just throw in an Any type! I had to do that recently with a function that handled different ROS services, which apparently do not share any base class.

Re: Ask HN: Which Python type checker should I use?

#75

Post by a-dub »

MrJohz wrote: post #69
a-dub wrote: post #65

Earlier quoted context omitted.

sortof. except two differences: 1) i'm primarily interested in data types, rather than objects, as i believe most runtime errors are the result of unexpected changes to data types. 2) i'm suggesting a type system that does away with hand annotations and rather builds them on the fly by keeping assignment history and using that to determine when statistically anomalous types appear.

That would be a structural type system with Hindley-Milner type inference. A good example of that might be OCaml - it's largely structurally typed, and the types can be completely inferred without type annotations. However, while Hindley-Milner will find all the type errors in your program, it won't necessarily report those errors in the most logical places. That's why it's generally still the norm to annotate functi…

cool library. although worth keeping in mind that strong typing doesn't solve all your problems.

here's an example, taking the second example on this page https://docs.pydantic.dev/latest/ with the external_data dictionary, if i add a collision like:

'wine': 9, 'wine': 1,

it seems to just silently pick the last one, which seems error prone to me. maybe there's a flag to fix it, but i couldn't find any documentation or switches that controlled this behavior. :/

with respect to the approach i mentioned above, it's more of an idea (or probably a research topic) to do quantitative analysis on the behavior of programs while they execute, rather than static analysis of the code that defines them. the latter is obviously preferable, assuming the approach can capture all of the types of analyses of interest, the quantitative approach may be more suitable for large or legacy systems where detecting changes in data shape (even if of valid type) and branching behavior may be useful for tracking down causes of, or detecting conditions for, errant behaviors.

Re: Ask HN: Which Python type checker should I use?

#76

Post by MrJohz »

a-dub wrote: post #75
MrJohz wrote: post #69

Earlier quoted context omitted.

That would be a structural type system with Hindley-Milner type inference. A good example of that might be OCaml - it's largely structurally typed, and the types can be completely inferred without type annotations. However, while Hindley-Milner will find all the type errors in your program, it won't necessarily report those errors in the most logical places. That's why it's generally still the norm to annotate functi…

cool library. although worth keeping in mind that strong typing doesn't solve all your problems. here's an example, taking the second example on this page https://docs.pydantic.dev/latest/ with the external_data dictionary, if i add a collision like: 'wine': 9, 'wine': 1, it seems to just silently pick the last one, which seems error prone to me. maybe there's a flag to fix it, but i couldn't find any documentation o…

Pydantic operates on Python dictionaries, so questions about duplicate keys will come down to whatever the Python behaviour is. (I.e. if you have a dictionary literal with duplicate keys, Python will select which key "wins" before Pydantic gets a chance to intervene.) When loading directly from JSON, it's possible to hook into Python's JSON-parsing mechanism and throw on duplicate keys, which probably ought to be the default given the number of security risks that come from this case.

Re: Ask HN: Which Python type checker should I use?

#77

Post by IshKebab »

mixmastamyk wrote: post #71
IshKebab wrote: post #70

Earlier quoted context omitted.

No it won't. It won't even catch trivial errors like `import os; os.sdfsdfsdf()`. Perhaps you haven't used static typing so aren't familiar with the kinds of bugs it can prevent?

Doctor, it hurts when I punch myself in the face. I need something to prevent that. I’m talking about realistic errors, not theoretical ones that would crash at first run. You have one test, right? Run the code once before deploying? I thought so. In which case, pyflakes and a few tests find a great majority (not all) of errors devs actually have.

> pyflakes and a few tests find a great majority (not all) of errors devs actually have.

No they don't. First of all people are really bad at writing tests. Secondly, writing tests sucks; I would much rather write static type annotations than tedious type checking tests: you know when to stop and you get the other benefits of static types like making the code easier to read and navigate.

Every piece of code I've added static typing to has revealed bugs. One we later hit in production (because the author was stupidly against static typing so I gave up trying to help him).

That bug was simply calling a method that didn't even exist. It was in a code path that wasn't tested, despite having some tests.

But I can see you're from the school of "I don't make mistakes" and convincing you otherwise is about as effective as trying to persuade Christians there isn't a god.

Re: Ask HN: Which Python type checker should I use?

#78

Post by v3ss0n »

MrJohz wrote: post #55
v3ss0n wrote: post #21

Earlier quoted context omitted.

Very good points. Did you also tried ignoring some rules that are overly strict? Or reduce them to warnings?

That was part of the wasted day I was talking about - trying to configure the system so that the rules would be more useful. I could disable the warnings per-line with comments - although this meant that my code was then filled with these lines and the formatting was awful - but I couldn't disable the warnings globally for some reason. I never managed to figure out why, unfortunately.

Per project is easy using PDM or Poetry.

Re: Ask HN: Which Python type checker should I use?

#79

Post by mixmastamyk »

IshKebab wrote: post #77
mixmastamyk wrote: post #71

Earlier quoted context omitted.

Doctor, it hurts when I punch myself in the face. I need something to prevent that. I’m talking about realistic errors, not theoretical ones that would crash at first run. You have one test, right? Run the code once before deploying? I thought so. In which case, pyflakes and a few tests find a great majority (not all) of errors devs actually have.

> pyflakes and a few tests find a great majority (not all) of errors devs actually have. No they don't. First of all people are really bad at writing tests. Secondly, writing tests sucks; I would much rather write static type annotations than tedious type checking tests: you know when to stop and you get the other benefits of static types like making the code easier to read and navigate. Every piece of code I've adde…

I’ve rescued two very large Python projects bordering on disaster, and have improved many more. In both cases the main problems were architecture-related, they were written at the wrong level of abstraction, then in the wrong language. Those need to be fixed first if you value your time. Typing errors were not a top-ten concern:

https://news.ycombinator.com/item?id=39161025

I.e. there are bugs, there are always bugs. But are they important bugs? Is the question. When faced with a house of cards, do you fix the nails, or start with a new foundation?

Also you don’t write tests just for types. You write tests to validate functionality as you would anyway. And they will find type errors on failure.

Black/white adolescent appeals to the one-true-way are not compelling and why you did not convince, then or now. Typing is simply another useful tool in the toolbox, and I never said it should be avoided—my statements were qualified.

Re: Ask HN: Which Python type checker should I use?

#80

Post by nickm12 »

mixmastamyk wrote: post #61
nickm12 wrote: post #30

Earlier quoted context omitted.

My experience couldn't be more different. I've had the experience of introducing type annotations into legacy Python 2 codebases and it's been a godsend and the only way to get them to Python 3. I've also had the experience of writing greenfield Python 3 code fully-typed from the start and its also been a useful tool there.

What does this mean, Python 2? Are those the old type comments?

Yes. Mypy supports type annotations provided in comments in both Python 2 and Python 3.

I have ported a codebase from Python 2 to 3 by annotating it with comment-syntax annotations and it helps a ton to keep your strings and bytes separate. Basically, you annotate with the binary_type, text_type types the six package and typecheck your code on both Python 2 and Python 3.

Basically, the steps involved are: 1. set up static analysis (linting and typechecking) for python 2 and python 3 2. get code passing static analysis on Py2 and Py3 3. get code passing tests on Py2 and Py3 4. get code running in production on Py3 5. stop checking and testing on Py2

It's not trivial and it requires some strong Python experience to write code that correctly "straddles" Py2 and Py3. But the advantage of the static checkers is you can slowly dial up their strictness and once they are dialed in, you minimize new regressions (new untyped code, new code that fails typechecking).

Post reply on HN