Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python Type Checker Comparison: Empty Container Inference

pyrefly.org

Post reply on HN
Print view View original

51–59 of 59 posts

Re: Python Type Checker Comparison: Empty Container Inference

#51

Post by maleldil »

nomel wrote: post #42
maleldil wrote: post #35

Earlier quoted context omitted.

Annotating empty collections is one of the few places you need to annotate outside function signatures. It's not a big deal. It doesn't happen that often.

And, when it does, you can just put them when the empty container is assigned: things: set[tuple[str, str, int]] = set() users: list[User] = [] Many people don't seem to know this exists.

Yes, that's what I was referring to. I get it that Pyrefly wanted to advertise their approach here, but it's weird that they didn't at least acknowledge this. It's what I use because it works on every type check, and I don't need to rely on their particular implementation for this.

In fact, I recently migrated a project from Pyright to Pyrefly for performance reasons, and there was very little I had to change between. The most annoying thing was Pyrefly's lack of exhaustive pattern matching for StrEnum and Literal[...]

Re: Python Type Checker Comparison: Empty Container Inference

#52

Post by maleldil »

yunnpp wrote: post #47
LtWorf wrote: post #45

Earlier quoted context omitted.

They can be used at runtime though. I wrote typedload, to load external data (json/bson/yaml) into python typed objects. In this way you know that if the data doesn't match the expectations you will have an exception at a specific point in the code, and after that it's safe to use the objects, rather than having to manually check at every access. Now there are several other libraries that do this thing, but at the ti…

That seems to handle deserialization? But would it protect you from assigning a value of the wrong type to the object later on?

That depends on what you're using. If you're using Pydantic, which lets you define a struct-like data type with validation, you can tell it to validate assignments as well [1]. Or you can set the class as frozen and forbid assignment entirely [2].

However, if you mean annotating a local variable with a type, then no, nothing will stop it at runtime. If you use a type checker, though, it will tell you that statically.

The ecosystem also offers other runtime validation options, such as beartype [3]. For example, you can annotate a function such that it always checks the data types of input parameters when called. You can even apply this to a whole module if you want, but I don't think that's commonly done.

[1] https://docs.pydantic.dev/latest/api/config/#pydantic.config...

[2] https://docs.pydantic.dev/latest/api/config/#pydantic.config...

[3] https://beartype.readthedocs.io/en/latest/eli5/

Re: Python Type Checker Comparison: Empty Container Inference

#53

Post by LtWorf »

maleldil wrote: post #52
yunnpp wrote: post #47

Earlier quoted context omitted.

That seems to handle deserialization? But would it protect you from assigning a value of the wrong type to the object later on?

That depends on what you're using. If you're using Pydantic, which lets you define a struct-like data type with validation, you can tell it to validate assignments as well [1]. Or you can set the class as frozen and forbid assignment entirely [2]. However, if you mean annotating a local variable with a type, then no, nothing will stop it at runtime. If you use a type checker, though, it will tell you that statically.…

Checking types on all function calls adds a considerable amount of extra work that I personally am not willing to pay, especially since static type checkers exist.

Re: Python Type Checker Comparison: Empty Container Inference

#54

Post by ocamoss »

maleldil wrote: post #51
nomel wrote: post #42

Earlier quoted context omitted.

And, when it does, you can just put them when the empty container is assigned: things: set[tuple[str, str, int]] = set() users: list[User] = [] Many people don't seem to know this exists.

Yes, that's what I was referring to. I get it that Pyrefly wanted to advertise their approach here, but it's weird that they didn't at least acknowledge this. It's what I use because it works on every type check, and I don't need to rely on their particular implementation for this. In fact, I recently migrated a project from Pyright to Pyrefly for performance reasons, and there was very little I had to change between…

It's acknowledged at the end of the "infer any" strategy, but perhaps worded poorly.

> To improve type safety in these situations, type checkers that infer Any for empty containers can choose to generate extra type errors that warn the user about the insertion of an Any type. While this can reduce false negatives, it burdens developers by forcing them to explicitly annotate every empty container in order to silence the warnings.

ie: "type checkers that don't infer container types can emit an error and require users to annotate"

Re: Python Type Checker Comparison: Empty Container Inference

#55

Post by jcgl »

tialaramex wrote: post #49
jcgl wrote: post #25

Earlier quoted context omitted.

> So, can we add two bools together? Adding booleans together is nonsense, but we've said these are a kind of integer so sure, I guess True + True = 2 ? And this cascades into nonsense like ~True being a valid operation in Python and its result is true... The bitwise negation is indeed janky and inaccurate, but True + True = 2 is absolutely a valid thing to say in boolean algebra. Addition mean "or", and multiplicati…

> True + True = 2 is absolutely a valid thing to say in boolean algebra Nope. The Boolean algebra only has two values, and it lacks the addition operation entirely.

I always remember learning that 2 was a legit enough way to represent the result of 1 + 1, but the internet seems to agree with you mostly. Though I contend that 1 + 1 = 2 is unambiguous, so is fine.

But multiplication and addition do work just fine for boolean arithmetic: https://en.wikipedia.org/wiki/Two-element_Boolean_algebra

Re: Python Type Checker Comparison: Empty Container Inference

#56

Post by linsomniac »

tl2do wrote: post #33
linsomniac wrote: post #32

Earlier quoted context omitted.

Correct, currently in Python the type checking is implemented more in a linting phase than in a compiling or runtime phase. Though you can also get it from editors that do LSP, they'll show you type errors while editing the code.

Thanks linsomniac and exyi. I didn't realize Python's type hints are checked by linters, not the compiler. Learned something today.

I really like them, I'm a very long time Python programmer ('97) and so the ability to just bang something simple out and not care about the typing is nice at times, but for anything very serious at all it's very nice to have the option to add the type annotations and get the bulk of the benefits from it.

Re: Python Type Checker Comparison: Empty Container Inference

#57

Post by tialaramex »

jcgl wrote: post #55
tialaramex wrote: post #49

Earlier quoted context omitted.

> True + True = 2 is absolutely a valid thing to say in boolean algebra Nope. The Boolean algebra only has two values, and it lacks the addition operation entirely.

I always remember learning that 2 was a legit enough way to represent the result of 1 + 1, but the internet seems to agree with you mostly. Though I contend that 1 + 1 = 2 is unambiguous, so is fine. But multiplication and addition do work just fine for boolean arithmetic: https://en.wikipedia.org/wiki/Two-element_Boolean_algebra

Huh, I learn something new, I was not aware of "Two element Boolean algebra" nor just how deep this particular rabbit hole goes.

It's fine that 1 + 1 = 2. That's just integer arithmetic. The problem is that the booleans are not "just integers" and so Python's choice to implement them as "just integers" while convenient for them has consequences that are... undesirable.

Re: Python Type Checker Comparison: Empty Container Inference

#58

Post by maleldil »

LtWorf wrote: post #53
maleldil wrote: post #52

Earlier quoted context omitted.

That depends on what you're using. If you're using Pydantic, which lets you define a struct-like data type with validation, you can tell it to validate assignments as well [1]. Or you can set the class as frozen and forbid assignment entirely [2]. However, if you mean annotating a local variable with a type, then no, nothing will stop it at runtime. If you use a type checker, though, it will tell you that statically.…

Checking types on all function calls adds a considerable amount of extra work that I personally am not willing to pay, especially since static type checkers exist.

Me neither! I was just mentioning it as a possibility. My main use of beartype is `is_bearable` for runtime checking of specific data structures, in cases where `isinstance` isn't quite enough. I would still explore turning full checks during tests, though [1].

[1] https://github.com/beartype/pytest-beartype

Re: Python Type Checker Comparison: Empty Container Inference

#59

Post by maleldil »

ocamoss wrote: post #54
maleldil wrote: post #51

Earlier quoted context omitted.

Yes, that's what I was referring to. I get it that Pyrefly wanted to advertise their approach here, but it's weird that they didn't at least acknowledge this. It's what I use because it works on every type check, and I don't need to rely on their particular implementation for this. In fact, I recently migrated a project from Pyright to Pyrefly for performance reasons, and there was very little I had to change between…

It's acknowledged at the end of the "infer any" strategy, but perhaps worded poorly. > To improve type safety in these situations, type checkers that infer Any for empty containers can choose to generate extra type errors that warn the user about the insertion of an Any type. While this can reduce false negatives, it burdens developers by forcing them to explicitly annotate every empty container in order to silence t…

I missed that. At least pyright will only emit an error if `typeCheckingMode` is strict (which forbids `Unknown`). It will happily treat `Unknown` as `Any` in basic mode.
Post reply on HN