Python: Even a Feature That You Do Not Use Can Bite You • Hacker News Board

Re: Python: Even a Feature That You Do Not Use Can Bite You

#41

Post by maxander » Sat, Feb 23, 2019, 6:01 PM UTC

goodside wrote: post #8

Understanding the basic syntax for assignment isn’t a high bar to clear. While it’s maybe not ideal to use colons for both the “assignment-like” separator between keys and values inside dictionaries and the annotation of types, introducing any new syntax features into a language that’s been around for decades is never without compromise. Colons are also used at the end of block statements and in index ranges and nobo…

There is a way to avoid this kind of problem; have sufficiently complicated syntax that most random permutations caused by "fat-fingering" are incorrect. Python's problem is that the syntax is so minimalist, and the semantics so rich, that any random typo is liable to do something.

For what it's worth, I think Python's approach here is the right one (at least, for Python.) As others have noted, the common answer to this among Python users is to just use a linter to catch these kinds of issues, although I find that unsatisfying- linters can't catch everything, and they require you to have your stuff together well enough to use them (e.g., you can't be a student who is already overwhelmed with the bare necessary stack.) There are things that seem like they could be done to alleviate these kinds of problems within the language itself; most obviously, for the particular problem in the OP, a variable annotation expression could require the right-hand to be a type expression. It's not clear to me why these kinds of moves aren't taken.

Re: Python: Even a Feature That You Do Not Use Can Bite You

#42

Post by Pxtl » Sat, Feb 23, 2019, 6:02 PM UTC

masklinn wrote: post #26

Pxtl wrote: post #19

Earlier quoted context omitted.

The question, then, is why that kind of analysis isn't built into the compiler?

> Despite considerable discussion about a standard type parameterisation syntax, it was decided that this should also be left to third-party libraries. ([7], [8], [9]). > Despite yet more discussion, it was decided not to standardize a mechanism for annotation interoperability. Standardizing interoperability conventions at this point would be premature. We would rather let these conventions develop organically, based…

That doesn't sound really "batteries included" to me.

Re: Python: Even a Feature That You Do Not Use Can Bite You

#43

Post by painful » Sat, Feb 23, 2019, 6:14 PM UTC

Pxtl wrote: post #42

masklinn wrote: post #26

Earlier quoted context omitted.

> Despite considerable discussion about a standard type parameterisation syntax, it was decided that this should also be left to third-party libraries. ([7], [8], [9]). > Despite yet more discussion, it was decided not to standardize a mechanism for annotation interoperability. Standardizing interoperability conventions at this point would be premature. We would rather let these conventions develop organically, based…

That doesn't sound really "batteries included" to me.

Is there a single programming language for which static analyzers are a part of the language rather than external tools?

Re: Python: Even a Feature That You Do Not Use Can Bite You

#44

Post by lukeschlather » Sat, Feb 23, 2019, 6:21 PM UTC

goodside wrote: post #34

masklinn wrote: post #25

Earlier quoted context omitted.

The problem is not understanding, it's fat-fingering (especially when dict literals and dict comps use `:`). > type annotations just delayed your runtime error by one line. Or it introduces a subtle bug because this was intended as a reassignment or somesuch.

The real problem, as others point out, is not using a linter to catch errors before runtime. Mypy flags this problem through static analysis. You’re right I’m being uncharitable by saying “understanding”. I’m annoyed he bothered writing an accusatory blog post about this aspect of the language without mentioning any of the alternative syntax proposals helpfully discussed in the PEP, or exploring why annotations work…

> Mypy flags this problem through static analysis.

The problem is that there's not really such a thing as static analysis in python. For example, if I wrap it in a function like this, mypy doesn't complain at all.

    def blah():
        ages = {}
        ages['John']: 42
        print(ages['John'])
        return ages

Re: Python: Even a Feature That You Do Not Use Can Bite You

#45

Post by goodside » Sat, Feb 23, 2019, 7:58 PM UTC

lukeschlather wrote: post #44

goodside wrote: post #34

Earlier quoted context omitted.

The real problem, as others point out, is not using a linter to catch errors before runtime. Mypy flags this problem through static analysis. You’re right I’m being uncharitable by saying “understanding”. I’m annoyed he bothered writing an accusatory blog post about this aspect of the language without mentioning any of the alternative syntax proposals helpfully discussed in the PEP, or exploring why annotations work…

> Mypy flags this problem through static analysis. The problem is that there's not really such a thing as static analysis in python. For example, if I wrap it in a function like this, mypy doesn't complain at all. def blah(): ages = {} ages['John']: 42 print(ages['John']) return ages

Mypy intentionally does not check annotations inside untyped `def` statements. This is a feature meant to make it easier to gradually introduce types in code that wasn't intended to be typechecked. If you call mypy with the flag `--check-untyped-defs` or just annotate the function with anything at all, you will see mypy checks the inside of the function:

    from typing import Any

    def blah() -> Any:
        ...

I'm not sure what you mean by "there's not really such a thing as static analysis" in Python -- that's exactly what Mypy does.

Re: Python: Even a Feature That You Do Not Use Can Bite You

#46

Post by goodside » Sat, Feb 23, 2019, 8:11 PM UTC

maxander wrote: post #41

goodside wrote: post #8

Understanding the basic syntax for assignment isn’t a high bar to clear. While it’s maybe not ideal to use colons for both the “assignment-like” separator between keys and values inside dictionaries and the annotation of types, introducing any new syntax features into a language that’s been around for decades is never without compromise. Colons are also used at the end of block statements and in index ranges and nobo…

There is a way to avoid this kind of problem; have sufficiently complicated syntax that most random permutations caused by "fat-fingering" are incorrect. Python's problem is that the syntax is so minimalist, and the semantics so rich, that any random typo is liable to do something . For what it's worth, I think Python's approach here is the right one (at least, for Python.) As others have noted, the common answer to…

> a variable annotation expression could require the right-hand to be a type expression. It's not clear to me why these kinds of moves aren't taken

Because when annotations were introduced in PEP 3107 they were not meant exclusively for type hinting. We're in a period now where non-type usage is deprecated but allowed for compatibility with old code. In 3.8 a non-type evaluation of the RHS will be disallowed.

Re: Python: Even a Feature That You Do Not Use Can Bite You

#47

Post by lukeschlather » Sun, Feb 24, 2019, 12:00 AM UTC

goodside wrote: post #45

lukeschlather wrote: post #44

Earlier quoted context omitted.

> Mypy flags this problem through static analysis. The problem is that there's not really such a thing as static analysis in python. For example, if I wrap it in a function like this, mypy doesn't complain at all. def blah(): ages = {} ages['John']: 42 print(ages['John']) return ages

Mypy intentionally does not check annotations inside untyped `def` statements. This is a feature meant to make it easier to gradually introduce types in code that wasn't intended to be typechecked. If you call mypy with the flag `--check-untyped-defs` or just annotate the function with anything at all, you will see mypy checks the inside of the function: from typing import Any def blah() -> Any: ... I'm not sure what…

Cool, I knew I had to be missing something. You're right of course.

Re: Python: Even a Feature That You Do Not Use Can Bite You

#48

Post by msluyter » Sun, Feb 24, 2019, 3:41 AM UTC

heavenlyblue wrote: post #35

msluyter wrote: post #32

It seems odder to me that 42 is a valid type hint than that the LHS can be an expression. But I share with most of the others here the opinion that this really isn’t a problem with Python. Every language makes trade offs, and python gives you incredible flexibility and expressiveness in exchange for a lack of certain guardrails. Once a coworker came to me with a program that was unexpectedly failing, and it turned ou…

PyCharm would highlight that error as a yellow warning if you correctly define the types of the arguments for "something". This would also not be accepted by mypy.

Iirc this was pre type hints, so the example might be invalid now, but I’m sure I could come up with other examples that demonstrate the same point.