PyAnnotate – Auto-generate type annotations for mypy • Hacker News Board

Re: PyAnnotate – Auto-generate type annotations for mypy

#31

Post by acdha » Thu, Nov 16, 2017, 8:27 PM UTC

crdoconnor wrote: post #28

acdha wrote: post #26

Earlier quoted context omitted.

Agreed — I'm just wondering about how to quantify the impact of various changes. A dynamic language project with no tests, etc. is going to look like a selling point for static typing but I suspect the real-world bug counts for, say, a Python project using mypy (or even flake8 + tests + coverage) is going to be a lot closer than you might think from how heated these discussions get.

There was a study that did a line by line translation of 4 python projects to haskell and caught some bugs (between 0 and 4 per project): http://evanfarrer.blogspot.co.uk/2012/06/unit-testing-isnt-e... I got the impression that the bugs found were either not at all serious (e.g. throwing a typeerror on malformed input instead of some other nicer kind of error) or were in areas of the code not covered by tests. Unfort…

Thanks - that's a lot like what I had in mind!

My gut feeling is that dynamic typing + tests & static analysis is faster than very heavyweight languages (e.g. Java) but probably near or less than languages with more advanced typing systems like Haskell or Rust, but I'd really like to see something more comprehensive than a subjective opinion.

Re: PyAnnotate – Auto-generate type annotations for mypy

#32

Post by yegle » Fri, Nov 17, 2017, 12:36 AM UTC

patrec wrote: post #14

yegle wrote: post #7

Earlier quoted context omitted.

(work for Google and uses pytype daily) Pytype is similar to mypy that it can do type checking with proper annotations. In addition to use annotations, pytype can also do inference based on static analysis. I don't have much experience with mypy but the last time I used it, it cannot infer from `return x == y` that the function returns a bool. Pytype can correctly infer many simple forms of function argument types an…

It's not possible to infer that the result `return x == y` is a bool, because python has rich comparisons (e.g. if x an y are numpy arrays you'll get an array of the same shape (after broadcasting) back). So either pytype uses additional information, or it's sometimes just wrong.

Technically true, since you'll need to check the return type of `__eq__()`. But the following code doesn't trigger any error using `mypy test.py --check-untyped-defs`.

  def foo():
    x = 1
    return x == 2


  def bar(x: bool):
    print(x)


  def baz(x: int):
    print(x)


  if __name__ == '__main__':
    bar(foo())
    baz(foo())

Re: PyAnnotate – Auto-generate type annotations for mypy

#33

Post by mdip » Fri, Nov 17, 2017, 1:45 PM UTC

hasenj wrote: post #8

Somewhat off topic but I think that more and more people are learning (the hard way, unfortunately) how important static typing is, and how dynamic typing makes it very difficult to develop and maintain large projects. I think the next generation of successful languages will all be statically typed (whether they will run natively or in a virtual machine is a different (even if related) question).

Have to agree with you, but with a little bit of additional context.

I get why people started rejecting statically typed languages. I work with many different languages, but started with statically typed languages with various forms of static typing (Pascal was my first, but then C and C++). Dynamic languages offer flexibility that statically typed languages lack. Most of the time, the flexibility rejected by static typing ends up being a good thing as it encourages sane code that can be analyzed by the compiler and prevent runtime bugs, so I've always accepted this sacrifice of flexibility as a feature rather than a hindrance.

However, I've found myself recently writing many solutions using TypeScript[0] and I'm finding its static type system, which allows one to configure the strictness, to be incredibly powerful. It's helped, first, by fantastic support for union types, type inference and duck typing. I find that the compiler, which is little more than a transpiler with a static analyser bolted on top of it, catches errors that I make and greatly reduces runtime WTFs while still allowing me to write terse, simple code that isn't littered with unnecessary type annotations. I add typings where either the implicit type detection can't predict the type for me or where it decreases cognitive overhead while reading code and leave them out when the opposite is true. This moves the bar back a little bit toward the flexibility side -- I can do things in code that are completely illegal in C# that result in less code, yet not increase my runtime bugs or decrease readability in the process. And if there's something truly harry that has to be done, I can tell the compiler to simply ignore the violation and give me the JavaScript output that I want regardless of what you think it's going to do.

...and I find myself longing for that type system everywhere else. So while I am still a huge proponent of statically typed languages and I don't see that changing any time, precisely how the static type system works and what features it supports is becoming very important to me.

[0] Which, considering how much I hate JavaScript, was a huge surprise considering it's basically ES6 JavaScript with optional type annotations.

Re: PyAnnotate – Auto-generate type annotations for mypy

#34

Post by yegle » Fri, Nov 17, 2017, 9:36 PM UTC

yegle wrote: post #7

ipsum2 wrote: post #2

For any dropboxers (or others), how does this compare with pytype? https://github.com/google/pytype .

(work for Google and uses pytype daily) Pytype is similar to mypy that it can do type checking with proper annotations. In addition to use annotations, pytype can also do inference based on static analysis. I don't have much experience with mypy but the last time I used it, it cannot infer from `return x == y` that the function returns a bool. Pytype can correctly infer many simple forms of function argument types an…

I think another good example of "observed types are not necessarily the intended types" is a Text parameter (unicode in python2, str in python3). The actual intent might be Iterable[Text].