Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tests aren’t enough: Case study after adding type hints to urllib3

sethmlarson.dev

Post reply on HN
Print view View original

171–180 of 205 posts

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#171

Post by tehnub »

kzrdude wrote: post #50
tester756 wrote: post #4

Earlier quoted context omitted.

It's $current_year and there's still debate whether checking stuff at compilation time is better than at runtime?

That's not really the debate in Python :) Almost every Python user now has to "deal" with type annotations. It's tempting to gradually add type annotations, it's nice documentation. But it also rubs me the wrong way to have annotations that are never checked(!). In many codebases, you might just have "casual" style type annotations in Python, and nothing ever asserts that they hold. That's nagging on me, a bit.

That's a good point. If they're never checked, then they're just like incorrect/outdated comments. They sort of get at this idea in the article, and sort of describe a compromise for it. They have a list of files that they've completely annotated, and only those files are checked by mypy. So in their case, they know which annotations to ignore, and which they can rely on.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#172

Post by Matheus28 »

skrimp wrote: post #152
slaymaker1907 wrote: post #113

Earlier quoted context omitted.

Please write a type for the following function: def compose(start, *args): def helper(x): for func in reversed(args): x = func(x) return start(x) return helper There is no mainstream typed language which can write a fully general type for the vararg compose function. TypeScript is probably the one that comes closest, but last I checked it still was unable to write a sufficiently powerful array type. You can write a t…

C++ is an extremely mainstream language that can write a fully general version of compose with variable arguments. https://godbolt.org/z/h7n8Y7qf1 Like sure, you can't write out a type for the entire overload set. Overload sets don't have types, but functions do. However, I don't think you'd ever actually want to write out the type of the compose function. Instead, I think it would be more reasonable to request that…

Great implementation of what the parent comment asked. Not only that, but the compiler managed remove all the abstractions.

(very minor nitpick: I'd pick `auto&& x` over `auto x`)

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#173

Post by Akronymus »

pansa2 wrote: post #130
adrianmonk wrote: post #65

Earlier quoted context omitted.

Consider these 4 possible combinations for programming languages: (1) Low-level, static types (2) Low-level, dynamic types (3) High-level, static types (4) High-level, dynamic types For whatever reason, historically #1 and #4 have been most popular. C, C++, Pascal, Ada, and Java are #1. Python, JavaScript, Perl, and BASIC are #4. There haven't been a lot of #2 or #3 languages. Some #3 languages (TypeScript and Python…

Exactly. IMO we still don't have a good #3 language - in particular, all of the popular languages that can be compiled into native code are in category #1.

I'd say that f# haskell ocaml and such labguages fall under #3.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#174

Post by Akronymus »

fiddlerwoaroof wrote: post #8
novok wrote: post #6

Earlier quoted context omitted.

People who don't think types are a good thing need to work in a statically typed language for a year or two and then see what a difference it makes in reality. Unproductive Java bureaucracy != static typing. I think the people debating it never tried it seriously.

I’ve done everything from Haskell to Java and I still strongly prefer Clojure and Common Lisp-style dynamic types.

LISPS are the only languages where I feel like dynamic typing works well.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#175

Post by IshKebab »

tunesmith wrote: post #158
IshKebab wrote: post #98

Earlier quoted context omitted.

Your criticism boils down to "it's possible to overcomplicate things". Sure if you completely remove static typing then you can't overcomplicate static typing. But is that really an argument against static typing?

No, and I am still in favor of static typing. But I also don't think it's purely up to team discipline. Something about typescript (or more aptly, javascript) incentivizes crazy typing more than other languages. Luckily, the last few years appears to have had more emphasis on designing languages to take these kinds of incentives into account, so I still think the future is bright.

> Something about typescript (or more aptly, javascript) incentivizes crazy typing more than other languages.

Definitely not more than other languages. Check out template metaprogramming in C++ or some of the OTT generics in Rust.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#176

Post by andybak »

layer8 wrote: post #83
andybak wrote: post #75

Having spent a decade with Python and more recently a few years with C# I still can't quite put my feelings into words but here's an attempt: "The benefits of explicit typing are obvious and clear but they downsides are subtle and hard to communicate" I still think typing in general is a net win but I'm not sure whether static typing is. You find yourself writing code that just wouldn't be neccesary in a dynamic lang…

Types are effectively assertions about the values they represent, and statically-typed code constitutes proofs that the assertions actually hold at runtime. The static typing forces you to be sufficiently rigorous in those proofs, which may require additional code as you mention. Without static typing, one has to rely on the "proofs" in one’s head to be correct (which humans aren’t really good at), instead of having…

I think this falls into the category of "The benefits of explicit typing are obvious and clear". It's the other side of the equation that I'm intrigued by and struggling most to formulate.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#177

Post by nybble41 »

slaymaker1907 wrote: post #113
tetha wrote: post #97

Earlier quoted context omitted.

Hm, growing somewhat experienced, I find myself adapating an old quote more and more: Sufficiently advanced static typing is indistinguishable from dynamic typing. Now, I know, it's not true. It's entirely possible to build weird things in python that are provably impossible to typecheck statically. But modern language servers and their type inference capabilities in rust, terraform, or even straight up python are ve…

Please write a type for the following function: def compose(start, *args): def helper(x): for func in reversed(args): x = func(x) return start(x) return helper There is no mainstream typed language which can write a fully general type for the vararg compose function. TypeScript is probably the one that comes closest, but last I checked it still was unable to write a sufficiently powerful array type. You can write a t…

This is really a fairly trivial exercise in Haskell, provided you pass the arguments as a heterogeneous list—which is semantically equivalent to a variable argument list. Here is my implementation: https://gist.github.com/nybble41/c459c6927a3bad8ec350d227193...

Here I defined a simple `Pipeline` GADT for the argument list, which is just a list of functions with some extra type constraints to ensure that they can be composed. You could do the same thing with a more general type like HList but the type signature for the `compose` function would be much more verbose since you would need to define the relationships between each pair of adjacent function types through explicit constraints involving type families, whereas the `Pipeline` type handles that internally.

Perhaps you don't consider Haskell "mainstream" enough?

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#178

Post by nybble41 »

dec0dedab0de wrote: post #54
jfabre wrote: post #28

Earlier quoted context omitted.

I never understood this argument. In what kind of shop are you working that passing a string named person to a method expecting an object is tolerated. Or even passing different types that don't share a common interface. This would never fly in a code review in any of the companies I've worked for.

I personally love it, and wish every library worked this way. My argument is why go out of my way to make it not work, when it would be easy to make it work. This is because I think of modules/packages as user facing programs that are easy to tie together, instead of simple building blocks. What I really wish existed was a built in way to cast and validate, or normalize and validate. I never care if something is a st…

> why go out of my way to make it not work, when it would be easy to make it work

The problem the DWIM approach to APIs is that when you go out of your way to "do something reasonable" with absolutely any kind of argument type, leaving the caller's intent implicit, you will sometimes run into combinations that "work" in unexpected—and often unwanted—ways.

For example, say you have a function which returns either a Person object or, in very rare cases, an error string. Moreover, you fail to check for the error string, and pass the result into another function which expects a Person object but will also take a name and look up the corresponding Person object in a table. Now if the first function fails you're left trying to look up an error string as a name, with no obvious signs (such as a type mismatch error) to show that anything is amiss.

It's important to make the intent explicit, and not just let the function guess. One option compatible with both statically- and dynamically-typed languages is to provide two functions, one requiring a Person object and another taking a name string. This is still perfectly ergonomic for the user and mitigates most of the potential for confusion.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#179

Post by native_samples »

tehnub wrote: post #166
phoneperson wrote: post #138

Earlier quoted context omitted.

You are probably thinking of Gödel's incompleteness theorems. I do not think that you can trivially derive it from the halting problem.

He just did derive it from the halting problem. In words: It's impossible to statically check the type of a function that takes a program P as its input, and returns the integer 1 as its output if P halts, and returns the string "1" if it does not halt. It would require solving the halting problem, which is undecidable.

How does that follow? The input type would be vague, e.g. ByteArray or Program or something.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#180

Post by madeofpalk »

jfabre wrote: post #28
david422 wrote: post #19

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

I never understood this argument. In what kind of shop are you working that passing a string named person to a method expecting an object is tolerated. Or even passing different types that don't share a common interface. This would never fly in a code review in any of the companies I've worked for.

We have tests, and static types, because developers are people and people make mistakes.

You can't say "we simply don't allow bugs!" because it's a lie. Why rely on a another person manually checking for silly mistakes when the computer can do it for you?

Post reply on HN