Live data from Hacker News

Tests aren’t enough: Case study after adding type hints to urllib3

sethmlarson.dev

111–120 of 205 posts

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#112
post #107
post #78

Earlier quoted context omitted.

I don't really see how Java is closer to C than it is to Python in terms of what level of abstraction it's working on, could you elaborate?

It's a matter of opinion whether Java's level is closer to C or Python. But I can name a bunch of high-level features in Python that aren't in Java: List literals, dictionary literals, tuples, bigint literals, byte strings, f-strings, sequence unpacking assignment, named parameters (kwargs), decorators, closures (functions within functions), metaclasses, generators, async, list/set/dict/generator comprehensions, mult…

However Java do support dynamic typing for every object even if it is a bit cumbersome so putting it at the same level as C++ isn't accurate either.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#113
post #97

Earlier quoted context omitted.

The main argument for dynamic typing is speed in prototyping but I find that's opposite for me. I'm much more comfortable rapid prototyping and ripping stuff apart when I have a strongly static typed environment telling me what I just broke. Doing radical refactoring often involves just making those changes and then fixing all the IDE or compiler errors until it runs again.

Hm, growing somewhat experienced, I find myself adapating an old quote more and more: Sufficiently advanced static typing is indistinguishable from dynamic typing. Now, I know, it's not true. It's entirely possible to build weird things in python that are provably impossible to typecheck statically. But modern language servers and their type inference capabilities in rust, terraform, or even straight up python are ve…

Please write a type for the following function:

  def compose(start, *args):
    def helper(x):
      for func in reversed(args):
        x = func(x)
      return start(x)
    return helper
There is no mainstream typed language which can write a fully general type for the vararg compose function. TypeScript is probably the one that comes closest, but last I checked it still was unable to write a sufficiently powerful array type. You can write a type for a version of compose with a fixed number of arguments, but not for one working over an arbitrary number of arguments.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#114

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

> some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a string - which makes all sorts of convoluted code paths

Do you have hints on how to avoid being one of those 10x clever programmers while programming a prototype? I find that I am most likely to write functions like that when there's some variables that I don't want to pass 5 layers down the call stack and then, in your example, would accept either a string (in which case those variables use their default values) or the Person object, where the variables are pulled from the Person's attributes.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#115

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

> I hate having to waste time figuring out the type of every variable and hold it in my head every single time I read a piece of code. If a codebase doesn't have static types, it damn well better be set up to be highly grep-able. Including dependencies and frameworks. This is why Rails pisses me off so much. No static types to help you out, and you can't grep (can barely google, even!) methods and properties that are…

That's some Rails stupidity there, not a dynamic language problem. Autoloading symbols by name is straight up dumb.

As for greppable though...then you may as well be using a static language. The point of a dynamic language is to be dynamic, ie you can do those things at runtime.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#116

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

I've been working in Clojure for the last few years, and what I learned is that the trick is to reverse the data dependencies, so that instead of your function asking: "What is a "person" and what attributes does it have if an object?". You have your function declaring: "I take a person as a map of keys :name and :age". And it is the caller who needs to ask itself: "What am I supposed to provide to this function?"

This is a very different mindset, but once you adopt this style, the lack of static types isn't as big an issue.

The reason you can do this in a dynamic language is that you can very easily adapt one structure to another, so its okay if not all your functions work directly on the same shared structures.

It also has the advantage that this style really favors making modular independent granular components that can be reused easily, because they aren't coupled to an application's shared domain structures, but to their own set of structures, creating a natural sub-domain.

There are other aspects to make this style work well, like keeping call-stacks shallow, and having a well defined domain model at the edge of your app with good querying capabilities for it.

Concretely it means say you need to add some feature X to the code, you might think, ok this existing function is one place where I could add the behavior, but for my new feature I need to have :age of "person", but I don't know if the "person" argument of this existing function would contain :age or not. Dammit, I wish I had static types to tell me.

Well, in this scenario, instead, what you do is that you don't add the behavior to that function. Instead, in my style you would have:

    A -> B
    A -> C
instead of:

    A -> B -> C
That means if after B is the right place for your logic, you don't do:

    A -> B -> B' -> C
And hope that the "person" passed to B had the :age key which is needed by B'.

Instead you would do:

    A -> B
    A -> B'
    A -> C
And when you implement B', you don't even care about "person", you can just say you need person-age, or that you need a Person object with key :age (which you don't care if it is the Person object shared in other places or not).

Finally, you modify A, where A was the function that creates the Person object in the first place, it has direct access to your actual database/payload and so finding whatever data you need is trivial in it.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#117
post #101
post #97

Earlier quoted context omitted.

Hm, growing somewhat experienced, I find myself adapating an old quote more and more: Sufficiently advanced static typing is indistinguishable from dynamic typing. Now, I know, it's not true. It's entirely possible to build weird things in python that are provably impossible to typecheck statically. But modern language servers and their type inference capabilities in rust, terraform, or even straight up python are ve…

I don't understand your argument. Could you please explain it?

Type systems simply have matured a lot.

It's not too long ago that you either had very clumsy type systems - C, Java. These type systems were more of a chore than anything else. Especially the generic transition in java was just tedious, you had to type cast a lot of stuff, and the compiler would still yell at you, and things would still crash.

Or you had very powerful and advanced type systems - Haskell and C++ with templates for example. However, these type systems were just impenetrable. C++ template errors before clang error messages are something. They are certainly an error message. But fixing those without a close delta what happened? Pfsh. Nah.

In those days, dynamic typing was great. You could shed the chore of stupid types, and avoid the really arcane work of making really strong types work.

However, type systems have matured. Today, you can slap a few type annotations on a python function and a modern type inference engine can give you type prediction, accurate tab-completion and errro detection. In something like rust, you define a couple of types in important locations and everything else is inferred.

This in turn gives you the benefit of both: You care about types in a few key locations, but everything else is as simple as a dynamically typed language. And that's when statically typed languages can end up looking almost - or entirely - like a dynamically typed language. Except with less error potential.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#118
post #109
post #78

Earlier quoted context omitted.

I don't really see how Java is closer to C than it is to Python in terms of what level of abstraction it's working on, could you elaborate?

And at what level of abstraction does ISO C work on? "C Is Not a Low-level Language, Your computer is not a fast PDP-11." https://queue.acm.org/detail.cfm?id=3212479

That would be a good point if you actually could write the final code for CPU's, but you can't since the CPU internals do that for you. So from an application programmers perspective machine code is as low as it gets and C maps really well to machine code so C is a low level language.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#119
post #97

Earlier quoted context omitted.

Hm, growing somewhat experienced, I find myself adapating an old quote more and more: Sufficiently advanced static typing is indistinguishable from dynamic typing. Now, I know, it's not true. It's entirely possible to build weird things in python that are provably impossible to typecheck statically. But modern language servers and their type inference capabilities in rust, terraform, or even straight up python are ve…

Please write a type for the following function: def compose(start, *args): def helper(x): for func in reversed(args): x = func(x) return start(x) return helper There is no mainstream typed language which can write a fully general type for the vararg compose function. TypeScript is probably the one that comes closest, but last I checked it still was unable to write a sufficiently powerful array type. You can write a t…

As I said, there are infinite possible programs that cannot be type checked statically. This is derived from the halting theorem trivially - lambda(p) = if p.halts() then A() else B().

However, the intersection of programs I encounter in practice with the number of programs that can be statically checked is rather large.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#120

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

I would argue that any function that branches on argument type is straight up doing dynamic typing wrong. Well branching may not be the right word. Something resembling pattern maching is fine, but like you say having a function that takes a string for lookup OR the object is just a disaster, particularly when you start stacking function calls. Dynamic types should closer resemble things that all share an interface, not totally different representations of that data based on the shape of your code.

Javascript is by far the worst offender here with its ignoring extra arguments. Javascript functions that totally change effective type signatures based on number of args are the devil's work.

I'd argue that if the types that a function accepts are not easily defineable than you're doing dynamic typing wrong.

Post reply on HN