Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tests aren’t enough: Case study after adding type hints to urllib3

sethmlarson.dev

Post reply on HN
Print view View original

51–60 of 205 posts

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#51

Post by david422 »

handrous wrote: post #21
gundamdoubleO wrote: post #2

I've seen a lot of push back on adding type checking to Python but we had a similar case at my company where we tried it out on a new project and the clarity and readability of the code was immediately beneficial to the entire team. Perhaps it's something well suited to larger codebases.

I want type checking on pretty much anything that will ever exceed about two screenfuls of code. If I can't keep the whole thing in my head at once, I want the computer to do it for me. That's the point, right? Making computers do stuff for us so we don't have to?

I kindof think of them as a giant set of unit tests. The compiler/linter etc. can check every variable and every function call to check to make sure you didn't mix up your types, which _will_ blow up at runtime if you got them wrong.

So rather than write them all by hand, just get your tools to do it.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#52

Post by agumonkey »

wvenable wrote: post #34
david422 wrote: post #19

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

The main argument for dynamic typing is speed in prototyping but I find that's opposite for me. I'm much more comfortable rapid prototyping and ripping stuff apart when I have a strongly static typed environment telling me what I just broke. Doing radical refactoring often involves just making those changes and then fixing all the IDE or compiler errors until it runs again.

I used to have loads of fun abusing Eclipse real time type checker, imagining software live with typed interfaces.

The IDE was my logical buddy, and every idea's possibility was rapidly shown with it. And I need to massage things a bit, I go faster because I know what's missing.

The only time I liked eclipse/java :)

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#53

Post by munk-a »

wvenable wrote: post #34
david422 wrote: post #19

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

The main argument for dynamic typing is speed in prototyping but I find that's opposite for me. I'm much more comfortable rapid prototyping and ripping stuff apart when I have a strongly static typed environment telling me what I just broke. Doing radical refactoring often involves just making those changes and then fixing all the IDE or compiler errors until it runs again.

When I'm prototyping I tend to go inside out in a layered fashion - some days I am really feeling the data layer - other days I like to work closer to the fringes. To this end type hinting serves as a quick and dirty code contract before all my pieces are in place. I can splat out a bunch of low level definitions that I know I'm going to need and then come back the next day to add in struts - remembering my choices easily as I go.

I know this isn't the approach of choice for most folks but hey - I'm working with ADHD so I've got to make some allowances for some neurodiversity.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#54

Post by dec0dedab0de »

jfabre wrote: post #28
david422 wrote: post #19

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

I never understood this argument. In what kind of shop are you working that passing a string named person to a method expecting an object is tolerated. Or even passing different types that don't share a common interface. This would never fly in a code review in any of the companies I've worked for.

I personally love it, and wish every library worked this way. My argument is why go out of my way to make it not work, when it would be easy to make it work. This is because I think of modules/packages as user facing programs that are easy to tie together, instead of simple building blocks.

What I really wish existed was a built in way to cast and validate, or normalize and validate. I never care if something is a string. I care that if I wrap it in str(), or use it in a fstring, the result matches a regex. Or if I run a handful of functions one of them returns what I need.

The only benefit I can see of type hints on their own is it makes it easy to change a callable's signature, but I think that's best avoided to begin with.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#56

Post by tabbott »

I agree with all the benefits of mypy cited in this article. For me, most important thing for the long-term health of a codebase is its readability/maintainability, and mypy static typing makes such a huge difference for that in large Python codebases. I'm really excited to see large libraries doing this migration.

I'll add for folks thinking about this transition that we took a pretty different strategy for converting Zulip to be type-checked: https://blog.zulip.com/2016/10/13/static-types-in-python-oh-...

The post is from 2016 and thus a bit stale in terms of the names of mypy options and the like, but the incremental approach we took involved only using mypy's native exclude tooling, and might be useful for some projects thinking about doing this transition.

One particular convention that I think many other projects may find useful is how we do `type: ignore` in comments in the Zulip codebase, which is to have a second comment on the line explaining why we needed a `type: ignore`, like so:

* # type: ignore[type-var] # https://github.com/python/typeshed/issues/4234

* # type: ignore[attr-defined] # private member missing from stubs

* # type: ignore[assignment] # Apparent mypy bug with Optional[int] setter.

* # type: ignore[misc] # This is an undocumented internal API

We've find this to be a lot more readable than using the commit message to record why we needed a `type: ignore`, and in particular it makes the work of removing these with time feel a lot more manageable to have the information organized this way.

(And we can have a linter enforce that `type: ignore` always comes with such a comment).

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#57

Post by tabbott »

wvenable wrote: post #34
david422 wrote: post #19

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

The main argument for dynamic typing is speed in prototyping but I find that's opposite for me. I'm much more comfortable rapid prototyping and ripping stuff apart when I have a strongly static typed environment telling me what I just broke. Doing radical refactoring often involves just making those changes and then fixing all the IDE or compiler errors until it runs again.

I would argue that the really big benefit of dynamic typing is that it enables a really nice interactive interpreter shell experience. I think it's also important from a prototyping standpoint that Python's static typing model does a lot of inference -- you don't have to add an explicit type annotation on every single variable.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#58

Post by Zababa »

taeric wrote: post #20
m12k wrote: post #11

I've only been in the industry for ~15 years, but it still feels like every year, some ecosystem discovers the value of something that another ecosystem has taken for granted for decades - type-checking, immutability, unidirectional data-flow, AOT-compilation, closures, pure functions, you name it. I'm glad we seem to be converging on a set of best practices as an industry, but sometimes I wish we were spending less…

I've been alive long enough to see that most things are useful, and all things are oversold. More, the nice easy things to build with major restrictions pretty much gets thrown out the window for complicated things that have constraints that most efforts don't have. This isn't just a software thing. Building a little shed outside? Would be silly to use the same rigor that goes into a high rise. Which would be crazy t…

The metaphor doesn't really works, as in software lots of high rise start as a little shed.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#59

Post by Zababa »

tunesmith wrote: post #45
dado3212 wrote: post #7

Honestly will never go back to languages without type checking, it prevents so many bugs and is a huge help in understanding code you haven’t worked with previously.

I've been a types advocate for years, but it wasn't until working with Typescript that I started experiencing some of the downsides... To me, ideally, types are supposed to be a benefit not only in safety, but in understanding the intent of a piece of code more quickly. For an api or library interface, review the types to see what its intentions are. But there's something about the typescript type system, with all th…

The problem is that typescript is here to type existing JS, and existing JS has been written without thinking about types. "fresh" TS might be better for that.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#60

Post by twelve40 »

malfist wrote: post #32
jfabre wrote: post #17

Earlier quoted context omitted.

I have to agree, I've done over 5 years of C# and then went to ruby and never looked back. Static type checking raises the floor on incompetence, but also lowers the ceiling on excellence. I have to admit I don't have experience with the extremes which would be Haskell and Clojure. The amount of cruft I had to type in C# just to get shit done... It's all implicit in ruby thank god for that. I never EVER have to check…

How in the world does type checking lower the ceiling on excellence?

i'm guessing maybe sometimes type checkers make you jump through the hoops to pass, and the OP finds that distracting? To me though, the benefits of type checking far outweigh the cost.
Post reply on HN