Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tests aren’t enough: Case study after adding type hints to urllib3

sethmlarson.dev

Post reply on HN
Print view View original

141–150 of 205 posts

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#141

Post by nitrogen »

blacktriangle wrote: post #115
handrous wrote: post #39

Earlier quoted context omitted.

> I hate having to waste time figuring out the type of every variable and hold it in my head every single time I read a piece of code. If a codebase doesn't have static types, it damn well better be set up to be highly grep-able. Including dependencies and frameworks. This is why Rails pisses me off so much. No static types to help you out, and you can't grep (can barely google, even!) methods and properties that are…

That's some Rails stupidity there, not a dynamic language problem. Autoloading symbols by name is straight up dumb. As for greppable though...then you may as well be using a static language. The point of a dynamic language is to be dynamic, ie you can do those things at runtime.

The point of a dynamic language is to be dynamic, ie you can do those things at runtime.

With Rails you have the option of pry-rails, and you can get a list of descendants of important parent classes like ActiveRecord with this: https://apidock.com/rails/Class/descendants

With the combination of vim, rspec, pry, fzf, and ripgrep, it's possible to become quite comfortable refactoring pure Ruby and Ruby+Rails code. But it does take some time to learn how to navigate the Rails runtime code generation magic. The more magic the code, the more you might have to use a debugger to break on method definition, but Ruby's dynamicism lets you do that.

On the topic of frameworks with a lot of magic, having used both Rails and Spring Boot (with Java and Kotlin), I'll take Rails any day. It was way easier to introspect Rails codegen magic with Pry, than Spring's codegen magic with IntelliJ. With Spring Boot, even with Kotlin, we had the burden of semi-manual typing, but lost a lot of the benefits because a lot of DB interaction and API payload handling was still only runtime checked.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#142

Post by merb »

wvenable wrote: post #124
Jensson wrote: post #108

Earlier quoted context omitted.

> Sufficiently advanced static typing is indistinguishable from dynamic typing. Static typing type checks are compile time, dynamic typing doesn't. I don't see how these two could be indistinguishable, in one you can't run the program with type errors, in the other you can.

I believe the author was talking about the act of writing the software. Modern type inference means that you can mostly code without needing to write down the types in many cases. This line of code is the same in JavaScript or C#: var instance = new SomeClass(); In function definitions, where you are definitely going to need to provide parameter types it's extremely common to document those types in a docblock in a d…

c# even has target-typed expressions which might be useful aswell, i.e. instead of var instance = you write SomeClass instance = new(); this might be preferable so that you have all types on the left.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#143

Post by jayd16 »

nayuki wrote: post #107
DylanSp wrote: post #78

Earlier quoted context omitted.

I don't really see how Java is closer to C than it is to Python in terms of what level of abstraction it's working on, could you elaborate?

It's a matter of opinion whether Java's level is closer to C or Python. But I can name a bunch of high-level features in Python that aren't in Java: List literals, dictionary literals, tuples, bigint literals, byte strings, f-strings, sequence unpacking assignment, named parameters (kwargs), decorators, closures (functions within functions), metaclasses, generators, async, list/set/dict/generator comprehensions, mult…

C# has most of this stuff and I think Java has at least a few of these but I wouldn't say any of these are really much of a differentiator.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#144

Post by nitrogen »

angelzen wrote: post #82
david422 wrote: post #19

I love static typing/type hints if for only 1 thing - code maintenance. Even code I wrote six months ago. Not having to dig through 6 functions deep to try to figure out whether "person" is a string, or an object, and if it's an object what attributes it has on it etc. is huge. And not to mention that some clever people decide - hey, if you pass a string I'll look up the person object - so you can pass an object or a…

This is doubly true as experienced programmers argue that designing the data structures is the hardest part of coding. Code follows semi-automatically.

I'd add data flows as another level above data structures. It helps to think about how data flows into, through, and out of a system, then it's more clear how the data needs to be packaged, and from there, the code follows semi-automatically.

Tangentially related: I think it'd be cool if there was a development environment that combined a node-based dataflow editor with normal text editing, so pure plumbing could be implemented visually, but embedded within (and translated to) textual code.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#145

Post by magicalhippo »

bmm6o wrote: post #121
jfabre wrote: post #64

Earlier quoted context omitted.

Tbh I'm conflating multiple things, I've heard a lot of good things about Haskell. But in C# for example, if the system was not designed with dependency injection and everything being an interface it's very hard to build a test harness since you can't mock anything. Which means everything has to be tested manually. (I haven't done any C# in a long time, maybe it's not the case anymore) So you have to create an interf…

> But in C# for example, if the system was not designed with dependency injection and everything being an interface it's very hard to build a test harness since you can't mock anything. Which means everything has to be tested manually. I wouldn't have put it like that, but I think I know what you mean. Mocks for unit testing do require that you have defined an interface to implement, which means that every class that…

> [...] which means that every class that you want to be mocked out needs to have an interface extracted. It is extra work.

If you define the interface first, then you can simply copy-paste that into the class definition and off you go implementing it. Hardly any more work at all.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#146

Post by SubjectToChange »

Jensson wrote: post #118
pjmlp wrote: post #109

Earlier quoted context omitted.

And at what level of abstraction does ISO C work on? "C Is Not a Low-level Language, Your computer is not a fast PDP-11." https://queue.acm.org/detail.cfm?id=3212479

That would be a good point if you actually could write the final code for CPU's, but you can't since the CPU internals do that for you. So from an application programmers perspective machine code is as low as it gets and C maps really well to machine code so C is a low level language.

> So from an application programmers perspective machine code is as low as it gets and C maps really well to machine code so C is a low level language.

C only "maps really well" to PDP-11 style machine code. If you want SIMD, parallel algorithms, heterogeneous programming, memory hierarchies/domains, etc then ISO C is completely useless.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#147

Post by magicalhippo »

fiddlerwoaroof wrote: post #69
arwhatever wrote: post #9

Earlier quoted context omitted.

Is it possible to elaborate in a comment? Honestly I probably wouldn’t take the time to read a lengthy article, but if there’s some elevator pitch then I’m all ears.

What makes CL/Clojure really work is that your editor (emacs usually, but there’s other options now) connects to the live program and has access to the entire runtime environment. So, you can do a lot of the things other languages need static types for via introspection (e.g. autocomplete: CL just asks the running program what functions are available that matches the current pattern and returns a list). Secondly, sin…

> This page surveys a bunch of studies on the question of dynamic v. static types and finds the evidence in favor of static types to be surprisingly small

Most of the studies seem to be rather poor though, so difficult to draw any solid conclusions from them. Almost all seem to drown in noise, or have flawed setups.

From personal experience, with a static type language I can jump into an unknown codebase and make non-trivial modifications much, much faster than if it's a dynamic type language codebase.

I've wasted soooo many hours doing print(dir(x)) in Python it's far beyond funny.

On the flip side, over the years I've helped countless people with their C/C++/Delphi code in minutes, frequently using libraries and API's I've never seen before.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#148

Post by fiddlerwoaroof »

magicalhippo wrote: post #147
fiddlerwoaroof wrote: post #69

Earlier quoted context omitted.

What makes CL/Clojure really work is that your editor (emacs usually, but there’s other options now) connects to the live program and has access to the entire runtime environment. So, you can do a lot of the things other languages need static types for via introspection (e.g. autocomplete: CL just asks the running program what functions are available that matches the current pattern and returns a list). Secondly, sin…

> This page surveys a bunch of studies on the question of dynamic v. static types and finds the evidence in favor of static types to be surprisingly small Most of the studies seem to be rather poor though, so difficult to draw any solid conclusions from them. Almost all seem to drown in noise, or have flawed setups. From personal experience, with a static type language I can jump into an unknown codebase and make non…

Yeah, the evidence here is mostly anecdotal but, while we’re trading anecdotes, I think you have to distinguish Smalltalk/Clojure/Common Lisp from other dynamic languages. Most dynamic language essentially work like statically-typed languages without typechecking: you put code in a file and then run it all at once (or run unit tests) and see what happens. The languages I mention actually bring your development environment to runtime (twisted manhole and pry are the closest things I can think of here) so, you don’t have to run the whole thing, you can just run the parts you care about and see what they do.

That being said, my experience isn’t the same: I’ve been able to make helpful changes to dynamically-typed codebases in roughly the same amount of time as to static codebases. I’ve never really identified what it is about how I approach code that makes a difference here, but I think it is because I think about changes in terms of operational equivalence (e.g. l.map(a).map(b) === l.map(compose(b, a)) ) rather than in terms of data types.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#149

Post by kennywinker »

freemint wrote: post #49
kennywinker wrote: post #37

Earlier quoted context omitted.

I've seen essentially this code in so many organically grown codebases (when they grew up without types). It's usually close the the UI, because someone had to quickly add an alternate path to support some new user interaction function find_user(person) { if user is string { query_by_name(person) } else { query_by_name(person.name) } } and yeah, we all know it's kinda messy, but also that logic has to live somewhere…

Sounds like a brilliant case for multiple-dispatch.

Right so we have:

    function find_user(person: string)
and also:

    function find_user(person: Object)
how long before someone writes this:

    find_user(person: { name: "dave" })
meanwhile, someone else, not suspecting that they'll be handed a weird half-formed `User` object adds `person.id` somewhere in the body of the Object version of `find_user` and now we have a weird edge-case where very rarely `find_user` panics because the user object we're handed doesn't have an id??? Great, I just lost an hour trying to dig that out of the logs, and the users are starting to think of the product as flakey because the bug has been in prod for over a month before we finally believed them enough to look into it.

Just. Use. Types. Multiple dispatch won't save you on its own. You NEED compile-time types.

Re: Tests aren’t enough: Case study after adding type hints to urllib3

#150

Post by magicalhippo »

fiddlerwoaroof wrote: post #148
magicalhippo wrote: post #147

Earlier quoted context omitted.

> This page surveys a bunch of studies on the question of dynamic v. static types and finds the evidence in favor of static types to be surprisingly small Most of the studies seem to be rather poor though, so difficult to draw any solid conclusions from them. Almost all seem to drown in noise, or have flawed setups. From personal experience, with a static type language I can jump into an unknown codebase and make non…

Yeah, the evidence here is mostly anecdotal but, while we’re trading anecdotes, I think you have to distinguish Smalltalk/Clojure/Common Lisp from other dynamic languages. Most dynamic language essentially work like statically-typed languages without typechecking: you put code in a file and then run it all at once (or run unit tests) and see what happens. The languages I mention actually bring your development enviro…

This is actual JavaScript code, from one of my projects:

    function processAudioData(data, callback) {
       // dump audio data to WAV file  
    }
It's implementing a callback from a library.

Even reading the source code of the library I had problems figuring this one out. Had it been say C# code I'm pretty certain I would have had it done in seconds.

How do you solve this in seconds? I'm genuinely curious as this is something I often struggle with when having to use say Python or JavaScript.

Post reply on HN