Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Analysis of whether unit testing obviates static type checking (2012)

evanfarrer.blogspot.com

Post reply on HN
Print view View original

41–50 of 65 posts

Re: Analysis of whether unit testing obviates static type checking (2012)

#41

Post by seanmcdirmid »

steve_adams_86 wrote: post #33

I’m surprised anyone thinks types make unit tests redundant. I can know a function returns a value adhering to the correct interface, sure, but that doesn’t tell me if the values are the correct ones given specific inputs. These are different things. I do think a type system dramatically reduces the size and complexity of unit tests, though. I don’t miss writing tests to check types.

It could happen given a sufficiently powerful type system…a type system no one would probably want to use (given that you would write more type-finessing code than real code, you might as well be programming with a theorem prover).

Re: Analysis of whether unit testing obviates static type checking (2012)

#42

Post by kelnos »

p1necone wrote: post #11

I'd be much more interested in needing less unit tests with stronger typing than avoiding needing stronger typing by writing more unit tests. The former seems like much less work.

Came here to say exactly this. Tests are more code that has to be maintained, and more code that can be wrong. I'm sure we've all encountered a test that didn't actually test what it was supposed to test, or wouldn't actually fail in the cases it was supposed to.

Obviously a compiler can't prove your code is correct in every possible way, but any time the compiler can prove something correct without me needing to write a test for it, that's a win. Stronger type systems and static type checking is a great way to do that.

Re: Analysis of whether unit testing obviates static type checking (2012)

#43

Post by kelnos »

steve_adams_86 wrote: post #33

I’m surprised anyone thinks types make unit tests redundant. I can know a function returns a value adhering to the correct interface, sure, but that doesn’t tell me if the values are the correct ones given specific inputs. These are different things. I do think a type system dramatically reduces the size and complexity of unit tests, though. I don’t miss writing tests to check types.

Types make some tests redundant. I've seen unit tests in Python code bases that were simply there to make sure that a function did something sane if a value of the wrong type was passed to it.

I think it's absolutely certain that, to achieve the same level of confidence in your code, you need more unit tests for, say, a Python code base, than for a Java code base.

Re: Analysis of whether unit testing obviates static type checking (2012)

#44

Post by kelnos »

efarrer wrote: post #36
wzdd wrote: post #7

The blog sets up to address the "claim by proponents of dynamically typed programming languages" that "once you have unit testing static type checking is redundant". This is a strawman and I'm fairly sure it was a strawman in 2012 as well. The more interesting claim, which this blog post doesn't address, is that dynamic typing plus unit testing lets you produce working programs faster, without having to wrestle a typ…

Hi I'm the author of the blog post and the paper. In the paper I provide a reference to such a claim by proponents of dynamic typing. From my paper: "Because some error detection can be done by both unit testing and static type checking, some proponents of dynamic type checking claim that static type checking is not needed [3]." and the reference: [3] J. Spolsky and B. Eckel, “Strong Typing vs. Strong Testing,” in Th…

> I do agree that development time is an important factor

In my experience, development time is much less of an important factor than management would like to think it is. I've seen companies burn customer goodwill by releasing a quickly-built, bug-ridden product too early, because they wanted to be first to market or whatever.

And I've also found that the companies that push for faster releases (without reducing scope, of course) end up shipping late anyway. If they'd accepted a later deadline in the first place, fewer coding mistakes would end up getting made along the way. (Stressed-out developers rushing toward an unachievable deadline make more mistakes than those whose time estimates are listened to.)

I know I don't represent all developers, but I do much better when I write in a "slower" language when that language's compiler verifies more things before we get to the point of running the code. The end product is more stable and more correct than what it'd be otherwise, and I don't think I deliver slower in a way that's significant to the business.

Re: Analysis of whether unit testing obviates static type checking (2012)

#45

Post by kelnos »

wzdd wrote: post #7

The blog sets up to address the "claim by proponents of dynamically typed programming languages" that "once you have unit testing static type checking is redundant". This is a strawman and I'm fairly sure it was a strawman in 2012 as well. The more interesting claim, which this blog post doesn't address, is that dynamic typing plus unit testing lets you produce working programs faster, without having to wrestle a typ…

> The more interesting claim, which this blog post doesn't address, is that dynamic typing plus unit testing lets you produce working programs faster

My feeling is that this is correct, but with a caveat big enough to drive a truck through: for often very different definitions of "working".

In a language with a strong type system, I can usually get something working (correctly!) fairly quickly with very few (and sometimes no) unit tests at all. In a dynamically-typed language, or one with a weak type system, unit tests quickly become essential to avoid simple errors that end up causing abnormal execution termination.

> without having to wrestle a type system for things you don't care about (e.g. malformed input leading to crashes when parsing machine-generated, standardised inputs, as is the case for three of the test programs

I've found that it's quite often that someone thinks a particular condition can't be triggered, when in fact it can be. Data gets corrupted, through coding errors, bad data entry, and many other things. And when you really do know that something can't ever happen (or have decided that the right thing to do is to terminate in that case anyway), most statically-typed languages have escape hatches, like `Option.unwrap()` in Rust.

> keeping in mind that, in Python, a crash due to a type error produces an orderly, if surprising, program exit rather than, for example, memory corruption

Most statically-typed languages out there will also abort cleanly if you do manage to trigger a type error. Languages like C & C++ are the exception here, not the rule. (Not to mention a type error is much much much less likely in a statically-typed language, so this is kinda a weird comparison.)

Re: Analysis of whether unit testing obviates static type checking (2012)

#46

Post by Tade0 »

I was part of a team maintaining 150k lines of ancient JavaScript. Doing this requires careful engineering, but is otherwise possible, though not recommended. A good friend of mine is still there.

I find conversations with him enlightening because there does not exist a mainstream type system which can clearly an concisely express what he's doing there - even though these are fairly mundane things in dynamically typed languages, like arguments with correlated types.

I'm a firm believer in exercise as a means of maintaining skill, so I've been doing a hobby project in vanilla JavaScript. Of course I make type errors, but they're usually the sort that makes me go "aw shit, this field is wrong" and move on. Meanwhile I relearned some characteristics of this environment, which are:

-Tests are mandatory.

-You have to be radically explicit, otherwise you lose track of what you've been doing.

-Documentation is the cornerstone of remaining on track in the long run.

-The biggest mistakes are not in the types, but, unsurprisingly, the logic behind the code.

-General advice, like short-circuiting edge cases, using pure functions and immutable data structures still applies.

-As per Grug (https://grugbrain.dev/) - "grug very like type systems make programming easier. for grug, type systems most value when grug hit dot on keyboard and list of things grug can do pop up magic. this 90% of value of type system or more to grug" - wise tell this, me concur

Some interesting features of the codebase so far:

-There's considerably less code than in my imagined type-annotated version.

-I still use JSDoc to label inputs and outputs.

-I'm sort of cheating, because I rely heavily on classes, which have the `myObject.constructor.name` property, which in turn serves as runtime type information.

-Data structures are flatter - I have e.g. an array (named "path") where even indexes are strings, odd - numbers. There might be a way in TypeScript to express this(and there's definitely one in some über "strongly" typed language that compiles to JS), but my experience is that finding this out would be the perfect nerd snipe for me. I could have an array of objects with proper fields, but these are not initialized in the same place and a "path" ending in a string is also valid, so I would have to deal with the bureaucracy of optional fields.

---

Overall you probably don't want to do the same in a team - unless you're doing a group exercise and this is not code meant for production. Static types are to me a whip for those who take the path of least resistance - every developer has such moments, some just more frequently than others. That being said you might want to occasionally try working without the whip cracking above you and see what you create then - my guess is that if you don't, you'll miss out.

Re: Analysis of whether unit testing obviates static type checking (2012)

#47

Post by chriswarbo »

xupybd wrote: post #15
pfdietz wrote: post #8

Earlier quoted context omitted.

Not that unit testing is enough, but the unit testing that these projects happened to do isn't enough. I suggest that testing that's enough to have confidence that a non-trivial program is correct is going to cover just about everything static type checking could find as well (except in dead code, and how does that matter?) Or, to put it another way, if you didn't care enough about the correctness of your code to tes…

Enough unit testing will test everything the type system can. However, that is way more work than a good type system.

> Enough unit testing will test everything the type system can

Depends on the type system. For example, if we have parametricity then we can constrain possible values quite a lot; e.g. a value with type 'forall t. t -> t' is the identity function (or diverges, if our language is non-total). If we don't have parametricity, such values could do all sorts of shenanigans, e.g.

  def f[T](x: T): T = x match {
    case n: BigInt if isOdd(n) && isPerfect(n) => n+1
    case _ => x
  }
This acts as the identity function for all values of all types; except if we give it a BigInt which is an odd perfect number. This is probably the identity function, but ensuring there are no edge cases would require our test suite to solve a famously hard problem https://en.wikipedia.org/wiki/Perfect_number#Odd_perfect_num...

(This sort of guarantee is known as a "free theorem", e.g. see https://bartoszmilewski.com/2014/09/22/parametricity-money-f... )

Re: Analysis of whether unit testing obviates static type checking (2012)

#48

Post by tome »

koito17 wrote: post #3

TL;DR: The blogger took 4 Python projects, rewrote them in Haskell, and 3 of them had type errors. Therefore unit tests are not replacements for static typing. --- I don't get how this post says anything about unit testing or static typing at all. Doesn't it just imply that bugs are easier to detect with one method than another method? Most statically typed languages out there have a type system too weak to make the…

> Doesn't it just imply that bugs are easier to detect with one method than another method?

Yes, but then how can you say

> I don't get how this post says anything about unit testing or static typing at all.

It does say something about unit testing and static typing!

Re: Analysis of whether unit testing obviates static type checking (2012)

#49

Post by tome »

LispSporks22 wrote: post #35
steve_adams_86 wrote: post #33

I’m surprised anyone thinks types make unit tests redundant. I can know a function returns a value adhering to the correct interface, sure, but that doesn’t tell me if the values are the correct ones given specific inputs. These are different things. I do think a type system dramatically reduces the size and complexity of unit tests, though. I don’t miss writing tests to check types.

When I was a Haskell noob there was this weird meme being propagated that if it compiles, it’s correct.

It's possible you heard that (there are 227 hits for that phrase on Google) but more likely you heard "if it compiles, it works" (35,000 hits).

"If it compiles, it’s correct" is obviously false, and no one should be saying it about Haskell code. "If it compiles, it works" is true in certain contexts and for suitably weak definition of "works" (such as, behaves in a sensible way and provides a sensible answer, just not necessarily a correct one) and is an experience attested to by many, many Haskell programmers.

Re: Analysis of whether unit testing obviates static type checking (2012)

#50

Post by pfdietz »

golergka wrote: post #25
pfdietz wrote: post #8

Earlier quoted context omitted.

Not that unit testing is enough, but the unit testing that these projects happened to do isn't enough. I suggest that testing that's enough to have confidence that a non-trivial program is correct is going to cover just about everything static type checking could find as well (except in dead code, and how does that matter?) Or, to put it another way, if you didn't care enough about the correctness of your code to tes…

> if you didn't care enough about the correctness of your code to test it adequately, why would you care that static type checking could find some bugs? I think you got this backwards. If you didn't care enough about the correctness of your code to even do something as simple and straightforward as using a proper type system, why would you spend enormous amount of time to write and then support unit tests just so tha…

I do care about the things type checking could find. I just don't care to use type checking to find them, because I'd find them by doing other activities (extensive testing) that I'd do anyway.
Post reply on HN