Analysis of whether unit testing obviates static type checking (2012) • Hacker News Board

Re: Analysis of whether unit testing obviates static type checking (2012)

#61

Post by wzdd » Mon, Jul 24, 2023, 5:17 PM UTC

efarrer wrote: post #36

wzdd wrote: post #7

The blog sets up to address the "claim by proponents of dynamically typed programming languages" that "once you have unit testing static type checking is redundant". This is a strawman and I'm fairly sure it was a strawman in 2012 as well. The more interesting claim, which this blog post doesn't address, is that dynamic typing plus unit testing lets you produce working programs faster, without having to wrestle a typ…

Hi I'm the author of the blog post and the paper. In the paper I provide a reference to such a claim by proponents of dynamic typing. From my paper: "Because some error detection can be done by both unit testing and static type checking, some proponents of dynamic type checking claim that static type checking is not needed [3]." and the reference: [3] J. Spolsky and B. Eckel, “Strong Typing vs. Strong Testing,” in Th…

I had a look at your referenced source (https://ia-petabox.archive.org/details/the-best-software-wri...), but I didn't find statements as strong as the summary you give in your paper or blog post. Here are some quotes to convey the tone:

"If your unit tests provide good code coverage, don't feel too paranoid about giving up compile-time type checking." (pp 69)

"The only guarantee of correctness [...] is whether it passes all tests which define the correctness of your program." (pp 75)

"To claim that static type checking constraints in C++, Java, or C# will prevent you from writing broken programs is clearly an illusion" (pp 76)

The closest I could find to your quote was at the end: "a dynamically typed language could be much more productive but create programs that are just as robust as those written in statically typed languages." (pp 77) In the context of the article (see previous quote), this is presumably referring to the goal of "defining the correctness of your program", and the article in fact makes the point which I made reference to, which is that incorrect inputs and API usage could well be out of scope for this goal (and perhaps were in the programs you tested). Or, to put it another way, it doesn't make sense to talk about robustness unless you also give a context.

The worst I could say about the article is that it is kind of tautological, because it's easy to no-true-Scotsman the concept of "sufficiently good" unit tests. But to be honest this also happens with type systems.

I realise there's a lot of ... let's say "enthusiasm" in this particular area, but I found the article quite a bit more measured than what was implied by the blog post.

Re: Analysis of whether unit testing obviates static type checking (2012)

#62

Post by ReflectedImage » Mon, Jul 24, 2023, 7:44 PM UTC

If you spent a day writing unit tests you will eliminate more bugs than if you spent a day writing types.

And that's all that matters. Unit tests are the more effective technique.

Yes obviously static typing doesn't catch all the bugs caught by unit tests or vice versa. But it's a meaningless thing to say.

Re: Analysis of whether unit testing obviates static type checking (2012)

#63

Post by ReflectedImage » Mon, Jul 24, 2023, 7:52 PM UTC

efarrer wrote: post #39

Jtsummers wrote: post #29

Earlier quoted context omitted.

Static typing, short of dependent types (and even then you have to use them to their full extent), does not obviate any form of testing unless your programs are trivial. For any non-trivial program, testing is still needed unless you're going to go through formal proofs (which could be embedded into dependent type systems). Otherwise, you can have something which type checks but which still contains logical errors.

Hi, I'm the author of the blog post and the paper. I certainly do not argue that static typing obviates unit testing. I agree that both are valuable together. I did however find examples where individual unit tests could be deleted because they didn't test anything that wasn't covered by the static type system. In other words static typing didn't eliminate the need for unit tests (for catching bugs), but it did reduc…

So I think you are running on a false premise here efarrer.

The false premises is that anyone ships bug free code. It can be done but it's stupidly expensive. Formal methods Z proofs, etc.

Unit testing and static typing are 2 different techniques which eliminate bugs.

The question that needs to be answered is which of the 2 techniques is the most effective per developer hour spent.

Unit tests do not caught all the bugs that static typing would catch nor does static typing catch all the bugs unit tests would catch. That's not an useful observation.

Nor is there a moral here that you should be doing both, doing both is expensive and costs $$$.

Re: Analysis of whether unit testing obviates static type checking (2012)

#64

Post by ReflectedImage » Mon, Jul 24, 2023, 7:54 PM UTC

v3ss0n wrote: post #31

Static type checking is way a lot easier, help with your code discovery and learning process . Less unit testing, better typing ftw.

Static type checking takes a long longer developer time wise and does a much worse job than unit tests.

Re: Analysis of whether unit testing obviates static type checking (2012)

#65

Post by golergka » Mon, Jul 24, 2023, 8:50 PM UTC

Jtsummers wrote: post #59

golergka wrote: post #57

Earlier quoted context omitted.

But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing? Automated testing is only useful (in the sense of making the software more reliable) if manual testing is inadequate. In which case, software is going to be crap anyway. You might argue that enforcing unit tests cause t…

> But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing? Unit testing is not the only form of automated testing. You can have automated regression tests, integration tests, and even automated exploratory tests (similar to what you get with the "million QAs typing on keyboar…

Yes, I completely agree with that. I provided that argument just to illustrate that exactly the same point you've just made works for static typing when you compare it to automatic testing.