Hasn't this long been settled? See e.g. https://www.destroyallsoftware.com/talks/ideology . Spoiler alert: We need both.
The article was published three years before this talk.
I know but it was resubmitted and upvoted to the Front page. :)
Analysis of whether unit testing obviates static type checking (2012)
51–60 of 65 posts
Re: Analysis of whether unit testing obviates static type checking (2012)
#52Earlier quoted context omitted.
> if you didn't care enough about the correctness of your code to test it adequately, why would you care that static type checking could find some bugs? I think you got this backwards. If you didn't care enough about the correctness of your code to even do something as simple and straightforward as using a proper type system, why would you spend enormous amount of time to write and then support unit tests just so tha…
I do care about the things type checking could find. I just don't care to use type checking to find them, because I'd find them by doing other activities (extensive testing) that I'd do anyway.
The main difference is, nobody checks if you wrote tests for every failure condition. Choose coverage tries, but it doesn't really come close enough to type checker.
Re: Analysis of whether unit testing obviates static type checking (2012)
#53Earlier quoted context omitted.
I do care about the things type checking could find. I just don't care to use type checking to find them, because I'd find them by doing other activities (extensive testing) that I'd do anyway.
The main difference is, nobody checks if you wrote tests for every failure condition. Choose coverage tries, but it doesn't really come close enough to type checker.
So, you're saying nobody actually cares if the code is correct. So why should I?
Re: Analysis of whether unit testing obviates static type checking (2012)
#54Earlier quoted context omitted.
The main difference is, nobody checks if you wrote tests for every failure condition. Choose coverage tries, but it doesn't really come close enough to type checker.
So, you're saying nobody actually cares if the code is correct. So why should I?
No, I'm saying that I would rather depend on an automated tool than human decision. That's why you use automated testing instead of manual, right? Static typing is just the next step in the same direction.
Re: Analysis of whether unit testing obviates static type checking (2012)
#55Earlier quoted context omitted.
So, you're saying nobody actually cares if the code is correct. So why should I?
No, I'm saying that I would rather depend on an automated tool than human decision. That's why you use automated testing instead of manual, right? Static typing is just the next step in the same direction.
But the automated tool (static type checker) does not actually produce reliable software. Only extensive testing does that. And if you do extensive testing, why do you need that automated tool?
The static type checker is only useful (in the sense of making the software more reliable) if the testing is inadequate. In which case, the software is going to be crap anyway.
You might argue that enforcing static type checking cause the software to be structured in some way that's helpful, but that's a different argument.
Re: Analysis of whether unit testing obviates static type checking (2012)
#56The answer is yes, obviously, due to the Curry-Howard isomorphism. Likewise static type checking obviates unit testing. Granted, most static type systems aren’t Turing complete so there are some assertions that can’t be made at the type level that can be made at the unit test level, however there are absolutely a handful of languages with type systems that are. Indeed, any language with first-class functions can impl…
Static typing, short of dependent types (and even then you have to use them to their full extent), does not obviate any form of testing unless your programs are trivial. For any non-trivial program, testing is still needed unless you're going to go through formal proofs (which could be embedded into dependent type systems). Otherwise, you can have something which type checks but which still contains logical errors.
Your comment reads like you’re opposing what I said but you’re not actually refuting or addressing any of the points I made..do you have anything to add to the conversation or are you just yelling in the opposite direction?
Re: Analysis of whether unit testing obviates static type checking (2012)
#57Earlier quoted context omitted.
No, I'm saying that I would rather depend on an automated tool than human decision. That's why you use automated testing instead of manual, right? Static typing is just the next step in the same direction.
But the automated tool (static type checker) does not actually produce reliable software. Only extensive testing does that. And if you do extensive testing, why do you need that automated tool? The static type checker is only useful (in the sense of making the software more reliable) if the testing is inadequate. In which case, the software is going to be crap anyway. You might argue that enforcing static type checki…
But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing?
Automated testing is only useful (in the sense of making the software more reliable) if manual testing is inadequate. In which case, software is going to be crap anyway.
You might argue that enforcing unit tests cause the software to be structured in some way that's helpful, but that's a different argument.
---
If you have a counter-argument against this in manual vs automated testing debate, then you also have a counter-argument against your own position above.
Also, from my personal experience: I've started to write more and more functional code lately. Not only static type checking, but "almost-Haskell" in Typescript with fp-ts. And I find that when I write code in this way, the only place where I actually find and fix bugs are... unit tests.
Re: Analysis of whether unit testing obviates static type checking (2012)
#58Earlier quoted context omitted.
But the automated tool (static type checker) does not actually produce reliable software. Only extensive testing does that. And if you do extensive testing, why do you need that automated tool? The static type checker is only useful (in the sense of making the software more reliable) if the testing is inadequate. In which case, the software is going to be crap anyway. You might argue that enforcing static type checki…
But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing? Automated testing is only useful (in the sense of making the software more reliable) if manual testing is inadequate. In which case, software is going to be crap anyway. You might argue that enforcing unit tests cause t…
I didn't specify automatic or manual testing, I just said testing. Automatic vs. manual testing is another topic entirely (albeit one I have opinions on).
Re: Analysis of whether unit testing obviates static type checking (2012)
#59Earlier quoted context omitted.
But the automated tool (static type checker) does not actually produce reliable software. Only extensive testing does that. And if you do extensive testing, why do you need that automated tool? The static type checker is only useful (in the sense of making the software more reliable) if the testing is inadequate. In which case, the software is going to be crap anyway. You might argue that enforcing static type checki…
But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing? Automated testing is only useful (in the sense of making the software more reliable) if manual testing is inadequate. In which case, software is going to be crap anyway. You might argue that enforcing unit tests cause t…
> But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing?
Unit testing is not the only form of automated testing. You can have automated regression tests, integration tests, and even automated exploratory tests (similar to what you get with the "million QAs typing on keyboards" approach of manual exploratory testing).
Automated tests are reproducible and reliable (particularly for regression testing or with tight timing constraints) in a way that manual testing will never be. You sound a bit like my old boss who had to be convinced to let us automate tests by sitting him at the relay box and telling him, "This test procedure says 'flip the switch 10 times per second for 10 seconds and a fault indicator will appear', can you do that?" The answer, of course, was no. No one can, automated tests are useful for making reliable software (this was a safety critical system, we cared very much about reliability and if you've ever flown you ought to be happy that we cared).
Re: Analysis of whether unit testing obviates static type checking (2012)
#60Earlier quoted context omitted.
But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing? Automated testing is only useful (in the sense of making the software more reliable) if manual testing is inadequate. In which case, software is going to be crap anyway. You might argue that enforcing unit tests cause t…
> But automated testing (unit tests) does not actually produce reliable software. Only extensive manual testing does that. And if you do extensive manual testing, why do you need that automated testing? Unit testing is not the only form of automated testing. You can have automated regression tests, integration tests, and even automated exploratory tests (similar to what you get with the "million QAs typing on keyboar…
Automated tests can also be provided in far higher volume than manual tests. This didn't used to be the case, back in the stone ages of computing, when computers were much more expensive, so you often see old literature criticizing automated testing as inefficient, wasting expensive computer time.
To give some idea of what automated testing can do these days, I point to the testing I regularly do on Common Lisp implementations, and particularly on the compilers of these implementations. I can quickly set it up to run some 200 tests per second in a single thread on SBCL, with randomly generated code of moderate size (a few hundred cons cells). Over the years I've run billions of distinct tests with this approach.
This sort of testing found large numbers of bugs in every implementation I've ever tried it on. Similar testing on compilers for other languages (like Csmith for C) found bugs in every compiler it was ever tried on, even those implemented in statically typed languages.