Live data from Hacker News

Most Unit Testing Is Waste (2014) [pdf]

rbcs-us.com

31–40 of 164 posts

Re: Most Unit Testing Is Waste (2014) [pdf]

#31
post #7

> If you want to reduce your test mass, the number one thing you should do is look at the tests that have never failed in a year and consider throwing them away. They are producing no information for you — or at least very little information. I feel like there is some subtlety here that most will miss. Specifically, "and consider". That is the part we are really bad at.

Those tests are helpful when you do major architectural changes, which may not happen every year. Tests also help in general development. Make some changes, then fix the tests. If you are running tests locally on your machine (and offline) how can you be sure that every time a test fails locally the failure is logged?

There's a difference between tests that haven't broken because they are in stable code, and tests that haven't failed because they are testing tautologies or even just completely failing to break when the underlying code is broken.

I've lost count of how many times I've gone into an old test suite at work and found that the tests were still passing even though the code they were testing had completely changed or been removed.

Sometimes tests are written very poorly. The codebase benefits from their removal.

Re: Most Unit Testing Is Waste (2014) [pdf]

#32
post #10
post #5

I've generally preferred integration tests to unit tests but Storybook has completely changed how I write React components, for the better.

Agreed on integration tests over unit tests. I get the most bang for my buck with them. Storybook is pretty nice. I use it for developing smaller components, more of the building blocks of the application. Have you ever checked out Kent C Dodd's react-testing-library? It was a refreshing approach to testing react components IMO and it has gained quite a bit of adoption.

The docs for react-testing-library look like they offer a great deal of wisdom in a tiny package. I find my React components tend to be simple enough that I'm generally far more concerned about whether the CSS works than the JS, leading me to probably keep my focus on Storybook for the moment but I suspect I'll be reaching for react-testing-library before too long. Thanks for the heads up about it.

Re: Most Unit Testing Is Waste (2014) [pdf]

#34
post #9

I got progressively more and more frustrated with this article, largely because he keeps making statements about the impossibility of covering all states a class may take on (true!) but then followed up with espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”. He also implied at the very beginning that somehow this was driven by the switch from FORTRAN to OO…

> espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”

The point is that when you do integration tests, you will test the underlying classes the way in which it actually matters. You can't test every possible condition a unit can have, but you can test for the most likely.

Re: Most Unit Testing Is Waste (2014) [pdf]

#35
post #34
post #9

I got progressively more and more frustrated with this article, largely because he keeps making statements about the impossibility of covering all states a class may take on (true!) but then followed up with espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”. He also implied at the very beginning that somehow this was driven by the switch from FORTRAN to OO…

> espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely” The point is that when you do integration tests, you will test the underlying classes the way in which it actually matters . You can't test every possible condition a unit can have, but you can test for the most likely.

This is my experience too. Realism in testing is criminally underrated while code coverage is criminally overrated.

IME unit tests can only effectively substitute for integration tests where you're testing logical/algorithmic code with simple function inputs/outputs.

Re: Most Unit Testing Is Waste (2014) [pdf]

#36
post #34

Earlier quoted context omitted.

> espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely” The point is that when you do integration tests, you will test the underlying classes the way in which it actually matters . You can't test every possible condition a unit can have, but you can test for the most likely.

This is my experience too. Realism in testing is criminally underrated while code coverage is criminally overrated. IME unit tests can only effectively substitute for integration tests where you're testing logical/algorithmic code with simple function inputs/outputs.

I somewhat agree, somewhat disagree. IMO the largest value I get from unit tests is the confidence that I can make changes and understand what breaks. Refactoring w/o unit tests makes me feel like I am flying blind

Re: Most Unit Testing Is Waste (2014) [pdf]

#37

Unit Tests = The assurance/confidence that you get, if something changes logic, this test will break & you will know it. Aside from just limiting bugs - this is more powerful.

95% of the time when a unit test fails for me, it is the test that need fixing instead of my code. It hardly inspires confidence.

Re: Most Unit Testing Is Waste (2014) [pdf]

#38
post #34
post #9

I got progressively more and more frustrated with this article, largely because he keeps making statements about the impossibility of covering all states a class may take on (true!) but then followed up with espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”. He also implied at the very beginning that somehow this was driven by the switch from FORTRAN to OO…

> espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely” The point is that when you do integration tests, you will test the underlying classes the way in which it actually matters . You can't test every possible condition a unit can have, but you can test for the most likely.

This is no more or less likely in an integration test vs a unit test. One of the first things you learn in writing unit tests is that you should be selective in picking your inputs. This is even taught in those “trade schools” he subtly derides. You regularly see both public test cases and hidden test cases run against problem sets, at least that’s what my interns tell me.

Re: Most Unit Testing Is Waste (2014) [pdf]

#39
post #9

I got progressively more and more frustrated with this article, largely because he keeps making statements about the impossibility of covering all states a class may take on (true!) but then followed up with espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”. He also implied at the very beginning that somehow this was driven by the switch from FORTRAN to OO…

I'm going to pile onto your comment, since I heartily agree. > In a given computing context, the exact function to be called is determined at run-time and cannot be deduced from the source code [in OOP languages] as it could in FORTRAN. This is not necessarily true, and I don't think this was true when this was written, either. (The Wayback Machine first saw this in 2014, and the paper doesn't date itself.) Most memb…

> then what amount of testing is going to catch that?

Mutation Testing could. :) https://ai.google/research/pubs/pub46584

Re: Most Unit Testing Is Waste (2014) [pdf]

#40
post #17

"The cross product of those paths with the possible state configurations of all global data (including instance data which, from a method scope, are global) and formal parameters is indeed very large. And the cross product of that number with the possible sequencing of methods within a class is countably infinite." Sounds more like a condemnation of OOP than of unit testing, and I do genuinely feel sorry for the unit…

Yeah a lot of the 90s/early 2000s OOP stuff I learned in school seemed to always result in really tightly coupled systems and bespoke webs of tests and fixtures that strung along weird dependency chains in unwieldy spaghetti piles that did no good. Following TDD has helped me land at decoupled functional interfaces like the ones you've described, and it all scales and composes so nicely, yet stays very tractable.

Robert Martin sketched 2 diagrams in [1] that elegantly illustrate these two different design patterns and how testable usually means composable and more tractable:

[1] http://blog.cleancoder.com/uncle-bob/2017/03/03/TDD-Harms-Ar...

Post reply on HN