Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Most Unit Testing Is Waste (2014) [pdf]

rbcs-us.com

Post reply on HN
Print view View original

51–60 of 164 posts

Re: Most Unit Testing Is Waste (2014) [pdf]

#51

Post by qznc »

quanticle wrote: post #46
chongli wrote: post #14

Earlier quoted context omitted.

I'll say that any unit test for a bug which would have been caught by a more sophisticated type system is a waste. I don't know how much time people spend writing such "obsolete tests", but I doubt it's insubstantial.

The problem is that sophisticated type systems only catch a subset of the bugs that a unit test can catch. For example, let's say I'm adding the ability to transfer funds from one account to the other in a banking application. I want to display a warning when the amount of money being transferred is over a certain percentage (let's say 95%) of the funds in the account. That's pretty easy to do in a unit test: create…

No mainstream type system can do that so GP would be fine with a unit test here.

You could use Coq, Isabelle, or Lean. Their type systems are powerful enough to allow such checks.

Re: Most Unit Testing Is Waste (2014) [pdf]

#52

Post by wilgertvelinga »

lstamour wrote: post #39
deathanatos wrote: post #25

Earlier quoted context omitted.

I'm going to pile onto your comment, since I heartily agree. > In a given computing context, the exact function to be called is determined at run-time and cannot be deduced from the source code [in OOP languages] as it could in FORTRAN. This is not necessarily true, and I don't think this was true when this was written, either. (The Wayback Machine first saw this in 2014, and the paper doesn't date itself.) Most memb…

> then what amount of testing is going to catch that? Mutation Testing could. :) https://ai.google/research/pubs/pub46584

Luckily it's not only a research result: actual libraries and tools exist for a lot of languages. See this list: https://github.com/theofidry/awesome-mutation-testing

Re: Most Unit Testing Is Waste (2014) [pdf]

#55

Post by winrid »

mobjack wrote: post #37
the_arun wrote: post #18

Unit Tests = The assurance/confidence that you get, if something changes logic, this test will break & you will know it. Aside from just limiting bugs - this is more powerful.

95% of the time when a unit test fails for me, it is the test that need fixing instead of my code. It hardly inspires confidence.

Is that because you're not writing tests while you're adding code and just breaking old tests?

Re: Most Unit Testing Is Waste (2014) [pdf]

#56

Post by kd5bjo »

MetalGuru wrote: post #45
war1025 wrote: post #27

Earlier quoted context omitted.

Some of our older tests at work are an absolute nightmare to deal with. We went through a phase where we basically just mocked everything non-trivial and ended up with a rewording of the code itself. Any time you make a change to that code, even if it still behaves exactly the same from a user point of view, the tests blow up into pieces and you end up having to rewrite or throw out most of them. We moved to a behavi…

What do you mean by behavior-focused testing style?

Presumably, testing that the API of each unit produces the desired result without consideration for how it accomplishes that internally. If it relies on some other component internally but never exposes that in the public API, it’s reasonable to consider that component as part of the unit under test and not try to mock it out.

Re: Most Unit Testing Is Waste (2014) [pdf]

#57

Post by danmaz74 »

sulam wrote: post #9

I got progressively more and more frustrated with this article, largely because he keeps making statements about the impossibility of covering all states a class may take on (true!) but then followed up with espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”. He also implied at the very beginning that somehow this was driven by the switch from FORTRAN to OO…

I saw cases where there really was a waste of stupid unit tests, and no significant tests for the interaction of those units, so I can see where the author is coming from. But my conclusion, after many years of developing with different levels of testing, is that it's a good idea to:

* Have at least one unit test for each nontrivial unit, but no unit tests for the trivial ones. But, in these unit tests, use mocking and stubbing only when using real objects isn't feasible, or very inconvenient - isolation is nice to have, but generally overrated in my experience

* Create as many unit tests as necessary to feel confident about the edge cases for the most critical units

* Implement higher level tests covering the main use cases for the interaction of all the units

Re: Most Unit Testing Is Waste (2014) [pdf]

#58

Post by hdfbdtbcdg »

sulam wrote: post #9

I got progressively more and more frustrated with this article, largely because he keeps making statements about the impossibility of covering all states a class may take on (true!) but then followed up with espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”. He also implied at the very beginning that somehow this was driven by the switch from FORTRAN to OO…

Ironically modern fortran is object orientated...

Re: Most Unit Testing Is Waste (2014) [pdf]

#59

Post by winrid »

kstenerud wrote: post #44

After 15 years of writing unit tests, I haven't reached the same conclusion as the author. What I've found in my time is that unit testing can be good, but like anything it's not a panacea. It requires discipline, and like normal code, it has code smells. Black box unit tests are the most likely to be good tests, and white box unit tests are the most likely to be bad tests. The more you depend on the inner workings o…

If you don't use mocks then won't you just end up with integration tests?

Re: Most Unit Testing Is Waste (2014) [pdf]

#60

Post by Nasrudith »

ChrisMarshallNY wrote: post #12

I've never tested 100% of my code. But I've also never written a unit test that didn't expose bugs. That's mainly because I know what parts of the code will be dicey, and focus unit tests on them.

In practice I personally tend to do the reverse. If I encounter a bug I create a unit test to replicate its circumstances and use it to ensure that the bug stays dead and gone.
Post reply on HN