Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Most Unit Testing Is Waste (2014) [pdf]

rbcs-us.com

Post reply on HN
Print view View original

71–80 of 164 posts

Re: Most Unit Testing Is Waste (2014) [pdf]

#71

Post by falsedan »

kd5bjo wrote: post #56
MetalGuru wrote: post #45

Earlier quoted context omitted.

What do you mean by behavior-focused testing style?

Presumably, testing that the API of each unit produces the desired result without consideration for how it accomplishes that internally. If it relies on some other component internally but never exposes that in the public API, it’s reasonable to consider that component as part of the unit under test and not try to mock it out.

And if you do need a mock (for something with hard-to-undo side effects), you set it up with stub response & canned data in the test fixture, and never assert anything against it in the test.

Part of the problem is that most test fake libraries support mocks (usually they even have it in the name), those mocks have complex “verify the 3rd call was function doDoAction with parameter (‘yes’, ‘really’)” and then people think that full-blown mocks are the only way to guarantee 100% correctness.

Re: Most Unit Testing Is Waste (2014) [pdf]

#72

Post by arendtio »

sulam wrote: post #9

I got progressively more and more frustrated with this article, largely because he keeps making statements about the impossibility of covering all states a class may take on (true!) but then followed up with espousing more use of integration and system tests, which are clearly combinatorial harder to test “completely”. He also implied at the very beginning that somehow this was driven by the switch from FORTRAN to OO…

Same here. That articles just mixes true facts and weird ideas without coming to a real punchline at the end.

At least, I didn't take away more than 'throw away tests that haven't failed in a year' and I don't agree.

Re: Most Unit Testing Is Waste (2014) [pdf]

#73

Post by falsedan »

aey wrote: post #13
ChrisMarshallNY wrote: post #12

I've never tested 100% of my code. But I've also never written a unit test that didn't expose bugs. That's mainly because I know what parts of the code will be dicey, and focus unit tests on them.

That’s funny. I only write tests to prove that the program does what it’s meant to do and not to find bugs. The goal is to prevent a future programmer, including myself, from breaking any of the declared properties.

Tests don’t prove anything, they just show the intended behaviour. Only full verification proofs show correctness.

Too often I’ve seen “tests show it’s correct” suites horribly fail to provide value when the behaviour changes in a unit of business logic and the brittleness needs to be unwound and replaced with robust assumptions.

Re: Most Unit Testing Is Waste (2014) [pdf]

#74

Post by falsedan »

winrid wrote: post #59
kstenerud wrote: post #44

After 15 years of writing unit tests, I haven't reached the same conclusion as the author. What I've found in my time is that unit testing can be good, but like anything it's not a panacea. It requires discipline, and like normal code, it has code smells. Black box unit tests are the most likely to be good tests, and white box unit tests are the most likely to be bad tests. The more you depend on the inner workings o…

If you don't use mocks then won't you just end up with integration tests?

Who cares what they’re called as long as they’re fast and don’t crash when running at scale/fill the disk or memory

Re: Most Unit Testing Is Waste (2014) [pdf]

#75

Post by kstenerud »

winrid wrote: post #59
kstenerud wrote: post #44

After 15 years of writing unit tests, I haven't reached the same conclusion as the author. What I've found in my time is that unit testing can be good, but like anything it's not a panacea. It requires discipline, and like normal code, it has code smells. Black box unit tests are the most likely to be good tests, and white box unit tests are the most likely to be bad tests. The more you depend on the inner workings o…

If you don't use mocks then won't you just end up with integration tests?

The higher up you go in your component hierarchy, the more likely it is that you'll need a mock component to test them, but I've found that it's possible to structure your architecture such that you don't need to do it as often as you'd think. If a component is using another component simply to fetch some data it needs (VERY common in my experience), then you could restructure things such that you just pass that data into the component from outside in the first place, possibly even treating it as a simple transformer instead of an actor.

There's a limit to this, of course, but keeping this principle in mind in your design also has the effect of bringing coupling to a minimum or even zero in some cases, and promoting re-use of these now completely independent components. When components don't care where or how they get their data, they become a lot simpler due to the elimination of state (implicit and explicit). Elimination of state also facilitates making functions idempotent, which massively reduces cognitive load when reasoning about a system.

Re: Most Unit Testing Is Waste (2014) [pdf]

#76

Post by alkonaut »

rileymat2 wrote: post #6

> If you want to reduce your test mass, the number one thing you should do is look at the tests that have never failed in a year and consider throwing them away. They are producing no information for you — or at least very little information. I feel like there is some subtlety here that most will miss. Specifically, "and consider". That is the part we are really bad at.

The tests that never failed for years are usually unit tests for fundamental things. Say your app has a math library at the bottom as in my example. It has thousands of years for things like vector additions, matrix multiplication and so on. And of course these never fail - they have no dependencies and we never modify it anymore. But we might modify it, say to plug in some other library to do math in 5 or 10 years. I don’t want to delete them for that reason. Luckily this kind of low level test usually takes very little time - we have a 15k test suite and the 5k “fundamental” tests take seconds while the whole integration level takes hours.

Re: Most Unit Testing Is Waste (2014) [pdf]

#77

Post by slifin »

quanticle wrote: post #46
chongli wrote: post #14

Earlier quoted context omitted.

I'll say that any unit test for a bug which would have been caught by a more sophisticated type system is a waste. I don't know how much time people spend writing such "obsolete tests", but I doubt it's insubstantial.

The problem is that sophisticated type systems only catch a subset of the bugs that a unit test can catch. For example, let's say I'm adding the ability to transfer funds from one account to the other in a banking application. I want to display a warning when the amount of money being transferred is over a certain percentage (let's say 95%) of the funds in the account. That's pretty easy to do in a unit test: create…

If anyone does have this case it's two distinct functions in my mind the first function does the comparison and returns the warning

You can then assert and type against the warning function, the printing function you could only assert against with output buffering or something like that but in my mind that's less likely to go wrong anyway

Keep your type system simple, custom types beyond in built primitives cause head aches imo

Re: Most Unit Testing Is Waste (2014) [pdf]

#78

Post by [deleted] »

cryptica wrote: post #64
njharman wrote: post #21

I did not read the article. I did read all comments Unit tests for the most part aren't about "testing". They are developer tool. To verify rthat modifications (refactoring, additions, bug fixes, etc) doesn't break contracts etc. Oh and showing that your code is a codependent mess of poorly isolated spagehtti, if your unit tests are hard to write, the code under test is a mess. Inittests are more useful in languages…

Unit tests don't guarantee that your code units are well designed. You can still write unit tests for spaghetti code. In fact, sometimes it encourages it because it encourages antipatterns like dependency injection; because developers get the urge to inject mock objects into the units from the unit tests. Just think about it at a high level. If your module depends on another module, is it really always necessary that…

[deleted]

Re: Most Unit Testing Is Waste (2014) [pdf]

#79

Post by dep_b »

Doing a typical "front-end" mobile application I write the following tests:

* Integration tests that test the API I get a ton of value out of it though it's sometimes hard to guarantee a certain state at the API's end. But I use them while building the API's but they also can detect any kind of problem after an API upgrade etcetera

* Unit tests that test data transformations This is stuff like date formatting, building headers into a request given certain input, building more complex ViewModels that take a few structs as input and turn them into something that reflects the actual process happening in the view. They're valuable while building the logic, help separating the logic since you need to make it testable and also help a ton if you find a bug since a bug simply means adding another test case to see if something goes wrong and then fix it.

I don't think I even get to 30% code coverage but I think what I cover is super valuable and the other 70% is usually mostly CRUD boiler plate.

Re: Most Unit Testing Is Waste (2014) [pdf]

#80

Post by revskill »

I agree. Unit Test is for testing API usage, not implementation. We can use it to figure out the API design good or not.

But the problem, is API change is very often, and we don't want to change both implementation and test just for the sake of API change.

Integration test is enough in most case.

Even API is the implementation details of an abstraction.

Post reply on HN