Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Most Unit Testing Is Waste (2014) [pdf]

rbcs-us.com

Post reply on HN
Print view View original

111–120 of 164 posts

Re: Most Unit Testing Is Waste (2014) [pdf]

#111

Post by kissgyorgy »

> If you find your testers splitting up functions to support the testing process, you’re destroying your system Why Most Unit Testing is Waste 4 architecture and code comprehension along with it. Test at a coarser level of granularity.

This is very much in line with DHH's opinion about testing in general (the discussion "TDD is dead" is about this). He said that he doesn't want to split logic for the sake of testing, so HE drives the design NOT tests. I very much agree with this. I think a human can design better code (meaning that's easier to use, so easier to consume by other humans) than any automated process (like TDD).

I noticed his blog post quotes this PDF: https://dhh.dk/2014/tdd-is-dead-long-live-testing.html

Re: Most Unit Testing Is Waste (2014) [pdf]

#112

Post by diego_moita »

BaronSamedi wrote: post #102

This paper is emblematic of a serious problem in the software development field: lack of empirical research. It is unsurprisingly filled with opinion and anecdote, with little mention of research in the area. And in the one actual study cited, "Does Test-Driven Development Really Improve Software Design Quality?", the author mis-characterizes the findings. Not that there is a huge set of research on unit testing to r…

I strongly agree but I warn that evidence is a lot harder than it seems because a lot of what we do is very hard to measure or even define. How do you reliably measure programmer productivity? Lines of code, inflection points, units of functionally... all have their flaws.

Re: Most Unit Testing Is Waste (2014) [pdf]

#113

Post by the8472 »

fogetti wrote: post #98

The thing is that most people don't understand that the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Now because you have to test it anyway, you can just simply spend the same time writing a unit test instead of executing the code with manually configured non-repeatable test cycles, a.k.a clicking through the UI, or sending Postm…

Not all forms of testing consist of separate test suites like unit tests. Assertions, contracts, encoding constraints in elaborate type systems and having them checked at compile time are also ways of coding repeatable tests.

And if it's some poorly coded legacy application or something that's mostly wrapping a proprietary black box, working around other peoples bugs, and things like that then system or integration tests may be easier than attempting to mock a gordian knot's tangles.

Unit tests work best for properly encapsulated pieces of code, utility classes, libraries and such.

Re: Most Unit Testing Is Waste (2014) [pdf]

#114

Post by Udik »

fogetti wrote: post #98

The thing is that most people don't understand that the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Now because you have to test it anyway, you can just simply spend the same time writing a unit test instead of executing the code with manually configured non-repeatable test cycles, a.k.a clicking through the UI, or sending Postm…

> the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do

First, in most cases you're going to test your code manually anyway- whether you write unit tests or not. So writing unit tests is just in addition to the time already spent testing manually.

But most importantly, unit tests often do very little to prove "your code actually does what it claims to do". They prove you have tested some cases, that's all. In many occasions, the code under test doesn't even "claim" to do anything in particular: the claim is that you're delivering a feature, not pieces of code that behave as expected. All your code fragments can behave exactly as expected and still the feature might be broken, or miss implicit requirements, or ill conceived.

Finally, unit tests as documentation are disastrous, as they document only details of specific implementations, as opposed to the feature you're expected to deliver. Good comments are a hundred times better that tests in that respect.

Re: Most Unit Testing Is Waste (2014) [pdf]

#116

Post by didericis »

phereford wrote: post #103
fogetti wrote: post #98

The thing is that most people don't understand that the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Now because you have to test it anyway, you can just simply spend the same time writing a unit test instead of executing the code with manually configured non-repeatable test cycles, a.k.a clicking through the UI, or sending Postm…

To add to this sentiment, tests also give teams confidence that is needed. It gives engineers confidence when developing new features because the test suite will break if you introduce a regression. It gives engineers confidence during deployment/CD. While the tangible benefits of unit tests are very important, there are other intangible benefits that are equally important.

I’d say that confidence is a tangible factor.

The single most important reason to test your code seems to be that it allows you to refactor your code while ensuring it still meets the original outside expectations.

The author’s main gripe seems to be about tightly coupled tests that make refactoring larger systems more difficult, and about prioritizing meeting arbitrary metrics (lines of codes covered) rather that thinking critically about the actual benefits.

This is in line with his emphasis on integration tests determined by the business. I think that same thought process can be applied to internal components of code as well, and you can empirically determine the quality of your approach (roughly) by evaluating how often your tests change. If nearly every change breaks a test, that probably means they’re low value/too tightly coupled.

Excessive mocking seems to be the biggest source of evil in that regard.

Re: Most Unit Testing Is Waste (2014) [pdf]

#117

Post by butterisgood »

fogetti wrote: post #98

The thing is that most people don't understand that the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Now because you have to test it anyway, you can just simply spend the same time writing a unit test instead of executing the code with manually configured non-repeatable test cycles, a.k.a clicking through the UI, or sending Postm…

Yes, code must be tested. Line and branch coverage just don’t have the actual impact that some people claim it does. It is, however, better than nothing at all.

The arguments in the paper are anecdotal and I took them to be “question the cargo cult” which is a valuable thing to do. The author is merely encouraging you to think because “automated garbage is still garbage.”

Good read.

Re: Most Unit Testing Is Waste (2014) [pdf]

#118

Post by ptx »

methyl wrote: post #110
fogetti wrote: post #98

The thing is that most people don't understand that the code needs to be tested anyway. If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Now because you have to test it anyway, you can just simply spend the same time writing a unit test instead of executing the code with manually configured non-repeatable test cycles, a.k.a clicking through the UI, or sending Postm…

> If you don't test it, you are just an idiot who has no proof that the code does what it claims to do. Does having tests make it any different? You would need to test those tests to make sure they are testing the right thing after all (ad infinity).

Untested tests are still a lot more useful than no tests, just like how an untested program (while probably worse than a tested program) is still usually better than no program at all.

Also, tests are actually subject to very basic testing in the red-green-refactor TDD cycle: they start out failing so you know that they actually test something when they start passing.

Re: Most Unit Testing Is Waste (2014) [pdf]

#119

Post by ptx »

walshemj wrote: post #108
raverbashing wrote: post #101

Earlier quoted context omitted.

That's because you (and many others) are drinking the static typing koolaid Now try serializing things in XML instead of JSON. But not one type of message, multiple types, with flexible formats. "Oh but this never happens" except it does. Not to mention the iteration speed whenever you need to fix something.

?? sorry what do you mean, I was happily reading multiple types of logs with multiple different format records for a Billing system using statically typed PL/1G in the 80's Apart from both json and XML are a PITA to work with in the first place

JSON is wonderfully simple to work with - in a dynamic language. Having to map it to a statically known schema, as you would in a statically typed language, loses that simplicity and can be, as you say, a PITA.

Re: Most Unit Testing Is Waste (2014) [pdf]

#120

Post by 0x445442 »

Frost1x wrote: post #104
BaronSamedi wrote: post #102

This paper is emblematic of a serious problem in the software development field: lack of empirical research. It is unsurprisingly filled with opinion and anecdote, with little mention of research in the area. And in the one actual study cited, "Does Test-Driven Development Really Improve Software Design Quality?", the author mis-characterizes the findings. Not that there is a huge set of research on unit testing to r…

I agree entirely. The more tendency there is to push development as an "engineering" discipline, the more evidence based research needs to be used to solidify or disprove certain development dogmas repeated over and over again, which in many cases, seemingly have no empirical evidence beyond a few anecdotal success cases. This is the norm in the industry, "you're doing it wrong, the best way is this way..."--based on…

Agree with you and parent poster. I remember 15 years ago or more going through CMMI certification and there was a push to get to level 3. Our company hired a consultant who came in every eight weeks or so to track our progress, give direction etc.

On one of his visits he started interviewing engineers individually to see how things were going and I asked him what the point of it all was and I could tell he was quite taken back. But then, not surprisingly, he said to be more efficient developing software. I then asked him if he or any organization he'd worked with had ever actually tracked the time it took to implement the process, to which he answered no. Then I asked him, then how the hell do you know if any of this is making software development more efficient?

Post reply on HN