It'll be fun to refactor a codebase without unit tests
Did that many times without problems.
But, I mostly code C++ and C#, both are strongly typed, so the compiler helps a lot with refactors.
Why Most Unit Testing Is Waste [pdf]
61–70 of 159 posts
Re: Why Most Unit Testing Is Waste [pdf]
#62The normal practice for large scale codebases in complex domains is "the code is the spec". That is, the only specification for how the system should work is how it worked yesterday. In that case, unit tests serve as a great specification. Even tests that just duplicate the business code under test and assert that it's the same (A huge waste in normal cases) is useful. Because a Unit test is much better than a word d…
> Even tests that just duplicate the business code under test and assert that it's the same (A huge waste in normal cases) is useful.
Maybe, but there's no reason to write such tests by hand. Just automatically test each method against the previous version and see what changed.
I think the generated results would be too noisy and developers would stop paying attention to test failures, but maybe a strict enough code review process could prevent that.
Re: Why Most Unit Testing Is Waste [pdf]
#63Earlier quoted context omitted.
How many times do you need to repeat the review of a given unit of code? (actually, that would have more chance of catching additional bugs than would rerunning the same unit tests.) Once you change the code, the existing unit tests are, by definition, invalidated (i.e. there is no value to being able to rerun them), while most higher-level testing will retain their validity. You have actually created an argument for…
No, changing the code doesn't invalidate the unit tests unless the API design contact changes.
By definition, unit tests test the implementation. A test at the API design contract level is an integration test. The only place where these are the same thing are pure (side-effect-free) functions that do not call other functions.
Re: Why Most Unit Testing Is Waste [pdf]
#64I pretty much agree with all of this. Unit tests are great when you have a large code base with dozens of apps and need to modify a core library to remove side effects. How do you know you didn't break one of the apps? But if we go deeper, why does the core library have side effects? Because it was a crummy, poorly designed piece of crap to begin with. If it was originally written with high quality it wouldn't need t…
Even the best code written by the best developers will eventually need to be refactored if the software lives long enough. Changes in requirements, environments, and tools will eventually invalidate the initial design. If the software will only be used for a couple years and then discarded then you can get away with minimal unit tests. But extensive unit tests are absolutely essential to successfully maintaining a system with a 10+ year lifecycle.
Re: Why Most Unit Testing Is Waste [pdf]
#65not only is it often waste but it can tremendously add to code bloat and 'dept'. I saw a dev spending 80% of his time 'fixing test setup' instead of fixing code.
To me that would be a red flag. If it's difficult to setup within your tests then it's difficult within your code and/or too much magic is happening.
Re: Why Most Unit Testing Is Waste [pdf]
#66The normal practice for large scale codebases in complex domains is "the code is the spec". That is, the only specification for how the system should work is how it worked yesterday. In that case, unit tests serve as a great specification. Even tests that just duplicate the business code under test and assert that it's the same (A huge waste in normal cases) is useful. Because a Unit test is much better than a word d…
> The normal practice for large scale codebases in complex domains is "the code is the spec".
”The code is the spec” is just a way of saying “we don't know what the code is supposed to do, and the intent was never, even initially, properly validated by domain experts, or, if it was, we tossed the knowledge generated in that process.”
This is especially the case if that phrase is used about a large system in a complex domain.
Re: Why Most Unit Testing Is Waste [pdf]
#67The normal practice for large scale codebases in complex domains is "the code is the spec". That is, the only specification for how the system should work is how it worked yesterday. In that case, unit tests serve as a great specification. Even tests that just duplicate the business code under test and assert that it's the same (A huge waste in normal cases) is useful. Because a Unit test is much better than a word d…
This. We tend to put a high value on code review, yet some don't value unit testing. Lean in and I'll tell you a secret: ... unit testing is automated, repeatable, code review!
Your unit tests check the architecture of the code and the implementation is maintainable? I say that cause that's what I use code reviews for :P
Re: Why Most Unit Testing Is Waste [pdf]
#68Earlier quoted context omitted.
Legacy code = code without tests. You might write HQ code today, but without tests it'll become technical debt within a month or even a week down the line. Programmers come and go, unit tests are omnipresent.
Tests can become technical debt themselves, there's nothing special about test code that makes it immune to all the programming pitfalls.
oh no! that consultant who I paid $100K to tell me that unit testing would solve all my problems didn't tell me that. maybe its because they have never written a line of code in their life let alone delivered a successful large scale software project, but who knows.
Re: Why Most Unit Testing Is Waste [pdf]
#69I pretty much agree with all of this. Unit tests are great when you have a large code base with dozens of apps and need to modify a core library to remove side effects. How do you know you didn't break one of the apps? But if we go deeper, why does the core library have side effects? Because it was a crummy, poorly designed piece of crap to begin with. If it was originally written with high quality it wouldn't need t…
Even the best code written by the best developers will eventually need to be refactored if the software lives long enough. Changes in requirements, environments, and tools will eventually invalidate the initial design. If the software will only be used for a couple years and then discarded then you can get away with minimal unit tests. But extensive unit tests are absolutely essential to successfully maintaining a sy…
The kind of refactor that you are talking about will require a rewrite of the unit tests. Unit Tests are a burden in restoring. They only stay the same if liskov substitution holds. Which is rare in any significant change.
Re: Why Most Unit Testing Is Waste [pdf]
#70The normal practice for large scale codebases in complex domains is "the code is the spec". That is, the only specification for how the system should work is how it worked yesterday. In that case, unit tests serve as a great specification. Even tests that just duplicate the business code under test and assert that it's the same (A huge waste in normal cases) is useful. Because a Unit test is much better than a word d…
The article addresses your first point in a big section: 1.4 The Belief that Tests are Smarter than Code Telegraphs Latent Fear or a Bad Process Your other point about only good programmers not needing unit tests is moot as you haven't followed it through to the conclusion. Namely, if bad programmers write bad code that needs unit tests, they're also going to write bad unit tests that don't test the code correctly. S…
>, if bad programmers write bad code that needs unit tests, they're also going to write bad unit tests that don't test the code correctly. So what's the point?
I don't think one necessarily follows the other. Consider the examples of test pseudo-code:
EXPECT[image_to_text("photo1.jpg") == "cat"]
EXPECT[image_to_text("photo2.jpg") == "dog"]
EXPECT[image_to_text("photo3.jpg") == "apple"]
EXPECT[image_to_text("photo4.jpg") == "Barack Obama"]
(Yes, there's also variability of skill in writing test code. An experienced programmer will think of extra edge cases that a novice will not and therefore, write more comprehensive tests.)
That still doesn't change the fact that writing code that tests to check for correct results is often easier than writing the algorithm itself.
It also contradicts the notion that refactoring code requires that the test code be refactored. That's not always true. Since the jpg file format came out in 1992, that test code could have been written in that same year and it would still be valid today, 25 years later. We still want to ensure the image_to_text() function will return "cat" for "photo1.jpg" even if we refactor the low-level code to use the latest A.I. algorithms.
Another example of a regression test to test performance:
TIMING[EXPECT[image_to_text("photo1.jpg") == "cat"] To do a meta-analysis of the debate with Coplien: a big reason we're arguing his conclusion is that he complains about this abstract thing he called "unit tests". If he had copy-pasted concrete examples of the bad unit testing code into his pdf, maybe more of us would agree with him? In other words, I'd like to see and judge for myself if Coplien's "unit tests" look like the regression tests that SQLite and NASA use to improve quality -- or -- if they are frivolous tests that truly wastes developers' time. Since he didn't do that, we're all in the dark as to the "test code waste" he's actually complaining about.