11–20 of 53 posts
There is actual empirical evidence that this is false.
Feels like you didn't the read the (whole) article. The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded . Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.
Earlier quoted context omitted.
Feels like you didn't the read the (whole) article. The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded . Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.
Telling the obvious from the true is the role of science.
There is actual empirical evidence that this is false.
Feels like you didn't the read the (whole) article. The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded . Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.
The author argues about the horrors of going from 80% to 79.9%, but I don't see how that is any different from the suggestion to explicitly exclude code. The excluded code is still in the gray area of code that is included but not covered by tests. What happens when someone starts making breaking changes to excluded code?
The title is disingenuous. A better title would be "100% test coverage should be the goal". Though that is laughably obvious and would generate no clicks, so I can see why the author made the choice they did. That said, if you have a single test, that better than no tests. And 90% is better than 80%. Yes, 100% is ideal, but if you let perfect be the enemy of good, than you waste time when low hanging fruit breaks.
Test the things that are high risk, that you are personally worried about, that have caused defects in the past.
Testing whether the language's getters and setters work is a total fucking waste of time and money.
For a long time, until my current project got off the ground and got stable funding, our entire test suite consisted of one integration test that made sure Spring Boot started up. As time allowed we back filled other tests. But when your ass is on the line you don't waste time.
Earlier quoted context omitted.
Feels like you didn't the read the (whole) article. The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded . Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.
Seems like 50% coverage is better than 0% coverage and 75% coverage is better than 50% coverage. The author argues about the horrors of going from 80% to 79.9%, but I don't see how that is any different from the suggestion to explicitly exclude code. The excluded code is still in the gray area of code that is included but not covered by tests. What happens when someone starts making breaking changes to excluded code?
You... don't. You un-exclude it first, and then, because of the rule, you're not allowed to commit the code until you write the tests that bring it back up to 100% coverage. Then you make breaking changes.
The solution to this dilemma is to make it easy to mark code as "OK not to be tested". It's annoying to have test annotations in code, but I'd expect mostly entire modules will be exempted, so it's not that big a deal. And this way, the coder has to either write tests or make a specific decision not to.
> I believe the only level of test coverage that's worth it is 100% of the lines you want to be covered.
This statement I vehemently disagree with. It seems like the kind of thing someone would hit you with as an excuse for why testing isn't useful at all because it's not perfect. I have found in my experience that simple end to end tests give me the best bang for my buck, and even a test suite that covers 80% of the code is pretty good.
To paraphrase from the Google code reviewer guide: No code is perfect, just try to make it better than it was.
100% test coverage doesn't tell you if you have race conditions, security problems, or networking issues. 100% test coverage doesn't matter much in distributed systems, where you can individually test each component to 100% but still have the system not work as a whole. 100% coverage doesn't mean you're checking that all your returned results are verified and what they should be.
> The problem with having a goal that is not 100% is that it leaves room for interpretation and negotiation.
By handing out edicts like these, you're taking away tools like negotiation and common sense and replacing them with blanket rules. Time is a finite resource and test coverage is only one small part of testing. Going from 95% to 100% might take more time and provide less value than other kinds of testing and other concepts (like UX, or market fit, where you test that what you're building is usable and useful).
Just because you answer all the questions right on the test doesn't mean you have perfect knowledge of the situation. Thinking such is hubris and leads to errors/defects.
By excluding lines and saying only make sure you cover what you're testing you're intentionally making blind spots. Just let the coverage number be what it is. Look at your coverage reports. Check the coverage of key components.
> I think it is a perfectly fine decision to exclude some code from coverage if the tests are too expensive to write (because of the required setup for instance).
No, no, no, no, no x a million. This is the fallacy that if you unit test each component, then when you put it together it's perfect because each part is perfect. You still have to do integration tests and complicated setup tests. If a test takes a long time, try to make it shorter, don't try to avoid testing it. Anything you avoid testing is where bugs will nest and code will churn.
To sum up, most of the important, complicated, and hairy bugs are not found by unit testing. It is the simple bugs that are found by unit testing. These are important to get out of the way because a system with a thousand cuts can still kill you, but in no way is 100% coverage and tests passing mean anything other than 100% coverage and tests passing.
This is totally ridiculous. Test coverage clearly matters even if it's not 100%, as anyone who has ever worked on an older codebase knows. As long as you're not causing coverage regressions, it is perfectly acceptable to leave untested code that "should" be tested, but hasn't been changed in years. Test it when you change it; don't test for the sake of achieving a coverage benchmark.
When writing test one should worry about the other registers too ...
Earlier quoted context omitted.
Feels like you didn't the read the (whole) article. The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded . Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.
Seems like 50% coverage is better than 0% coverage and 75% coverage is better than 50% coverage. The author argues about the horrors of going from 80% to 79.9%, but I don't see how that is any different from the suggestion to explicitly exclude code. The excluded code is still in the gray area of code that is included but not covered by tests. What happens when someone starts making breaking changes to excluded code?
The point you're making is actually irrelevant to the discussion. If you explicitly excluded code, you probably have a reason for that and you take your chances. Ideally, in a perfect world, you'd have no excluded code.