31–40 of 53 posts
Is there a difference between 79.9% and 80%? Maybe not. My team is increasing test coverage for a legacy code base. 60% to 80% DOES matter. And it’s not just the fact that the tests flags regression, but the very process of increasing test coverage uncovers bugs / dead code.
The irony to me is that the examples he provides for things not to cover are exactly things I think should be covered. Short-circuit logic is ripe for bugs and the code that causes the short-circuit should definitely be tested!
raise ValueError(' '.join('meaningful message with:', parameter)
Test coverage is a very dangerous metric. IMO it will only be useful the day that we find a way to not take into account those tests that are testing implementation details. Every time that I see a high number being enforced, the end result is always a disaster: lots of poor quality tests that are only there in order to bump up that metric. Those tests are poison: they don't aim to test correctness, they make refacto…
Your team ends up spending half their time writing painful depressing tests that are bad afterthoughts. Generally with TONS of mocks and little real meaning.
Some lines of code should be tested twice to ensure they're correct.
I'm serious. I think we're being trolled here everyone.
While that statement is itself is somewhat questionable, it's a little better and IMO reflects the purpose of unit tests better. Usually, if I'm finding it hard to write or setup a test, that's a signal that something's off. Also, UTs are absolutely lovely when you're refactoring code - as long as code tests invariants and publicly visible behavior instead of implementation. So even if the part you're refactoring has say 60%, you'd still be in a much more confident of making changes.
Code coverage under anything other than unit tests can be very misleading - for ex I've seen places where system & integration tests are run on instrumented code (and little to no unit tests) and since a huge swathe of code will be hit (mostly happy path), there's a false sense of code quality and safety which is probably more dangerous than no tests.
Overall, "When a measure becomes a target, it ceases to be a good measure." applies most often to test coverage.
As someone who worked somewhere where 100% code coverage was practically required, while it does point out flaws in your testing where you may have "missed a spot." It still doesn't prove that you're testing everything. You never know if a library function you are calling beneath your code has a different path it can also follow. > I believe the only level of test coverage that's worth it is 100% of the lines you wan…
if myTest:
myVar := “foo”
print(myVar)
As someone who worked somewhere where 100% code coverage was practically required, while it does point out flaws in your testing where you may have "missed a spot." It still doesn't prove that you're testing everything. You never know if a library function you are calling beneath your code has a different path it can also follow. > I believe the only level of test coverage that's worth it is 100% of the lines you wan…
This is totally ridiculous. Test coverage clearly matters even if it's not 100%, as anyone who has ever worked on an older codebase knows. As long as you're not causing coverage regressions, it is perfectly acceptable to leave untested code that "should" be tested, but hasn't been changed in years. Test it when you change it; don't test for the sake of achieving a coverage benchmark.
And implying that there is 100% coverage just because the instruction register had all valid values is silly. When writing test one should worry about the other registers too ...
This concern has been nagging me in the back of my mind, but I hadn't convinced myself that code-coverage was insufficient (in practice) until you framed it this way.
Now all kinds of supporting examples come to mind:
- divide-by-0 errors
- floating-point exceptions
- numeric underflow / overflow (both silent and signaled)
The alternative code paths exercised by these situations are generally outside the purview of code-coverage tools, because they reside in the silicon / firmware, process' default signal handlers, etc.