Test coverage only matters if it's at 100%

The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded. Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.

Re: Test coverage only matters if it's at 100%

#6

Post by blumomo » Fri, Sep 06, 2019, 5:40 PM UTC

wvenable wrote: post #5

mpweiher wrote: post #2

There is actual empirical evidence that this is false.

Feels like you didn't the read the (whole) article. The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded . Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.

That's exactly what we practice. And it works quite well. Can only recommend this methodology.

Re: Test coverage only matters if it's at 100%

#7

Post by mpweiher » Fri, Sep 06, 2019, 5:44 PM UTC

wvenable wrote: post #5

mpweiher wrote: post #2

There is actual empirical evidence that this is false.

Feels like you didn't the read the (whole) article. The author makes a good point -- everything should either be covered by tested or explicitly marked as excluded . Having a gray area of code that is included but not covered by tests is a recipe for missing something. That much seems obvious.

Telling the obvious from the true is the role of science.

Re: Test coverage only matters if it's at 100%

#8

Post by aluminum96 » Fri, Sep 06, 2019, 5:46 PM UTC

This is totally ridiculous.

Test coverage clearly matters even if it's not 100%, as anyone who has ever worked on an older codebase knows.

As long as you're not causing coverage regressions, it is perfectly acceptable to leave untested code that "should" be tested, but hasn't been changed in years.

Test it when you change it; don't test for the sake of achieving a coverage benchmark.

Re: Test coverage only matters if it's at 100%

#9

Post by kempbellt » Fri, Sep 06, 2019, 5:47 PM UTC

So, don't bother writing any tests unless you can test everything?

At a previous shop, we had a simple test that ensured user could log in for a while, until we created more coverage. We didn't test every aspect of the code. This simple sanity check saved us once or twice.

Should we have not written this test just because we couldn't test every conceivable piece of code?

Don't let perfect be the enemy of good...

Re: Test coverage only matters if it's at 100%

#10

Post by amflare » Fri, Sep 06, 2019, 5:47 PM UTC

The title is disingenuous. A better title would be "100% test coverage should be the goal". Though that is laughably obvious and would generate no clicks, so I can see why the author made the choice they did.

That said, if you have a single test, that better than no tests. And 90% is better than 80%. Yes, 100% is ideal, but if you let perfect be the enemy of good, than you waste time when low hanging fruit breaks.