The Myth of Code Coverage

I don't think so. I once took on the challenge of writing as many unit tests as possible for a project at work (the project did not have unit tests - but was well covered with other types of tests).

The two key takeaways I got from the effort:

First, I had no idea how coupled my code was until I tried writing many unit tests for it. If it's hard for you to instantiate your class without involving N other libraries/objects, your code is very coupled. No one in my team would have looked at the code and said "It's highly coupled". The real proof was "Can you test this in isolation?" If not, you're strongly coupled.

I had to redesign a lot of bits to succeed, and as another commenter pointed out, in my attempts to do so my code really was a lot better. I discovered good principles of design in doing it.

The second thing I learned, which may appear to disagree with the first: There are always easy ways to write code to be unit testable. Most of those easy ways are bad and reduce the "quality" of your code. Forcing yourself to not redesign just "for the sake of writing tests", while still ensuring you have 100% code coverage, will really force you to think heavily about your code, architecture, failure points, etc.

So a 100% code coverage really doesn't tell you if you have good code. But less than 100% does indicate potential problems.

(I personally did not go for 100%, FYI).

I think code coverage (particularly condition coverage) is mostly useful for one purpose:

You need to get it close to 100% before you can use mutation testing.

Without mutation testing you really shouldn't be particularly confident that your tests are testing much of anything useful.

> Could not figure out how to trigger a malloc failure inside of a imported library

Glibc has (or at least has had) special (unsupported) instrumentation that lets you substitute out the libc malloc at runtime with another function.

You can use this to shim in a version that fails after a specific number of executions in order to inject malloc faults.

E.g. https://github.com/xiph/opus/blob/master/tests/test_opus_api...

All these discussions in the comments about coverage percentages would be so much more insightful if everyone mentioned the kind of code they work on. Client-side JS? Real-time video processing firmware? A language support library? Automotive middleware? Java for an ATM? An ML library wrapper? Personal website backend in Rust? Android game engine? Cosmic particle simulation?

I don't even care about the implied "what could go wrong" but about the size of the input spaces and the complexity of the algorithms behind them. I certainly don't subscribe to the idea that there is a one-size-fits-all testing strategy for all the above, and I see little gain in arguing for or against a certain approach with someone who has entirely different constraints and code architectures to work with.

Require 100% code coverage on an async task library? Sure, go for it. 100% coverage on a (deep-learning-free) algorithm identifying constellations in pictures of the night sky? Do you really want to have to violate every single assumption one by one in your unit tests? How do you even get that input data (and who will create more nonsense data when you improve the algorithm)?

Or in other words: For any reply in this thread you can probably come up with a scenario where it makes sense to operate as described in it. There is some nice perspective but overall I find it a bit unactionable...

Code coverage as a single percent is low value and mostly management fodder.

Code coverage as a percent per module has more value.

Code coverage as highlighted source calling out uncovered code, missed branches, etc is useful during development to help you remember to test stuff.