It resulted in a mug being made in the same style as our company mugs but with the words "Fuck Coverage" instead of the company name.
Eyebrows were raised when it was mistaken for a 'real' mug and taken in to a board meeting by a manager however..
21–30 of 53 posts
It resulted in a mug being made in the same style as our company mugs but with the words "Fuck Coverage" instead of the company name.
Eyebrows were raised when it was mistaken for a 'real' mug and taken in to a board meeting by a manager however..
So, don't bother writing any tests unless you can test everything? At a previous shop, we had a simple test that ensured user could log in for a while, until we created more coverage. We didn't test every aspect of the code. This simple sanity check saved us once or twice. Should we have not written this test just because we couldn't test every conceivable piece of code? Don't let perfect be the enemy of good...
Are you covering 100% of code interactions? How much of your integration logic are you mocking? Are you covering 100% of HTML and interface logic? How much code is not in your code? What about packages and imports? It's incredibly arbitrary for no good reason except to slow development of a product that may depend on fast development for survival in trade to test for bugs that honestly, actually, may never happen. To be clear, testing is good, but you write tests to serve the code and product, not the other way around.
Eg. complex math functions, it'd be very easy for them to be not quite right and you would never notice without tests, and also they will probably never change.
An area that would be stupid to test (with automated tests) for example, would be a UI menu.
* The test (ie. replicating a user pressing buttons via code) is probably gonna be more complex and less maintainable than the code its testing.
* If there is a bug it will be obvious in human acceptance testing.
* UI menus are gonna change drastically during development.
Anyway, I could see a misguided requirement for 100% code coverage massively hampering a project and potentially killing it.
Test coverage matters even if it is not 100%, especially if it covers the most important use-cases. Cost and time always needs to be considered. 100% coverage may not be worth it. Coverage based on LOC is also an arbitrary metric in some sense, the author himself has examples of that. From a business POV, use-case coverage is more important.
Conversely perfect or high coverage is not a positive signal per se, but rather just the absence of a negative signal
The author is either crazy, or else maybe works in a very niche area where this makes sense. I work in games, and some modules are very conducive to automated testing. Eg. complex math functions, it'd be very easy for them to be not quite right and you would never notice without tests, and also they will probably never change. An area that would be stupid to test (with automated tests) for example, would be a UI menu…
You can even do model based testing.
For example, you're testing a word processor. You know that your state is no document. Then you click the open file button. Press some other buttons, and knowing what button it is, you should expect some change.
That being said, usually you can "press" the buttons virtually, for example sending window messages or calling the same functions as the buttons wire up to, rather than knowing the pixel layout.
People have been doing these things for years.
Earlier quoted context omitted.
Seems like 50% coverage is better than 0% coverage and 75% coverage is better than 50% coverage. The author argues about the horrors of going from 80% to 79.9%, but I don't see how that is any different from the suggestion to explicitly exclude code. The excluded code is still in the gray area of code that is included but not covered by tests. What happens when someone starts making breaking changes to excluded code?
Lots of code bases have code coverage in some middle range, lets say it's 82.34%. What does that number tell you? Did the team write 7.64% more code this month that doesn't have test cases? The point you're making is actually irrelevant to the discussion. If you explicitly excluded code, you probably have a reason for that and you take your chances. Ideally, in a perfect world, you'd have no excluded code.
Also, this method doesn't show when you remove excluded code or add coverage for it. In a perfect world you should have no excluded code, and in a perfect world you should have 100% coverage.
World isn't perfect though, and a good working product with critical features is better than one lacking important features but with '100%' coverage.
Well, with Python, the compiler doesn't do type checking, so 100% coverage (or close) is very important. One of his cases of "what do you not need to check" is error-handling. Yet, having something in production that should raise an exception that gets handled versus having the exception blow up and crash the system is a critical system stability defect. And in fact, in my personal code, since I tend to try to create exception messages that are as specific as possible, I must admit with chagrin that screwing up a ''.join() to create the exception message is one of my most common errors. I at least touch-test my exception code, without exception.
And then... line coverage? Oh my, that is just the beginning. I pretty much always use the hypothesis package so that everything get exercised with a large variety of inputs. Inputs that I didn't have to think up, and avoid my pre-existing biases of what might get thrown at the function.
Line coverage does do much to really ensure that you have reached "interesting conditions". I wish Python had a good tool for that kind of instrumentation. Line coverage is really only a "touch-test" and mainly useful as a check-in gate, not as production testing.