Live data from Hacker News

The tragedy of 100% code coverage (2016)

labs.ig.com

151–160 of 346 posts

Re: The tragedy of 100% code coverage (2016)

#151
post #141

A lot of people here seem to have strong opinions against 100% coverage, so I'll risk their ire with my strong opinion in favor. If you have, say, 95% coverage -- and most corporate dev orgs would be thrilled with that number -- and then you commit some new code (with tests) and are still at 95%, you don't know anything about your new code's coverage until you dig into the coverage report. Because your changes could…

The problem with the "100%" goal is that it provides a false sense of safety. There can still be plenty of bugs in code that has "100%" test coverage. The 100% are an illusion. To truly get to 100%, it does not suffice that every line of code is executed, you must also test all valid inputs - which is utopic.

Re: The tragedy of 100% code coverage (2016)

#152

Earlier quoted context omitted.

I don't see how that's a "tragedy", as in " an event causing great suffering, destruction, and distress, such as a serious accident, crime, or natural catastrophe ". You're also making it sound like somebody promised you that tests can prove the absence of bugs, when that was never the bargain and smart people have already told you so, probably before you were born even. > Testing shows the presence, not the absence…

If we're being pedantic, a tragedy is _a drama or literary work in which the main character is brought to ruin or suffers extreme sorrow, especially as a consequence of a tragic flaw, moral weakness, or inability to cope with unfavorable circumstances_. Definitely not a tragedy.

If we're being pedantic, there's a second common definition of the word tragedy that does not refer to a drama or literary work, but "an event causing great suffering, destruction, and distress, such as a serious accident, crime, or natural catastrophe."

I think when you take all of the time wasted on useless tests written merely for the sake of having tests, that waste is tragic. You could be doing anything else with that time.

Re: The tragedy of 100% code coverage (2016)

#153

The worse the developer, the more tests he'll write. Instead of writing clean code that makes sense and is easy to reason about, he will write long-winded, poorly abstracted, weird code that is prone to breaking without an extensive "test suite" to hold the madness together and god forbid raise an alert when some unexpected file over here breaks a function over there. Tests will be poorly written, pointless, and give…

This hits home with me. Incompetent developers need dogmatisms (100% unit coverage!!!! etc)

No, inexperienced developers need dogmatisms. Subtle difference perhaps, but it's worth remembering that every competent developer started out being inexperienced, and that it's nothing to be ashamed of.

Re: The tragedy of 100% code coverage (2016)

#154
I noticed the author was speechless in two situations, both of which involved "but we write all our tests in ." This is legitimate and should be taken more seriously by the author.

Codebases serve businesses and businesses value legibility over efficacy. It's more important to them to have control over their assets than to have better assets. Using one test framework is in perfect service of that goal.

It's inefficient in that it will take future developers more time to understand that code. But fewer architectural elements means that you can get by with less senior programmers.

Imagine if you went onto a software project and they were using 6 different databases because every time they had a new kind of data that they wanted to access differently, they reached for another database rather than use the one they had.

Of course nobody would ever do that, well I hope anyway, but I do see a lot of unnecessary architectural complication in projects in service of "using the right tool for the job." And it can balloon. A new test framework has to work in your CI framework. You need to decide how to handle data. It's not a huge decision, but it's more complicated then most devs would think and it'll take up more of your time than you'll expect.

You can generalize this to the the main thrust of the article. 100% code coverage is not a bad goal to want to hit. Sure, you're going to get a lot of waste. But you're not paying for it, your employer is. And your employer might have a different idea of which side of the tradeoff he wants to be on and where to draw the line. You know the code way better than they will, but they know the economics far better than you ever could.

Re: The tragedy of 100% code coverage (2016)

#155
post #126

The article illustrates what happens when you have inexperienced or poor developers following a management guideline. To see how 100% coverage testing can lead to great results, have a look at the SQLite project [1]. In my experience, getting to 100% takes a bit of effort. But once you get there it has the advantage that you have a big incentive to keep it there. There is no way to rationalise that a new function doe…

Doesn't what you're writing actually influence whether 100% coverage is a worthy goal or not?

I mean, SQLite is a good example of something where 100% coverage would actually be useful, because it tries to maintain compatibility with the SQL spec and with Postgres (largely because Postgres complies with the spec). Testing that 100% makes a lot of sense.

Suppose you're instead doing something very UX/UI driven. Why bother trying to cover that 100% with automated tests, when change is going to be driven by the whim and fancy of anyone who sits down in front of it?

Re: The tragedy of 100% code coverage (2016)

#156
But remember nothing is free, nothing is a silver bullet. Stop and think.

I'm going to be the one to point at the elephant in the room and say: Java. More precisely, Java's culture. If you ask developers who have been assimilated into a culture of slavish bureaucratic-red-tape adherence to "best practices" and extreme problem-decomposition to step back and ask themselves whether what they're doing makes sense, what else would you expect? These people have been taught --- or perhaps indoctrinated --- that such mindless rule-following is the norm, and to think only about the immediate tiny piece of the whole problem. To ask any more of them is like asking an ostrich to fly.

The method names in the second example are rather WTF-inducing too, but to someone who has only ever been exposed to code like that, it would probably seem normal. (I counted one of them at ~100 characters. It reminds me of http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom... )

Many years ago I briefly worked with enterprise Java, and found this sort of stifling, anti-intellectual atmosphere entirely unbearable.

Re: The tragedy of 100% code coverage (2016)

#157
To paraphrase: "Premature testing is the root of all evil".

How I do it is going from rough testing of pages and components to granular testing of those parts which had some error.

For pages, I just run them to see if they display without producing errors, same goes for critical components. This gets me the feeling of roughly tested and from the user perspective working system with little time investment.

Then I test critical business logic, but usually only after some error was reported.

Mind though that I am freelance developer unconstrained by organizational rules.

Re: The tragedy of 100% code coverage (2016)

#158
I don't think I know anyone that do TDD. Uncle Bob has indoctrinated a few zealots into that mindset, but it all comes off as crazy to me. A germ of a good idea taken way too far.

People of that school tend to write tests that test implementation rather than functionality. As a result you get fragile tests that break not telling you what went wrong but how the implementation has changed.

Good tests should test behavior. A change in implementation shouldn't break the test.

Re: The tragedy of 100% code coverage (2016)

#159

But remember nothing is free, nothing is a silver bullet. Stop and think. I'm going to be the one to point at the elephant in the room and say: Java. More precisely, Java's culture . If you ask developers who have been assimilated into a culture of slavish bureaucratic-red-tape adherence to "best practices" and extreme problem-decomposition to step back and ask themselves whether what they're doing makes sense, what…

Method names got my eyebrows up as well and I spend a lot of time in Java. I should consider myself lucky.

Re: The tragedy of 100% code coverage (2016)

#160

The worse the developer, the more tests he'll write. Instead of writing clean code that makes sense and is easy to reason about, he will write long-winded, poorly abstracted, weird code that is prone to breaking without an extensive "test suite" to hold the madness together and god forbid raise an alert when some unexpected file over here breaks a function over there. Tests will be poorly written, pointless, and give…

> The worse the developer, the more tests he'll write. As always, generalization is the tool of the fool (sorry for the fool part, but it rhymes ;) ). Writing pointless stubs / mocks and testing execution order of statements is definitely a bad pattern, writing many and good functional, e2e and integration tests however is not.

100% agree. Full code coverage with acceptance and integration tests is really good. If you can get a test case for every feature that your app is supposed to have you can develop quickly and ship often knowing that nothing will break. Extensive unit tests that don't test what features do but test how features are implemented are usually a waste of time in my opinion, unless there is some complicated and critical algorithm that you need more visibility and protection for. Otherwise the only purpose of those unit tests is to break for good refactors and double dev time.

When You are cleaning up tech debt and make hundreds of lines of changes and no acceptance tests break (because everything is still working as it should) but 50 unit tests break... I've wasted so much of my life

Post reply on HN