>Testing is usually regarded as an important stage of the software development cycle. Testing will never be a substitute for reasoning. Testing may not be used as evidence of correctness for any but the most trivial of programs. Software engineers some times refer to "exhaustive" testing when in fact they mean "exhausting" testing. Tests are almost never exhaustive. Having lots of tests which give the right results may be reassuring but it can never be convincing. Rather than relying on testing we should be relying in reasoning. We should be relying on arguments which can convince the reader using logic.
http://www.soc.napier.ac.uk/course-notes/sml/introfp.htm
The tragedy of 100% code coverage (2016)
281–290 of 346 posts
Re: The tragedy of 100% code coverage (2016)
#282I think a bigger epidemic is we're putting too much emphasis on "do this" and "do that" and "if you don't do this then you're a terrible programmer". While that sometimes may be true, much more importantly is to have competent, properly trained professionals, who can reason and think critically about what they're doing, and who have a few years of experience doing this under their belt. Just like other skilled trades…
That's why I'm highly skeptical when I hear the word "best practices". Sure, the intention is good, but it promotes mindless repeating of patterns over thinking about what really helps.
On the other hand, every discipline that I can think of has its own set of best practices. Why should software development be any different? I know a lot of people are susceptible to the mindless repeating, but that's not a fault of best practices.
Re: The tragedy of 100% code coverage (2016)
#283Earlier quoted context omitted.
"Does static typing stunt prototyping and exploration?" Not in my experience. It is probably about what you personally find hard/uncomfortable/unfamiliar.
But you're kind of implying that I feel that way because either I don't use static typing much, or I haven't learned it well, aren't you? I've used static typing much more than dynamic. I'll admit confusion on things like ocaml polymorphic variants and haskell monads, nevertheless I wouldn't say I find static typing hard as a rule. But surely the point of static type checking is to constrict what you can do for safet…
It depends on how often that freedom is used for a benefit rather than by accident and thus foot shooting.
Re: The tragedy of 100% code coverage (2016)
#284The tragedy he mentions aren't 100% code coverage. It's more about people using the wrong tools for the job and using 100% coverage as an indication that everything is fine. 100% coverage for my team means that we were intentional about our code. It's not hard at all to have 100% coverage in a Ruby application as it is possible to do a lot with very little code. Furthermore it allows us to bring in a junior on the te…
Exactly this. One problem with TDD, etc. is that you are limited by the quality of your tests. 100% means nothing if you don't have a try/catch around a cache lookup that could should never fail but could and not get handled.
No, that isn't a limitation with TDD. That is a limitation of imagination, and can happen to any programmer. (proving the code correct is most likely to catch this - but even that can fail in some cases)
TDD just means that when discover the bug and add the missing try/catch you don't accidentally break something else.
Re: The tragedy of 100% code coverage (2016)
#285Earlier quoted context omitted.
There can be circumstances in which the substituted operator is just as valid as the operator being substituted for, resulting in false positives.
How would that be a false positive? If the substituted operator does not cause a test to fail, then your tests don't cover it. If the function of the program is not changed by changing the operator, then the operator does nothing and should be removed. That's the idea at least. All focus on high coverage, whether line coverage or mutation testing, creates an incentive to remove redundant robustness checks, and maybe…
I was thinking that, for example, that a >= test is as valid as == in some cases. If == is actually used and is correct, substituting >= would not cause any valid test to fail. As I wrote this, however, I realized that if you substitute the inverse operator (!= for ==, = etc.) and it is not caught, you can infer it is not covered. (edit: or that, in the context of the program as a whole, the result of the operation is irrelevant. I imagine this might legitimately happen, e.g. in the expansion of a macro or the instantiation of a template.)
Re: The tragedy of 100% code coverage (2016)
#286Build tests against your app`s public interface. On a web app, that would be your controllers or API. That will give you good coverage, while avoiding too simple to be useful unit tests. It's really hard to foresee all possible input variations and business logic validations, but that doesn't mean your test suite is useless. It just means it will grow everytime you find a new bug and you are guaranteed that one won't…
On a small app that is a good idea. However on a massive app you need to break things down more. I work on a project with > 10 million lines of code, there is no way any human can understand everything in detail. As a result I need to test at a smaller scale which is not my public interfaces, but my interfaces to other people.
Re: The tragedy of 100% code coverage (2016)
#287I once joined a company that had 90% code coverage. After a while it became clear that there were all vanity tests: I could delete huge swathes of code with zero test failure. We let the contractors that wrote it move on, and we formed a solid team in house. We don't run code coverage any more because it makes the build run four times slower. Instead, I trust our teams to write the good tests. Sometimes that means Some feedback on the article:
>Test-driven development, or as it used to be called: test-first approach
Test-first is not the same as Test-Driven. The test-first approach includes situations where a QA dev writes 20 tests, and then hands them to an engineer who implements them. Thats not TDD.
>"But my boss expects me to write test for all classes," he replied.
That's very unlikely to be TDD. "Writing tests because I've been told to" is never likely to be "I'm writing the tests that I know to be necessary", and that's all TDD is: writing necessary tests. If the test isn't necessary, then neither is the code.
>Look, if that imaginary evil/clueless developer comes and breaks that simple code, what do you think he will do if a related unit test breaks? He will just delete it.
Sure. But then their name is on that act in the commit log. The test is a warning. I've been lucky not to have worked with evil developers, but I have worked with some clueless ones, and indeed some have just deleted tests. Thats an opportunity for education, and quality has steadily improved.
>The tragedy is that once a "good practice" becomes mainstream we seem to forget how it came to be, what its benefits are, and most importantly, what the cost of using it is.
Totally agree. So many programmers and teams practice cargo cult behaviors. Unfortunately, this article is one of them: making claims about TDD, and unit tests in general, without understanding "why" TDD is effective.
Re: The tragedy of 100% code coverage (2016)
#288Earlier quoted context omitted.
By far the best codebase I ever worked with had zero test cases, was nearly 30 years olds, been worked on by not just several teams but several companies, had basically zero documentation, and was still easy to read and understand. Unit tests only really catch a tiny fraction of bugs relating to defective code, not poor design, poor market fit, poor understanding of requirements, or any of the other bugs that most of…
I worked on one such system (written in Ada, of all things) back when I was a fairly junior developer. I was hired to work on a major effort to port the system from SPARC/Solaris to RHEL/x86 and add some new features. It was designed by a crusty (and eccentric) old architect (who was still around to manage this major refresh project) who ruled with an iron fist, and the project was managed in classic waterfall fashio…
That would be an ... interesting work environment. Probably the dev version of benevolent dictator, works great until the leader leaves, and the team is left with weaker control structures.
Re: The tragedy of 100% code coverage (2016)
#289Most of "we should go for 100% coverage" is simply cargo-culting (pushed by "gurus" like Uncle Bob - the negative aspects of the word guru implied)
Not to mention 100% coverage is not guarantee the system works, in practice quite the opposite
Not to mention this BDD crap which only makes my blood boil, it's syntactic yuck disguised as syntactic sugar
Re: The tragedy of 100% code coverage (2016)
#290Most of "we should go for 100% coverage" is simply cargo-culting (pushed by "gurus" like Uncle Bob - the negative aspects of the word guru implied) Not to mention 100% coverage is not guarantee the system works, in practice quite the opposite Not to mention this BDD crap which only makes my blood boil, it's syntactic yuck disguised as syntactic sugar
>Most of "we should go for 100% coverage" is simply cargo-culting (pushed by "gurus" like Uncle Bob - the negative aspects of the word guru implied)
I came here just to search for Uncle Bob to see the fun comments talking about him.
No, he does not recommend 100% coverage.
He is against writing tests for "basic" code like getters/setters.
I can understand the irritation with Uncle Bob - especially if you've read his blog. But everyone seems to get his stance on unit tests wrong - including Uncle Bob fans.
His fans say he advocates for 100% code coverage.
His detractors say he advocates for 100% code coverage.
All while Uncle Bob is saying he doesn't cover "trivial" code.