Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The tragedy of 100% code coverage (2016)

labs.ig.com

Post reply on HN
Print view View original

31–40 of 346 posts

Re: The tragedy of 100% code coverage (2016)

#31

Post by devrandomguy »

If you can prove that your testing process is perfect, then your entire development process can then be reduced to the following, after the test suite is written:

  cat /dev/random | ./build-inline.sh | ./test-inline.sh | tee ./src/blob.c &&
  git commit -Am "I have no idea how this works, but I am certain that it works perfectly, see you all on Monday!" &&
  git push production master --force
When presented like this, relying on human intelligence and experience doesn't seem like such a bad thing after all.

Just so we're clear, my username was not inspired by this scheme.

Re: The tragedy of 100% code coverage (2016)

#32

Post by sbov »

cbanek wrote: post #27
pacaro wrote: post #25

Earlier quoted context omitted.

This was one of the fun things about working on Midori, the whole system was available to us, components all communicated through well defined interfaces, you really could (if necessary) mock the interfaces above and below and be sure that your logic was being exercised. The browser team, in particular, pushed hard to be close to 100% coverage. The overall number for the project was (IIRC) in the 85% range. When I'm…

> As an aside, while I'm rambling, all the examples in the article appeared to represent unnecessary abstraction, which is the opposite problem. One other thing about code coverage, unit testing, and other testing fads is I think they actively affect the architecture, and usually in the overthinking it way. Instead of having one tight bit of procedural code (which may have some state, or some dependency calls), peopl…

It's strange to me that we continue to make languages that force us to make certain architectural choices to help facilitate testing.

Re: The tragedy of 100% code coverage (2016)

#33

Post by mdpopescu »

pacaro wrote: post #25
cbanek wrote: post #18

I've had to work on mission critical projects with 100% code coverage (or people striving for it). The real tragedy isn't mentioned though - even if you do all the work, and cover every line in a test, unless you cover 100% of your underlying dependencies, and cover all your inputs, you're still not covering all the cases. Just because you ran a function or ran a line doesn't mean it will work for the range of inputs…

This was one of the fun things about working on Midori, the whole system was available to us, components all communicated through well defined interfaces, you really could (if necessary) mock the interfaces above and below and be sure that your logic was being exercised. The browser team, in particular, pushed hard to be close to 100% coverage. The overall number for the project was (IIRC) in the 85% range. When I'm…

> If you have many methods in a class with only one basis path, what purpose does the class serve.

It ties together related algorithms. You want "addDays(n)", "addHours(n)", "addMinutes(n)" and so on to be in the same class, even if they're one-liners.

Re: The tragedy of 100% code coverage (2016)

#34

Post by lacampbell »

I feel like this high test coverage thing can only work if you have tight modules, tight interfaces, and you only bother testing at module boundaries. So the test cases almost function as a bit of executable API documentation - here's the method name, here's what it does, here's the contracts and/or static types, and.... given this input, you should get this output.

Do it for the high level bits you actually expose. If you're exposing everything, tests won't really save you - architecture and modularity are more fundamental and should be tackled first. If you're writing a big ball of mud, what benefit do you get testing a mudball?

Re: The tragedy of 100% code coverage (2016)

#36

Post by cbanek »

sbov wrote: post #32
cbanek wrote: post #27

Earlier quoted context omitted.

> As an aside, while I'm rambling, all the examples in the article appeared to represent unnecessary abstraction, which is the opposite problem. One other thing about code coverage, unit testing, and other testing fads is I think they actively affect the architecture, and usually in the overthinking it way. Instead of having one tight bit of procedural code (which may have some state, or some dependency calls), peopl…

It's strange to me that we continue to make languages that force us to make certain architectural choices to help facilitate testing.

Indeed. It's almost as if testing should be built into the language itself. (I'm always a big fan of including self-test code in projects)

Re: The tragedy of 100% code coverage (2016)

#37

Post by state_less »

Unit tests are a poor substitute for correctness. Many unit tests does not a strong argument make.

Unit tests are typically inductive. Developer shows case A, B and C give the expected results for function f. God help us if our expectations are wrong. So, you're saying since A, B and C are correct therefore function f is correct. Well that may be, or maybe A, B and C are trivial cases, in other words, you've made a weak argument.

100% test coverage sounds like lazy management. Alas, the manager may have worked their way via social programming rather than computer programming. In such cases, better to say you have 110% test coverage.

Re: The tragedy of 100% code coverage (2016)

#38

Post by jowiar »

lacampbell wrote: post #24
ben_jones wrote: post #14

Earlier quoted context omitted.

IMO ruby is the worst because its so easy, and often times encouraged, to write layers of useless tests that ultimately give only a false sense of security. I recently re-did the Michael Hartl tutorial to catch up with Rails5 and it gives examples where you test proper template rendering. I love that book but I'd rather shoot myself in the foot then spend dev time writing tests to check if the right html title attrib…

Curious as to why you're blaming a language (ruby) for a bad practice (writing layers of useless tests). Excuse my pedantry, but it wouldn't it be more accurate to blame a culture that you feel has grown up around that language?

In Scala, "does it compile/typecheck" will generally catch typos in code that isn't frequently executed, and raise warnings for unhandled cases. For Ruby, you pretty much need to execute the code to have any inkling if it goes kaboom -- the language (and how people write it) have so much magic that short of executing things, the computer can't tell you anything useful.

Re: The tragedy of 100% code coverage (2016)

#39

Post by waibelp »

This remembers me of recent projects where developers started to mock every piece of code.. The result was that all tests passed while the codebase exploded in real environments.

In my opinion the best advice is to force developers to use their brains. I know, there are a lot of sh*tty CTO/CEO/HoIT/SomeOther"Important"Position people out there seeing them as code monkeys and saying that developers are not paid to think but in that case the best thing developers could do is learn to say "NO"... My experience with that kind of people is that they need to learn the meaning of "NO" instead of wasting time and money in the end of the day.

Post reply on HN