Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The tragedy of 100% code coverage (2016)

labs.ig.com

Post reply on HN
Print view View original

321–330 of 346 posts

Re: The tragedy of 100% code coverage (2016)

#321

Post by lacampbell »

plinkplonk wrote: post #245
lacampbell wrote: post #86

Earlier quoted context omitted.

But you're kind of implying that I feel that way because either I don't use static typing much, or I haven't learned it well, aren't you? I've used static typing much more than dynamic. I'll admit confusion on things like ocaml polymorphic variants and haskell monads, nevertheless I wouldn't say I find static typing hard as a rule. But surely the point of static type checking is to constrict what you can do for safet…

"you're kind of implying that I feel that way because either I don't use static typing much, or I haven't learned it well, aren't you?" No. I didn't mean to imply or assert anything of the sort. Apologies if I inadvertently gave offense. You seem to be reading meanings into my reply that aren't there. I don't know you from Adam. You asked a question in your post. I answered as best as I could. That said, your latest…

I'm not offended. I am happy to talk to open minded static typing advocates. I myself am not really sure where I sit.

You're right, I haven't used ocaml or haskell in production. I did use F# though, which seems to be in the same ballpark as those languages in terms of having algebraic data types and inference and all the rest. I suppose the fact that I still don't really grok polymorphic variants after reading the real world ocaml a few times may say something about my ability, motivation, or at the very least how my brain is wired.

Fundamentally though, a type system is a restriction meant to help the programmer. This restriction must inevitably put you down a certain road when you explore stuff, right?

Re: The tragedy of 100% code coverage (2016)

#322

Post by pkroll »

realusername wrote: post #292
vincnetas wrote: post #47

Earlier quoted context omitted.

What are the languages that have testing not as afterthought?

Definitely Ruby in general, testing is a core of the language culture.

That doesn't mean testing isn't an afterthought in Ruby, the language. It means the culture built up a defense against writing buggy code.

Re: The tragedy of 100% code coverage (2016)

#323

Post by vram22 »

mannykannot wrote: post #319
vram22 wrote: post #315

Earlier quoted context omitted.

You raise some interesting points about issues with the method. I had used the approach in your second paragraph above. I agree that it can have the issue you mention. But is this not more or less the same as the issue that even test code can have bugs in it? But we still use test code. For that matter, even human testers doing manual testing can make mistakes. But we still do manual testing.

I agree that it is a valid technique, and in fact it is widely used: a minimal case is when we use a test account ID to stand for all accounts. The pessimist in me was looking for the exceptions, which is actually not a bad trait when you are testing.

> The pessimist in me was looking for the exceptions, which is actually not a bad trait when you are testing.

Agreed :) Good discussion.

Re: The tragedy of 100% code coverage (2016)

#324

Post by dllthomas »

sidlls wrote: post #300
sqeaky wrote: post #297

Earlier quoted context omitted.

The actual code is the check on the tests. They cover each other, acting as though they are and there is some infinite recursion dishonest. Tests do have a maintenance cost, but it is much lower than the maintenance cost of code without tests. Without manual tests if there a bug it must be caught manually, this costs human time every time the tests are run. That also presumes the humans do the tests correctly, have a…

I don't disagree that tests are good and necessary. I take issue with the view that tests are somehow free, that more testing is better, or that tests are of such marginal cost that more effort developing tests than the product is considered a reasonable use of resources. Also, the notion that "they test each other" is likely to be dangerous.

> Also, the notion that "they test each other" is likely to be dangerous.

About as dangerous as double entry bookkeeping. Of course it doesn't provide any absolute guarantees, but having people state things in multiple contexts and checking their consistency is one of the better approaches we have for finding errors.

Re: The tragedy of 100% code coverage (2016)

#325

Post by clay_to_n »

bpicolo wrote: post #271
mdpopescu wrote: post #267

Earlier quoted context omitted.

> I've never seen a real case for excluding private methods as a testable unit. Here's the reason: We're not writing tests; we're writing SPECIFICATIONS. Private methods are implementation details ; they're not part of the specification.

That's potentially one philosophy on it, sure. Yet that still doesn't preclude private methods from being an independently testable unit. Implementation details are the meat of the whole thing, and so they seem inherently testworthy. The glue code itself, less so. Thanks for that thought though, definitely something I'll think on :)

You should be able to get the code coverage you need (even if 100%) by testing the public API of any class. If this isn't true, it means you have some private methods you can delete.

That's how the unit you're testing is used - through it's public API. The public API is the specification of how it works.

By only testing the public API, you allow yourself maximum ability to refactor in the future, while still maximizing code coverage. It means that simple refactoring (inlining methods, for example) won't break tests. More importantly, a failing test means something is wrong that is potentially relied upon elsewhere. If you test private methods, you will get test failures without the public API of the unit having changed at all.

Your point about "each function is a unit" is fine, you can justify testing private methods with that - but it's inefficient. If it's not necessary to be in the spec (public API), why have you made it so? You're over-complicating the design by locking yourself into implementation details in places where you don't need to.

In my experience I've found that testing private methods directly is a code smell. It shouldn't be necessary.

Re: The tragedy of 100% code coverage (2016)

#326

Post by BigJono »

tumblen wrote: post #242
BigJono wrote: post #71

Earlier quoted context omitted.

I'd recommend aiming for 10-20% on those projects, and also for startups trying to rapidly push an MVP out. Tests have diminishing returns. You want to hit the absolute most crucial ones that give you plenty of bang for buck and even save you time. That means finding the (usually small handful) of functions that implement your most crucial and most complicated business logic, and writing tests for them. Anything past…

How do you know what 10-20% to test? This sort of basic level of decision-making for testing is something I wish I had, but all the tutorials and guides are about 100% code-coverage TDD so it's hard to find a path to to learn reasonable, high ROI testing.

It varies wildly on the type of application you're building. I can only speak for front-end development of complex SPAs w/ React.

Generally, most architectures in this domain have a combination of UI components, a data store, a set of update logic for the data store, and a set of asynchronous controllers that respond to events, interact with APIs, and call the aforementioned update logic.

In React the UI components are declarative, they (generally) contain no logic or algorithms, just a mapping from state to DOM. I see basically zero value in testing these. Bugs are almost always of the 'forgot to actually implement' variety, or are related to the way the page is rendered in a particular browser, rather than the DOM output the components are responsible for.

The data store update logic is usually either simple setters/getters (which don't need testing) or complex data transformations (which do).

The controllers also come in simple and complex varieties. Simple ones (one API call, one data store update once it's resolved) don't need testing. Anything more complex than that probably does.

So those are the two main targets for testing in the apps I build. I generally don't bother with anything else.

There are exceptions though. For example, here's an accordion UI component I built which relies on an asynchronous manual DOM update after the React DOM update has finished resolving. This could almost definitely use tests, if only to help any maintenance developers understand what it's doing.

https://gist.github.com/JonathonAshworth/b401810b965149348d0...

Basically, as long as you have some sort of sane architecture, there should only be a few potential targets for testing, and they should be easily identifiable.

Re: The tragedy of 100% code coverage (2016)

#327

Post by Nomentatus »

dublinclontarf wrote: post #124
iamleppert wrote: post #114

The worse the developer, the more tests he'll write. Instead of writing clean code that makes sense and is easy to reason about, he will write long-winded, poorly abstracted, weird code that is prone to breaking without an extensive "test suite" to hold the madness together and god forbid raise an alert when some unexpected file over here breaks a function over there. Tests will be poorly written, pointless, and give…

I inherited a code base with LOTS of test coverage. Made one small change and everything broke. Looked into the tests, oh the humanity!!! Dropped the test suite entirely, haven't had any issues since. We have VERY good logging and notifications though so anything goes wrong we know all about it.

I knew a guy who used to wire down safety features on old gas stoves. He said similar stuff about how unnecessary these were. He died (along with his daughter) a year or two later because he disregarded what ski areas were and weren't off limits. Something he'd no doubt done before without any problems. You're fine until you aren't.

Re: The tragedy of 100% code coverage (2016)

#328

Post by johnwatson11218 »

LgWoodenBadger wrote: post #230
voidpointer wrote: post #188

Earlier quoted context omitted.

passing 0 to f returns infinity (IEEE 754) Not arguing the point here. It's just a terrible example.

I thought you'd get NaN, not Inf. Of course I'm probably misremembering.

I think what the above returns depends on the language. In java I think there is an actual run time exception for this. The point of the original comment is to show how 100% coverage is easy to achieve but often meaningless.

There is some other more abstract concept related to the classes of input that can possibly be passed to a method. IMHO 100% coverage and "test first" have done more harm than good to the cause of automated testing.

Re: The tragedy of 100% code coverage (2016)

#329

Post by ubernostrum »

voidpointer wrote: post #181
ubernostrum wrote: post #90

Earlier quoted context omitted.

I only have one piece of code for which I can say true 100% coverage exists: a library that works with HTML/CSS color values, and which ships a test that generates all 16,777,216 hexadecimal and integer rgb() color values, and runs some functions with each value. However, I don't run that as part of the normal test suite. It only gets run when I'm prepping a new release, as a final verification step; the normal runs-…

Just out of curiosity... What is the point in testing all 2^24 possible color values?

The point is being certain I haven't missed an edge case somewhere.

The hard part is the percentage rgb() values, of which there are technically an uncountably infinite number (since any real number in the range 0-100 is a legal percentage value). For those I generate all 16,777,216 integer values, and verify that converting to percentage and back yields the original value.

Re: The tragedy of 100% code coverage (2016)

#330

Post by ubernostrum »

LgWoodenBadger wrote: post #228
ubernostrum wrote: post #90

Earlier quoted context omitted.

I only have one piece of code for which I can say true 100% coverage exists: a library that works with HTML/CSS color values, and which ships a test that generates all 16,777,216 hexadecimal and integer rgb() color values, and runs some functions with each value. However, I don't run that as part of the normal test suite. It only gets run when I'm prepping a new release, as a final verification step; the normal runs-…

how do you know the test that generates those values is correct?

Python's test suite.
Post reply on HN