How deep are your unit tests? • Hacker News Board

Re: How deep are your unit tests?

#61

Post by [deleted] » Mon, Dec 10, 2012, 4:33 PM UTC

pekk wrote: post #46

stonemetal wrote: post #42

Earlier quoted context omitted.

Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." Oh good, evidence against is just as preachy and unscientific as the evidence for. http://evidencebasedse.com/?q=taxo…

Your requirement of 'evidence against' isn't how science works. The burden of evidence rests with the people making the claim. If you make a claim and do not provide evidence on request, that claim is not supported; it is not necessary to supply 'evidence against' for the claim to be unsupported, and there's no good reason to believe unsupported claims.

[deleted]

Re: How deep are your unit tests?

#62

Post by [deleted] » Mon, Dec 10, 2012, 4:39 PM UTC

pekk wrote: post #46

stonemetal wrote: post #42

Earlier quoted context omitted.

Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." Oh good, evidence against is just as preachy and unscientific as the evidence for. http://evidencebasedse.com/?q=taxo…

Your requirement of 'evidence against' isn't how science works. The burden of evidence rests with the people making the claim. If you make a claim and do not provide evidence on request, that claim is not supported; it is not necessary to supply 'evidence against' for the claim to be unsupported, and there's no good reason to believe unsupported claims.

[deleted]

Re: How deep are your unit tests?

#63

Post by meaty » Mon, Dec 10, 2012, 4:41 PM UTC

jiggy2011 wrote: post #60

meaty wrote: post #59

Earlier quoted context omitted.

That's vertical decoupling. Well the point of the abstraction is that you don't care about the contents, hence the term "black box abstraction". It doesn't matter to the interface consumer what the implementation is, just that it does what the interface says it will do. Our VFS layer has various backends which can be picked based on the client's requirements and this covers REST APIs, RDBMS and filesystems. There are…

True, if all of your classes can conform to the same interface. When you integrate some third party black box with a different interface you're probably going to have to write at least one wrapper class.

True also. We wrap Amazon S3's client behind our VFS layer.

Re: How deep are your unit tests?

#64

Post by Silhouette » Mon, Dec 10, 2012, 6:29 PM UTC

meaty wrote: post #57

Silhouette wrote: post #54

Earlier quoted context omitted.

That may be true, but sometimes it can also be a pain when you are talking to everything through a load of abstraction layers. Exactly. Sometimes the benefit of a certain kind of test does not outweigh the cost of distorting a useful design so it will support the test code. That cost might be directly in terms of performance overhead, or it can be a more subtle overhead through (for example) weakening language-enforc…

Hint: You never have to distort a design for testing, if it's not a crock. It's a great indicator of brokeness.

That's not a hint, it's a conceit, born in the kind of fantasy world that drinks whatever folks like Bob Martin are selling and loves names like SimpleBeanFactoryAwareAspectInstanceFactory and functions no more than five lines long.

In the real world, software architecture is the servant of many masters, often including multiple kinds of testing, and sometimes you can't do everything that each of them would like all at the same time. For a start, they can be objectively contradictory: every time you add a dedicated test hook or introduce inversion of control to support unit testing, for example, you have also created additional cases that need to be considered during a code review.

However, it's important to remember that all these testing methods and design techniques are merely means to an end. They are valuable precisely to the extent that they help us build software to do useful things, and no more. If they sometimes conflict, that's OK, as long as we favour the one that most helps us to get the job done. However, things tend to go wrong if we start nerfing one aspect just because dogma says we must not compromise on another.

Re: How deep are your unit tests?

#65

Post by ruggeri » Tue, Dec 11, 2012, 6:44 AM UTC

Is it me, or does Bill the Lizard close every SO thread linked from HN?

Re: How deep are your unit tests?

#66

Post by stonemetal » Tue, Dec 11, 2012, 1:41 PM UTC

pekk wrote: post #46

stonemetal wrote: post #42

Earlier quoted context omitted.

Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." Oh good, evidence against is just as preachy and unscientific as the evidence for. http://evidencebasedse.com/?q=taxo…

Your requirement of 'evidence against' isn't how science works. The burden of evidence rests with the people making the claim. If you make a claim and do not provide evidence on request, that claim is not supported; it is not necessary to supply 'evidence against' for the claim to be unsupported, and there's no good reason to believe unsupported claims.

This is a discourse not science class both sides of the argument should have evidence.

Re: How deep are your unit tests?

#67

Post by dalke » Tue, Dec 11, 2012, 3:03 PM UTC

stonemetal wrote: post #66

pekk wrote: post #46

Earlier quoted context omitted.

Your requirement of 'evidence against' isn't how science works. The burden of evidence rests with the people making the claim. If you make a claim and do not provide evidence on request, that claim is not supported; it is not necessary to supply 'evidence against' for the claim to be unsupported, and there's no good reason to believe unsupported claims.

This is a discourse not science class both sides of the argument should have evidence.

Who says there are only two sides? There are dozens or hundreds. I can make a new one - programming is best done with a candle burning on the table and a glass of wine or whisky available. It works for me, so I say it should work for everyone. I call it "Romantic Driven Programming" or "RDP." Now, what's the evidence that you have against RDP as a best practice?

You don't have any, do you? So obviously I can now promote it as the new universal methodology.

(Or more directly, you think the argument is that there is evidence for and evidence against TDD. My argument is that there isn't enough evidence to make a conclusion.)

Given that this is a discourse, have you read any of the discussions of the topic of TDD? For example, "Making Software" (e.g Oram and Wilson) has an excellent chapter summarizing the research results. If not, then it's a rather unbalanced discourse, no? You pointed to a list of papers, but didn't describe what you wanted to discuss from them.

For example, some of the research compares test-first with test-last. I'm a proponent of test-during, and don't think that test-first and test-last are the only two ways to compare things.

Re: How deep are your unit tests?

#68

Post by dalke » Tue, Dec 11, 2012, 3:19 PM UTC

jiggy2011 wrote: post #47

dalke wrote: post #36

Earlier quoted context omitted.

"... because it works better..." Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." There is very little research which shows that TDD is better than other approaches.…

I'm not sure how one would do such research. You would need to compare teams of equal skill level who were working on very similar software over quite a long period of time. What would the benchmarks be? Number of defects in production code? Time to ship? Severity of defects? Time spent debugging?

It's not easy research, but there have been some papers published on the topic. The problem of course is that there are a lot of confounding factors.

Besides, that my point, isn't it? Promotors of TDD argue that their personal intuition suffices, despite having little evidence to back their claims.

Re: How deep are your unit tests?

#69

Post by pekk » Tue, Dec 11, 2012, 7:24 PM UTC

TeMPOraL wrote: post #51

pekk wrote: post #48

Earlier quoted context omitted.

Your tests routinely fail because of typos ?

In dynamic languages? Sure they may. The PHP script you thought works crashes, the Lisp routine out of the sudden enters the debugger, things like that.

The question isn't what "may" happen, but what routinely does happen for you.

If typos are even a significant problem in your code then the solution is to type more slowly, have your IDE flag unknown symbols, use autocomplete, etc. As a solution to this problem, strong typing is both disproportionate and inadequate.