Live data from Hacker News

How deep are your unit tests?

stackoverflow.com

41–50 of 69 posts

Re: How deep are your unit tests?

#41
Well. I use a lot of unit testing because the idea of testing small pieces of code works for the way I develop software at work - modified pair programming. Instead of reviewing Holmes's code Watson writes tests (mostly unit ones).

At spare time, tho, I tend to be too lazy to write a lot of tests(most of my "projects" are too small and never get released to the public anyway)

Re: How deep are your unit tests?

#42
post #36
post #5

Earlier quoted context omitted.

I suspect there are some subtleties to that first answer which you've overlooked. It is written by Kent Beck. He pretty much invented the concept of test-driven development. He's probably the most well-known advocate of developers writing their own tests there is. What his answer demonstrates (to me) is that the conclusions he and the other XP guys arrived at - to write code in tiny chunks, each one preceded by a tin…

"... because it works better..." Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." There is very little research which shows that TDD is better than other approaches.…

Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html: "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile."

Oh good, evidence against is just as preachy and unscientific as the evidence for.

http://evidencebasedse.com/?q=taxonomy/term/2

Research suggests that unit testing has a very strong correlation with correct software. Research suggests TDD has no correlation with correct software, except for the fact that TDDers tend to write 2x more tests than non TDDers.

Re: How deep are your unit tests?

#43

> "If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it." Hmm, that sounds like the old "You don't need to test , just be an awesome programmer duh!" argument. The kind of mistakes you never anticipate making are probably the sort of stuff you should be testing for. The amount of times I've had something fail on an edge case that turned out to be a typo…

don't know why this has so many downvotes.

Re: How deep are your unit tests?

#44

The thing I really dislike are unit testing styles that use mock objects a lot. I see some people say that you should test each object in isolation, mocking out any dependencies. That seems wrong headed to me. I prefer my unit tests to test everything all the way down. The only thing I would mock is the file system, which is useful for testing file loading code.

A unit test tests a unit of code. An integration or functional test tests the interaction between multiple units of code, usually from the point of view of "user needs functionality X, functional test Y ensures that X works". Unit testing is not supposed to catch bugs, it is primarily to ensure a solid design. Integration/functional tests are primarily there to ensure that the code works.

It only makes sense to unit test code which is in the domain of architecture. This ensures that your domain model is solid.

It only makes sense to functional test code which is a concrete implementation running on top of said architecture. This ensures that the end user sees what they need to see.

One of the often quoted benefits of unit testing is that they run fast. That fact is only incidental. Not to mention the insanity of unit testing as much as you can just because they're faster to run - they are also an order of magnitude slower to write than functional tests.

tl;dr - use the right tool for the right job. Unit testing is for design/architecture. Functional testing is for concrete functionality.

Re: How deep are your unit tests?

#45
post #37

The thing I really dislike are unit testing styles that use mock objects a lot. I see some people say that you should test each object in isolation, mocking out any dependencies. That seems wrong headed to me. I prefer my unit tests to test everything all the way down. The only thing I would mock is the file system, which is useful for testing file loading code.

If you need lots of mocks, it's a sign of coupling which means a bad fundamental design. You need to test both the components individually (unit tests) and their interaction (integration tests). We use Commons VFS for filesystem abstraction ( http://commons.apache.org/vfs/ ) and our own port of the API for .Net. In fact we have our own systems API (more Google AppEngine style thank say the .Net Framework or Java libr…

That may be true, but sometimes it can also be a pain when you are talking to everything through a load of abstraction layers.

When something blows up it can be nice just to get a simple error rather than a 20 page stack trace.

Re: How deep are your unit tests?

#46
post #36

Earlier quoted context omitted.

"... because it works better..." Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." There is very little research which shows that TDD is better than other approaches.…

Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." Oh good, evidence against is just as preachy and unscientific as the evidence for. http://evidencebasedse.com/?q=taxo…

Your requirement of 'evidence against' isn't how science works. The burden of evidence rests with the people making the claim. If you make a claim and do not provide evidence on request, that claim is not supported; it is not necessary to supply 'evidence against' for the claim to be unsupported, and there's no good reason to believe unsupported claims.

Re: How deep are your unit tests?

#47
post #36
post #5

Earlier quoted context omitted.

I suspect there are some subtleties to that first answer which you've overlooked. It is written by Kent Beck. He pretty much invented the concept of test-driven development. He's probably the most well-known advocate of developers writing their own tests there is. What his answer demonstrates (to me) is that the conclusions he and the other XP guys arrived at - to write code in tiny chunks, each one preceded by a tin…

"... because it works better..." Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html : "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile." There is very little research which shows that TDD is better than other approaches.…

I'm not sure how one would do such research.

You would need to compare teams of equal skill level who were working on very similar software over quite a long period of time.

What would the benchmarks be? Number of defects in production code? Time to ship? Severity of defects? Time spent debugging?

Re: How deep are your unit tests?

#48

> "If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it." Hmm, that sounds like the old "You don't need to test , just be an awesome programmer duh!" argument. The kind of mistakes you never anticipate making are probably the sort of stuff you should be testing for. The amount of times I've had something fail on an edge case that turned out to be a typo…

don't know why this has so many downvotes.

Your tests routinely fail because of typos?

Re: How deep are your unit tests?

#49
post #31

The thing I really dislike are unit testing styles that use mock objects a lot. I see some people say that you should test each object in isolation, mocking out any dependencies. That seems wrong headed to me. I prefer my unit tests to test everything all the way down. The only thing I would mock is the file system, which is useful for testing file loading code.

To avoid getting hung up on taxonomy I much prefer defining the scope of tests with the control theory terms of Observability and Controllability, http://en.wikipedia.org/wiki/Observability . That makes it way easier to categorize tests in terms of how effective they are and gives useful starting point to determine whether the granularity of the test really is able to adequately test the desired behaviour.

Control theory also tells us how feedback loops are related to those two terms; a lot of discussions about Agile, etc. would be much more productive if people understood that it is the feedback loop that matters (as opposed to particular methodology flavour) and why it does.

Re: How deep are your unit tests?

#50
post #48

Earlier quoted context omitted.

don't know why this has so many downvotes.

Your tests routinely fail because of typos ?

I've seen production code fail because of typos after months in deployment (because that's how long it took that else { branch to be hit in practice).

The code in question was only subject to very high level smoke tests that looked like they were basically cherry picked best case scenarios intended to pass.

A unit test would have picked this up before it even got to production.

Post reply on HN