Unit testing in Coders at Work

The plural of anecdote is not data, but when I worked at ThoughtWorks, Martin sent me some code to look at. It was very clean OO code (very smalltalk-ey in design and form, thought it was written in Ruby). The man can code well (not to the level of say Peter Norvig, but then not many of us can) and competently and is very sharp and when he gets hands on, solves problems fast.It is true that he doesn't have any code online where we can look at it.

Another admirable trait I've witnessed is his willingness to say "I don't know" when asked questions outside his circle of competence. He often says it very bluntly (which has caused some heartburn to some ThoughtWorkers once in a while, but that is another story).

His "Enterprise Patterns" book is very useful for anyone working on enterprise software. I believe DHH was influenced by the book when he was hacking Rails.

His upcoming book on rich gui apps seems promising too. Also he doesn't make outragoeus claims or adopt a condescending atitude to programmers who don't work the way he does/advocates (unlike say Robert Martin). I wouldn't dismiss him as a "know nothing" talker/blowhard.

As I said, just anecdotes. Make of them what you will.

I was amused by his comment on Knuth:

"So Knuth too disagrees with the notion that unit testing always makes you go faster. Maybe he too is living in the stone age."

This follows him describing writing a program, in pencil, in 1977.

I got all this directly from the linked article:

Zawkinski: created automated tests for a complicated corpus of quasi-standard email headers that he triggered manually. When he rewrote the mailer in Java he unit-tested because it was easier in a Java OO environment. He says there's specific areas where unit-testing would have sped them up, but that they were under so much time-pressure, with short-term ROI horizons that they didn't bother, they just went with the gamble of "getting it right first time". If they had more time then unit-testing was "certainly the way to go". He specifically states that they traded quality for a ship date.

Fitzpatrick: Does "a lot of testing". Tests anything half-way "clever" because others will break it. Forces people who work for him to write tests. "At a certain point, people realize, “Holy crap, it does pay off,” especially maintenance costs later."

Joshua Bloch: automated testing of transactional shared-memory with a "basher". When it failed at high concurrency he had to use unit tests to locate the bug, which wasn't in his code but the underlying Solaris assembly code. This was early 90s and he implies that finding a bug in such code today would probably result in the engineer being shouted at for inadequate testing, though perhaps not in concurrency areas as testing those is still an "art". Argues that writing an API before the code, which he does, isn't technically TDD though I'd say it's in the same ballpark.

Knuth: is "a fan of after-the-fact torture tests".

The blog author interprets the other comments as anti-TDD, though they seem unrelated to me.

Joe Armstrong: Actually does TDD.

Norvig: "I think test-driven design is great. I do that a lot more than I used to do."

I don't understand how you could have come up with a different reading. The originally post seems to be half-heartedly questioning TDD (without much support from the quoted sections), but that is on the far end of a continuum that has automated tests, then unit tests, on the way towards it. Which I thought I made clear with my list where the near end of that spectrum (automating tests for complicated areas) is better supported than the far end (TDD according to the book). Many responses I see here seem to reject testing entirely.

(edit: just to be clear when I say "automated" I mean they wrote a program to test their program. It doesn't necessarily mean that they run it on every change, or check-in or whatever. I just mean it runs and says: pass or fail without a human being having to examine the output, though that output may be helpful when it does fail.)

You should have a look at QuickCheck. It's a Haskell library where you (more or less) specify laws that your program should satisfy, and it generates test cases on its own. I found it very useful.

51/2 for 6?

(EDIT: you first claimed that of the 6 developers from CaW 5-1/2 of 6 supported your claims. Knuth being the "1/2" You've since edited your post)

51/2 for 6?

You must be kidding. You are cherry picking statements that support your viewpoint and ignoring those that go in an opposite direction.

these were what you said "programming legends" had a consensus on

"• (semi-)automated testing for any complex code: essential • unit tests for any long lived code: great • TDD (writing the tests before the code): good in theory but alien to most folks"

Let's see now.

"after the fact torture tests" (Knuth) is hardly support for "TDD' or "semi automated testing is essential". SO that's one down (or 1/2 down :-P) So that yo are down to 5 of 6 right there.

BTW Knuth also said (http://www.informit.com/articles/article.aspx?p=1193856)

"As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."

Zawinski, Knuth, Norvig etc don't say anything like "semi automated testing is essential" (one of your conclusions) or "TDD is good in theory".

It is one technique among many they sometimes use,certainly.

Norvig (for example) specifically says

"Seibel: What about the idea of using tests to drive design?

Norvig: I see tests more as a way of correcting errors rather than as a way of design. This extreme approach of saying, “Well, the first thing you do is write a test that says I get the right answer at the end,” and then you run it and see that it fails, and then you say, “What do I need next?”—that doesn’t seem like the right way to design something to me"

So that is 4 of 6.

Zawinski says

"Seibel: In retrospect, do you think you suffered at all because of that? Would development have been easier or faster if you guys had been more disciplined about testing?

Zawinski: I don’t think so. I think it would have just slowed us down. There’s a lot to be said for just getting it right the first time. In the early days we were so focused on speed. We had to ship the thing even if it wasn’t perfect. We can ship it later and it would be higher quality but someone else might have eaten our lunch by then."

In other words in retrospect, If speed is a concern, "being disciplined about testing" is NOT important which is hardly "consensus" about the centrality of testing.

So that is 3 of 6.

About Bloch, you say "Argues that writing an API before the code, which he does, isn't technically TDD " and you concluded "it is in the same ballpark" ?! You conclude something 180 degrees opposite to what the interviewer understood?

Josh said

" I do disagree with Martin (Fowler) on one point: I don’t think tests are even remotely an acceptable substitute for documentation. Once you’re trying to write something that other people can code to, you need precise specs, and the tests should test that the code conforms to those specs. "

So write precise specs, write code and then add tests that the code confirms to written specs is "in the same ballpark" as TDD? This is how sw is being developed for ages.

By this logic, any thinking about code before coding is "in the same ballpark" as TDD. A strange argument.

Besides Josh Bloch wrote a test when everything else failed. And when specifically asked if an automated test would have been useful, he says

"I think a good automated unit test of the mutex facility could have saved me from this particular agony, but keep in mind that this was in the early ‘90s. It never even occurred to me to blame the engineer involved for not writing good enough unit tests. Even today, writing unit tests for concurrency utilities is an art form."

More nuanced than your claim that Josh was all for automated testing don't you think?

So 2 of 6.

So how did you come to your conclusions "TDD is great in theory but too strange for most folks" as something they concluded?

You are concluding a few things and cherry picking statements to support this.

Every developer in the world would love an extensive (semi) automated test suite if they didn't have a cost to write and maintain.

You can't take a statement from someone that "tests are great" and ignore "tests are for correcting errors" or "when speed is critical, tests are not important".

My point was that you say "programming legends" conclude various things. As I see it they don't conclude anything beyond "tests are good in certain circumstances for specific purposes". They make some generic statements in support of "testing" but are very cautious of "test driven development" or "you must test every bit of code" por even "you must have an automated test suite".

There is no consensus for the stronger statements you put forward. Nobody says "TDD is good in theory but most people don't get it". Almost everyone says it is a bad idea.

The second part of your original post , that the consensus evolving here on HN is "anyone who advocates testing is abda programmer" is still unsupported. Please, show me links.

EDIT: I see you've edited your post and are now proposing a weaker claim "testing" vs "tdd". Sure if you are saying "programs that test other programs" are a useful technique to have in your arsenal, sure thing.

Programmers have written tests drivers (in other words programs that tests programs) from the beginning of programming. This is hardly an argument for "there is a consensus among programming legends different from teh consensus here on HN" .