Live data from Hacker News

An External Replication on the Effects of Test-driven Development [pdf]

people.brunel.ac.uk

311–320 of 332 posts

Re: An External Replication on the Effects of Test-driven Development [pdf]

#311
post #307

Earlier quoted context omitted.

>You can have tests without TDD ? I've worked on several projects without this and the tests done without TDD tend to be of higher quality. I noticed a common anti-pattern of "write the code, run the code, copy the output of the code, paste it into a test and write an assert to check that the output was precisely what came out". This was brittle, it killed the self-documenting aspect of the tests and it often conceal…

> I've worked on several projects without this and the tests done without TDD tend to be of higher quality. Do you mean "with TDD", by any chance? The rest of your post says that you observed better results when tests were written before the code (rather than later). But as far as I understand, writing the tests before the code is indeed one of TDD's commandments.

Yes. Dumb mistake.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#312
post #279

Earlier quoted context omitted.

Although, isn't this more true for writing tests at all versus writing tests first or last in the coding process? In terms of technical debt, does it matter when the tests are written?

Yes. Some people do not know how to write extensible software. They solve the issue at hand without any regard to big picture, so when something needs to be added (like test) the code needs to be heavily refactored to accommodate the change. As others have stated, this usually(!) isn't a big deal when the project is small with a single contributor. However, writing the tests last without any forethought to what the t…

I don't do 100% TDD, but do write code to be testable, and a lot of that mindset and experience came from trying TDD.

Forcing myself to think "how would this be tested?" helped, but doing some of the tests first got me in that frame of mind.

And by practicing that, you're writing code that - by definition - is reusable - you're using it in the tests, and in your production code. The 'reuse' thing doesn't have to mean 'reuse on 8 other projects' (a common rebuttal I've heard). Reusing the functionality in another part of your project 3 months from now is reuse too.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#313

Earlier quoted context omitted.

This benefit of test-driven design is not obvious (at least it wasn't for me) until you really experience it. Several years back I wasn't aware of the concept of dependency injection, but independently discovered it while trying to unit test some error logging functionality. Previously I'd always used a global/singleton logger object. That always felt wrong, but I always assumed it was necessary until writing unit te…

Well, I practiced TDD for no less than 10 years, and its benefits are still not obvious, more like something that needs to be continuously reviewed in the context of the current work. Once you've understood all the design patterns that TDD enforces, you don't need TDD to use them. You just think how you _would_ test your code and defer test implementation for future.

Why not just learn the patterns directly?

Re: An External Replication on the Effects of Test-driven Development [pdf]

#314

Earlier quoted context omitted.

I agree with you on this. The study is irrelevant from the statistical point of view. However, there are other considerations with TDD, probably on a more theoretical level that: * There is no other major engineering branch that uses testing this way in production (would you bang a car against the guard rail to see if it works?) * There is one thing common for sure with all of the bugs you catch in production, they a…

If you could bang a car against a guard rail without incurring the cost of a car, you would.

I mean, you do incur a cost for programming with TDD though. The benefits may repay the cost in the end, but the cost is still there. And for that matter cars do undergo destructive testing, just not TDD--and the problem there is presumably that the car benefits from taking a holistic component-driven approach, "here is what I want the car overall to look like, and what components I want to fit into the space inside, now how can I reduce the rollover risk?" rather than "here are some basic expectations of a car, I expect it to be able to move forward and hold a passenger, let's make the smallest thing which causes that to happen. OK now let's add a single passenger, and let's test that bugs aren't flying into your face at highway speeds so that we remember to add a windshield eventually. (1000 steps later) Crap now we're testing the ability to shift into reverse and none of my homespun bare-minimum engine design is going to work there."

TDD probably pays its best rewards with relatively simple well-defined projects, and needs to be deployed with a modularization strategy. Possibly the wishful-thinking strategy of top-down design would be a natural complement, since when you wishfully think of a function you can hopefully quickly write a test or two for it.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#315

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

From experience, the value of TDD (or any style of automated tests, really, you don't really have to be driven by them) only really kicks in when the code is of a certain size, complexity and age. They are much closer to a tax in the early phases when you can reasonably keep the full scope readily in your mind.

I would say the cost of TDD only kicks in at scale... unit tests are a real cost that plague a project and make devs ultra conservative.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#316

Earlier quoted context omitted.

From experience, the value of TDD (or any style of automated tests, really, you don't really have to be driven by them) only really kicks in when the code is of a certain size, complexity and age. They are much closer to a tax in the early phases when you can reasonably keep the full scope readily in your mind.

>or any style of automated tests, really, you don't really have to be driven by them Looks like you misunderstood what TDD is. TDD is not a method of testing, it's the method of design of system architecture, in which you formalize the contracts and design APIs by writing examples of their use in tests. Because of that TDD is applicable on every scale (e.g. even when you design complicated algorithm that can fit in j…

While you are right that TDD is about design, and that it creates examples of use and contracts in a sense. There are better methods for this purpose like Design by Contract which in my opinion is a superior method. Why?

Mostly because TDD process of red-green-refactor is an evolutionary process. Like taking mother nature to design.

It's un-intelligent design....hahahha

Re: An External Replication on the Effects of Test-driven Development [pdf]

#317
post #315

Earlier quoted context omitted.

From experience, the value of TDD (or any style of automated tests, really, you don't really have to be driven by them) only really kicks in when the code is of a certain size, complexity and age. They are much closer to a tax in the early phases when you can reasonably keep the full scope readily in your mind.

I would say the cost of TDD only kicks in at scale... unit tests are a real cost that plague a project and make devs ultra conservative.

What do you mean ultra conservative? I can see that the opposite is true: when there are no tests, devs are conservative, ie. they are afraid to change anything. Having at least some tests, they have the possibility to change and refactor more safely.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#318
post #145
post #136

Earlier quoted context omitted.

Nobody needs an army of test subjects - just statistically valid numbers. But even with 20 test subjects the study fails at trying to make that 20 representative of any meaningful population.

Out of curiousity, how many subjects would you say are necessary to start being statistically valid?

[deleted]

Re: An External Replication on the Effects of Test-driven Development [pdf]

#319

Earlier quoted context omitted.

If you could bang a car against a guard rail without incurring the cost of a car, you would.

I mean, you do incur a cost for programming with TDD though. The benefits may repay the cost in the end, but the cost is still there. And for that matter cars do undergo destructive testing, just not TDD--and the problem there is presumably that the car benefits from taking a holistic component-driven approach, "here is what I want the car overall to look like, and what components I want to fit into the space inside,…

Your example posits an evil developer with a BOFH mindset. S/he is trying very hard to get away with doing as little work as possible while meeting absolute minimum requirements. I know that you are parodying the Agile mindset but any philosophical framework within which you make software has edges. We try to not make them count by having good hiring practices. I'll be the first to grant you that hiring is an unsolved problem in software development.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#320
post #204

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

Promotion of schemes akin to TDD tend to build their foundations on even smaller sample sizes, usually of those close in some fashion to the new wheel's reinventor; the naïve solving text-book-example problems. But maybe I missed something.

You didn't.

The top comment is a book writer on TDD who is shocked and downright appalled

TDD provides absolutely no value over TLD.

Post reply on HN