Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An External Replication on the Effects of Test-driven Development [pdf]

people.brunel.ac.uk

Post reply on HN
Print view View original

231–240 of 332 posts

Re: An External Replication on the Effects of Test-driven Development [pdf]

#231

Post by raverbashing »

ericdykstra wrote: post #210
tbrownaw wrote: post #121

Earlier quoted context omitted.

the conclusion is always the same: "it depends." Er, no. The studies I've read all end up showing that principled testing helps, but test-first and TDD (strict red/green cycle, code only enough to pass the new test, etc) provide no additional benefit over anything else that gets the tests written. The "it depends" always comes from the echo chamber trying to justify their desire to believe that TDD isn't completely u…

I really don't believe any study or meta-study could come close to being able to suss out the nuance of when TDD may provide an advantage and when it doesn't. I'd rather just trust programmers to consider what approach works best for their problem and mindset and go from there. I personally don't TDD most things, but it's a tool I have available and I bring it out when a situation arises.

Well, with TDD you waste time writing red tests first, then with tests that "just pass" and finally make the thing as it is supposed to be

I find it surprising that's not slower than Test-Last (maybe if you really leave it for last then you'll need some time to fit your functions to your test)

Re: An External Replication on the Effects of Test-driven Development [pdf]

#233

Post by ivan_gammel »

Confusion wrote: post #160

The comments to that story are pretty good. An interesting question is: why does TDD fail in such experiments (it does so unexpectedly consistently), even when many developers feel it has benefits when they practice it? There is no silver bullet, so there must be circumstances in which TDD does not work. And conversely, the central question is: under what circumstances does TDD work? What are the preconditions?

The answer is likely in the studies, which have declared TDD efficient practice, and in the works that specified TDD approach. These works and studies tried to solve specific problems and I'm not sure they followed really scientific process before declaring that TDD is a solution, not a by-product of some solution that passed unnoticed during the research (e.g. education of developers on software architecture).

Re: An External Replication on the Effects of Test-driven Development [pdf]

#234

Post by reitoei »

hliyan wrote: post #61
tspike wrote: post #14

My experience has been that TDD is worthwhile when working with notoriously slippery whack-a-mole functions like handling time or money. The time saved by catching regressions vastly outweighs the time taken to implement the tests. In contrast, TDD has been a waste of time for me for UI-based work, as the effort needed to properly expose the functionality under test is too great and the requirements and design change…

Exactly my experience - I've done TDD for 3 different types of calculation functions in 3 different languages over the years, and all were beneficial: financial risk calculations in C++, report data calculations in PHP and billing calculations in JavaScript. On the other hand, TDD for operations that involve I/O (including user interactions) were not helpful at all.

That would probably be better handled by functional/e2e testing?

Re: An External Replication on the Effects of Test-driven Development [pdf]

#236

Post by jacquesm »

hackuser wrote: post #98
jdlshore wrote: post #71

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

It may be unfair to say this in response to the parent comment, but the great majority of HN discussions start with a comment like this one: It's seriously flawed, etc. Occasionally it's true, but the noise drowns out the signal. In a graduate-level engineering class, the students were making similar statements about all the studies we read. One day the professor said: It's easy to find flaws in someone else's work;…

Well, he did find value in the work, just not as much as the hyped title would have you believe and I'm perfectly ok with that.

The fact that it matches my experience (that TDD has benefits but only for projects of a certain size with programmers of a certain minimum experience level) helps with that, but it is always good to be reminded of the importance of sample size and other priors related to a study that makes a very bold claim.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#237

Post by crdoconnor »

rubber_duck wrote: post #188
blazespin wrote: post #177

Earlier quoted context omitted.

The study is huge huge HUGELY flawed. TDD only comes into value when you start having to refactor extremely large projects that you don't understand. You need the tests in order to refactor with confidence.

You can have tests without TDD ? TDD is a process where you write empty shims for your code and tests for it first, failing because there is no implementation, and then you write your code that passes the tests. Frankly I find this style to be completely opposite of how I code - getting something working ASAP, plugging it in to the big picture and then figuring out the problems with my approach and designing with the…

>You can have tests without TDD ?

I've worked on several projects without this and the tests done without TDD tend to be of higher quality.

I noticed a common anti-pattern of "write the code, run the code, copy the output of the code, paste it into a test and write an assert to check that the output was precisely what came out".

This was brittle, it killed the self-documenting aspect of the tests and it often concealed various bugs.

As soon as the same team members started writing the test first, it stopped happening.

I also personally felt better about doing this since it made it easier to correct API design mistakes before implementing the code and baking them in.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#238

Post by godmodus »

"YOLO" based Dev work on the otherhand is where it's at, right?

On the other hand I can see where students and new learners might falter. TDD requires u know a bit about what ur doing,and if you're new to programing, ut just costs more time to compensate for not having a healthy intuition.

Still tho, if you want to run maintainable code, that's somewhat future proof and not disposable - test it and keep it clean.

I mean it's like arguing sharpening ur katana while u fight is detrimental to duel survival. Which is true.. But...

Re: An External Replication on the Effects of Test-driven Development [pdf]

#239

Post by mcv »

geerlingguy wrote: post #5

Could it be that TDD vs. tests-after-code is a highly personal thing? I personally find it easier to write good tests after I've coded something functional. Before hand, I know one or two fuzzy ideas of what I want to accomplish, but I can't list out the concrete, real-world test scenarios until after I've coded something, poked and prodded it, etc. But I know some people are wired differently; they'll think a lot mo…

To me, it depends on the problem. With some problems, I'm not sure yet how to implement it, but I do already know what the result should look like. That's an excellent case for TDD. You get to work on the problem, refine your ideas about it, and you start with the part you already know: the results.

If you have no idea what the results should look like, then TDD is pointless.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#240

Post by regularfry »

StreamBright wrote: post #226
jdlshore wrote: post #71

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

I agree with you on this. The study is irrelevant from the statistical point of view. However, there are other considerations with TDD, probably on a more theoretical level that: * There is no other major engineering branch that uses testing this way in production (would you bang a car against the guard rail to see if it works?) * There is one thing common for sure with all of the bugs you catch in production, they a…

To your first point, the usual response is that there's no other major engineering branch which demands quite such extensive changes be possible after you've broken ground. To extend your analogy a little, nobody takes a car production line and expects the people running it to be able to switch to helicopters without starting over.
Post reply on HN