Based on my own experience working in teams using TDD and teams not using TDD, I cannot agree.
An External Replication on the Effects of Test-driven Development [pdf]
221–230 of 332 posts
Re: An External Replication on the Effects of Test-driven Development [pdf]
#222The research seems low quality. Whenever i try creating something more complex than just a CRUD webapp, i'm always relieved after getting a significant code coverage. It may be because i'm a medicore programmer (i mostly do hobby projects), but getting assurance that my 'small change here' didn't mess up anything major in a distant part of the system is quite relaxing. Obviously i only test logic and usually write th…
Then you're not talking about TDD.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#223Earlier quoted context omitted.
The criticism was very constructive though. Increase the sample size, put it in a more realistic setting.
How do you propose to gather more experienced, professional developers into the same location and get them to work on a topic that isn't making them tons of money? They can't be left to do the problems in their own workplace, or the next criticism will be "uncontrolled variables!". They also have to be vetted for minimum skills (there are plenty of experienced, professional devs out there who aren't worth a second lo…
You could start by finding a corporate sponsor with a lot of developers and a vested interest in finding out which methods will be best for them. The advantage of that being that you could even test it on a real project (the sponsor would "just" need to be willing to dedicate twice the number of developers to a suitably small project).
It'd still not be easy, and of course there'd still be issues (e.g. is there anything about the corporate culture or training in that company that would affect the result?), but it'd still be far better than a bunch of students and toy problems.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#224I've always made data to test if my functions work but now I write that data down in other programs for the future to keep checking my functions. What's the big deal... sure TDD is about the future not the time or development quality today.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#225Earlier quoted context omitted.
If studies don't need to be rigorous, what's the point of doing them? Why not just write a persuasive essay instead? Is the format of a "study" just a rhetorical device, a glammed-up appeal to authority? The whole point of careful statistics and well-designed experiments is so that we can learn whether a premise is true or not. Without rigor, we prove nothing; this study for example, neither proves NOR disproves anyt…
How short the internet's memory is. Le mieux est l'ennemi du bien. It was right here not a couple of days ago. Sheesh why do I bother?
the point is that the study appears flawed, so much that it is not even good. let alone perfect.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#226This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…
I agree with you on this. The study is irrelevant from the statistical point of view.
However, there are other considerations with TDD, probably on a more theoretical level that:
* There is no other major engineering branch that uses testing this way in production (would you bang a car against the guard rail to see if it works?)
* There is one thing common for sure with all of the bugs you catch in production, they all passed the unit tests
I think testing has it benefits but driving(!) your software development efforts is probably too much, at least for me.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#227Earlier quoted context omitted.
If your code doesn't have tests then you can't refactor because you don't know if you broke something. Getting the right level of tests is important, they should check the interface and not the implementation, otherwise I agree, you can't refactor anything. But if you test your interfaces the same way that real code would use it then it actually makes you more agile, because it gives you the peace of mind to do major…
"Tests can make refactoring much easier" is a statement I could easily buy into. "If your code doesn't have tests then you can't refactor" is pure dogma and trivial to falsify (go on, refactor some untested code now!). More likely to introduce bugs? Perhaps. But relying on that absolutist statement about refactoring will alienate me every time.
Let me clarify: you can't refactor without a very awake QA team, because you have no idea what your changes may have broken.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#228Earlier quoted context omitted.
I have a somewhat different experience. I've found TDD is useful on even tiny projects because using TDD forces you to write better code. It's really hard to reliably test things that use "magic" and side effects that affect parts of your code they shouldn't really be affecting, and really easy to test things that have well-defined and properly documented interfaces that only do one thing. Consequently writing tests,…
I would humbly suggest that because you and parent (and other people here) have such a different positive experience with TDD, that you cannot quite agree when it works, only that it sometimes works, then this is a big indication that TDD is just a placebo. But maybe that's what's needed - a placebo to feel you better as a programmer, and thus making you more productive through your feelings.
I have the same positive experience as both of the above commenters. I've found that there are many other variables which can change its effectiveness though:
* The type of code I'm working on
* How well I remember/understand the full requirements.
* How much of the testing framework is built already and how much I have to build.
* The type of test I'm writing.
* How new the project is.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#229Earlier quoted context omitted.
From experience, the value of TDD (or any style of automated tests, really, you don't really have to be driven by them) only really kicks in when the code is of a certain size, complexity and age. They are much closer to a tax in the early phases when you can reasonably keep the full scope readily in your mind.
I have a somewhat different experience. I've found TDD is useful on even tiny projects because using TDD forces you to write better code. It's really hard to reliably test things that use "magic" and side effects that affect parts of your code they shouldn't really be affecting, and really easy to test things that have well-defined and properly documented interfaces that only do one thing. Consequently writing tests,…
> Consequently writing tests, even if you never actually run them, makes your software better.
It could be argued that an experienced programmer doesn't need TDD to write the same code that TDD could have produced.
Re: An External Replication on the Effects of Test-driven Development [pdf]
#230Earlier quoted context omitted.
You can also add a coverage tool to your CI and get the same result (tests get written) without any of the ideology (TDD fairies sprinkle unicorn dust everywhere).
Coverage tools are not sufficient for getting good test coverage. One can easily make code 'covered' without having proper tests for them. By writing tests early, you make sure the code you are testing is testable and your knowledge about the code is fresh.
> By writing tests early, you make sure the code you are testing is testable and your knowledge about the code is fresh.
IME you also end up writing tests which are far too tied to the implementation. (With the resulting churn that that implies when the implementation changes.)
You get far more mileage from QuickCheck-type tests IME. Granted, not everything is very amenable to testing using QC-type tests, but a lot of stuff is.