Live data from Hacker News

An External Replication on the Effects of Test-driven Development [pdf]

people.brunel.ac.uk

121–130 of 332 posts

Re: An External Replication on the Effects of Test-driven Development [pdf]

#121

Earlier quoted context omitted.

It may be unfair to say this in response to the parent comment, but the great majority of HN discussions start with a comment like this one: It's seriously flawed, etc. Occasionally it's true, but the noise drowns out the signal. In a graduate-level engineering class, the students were making similar statements about all the studies we read. One day the professor said: It's easy to find flaws in someone else's work;…

The conversation around TDD is tired, and the conclusion is always the same: "it depends." It depends on the person writing the code, the type of problem they're tackling, the language they're using, the needs of the business, etc. This study doesn't bring anything new to the table except: "in this manufactured environment we found a single point of data that equates to noise." I'm guessing the only reason the story…

the conclusion is always the same: "it depends."

Er, no. The studies I've read all end up showing that principled testing helps, but test-first and TDD (strict red/green cycle, code only enough to pass the new test, etc) provide no additional benefit over anything else that gets the tests written.

The "it depends" always comes from the echo chamber trying to justify their desire to believe that TDD isn't completely useless. It actually feels quite similar to the claims I've seen from practitioners that reikei, faith healing, etc aren't complete bunk.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#122

This is a misleading title and conclusion. The study showed a huge benefit of TDD over Waterfall, and it is only when compared to ITL that it was found to not be better. But moreover, I think it's important to understand why Beck pushed for TDD. TDD is like saying "I'm going to floss before I brush every time, no matter what." But, when people don't do TDD they typically aren't all saying "I'm going to brush and flos…

The extra complicated architectural refactors I've seen done in the name of 'test-ability' have been eyebrow raising. TDD isn't a guarantee in inducing engineers to KISS. You can still make overengineered crap tests first / TDD or not

Re: An External Replication on the Effects of Test-driven Development [pdf]

#123

Could it be that TDD vs. tests-after-code is a highly personal thing? I personally find it easier to write good tests after I've coded something functional. Before hand, I know one or two fuzzy ideas of what I want to accomplish, but I can't list out the concrete, real-world test scenarios until after I've coded something, poked and prodded it, etc. But I know some people are wired differently; they'll think a lot mo…

TDD vs. test-after-code is a small distinction. 80% of software development is designing correct abstractions/interfaces/APIs. If you have the correct abstractions, everything else is easy by comparison. And both tests and code are fundamentally founded on these early design decisions. So whether I do TDD or tests-after-code, I'm confronted with the 80% first: designing the interfaces (either in writing or mentally).…

I've found implementing test as I go along writing a new API help code quality and speed in two ways: I catch my own logic bugs early and the self-feedback I get from writing the tests helps me to write a more usable interface.

I never catch all the design errors with non-coding design work - this gives me the high level view which is critical - but there are always some logical errors and bugs I've failed to model correctly which early tests catch.

But yeah, the optimal design-implememt-test probably depends probably very much on the implememter and on the problem to be solved.

The third meta-advantage is that if I'm writing hobby code for myself and the profject is large compared to the weekly time allocations available in my sparetime I get something to help me remember where I was (say a month back) and gives me something to compile and run immediately (which is a huge motivational booster for me).

Re: An External Replication on the Effects of Test-driven Development [pdf]

#124

The * fire your QA team * dev team is the level 2 production support, and * get to continuous integration nirvana management fads have been sweeping through my Scrum enterprise for the last 18 months. Teams that aren't testing constantly, well, they've got tons of escape defects on every release. And those devs are constantly in fire-fighting mode, it's miserable for them. And I see that leading to compressed schedul…

  > You should be spending 70% of your dev time writing tests and doing devops and 30% writing features.
I laugh, but the tears are real.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#125

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

It may be unfair to say this in response to the parent comment, but the great majority of HN discussions start with a comment like this one: It's seriously flawed, etc. Occasionally it's true, but the noise drowns out the signal. In a graduate-level engineering class, the students were making similar statements about all the studies we read. One day the professor said: It's easy to find flaws in someone else's work;…

If it is unfair to say in here then you should have waited for a better time to write this comment instead of just venting yourself on the first comment you see like it.

This study has the potential to be misused by people to get their senseless argument across and could cause a lot of headache for people in the industry. It's hard enough as it is for a consultant to get people to pay for testable software.

That line about your professor is cute, but the issue is the media gobbles up studies and reports on them no matter what the quality leaving large chunks of the population misinformed. So publishing a study that is inaccurate is frankly irresponsible.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#126

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

From experience, the value of TDD (or any style of automated tests, really, you don't really have to be driven by them) only really kicks in when the code is of a certain size, complexity and age. They are much closer to a tax in the early phases when you can reasonably keep the full scope readily in your mind.

I have a somewhat different experience. I've found TDD is useful on even tiny projects because using TDD forces you to write better code. It's really hard to reliably test things that use "magic" and side effects that affect parts of your code they shouldn't really be affecting, and really easy to test things that have well-defined and properly documented interfaces that only do one thing. Consequently writing tests, even if you never actually run them, makes your software better.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#127

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

The study presents imperfect data. You have presented no data. Which is more credible? Why do comments like this get upvoted so much?

Re: An External Replication on the Effects of Test-driven Development [pdf]

#128
Does this study assess the long term cost of software? It may be true that this has little benefit in writing code from scratch, and my experiences are that TDD definitely takes longer when writing code than not doing it. But, how does it evaluate claims that 90% of the cost of code comes in the maintenance, not the initial creation of it.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#129
post #51
post #14

My experience has been that TDD is worthwhile when working with notoriously slippery whack-a-mole functions like handling time or money. The time saved by catching regressions vastly outweighs the time taken to implement the tests. In contrast, TDD has been a waste of time for me for UI-based work, as the effort needed to properly expose the functionality under test is too great and the requirements and design change…

Can you elaborate more? Im a frontend guy and sometimes I write JS plugins for the browser (that run after page load, and apply based on how things render on the page, or bawed on user interaction with the page) and non-frontend folks tell me I need tests for my code, or dont want to look at it until I have test. How would I build a tewt for this, other than a functional test that runs in-browser and the test is whet…

Here are two strategies. First, you can go "outside" what you've built, simulating a user's interaction and examining the resulting DOM. In general this is a tool-centric approach.

Another approach is to partition your code into what Gary Bernhardt calls an "imperative shell" of code that touches the outside world and a "functional core" that does not. Then unit test the functional core, which shouldn't require special testing libraries, and validate that the imperative shell works by the occasional manual or "outside" test.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#130

This study, like most software development studies I've seen, is seriously flawed. It doesn't justify the sensational title here on HN. * The sample size was tiny. (20 students) * The participants were selected by convenience. (They were students in the researcher's class.) * The majority of participants had no professional experience. (Six students had prior professional experience. Only three had more than two year…

It may be unfair to say this in response to the parent comment, but the great majority of HN discussions start with a comment like this one: It's seriously flawed, etc. Occasionally it's true, but the noise drowns out the signal. In a graduate-level engineering class, the students were making similar statements about all the studies we read. One day the professor said: It's easy to find flaws in someone else's work;…

We live in a world with too much information. There are probably more words written everyday than we can read in a lifetime.

The sooner you can remove worthless information the better.

Post reply on HN