Live data from Hacker News

An External Replication on the Effects of Test-driven Development [pdf]

people.brunel.ac.uk

81–90 of 332 posts

Re: An External Replication on the Effects of Test-driven Development [pdf]

#81
post #8
post #3

Earlier quoted context omitted.

Can you elaborate?

If you write testable software and actually write the tests, the end result is the same whether you test first or test later. You're designing with testing in mind and creating useful tests either way. It's really a matter of code that lacks tests that's an issue and especially of code which isn't designed with testing in mind. I think they overinterpreted the value of TDD as simply test-first. Test first can be good…

tests can do alot of things, I wish there'd be a more functional name for them, ie. regression stoppers.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#82
post #57
post #36

Earlier quoted context omitted.

Only if you think unit testing necessarily requires TDD (which is defined as writing tests before writing the code that passes those tests). It doesn't. All the article says is TDD has little or not impact... it does not say unit testing as a practice has no impact. Seems to be a common misunderstanding around here that you, too, have fallen victim to...

>Seems to be a common misunderstanding around here that you, too, have fallen victim to... I work with people who openly believe that unit testing is a waste of time. Personally, I think they are afraid, because they don't even know how, and don't care to learn. They would see that title, and that's all the confirmation bias they would need. Thanks for assuming the worst about me though.

Err, my apologies if that came across as a slight, I literally meant you and a lot of other folks made the exact same mistake. No judgement intended, it's clearly a common misapprehension.

My guess is that a lot of folks got introduced to unit testing through test-driven development and, as a result, conflate the two, assuming the former necessarily implies the latter.

Speaking for myself, I was writing UTs long before the TDD fad landed and so I never picked up the habit. It's just not the way my developer mind works, and I've not found it compelling enough to try and re-train myself.

That said, not believing in the value of automated regression test (of which UTs are the lowest hanging fruit) is utter madness...

Re: An External Replication on the Effects of Test-driven Development [pdf]

#83
It's great such studies exist, but there might be many reasons why they are incorrect (they are testing on students, probably the students don't understand how to apply TDD, or other way around, they are so good that their coding approach provides all the benefits without TDD; the numeric metrics used in study might not adequately reflect the interesting characteristics of the code base, the payback of TDD might show up in later stages of the product life when we refactor or extend it, etc).

Probably TDD can speedup people who otherwise aren't used to iterative bottom-up approach - TDD will encourage short cycle of "change - run and see how it works" loop. Especially in non-interactive languages like C or Java.

Also, if we write tests after functionality is implemented, how do we know why our test passes: is it because the functionality is correctly implemented or it's because the test doesn't catch errors? To ensure test catches errors we need to run it on a buggy version of code. Implement functionality, write test, introduce errors in functionality to ensure the test catches them - that's 3 steps. Run test in the absence of correct code and then implement the code - 2 steps. That's where "test first" might be efficient.

But often that might be achieved other way. Suppose I'm writing a function to merge two lists. I will just do in REPL (merge '(a b c) '(1 2 3)) and see by eyes that it returns (a 1 b 2 c 3). I will then just wrap it into assert: (assert (equal `(a 1 b 2 c 3) (merge '(a b c) '(1 2 3))). Run this and see it's passes - that all, I'm sure it's an OK test.

In short, I think there is a certain truth in TDD, but it shouldn't be taken with fanaticism. And it can even be applied with negative effect (as any idea).

Suppose I want to develop a class (defclass user () (name password)).

I personally will never write tests for make-instance, (slot-value ... 'name), (slot-value ... 'password) before creating the class, the see how tests fail, then creating the class and see how tests pass.

Tests take time and efforts for writing them, and then for maintenance and rewriting when you refactor code. If a test captures an error then the test provides some "return of investments". Otherwise writing this test was a waste.

The tests in the above example will never capture anything.

I tend to create automated tests for fragile logic which is relatively easy to test, so that the efforts spend are justified by the expected payback.

But all my code is verified. Write several lines, run and see what doesn't work, fix that.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#84

Up until now it has mostly been opinions and biases and even though many popular programmers[1] have been saying this for a very long time, it's great to see a controlled study done about it. This makes it a fact and a great counter argument for helping a lot of programmers who are being forced to practice TDD because of the generally accepted claims in productivity and code quality associated with doing it. [1] http…

Keep in mind though, that the study shows that it's no better than testing after iterative development. Testing is still required, and I'd wager that the study participants following the ITL process didn't have external business pressure to skip the "test-later" bit...

Re: An External Replication on the Effects of Test-driven Development [pdf]

#85

Could it be that TDD vs. tests-after-code is a highly personal thing? I personally find it easier to write good tests after I've coded something functional. Before hand, I know one or two fuzzy ideas of what I want to accomplish, but I can't list out the concrete, real-world test scenarios until after I've coded something, poked and prodded it, etc. But I know some people are wired differently; they'll think a lot mo…

TDD vs. test-after-code is a small distinction.

80% of software development is designing correct abstractions/interfaces/APIs. If you have the correct abstractions, everything else is easy by comparison. And both tests and code are fundamentally founded on these early design decisions.

So whether I do TDD or tests-after-code, I'm confronted with the 80% first: designing the interfaces (either in writing or mentally).

Naturally, I never get this part right at first, and so I wind up refactoring a lot as I go along. ("Build one to throw away", I believe this has been called.)

Now in my experience, TDD requires me to refactor more during this process than tests-after-code. But suit yourself. Changing the order within the last 20% won't be earth-shattering one way or the other.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#86

Could it be that TDD vs. tests-after-code is a highly personal thing? I personally find it easier to write good tests after I've coded something functional. Before hand, I know one or two fuzzy ideas of what I want to accomplish, but I can't list out the concrete, real-world test scenarios until after I've coded something, poked and prodded it, etc. But I know some people are wired differently; they'll think a lot mo…

> But don't force it down my throat or act like it's the One True Path to clean code.

When I was a programming beginner and started learning about TDD, I was always thinking how soon to start using it because everyone says it's such a good idea. If you would search anything about it, you'd rarely find a resource not marvelling at its surprising benefits (at least, that is how it was few years back). Granted, it may have its merits but is it really a good idea for everything? No way!

One of the anti-patterns I have seen in programming world is how everyone jumps on adopting / advocating a 'good' practice without understanding how it fits the project. I wrote about this a while back[1]. Cue in — git flow, TDD, not using goto, DOM Parser instead of RegEx.

[1]: https://shubhamjain.co/2015/10/26/imperfect-best-practices/

Re: An External Replication on the Effects of Test-driven Development [pdf]

#87
post #47
post #8

Earlier quoted context omitted.

If you write testable software and actually write the tests, the end result is the same whether you test first or test later. You're designing with testing in mind and creating useful tests either way. It's really a matter of code that lacks tests that's an issue and especially of code which isn't designed with testing in mind. I think they overinterpreted the value of TDD as simply test-first. Test first can be good…

> If you write testable software and actually write the tests, the end result is the same whether you test first or test later. This is definitely not my experience. As Kent Beck says, TDD is a design technique. It forces you to always start thinking of the code from the outside of the unit. If I build the unit first and add tests later, it's more likely I'll end up with something where the API reflects the implement…

Perhaps it depends on how you implement each. If your idea of test last is writing a full large module and then trying to test that all at once, you might wind up with a different result than if you wrote an individual function or two and tried to test only those at once.

Getting the testing as close in time to the implementation as possible probably makes a bigger difference than first vs last.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#88

Could it be that TDD vs. tests-after-code is a highly personal thing? I personally find it easier to write good tests after I've coded something functional. Before hand, I know one or two fuzzy ideas of what I want to accomplish, but I can't list out the concrete, real-world test scenarios until after I've coded something, poked and prodded it, etc. But I know some people are wired differently; they'll think a lot mo…

TDD vs. test-after-code is a small distinction. 80% of software development is designing correct abstractions/interfaces/APIs. If you have the correct abstractions, everything else is easy by comparison. And both tests and code are fundamentally founded on these early design decisions. So whether I do TDD or tests-after-code, I'm confronted with the 80% first: designing the interfaces (either in writing or mentally).…

This is why designing the interface in a language with a powerful type system provides the same quoted benefit of TDD. It helps you think about the interface before you move to implementation.

In my experience, using a type system to do this requires much less effort and refactoring.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#89

Earlier quoted context omitted.

That means you can't use coverage tools, so this approach makes the tests unquantifiable. It is not how most major dev shops work, because of the bean counters and the "senior" developers/leads/managers/... who seem to want nothing but appeasing beancounters. It also produces superior software in my experience.

It's been very YMMV in mine. For some things, it's indispensable. Others it's tedious, repetitive and of questionable benefit.

It's tedious and repetitive even when the benefit is obvious and substantial, unfortunately.

Re: An External Replication on the Effects of Test-driven Development [pdf]

#90
I'm not commenting on the TDD studies in terms of its effectiveness but I do know that a project that takes longer brings more programming hours which results in larger budgets. If you were a company selling your services, you would be a bit more motivated to include things that take longer especially if this tugged at the emotional sense of assurance in your clients. You would also preach it to your programmers as a core practice and they would happily be converts. This goes for all the structure surrounding your project as well. I tend to see more structure in outsourcers these days and a smugness along with it. I wonder how much of it is bloatware though.
Post reply on HN