Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TDD did not live up to expectations

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

201–210 of 450 posts

Re: TDD did not live up to expectations

#201

Post by agentultra »

nostrademons wrote: post #37

TDD failed for economic reasons, not engineering ones. If you look at who were the early TDD proponents, virtually all of them were consultants who were called in to fix failing enterprise projects. When you're in this situation, the requirements are known. You have a single client, so you can largely do what the contract says you'll deliver and expect to get paid, and the previous failing team has already unearthed…

You're assuming TDD has failed and are reaching for evidence to make your case.

From the SWEBOK [0]:

  5.1.5 Test-Driven Development

  [1, c1s16]

  Test-driven development (TDD) originated as one of the core XP (extreme programming) practices and consists of writing unit tests prior to writing the code to be tested (see Agile Methods in the Software Engineering Models and Method KA).
  In this way, TDD develops the test cases as a surrogate for a software requirements specification document rather than as an independent check that the software has correctly implemented the requirements.
  Rather than a testing strategy, TDD is a practice that requires software developers to define and maintain unit tests; it thus can also have a positive impact on elaborating user needs and software requirements specifications.
It really depends on many more factors than getting to market first. If we developed software in this way for fighter planes or search engines I think we wouldn't find them nearly as useful. Perhaps even dangerous.

In the absence of a more formal software specification or testing strategy TDD is the best way for a self-directed team of software developers to maintain that specification.

Beyond that it also has benefits for practitioners working in languages that lack a sound type system to catch domain errors for them at compile time. Coupled with property-based testing TDD can be extremely effective.

[0] http://swebokwiki.org/Chapter_4:_Software_Testing

Re: TDD did not live up to expectations

#202

Post by the_af »

enraged_camel wrote: post #155
taeric wrote: post #64

Earlier quoted context omitted.

No. TDD failed for engineering reasons in addition to economic ones. The main failing of TDD is the assumption that you know all of the tests that you will need before you know your software. This is true in bridge building as much as it is in anything else. Until you fully know the domain of what you are building, you cannot possibly know all of the things you need to test for. To that end, if TDD were focused aroun…

Yes, precisely. TDD works for experienced developers who know the problem domain well. It does NOT work for that new junior dev you hired straight out of college. They need to write functions and features first and see the spectacular ways in which those functions and features can fail (in a dev/test environment, hopefully), and then use that experience to switch over to TDD when they write a similar feature/function…

> TDD works for experienced developers who know the problem domain well

This is important: it's not only junior devs that fail -- TDD also does NOT work for experienced developers who do NOT know the problem domain well, as the infamous Sudoku solver debacle showed, where Ron Jeffries got too caught up in TDD/refactoring to make any serious progress in actually solving Sudoku.

Re: TDD did not live up to expectations

#203

Post by velox_io »

TDD is a nice idea, however TDD can add quite a bit of overhead. Upfront overhead, before writing any code. This isn't a problem if it is justified/ needed. Plus the tests can become more cumbersome when code changes, more baggage to carry.

Testing often becomes a KPI, and therefor is commonly gamed. Doing the bare minimum to tick '100% code converge!'. I'm a fan of contracts to test the spec and boundaries(whether human or other application) of software.

TDD requires discipline and experience; You could spend an infinite amount of time writing tests and never deliver, or the other extreme becoming incapacitated fighting bugs.

Our first priority should be crash-free software, THEN start to think about making it bug free.

Re: TDD did not live up to expectations

#204

Post by yebyen »

taeric wrote: post #64
nostrademons wrote: post #37

TDD failed for economic reasons, not engineering ones. If you look at who were the early TDD proponents, virtually all of them were consultants who were called in to fix failing enterprise projects. When you're in this situation, the requirements are known. You have a single client, so you can largely do what the contract says you'll deliver and expect to get paid, and the previous failing team has already unearthed…

No. TDD failed for engineering reasons in addition to economic ones. The main failing of TDD is the assumption that you know all of the tests that you will need before you know your software. This is true in bridge building as much as it is in anything else. Until you fully know the domain of what you are building, you cannot possibly know all of the things you need to test for. To that end, if TDD were focused aroun…

> The main failing of TDD is the assumption that you know all of the tests that you will need before you know your software.

You know what makes this work for me (I am a TDD & BDD dev, and I don't think it's dead) is the fact that you can reorder commits. When I hit a case like this, where the correct code is easier to write than the correct test, I still test it.

I write the test after, then I reorder the commits once the test passes, and make sure it fails predictably in the way you were expecting it to fail. Sometimes you find your test doesn't fail correctly with the codebase you had before the feature was implemented, and then you amend the commit and use rebase or cherry-pick.

Adding BDD to the mix helps a lot too. Sometimes you can write the whole test up-front! Sometimes you can write pseudocode for the test, but you don't have enough information about the details of the implementation to write what you'd call a complete test before the feature is born. Usually you can at least get a vague, english-language description of what the feature is supposed to do.

Then you have, at least, a foundation for your test (which can ultimately be upgraded to an executable proof/regression test), and you have a way to measure how far from the vague, usually simple, plain-english description of the feature you've had to go in order to produce "correctly behaving" code. (And you can amend the behavioral spec to correctly explain the edge cases.)

I am in the habit of telling my team members with less TDD and BDD experience, who are just trying to get the hang of building up a codebase with strong regression tests, that they should always try to write a Cucumber Scenario before they wrote the feature, even if they don't write step definitions until (sometimes a great while) after the feature is done.

I agree that the burn-down chart is not the end goal of TDD, and you should not optimize your process simply around making sure you have a progress chart and a great estimate up-front. Complete coverage of the codebase is more important! And surely there are other things even more important than that.

Re: TDD did not live up to expectations

#205

Post by jldugger »

S_A_P wrote: post #80

One thing I've noticed. TDD takes longer. It just does. You can argue that you are racking up less technical debt in the long term but every consulting gig I've been on where TDD was the "directive" often deteriorated because the business does not want to factor in between 40 and 100% extra time to allow proper TDD coding. They want the same somewhat arbitrary and bonus driven deadlines that they always do, and in or…

> One thing I've noticed. TDD takes longer.

It's not just you, there's ample evidence[1][2] that TDD is a tradeoff between delivery time and quality. I think most experienced practitioners would agree this makes sense: when the schedule slips, you can cut testing but you can't cut implementation. Works at all beats trumps works in all cases.

In the TDD model, you make that decision up front, as well as the investment in tests. When the schedule slips in development, there's nowhere to cut but the most painful part, so you don't. The schedule slips.

To take a contrarian position, TDD fails because it doesn't allow management to revisit decisions as reality deviates from the plan N months ago. This seems like a surprisingly non-agile approach at the high level.

[1]: https://pdfs.semanticscholar.org/4dcf/5e7eed29c6707a8e1a415c...

[2]: https://www.infoq.com/news/2009/03/TDD-Improves-Quality

Re: TDD did not live up to expectations

#206

Post by lgunsch »

dtech wrote: post #183
lgunsch wrote: post #158

Earlier quoted context omitted.

It's exactly the opposite. TDD was a put forward as a solution to unclear, changing business requirements. It allows you to adjust course very quickly when the business realizes its requirements were off and new set of requirements are needed. Robert Martin has even mentioned that it's a waste of time if you have a full 100% specification available to you, or when you already know what the final solution is and simpl…

I'm not an expert on TDD, but how does TDD help in this situation? If your requirements change, your existing tests and code become (partly) useless. This would indicate the most efficient way is the exact opposite where you test minimally until the requirements have stabilized.

The goal of TDD is not to have tests, or a huge test coverage. Those are just important side-effects. One goal, ignoring how it improves focus and divides complexity up nicely, is to make cleaner code. If your code respects the single-responsibility-principle, is not duplicated, and doesn't have dependency issues, you can adjust the requirements faster. The test suite allows you to be confident your changes to some parts of the system don't break others.

You can keep more code, not even having to touch many parts of the system. You know that by changing x, you don't also have to change w, y and z. Of course code still must change, but its vastly simpler and quicker to change.

Now, that doesn't mean you can't have those benefits without TDD, it's just much easier and clear.

Re: TDD did not live up to expectations

#207

Post by rsyntax »

raverbashing wrote: post #78
atonse wrote: post #8

In the earlier days of the ruby community, I feel TDD was seen as gospel. And if you dared say that TDD wasn't the way (which I always felt), you'd feel like you were ostracized (update: just rewording to say, you'd worry that it would hurt you in a job search, not that people were mean to you). So I never spoke up. I feel like I was in the TDD-bad closet. I absolutely think _tests_ are useful, but have never found a…

Yes a lot of people kept parroting TDD (as most people parrot the latest fad without thinking for 3 seconds if they should actually do that) > I absolutely think _tests_ are useful, but have never found any advantages to test-DRIVEN-development (test-first). Seconded. Especially the crap about "make the test fail first" (making the test fail is useful, especially if you're unfamiliar with some aspect of it), but it s…

Yeah I dislike that mantra, but I think it was a guideline for novice. I think the phrase should be updated to "make the test fallible".

Re: TDD did not live up to expectations

#209

Post by lgunsch »

dtech wrote: post #183
lgunsch wrote: post #158

Earlier quoted context omitted.

It's exactly the opposite. TDD was a put forward as a solution to unclear, changing business requirements. It allows you to adjust course very quickly when the business realizes its requirements were off and new set of requirements are needed. Robert Martin has even mentioned that it's a waste of time if you have a full 100% specification available to you, or when you already know what the final solution is and simpl…

I'm not an expert on TDD, but how does TDD help in this situation? If your requirements change, your existing tests and code become (partly) useless. This would indicate the most efficient way is the exact opposite where you test minimally until the requirements have stabilized.

TDD allows you to make your abstractions based on real evidence, rather than conjecture.

Re: TDD did not live up to expectations

#210

Post by seanmcdirmid »

kevwil wrote: post #192

This logic is flawed, and I'm not surprised it's coming from Microsoft. If the expectation is that (repeatedly?) making blind guesses quickly and (optionally) cleaning up the mess later is better than expressing an understanding of the problem domain before writing 'real' code, then yes TDD will not live up to those expectations.

Note the site is http://blogs.msdn.microsoft.com. Any MS employee can start a blog here, and none of the content posted by an employee goes through a vetting process.

That being said, the logic is not really flawed at all. There will always be the real scottsman fallacy to deal with (what is real code? When does code become real?). TDD promotes writing tests first, whereas you might not even know what the spec is until you've gone through multiple iterations of the code, and along the way you'll have to refactor and rewrite to reduce coupling and reflect better understanding of the spec. Tests aren't bad, but writing the tests at the beginning doesn't really make sense in that context.

Post reply on HN