Live data from Hacker News

TDD did not live up to expectations

blogs.msdn.microsoft.com

61–70 of 450 posts

Re: TDD did not live up to expectations

#61

My day job is teaching TDD. Just like other agile rhetoric I've found the benefits are not what the proponents advertise. I teach it through pairing and here's what I find. TDD provides two things. 1. Focus Focus is something I find most programmers struggle with. When we're starting some work and I ask, "ok what are we doing here" and then say "ok let's start with a test" it is a focusing activity that brings clarit…

I would agree with those two statements. They are even mentioned by Kent Beck, who created the technique in the first place, in his book Test Driven Development: By Example.

He spends a whole bunch of time hammering home the point that TDD is to help you manage complex problems and focus. He even mentions that if you think you can just power through the code and write it correctly in one swoop, then just do it. Skip TDD - it's not helping you then.

There are also other techniques he talks about regarding focus. For example, leaving a failing test as your last "bookmark" to where you left off for when you come back the next day. That allows you to jump right into where you left off, no ramp-up time at all.

Re: TDD did not live up to expectations

#62
post #8

In the earlier days of the ruby community, I feel TDD was seen as gospel. And if you dared say that TDD wasn't the way (which I always felt), you'd feel like you were ostracized (update: just rewording to say, you'd worry that it would hurt you in a job search, not that people were mean to you). So I never spoke up. I feel like I was in the TDD-bad closet. I absolutely think _tests_ are useful, but have never found a…

Like the author in the original article, I used to be all about TDD. Now, I like to tell people that "I'm apostate in the ways of TDD". I specifically use the language of religion because I think strict adherence to TDD is itself a religion.

I somewhat disagree with the statement he makes, "we know that most developers do not have great design/refactoring skills". I've certainly worked at places that all but ordered me to never refactor code, and I suspect my experience is very far from unique. They thought "refactoring" was a made-up word that programmers used to cover up dicking around and wasting time. From a non-programmer's perspective, all they see is the programmer spending several hours with the end result being that nothing has visibly changed. Mind you, no visible changes is the ideal case for refactoring, so you have no easy means to explain why that is wrong.

He says that TDD only works if the programmer is good, but I think it's more the case that test-first only works in cases where we have very well-defined problems. For these TDD trainers that he's talking about, they've gone through the same problem set over and over again. For an experienced programmer, if you're encountering the same sorts of problems, you know what the pitfalls are and can look ahead, even if you aren't violating YAGNI, you can still do yourself some favors.

To put it to a specific metric, if you have a specification from which you're working, you basically have an answer sheet for your code. You can then write code that automatically verifies that your code matches that answer sheet. The more like "having a spec" your problem is, the more likely TDD will work very well for you.

But I don't do that kind of work very frequently. Most of my work is extremely experimental, fluid, and in a lot of ways, its behavior is arbitrary. Does a padding of 5px or 10px look better? Should the surface of a polygon be more or less smooth? Should running over the ground be 3m/s or 5m/s? Should the user have two hand tools that are identical, or should they be different tools? A lot of it has to be visually verified, because it would otherwise require a computer vision algorithm of some kind to know that the code I wrote to go from text to screen pixels worked correctly.

It's much more design than development, just that design is expressed in code rather than in Photoshop PSDs.

As a result, I focus much more on REPLs, saved REPL sessions, demo code, and making the iteration time between code change and test run as narrow as possible. What I think TDD provides (poorly) for people in this situation is this REPL-like behavior, in languages that don't particularly provide a good REPL (let us consider RoR a separate "language" from Ruby, in this case, as the expectations tend to be completely separate).

So if I'm writing an implementation of a rope data structure as part of a syntax highlighting text editor that renders into a WebGL texture, yeah, I have tests for that. Ropes and text editors are well-understood data structures, there's lots of information available about them, and there are right and wrong answers to what operations each should have and how they should behave. But that's not where the work ends. I still need to figure out how to let people interact with that text editor, be it with touch gestures or motion controllers or a keyboard or a gamepad or what have you, and that has no easy-mode.

Re: TDD did not live up to expectations

#64

TDD failed for economic reasons, not engineering ones. If you look at who were the early TDD proponents, virtually all of them were consultants who were called in to fix failing enterprise projects. When you're in this situation, the requirements are known. You have a single client, so you can largely do what the contract says you'll deliver and expect to get paid, and the previous failing team has already unearthed…

No. TDD failed for engineering reasons in addition to economic ones.

The main failing of TDD is the assumption that you know all of the tests that you will need before you know your software. This is true in bridge building as much as it is in anything else. Until you fully know the domain of what you are building, you cannot possibly know all of the things you need to test for.

To that end, if TDD were focused around building things to break them and analyze them, it would be wise. However, in software it is often taught to build a wall of failing tests, and you can then use that as a sort of burn down chart for progress. Note that you don't learn about the software you are building from this burndown. Instead, you only get a progress report.

And obviously, this is all my opinion. I'm highly interested in exploring the ideas. And I'm highly likely to be wrong in my initial thoughts. :)

Re: TDD did not live up to expectations

#65

TDD failed for economic reasons, not engineering ones. If you look at who were the early TDD proponents, virtually all of them were consultants who were called in to fix failing enterprise projects. When you're in this situation, the requirements are known. You have a single client, so you can largely do what the contract says you'll deliver and expect to get paid, and the previous failing team has already unearthed…

I have a colleague who is not a household name but still fairly widely known in the software process space and he has insisted to me multiple times, for more than ten years now, that this is a west coast thing. Other parts of the US are much less allergic to discipline in development.

Re: TDD did not live up to expectations

#66

No article about TDD, particularly one that shouts out to the respected Ron Jeffries http://ronjeffries.com/ , is complete without mentioning the TDD Sudoku Fiasco :) Ravi has a nice summary: http://ravimohan.blogspot.se/2007/04/learning-from-sudoku-so... Peter Norvig's old-fashioned approach is excellent counterbalance: http://norvig.com/sudoku.html

Playing devil's advocate here: I'd say that algorithm design (Which is what a sudoku solver is) is a bad fit for TDD. However, I'd argue that most problems in commercial coding are of a more engineering/architectural type of nature. Here, outside-in TDD has it's place. Outside-in TDD can help you tackle problems that seem enormous, and slowly but steadily break them into smaller components.

I find TDD useless as a way of poking at problems I don't actually know how to solve (as Jeffries found with Sudoku), in the way and slowing down when writing large new systems, and excellent when bug-fixing maintenance. Test Driven Debugging!

Re: TDD did not live up to expectations

#67

TDD failed for economic reasons, not engineering ones. If you look at who were the early TDD proponents, virtually all of them were consultants who were called in to fix failing enterprise projects. When you're in this situation, the requirements are known. You have a single client, so you can largely do what the contract says you'll deliver and expect to get paid, and the previous failing team has already unearthed…

That's a great write-up on the situation. There's one thing I'll counter:

"And so economically, the only companies to survive are those that have built a steaming hunk of shit,"

This is true in average case because most of these startups don't care about quality that much since they don't have to for reasons you describe. However, if they did, then they could at least document things, do limited amount of Design-by-Contract, and/or do assertion/property-based testing as they went along. They can also follow principles like cleanly separating modules, protocols that have an update feature, standardized formats for data, and so on that can make rewriting a lot easier later. Easier, not easy, as rewriting is also difficult. That's why I mentioned documentation and Design-by-Contract as they take very little time. Essentially just write down whatever was in your head at that moment.

Most of the time will still go to experimentation and debugging which... wait, styles like Design-by-Contract in safe languages knock out a lot of that debugging. Good docs do, too. Now, we've gotten to the point illustrated by methods like Cleanroom where the QA might cost a little extra, might cost the same, or might even save time/money due to less debugging of new or old code.

Re: TDD did not live up to expectations

#68
post #53
post #40

Earlier quoted context omitted.

My experience is that whether one is writing tests first or not, the code that is being written does get tested one way or the other. In other words, people (for example web-devs) who don't write tests first tend to "test" their code by clicking around in the browser to see whether they get the desired effect. Writing tests, running them and then iteratively updating/fixing the code is IMO so much more efficient than…

Clicking around in your browser is more akin to an integration test though. TDD is about unit tests, not integration tests.

I disagree. I and others (such as Thoughtbot) often start by writing integration tests first. It’s known as outside-in testing.

TDD by started by with low level tests is just backwards. Outside-in helps you clarify the design from the perspective of the user.

https://robots.thoughtbot.com/testing-from-the-outsidein

Re: TDD did not live up to expectations

#69

I've also never found TDD to really be very beneficial except for all but the most trivial utility libraries. Most of the time, I have an idea of where I want to go, but not necessarily exactly what my interface will look like. Writing tests beforehand seems to never work our since nearly always, then will be some requirement or change that I decide to make that'd necessitate re-writing the test anyway, so why write…

> necessitate re-writing the test anyway, so why write it to begin with

Because now you have an unambiguous record of the conscious decision to change your interface, because the test demonstrates the correct change of that interface.

I don't write tests first for everything I do, but I try very hard to when I'm writing code that other people may read for this exact reason. Otherwise they have to divine my intent through less significant means.

Post reply on HN