Giving Up on TDD • Hacker News Board

Re: Giving Up on TDD

#71

Post by jacques_chester » Sat, Mar 19, 2016, 8:34 PM UTC

ssmoot wrote: post #60

c0achmcguirk wrote: post #18

Earlier quoted context omitted.

> As usual my advice on this is: look at the people who build things you find highly impressive, and study how they did it. This is much more fruitful than reading the output of people who want to spend all day telling you how to program (which leaves very little time for them to build software that is impressive, i.e. they never even test their own ideas!) "Uncle Bob" Martin has built a lot of impressive things. Mos…

Back when TDD started, it was often referred to as Test Driven DESIGN by the same people you cite. I do it, and I find it valuable. But I think if you're not using it as a design tool, you're missing most of the benefit. You don't do TDD (IMO) to prove correctness. You do it to achieve a level of composition and design that makes later requirements changes and refactoring less painful. Proving that your stuff actuall…

> BDD and the culture that sprang up around it (at least in the Ruby community) was such a disappointment after having done TDD for a few years previous. It's like every known anti-pattern was adopted as a core deliverable and the entire point of the exercise (IMO, solving the "blank page" problem in building systems; Design) was forgotten entirely.

Could you elaborate?

Speaking for myself, I thought that the main useful thing to come out of BDD was working outside-in and the given/when/then convention.

But I certainly didn't get it on my first pass. Looked at Cucumber, soldiered mightily, gave up in disgust.

Re: Giving Up on TDD

#72

Post by ArkyBeagle » Sat, Mar 19, 2016, 8:36 PM UTC

simula67 wrote: post #26

jacques_chester wrote: post #22

Earlier quoted context omitted.

> Because the benefits aren't higher, in fact they are lower It depends on what you think the benefits are. > As usual my advice on this is: look at the people who build things you find highly impressive, and study how they did it. Good idea. Researchers at Microsoft and IBM conducted studies of applying TDD to real, commercial projects of varying sizes, languages, team structure and complexity. They found that defec…

Each chapter in the book 'Making Software' by Greg Wilson and Andy Oram, takes one claim usually made by software developers and examines the body of research surrounding that claim. Chapter 12 is on TDD. The conclusion was that the evidence is not conclusive.

Can we then conclude that there's not a way to balance the 15 to 35 % rise in (one) cost against the 40 to 90% drop in defects ( if those are the measurements )?

That's really interesting if it is the case.

Re: Giving Up on TDD

#73

Post by jacques_chester » Sat, Mar 19, 2016, 8:48 PM UTC

facepalm wrote: post #31

And communism would work if only people would do it properly. It seems to me that kind of argument could be made about anything. "It only didn't work because you didn't do it properly". Doesn't it work for Homeopathy, too? "It would have worked if you hadn't passed it through the x-ray scanner at the airport" or "it would have worked if you had taken the medicine while doing a handstand". In the end doing it right be…

The communism analogy is better than the homeopathy analogy. The former makes problematic assumptions about human nature, the latter impossible assumptions about physical reality.

That said, I work in a company where TDD is the default. So far everything still works and there are no tanks rolling into any public squares to squash the bourgeois scourge.

Re: Giving Up on TDD

#74

Post by jacques_chester » Sat, Mar 19, 2016, 9:00 PM UTC

ArkyBeagle wrote: post #72

simula67 wrote: post #26

Earlier quoted context omitted.

Each chapter in the book 'Making Software' by Greg Wilson and Andy Oram, takes one claim usually made by software developers and examines the body of research surrounding that claim. Chapter 12 is on TDD. The conclusion was that the evidence is not conclusive.

Can we then conclude that there's not a way to balance the 15 to 35 % rise in (one) cost against the 40 to 90% drop in defects ( if those are the measurements )? That's really interesting if it is the case.

I think it can be rationally traded off, so long as everyone is clear about how to measure and what the decision criteria are.

I think that for products with long release latency (ie the kind of shrink-wrapped software Microsoft specialises in), the accounting very much works against TDD on these findings, because the incentives are to front-load features and catch up on bugs with service patches. If you miss your release window, your feature will be bumped by years.

For products with continuous deployment, the economics shift and invert. Blasting out code willy-nilly will eventually cause engineering to steadily be diverted to firefighting. Eventually forward progress stalls.

Re: Giving Up on TDD

#75

Post by ArkyBeagle » Sat, Mar 19, 2016, 9:20 PM UTC

jacques_chester wrote: post #74

ArkyBeagle wrote: post #72

Earlier quoted context omitted.

Can we then conclude that there's not a way to balance the 15 to 35 % rise in (one) cost against the 40 to 90% drop in defects ( if those are the measurements )? That's really interesting if it is the case.

I think it can be rationally traded off, so long as everyone is clear about how to measure and what the decision criteria are. I think that for products with long release latency (ie the kind of shrink-wrapped software Microsoft specialises in), the accounting very much works against TDD on these findings, because the incentives are to front-load features and catch up on bugs with service patches. If you miss your re…

This just confirms my understanding that nobody really understands the cost of defects.

Re: Giving Up on TDD

#76

Post by 0xcde4c3db » Sat, Mar 19, 2016, 9:26 PM UTC

c0achmcguirk wrote: post #18

jblow wrote: post #5

For many paragraphs, I thought this was a parody of TDD defense, but then it turns out it wasn't. His 'defense' of the point is basically: Look, when you do TDD you have to put a lot more work into the tests than you thought! It is not just a simple thing! Okay, fine, but ... Before embarking on TDD, the programmer had a picture in his head of what the costs+benefits of this change would be. Now you are telling him t…

> As usual my advice on this is: look at the people who build things you find highly impressive, and study how they did it. This is much more fruitful than reading the output of people who want to spend all day telling you how to program (which leaves very little time for them to build software that is impressive, i.e. they never even test their own ideas!) "Uncle Bob" Martin has built a lot of impressive things. Mos…

> I've yet to meet an anti-TDD zealot who has actually spent a month developing the art of TDD. The detractors tend to be people who write really brittle tests that are a pain to maintain.

I guess I loosely fall into the category of "anti-TDD". I don't generally go around badmouthing it, but I'm not really sold on the idea that it would save me significant time or effort. I certainly encounter things that TDD would likely catch earlier, but those are typically easy fixes and caught in QA or beta testing. The majority of my bugfixing hours are eaten up by things for which I don't even know what it would mean to write tests, such as:

- reproducing rare hardware-involved errors (often race conditions, often based on the variability of random-ish real-world events) in a way that allows getting relevant information logged/dumped

- clarifying product definition/spec, usually to resolve some inconsistency where the behavior isn't invalid per se but implicitly conflicts with some other requirement (i.e. we basically decide which behavior to call a bug and change that one)

- reverse-engineering undocumented design decisions of some third-party product that our product is expected to work with, because we foolishly believed that there was a standard (overlaps somewhat with #1)

So while I don't claim that TDD has no value, I admit to rolling my eyes a bit at the more fervent evangelists.

Re: Giving Up on TDD

#77

Post by tracker1 » Sat, Mar 19, 2016, 9:29 PM UTC

c0achmcguirk wrote: post #18

jblow wrote: post #5

For many paragraphs, I thought this was a parody of TDD defense, but then it turns out it wasn't. His 'defense' of the point is basically: Look, when you do TDD you have to put a lot more work into the tests than you thought! It is not just a simple thing! Okay, fine, but ... Before embarking on TDD, the programmer had a picture in his head of what the costs+benefits of this change would be. Now you are telling him t…

> As usual my advice on this is: look at the people who build things you find highly impressive, and study how they did it. This is much more fruitful than reading the output of people who want to spend all day telling you how to program (which leaves very little time for them to build software that is impressive, i.e. they never even test their own ideas!) "Uncle Bob" Martin has built a lot of impressive things. Mos…

I don't necessarily think it's even TDD vs not... I'm not big on TDD, but always try to design my code so that it is very module, and as a side effect easier to test. At least beyond some cross cutting edge cases, which are usually easy enough to isolate.

The act of TDD, or test compliance is only that you've looked at everything twice. It doesn't guarantee quality. Writing modular code means less friction over time, and TDD encourages this, but it's not necessarily a requirement imho.

Re: Giving Up on TDD

#78

Post by aikah » Sat, Mar 19, 2016, 9:33 PM UTC

ryanbrunner wrote: post #6

There's definitely some good points here, but I have a huge issue with this point: > Something that is hard to test is badly designed. In my experience, TDD has a natural tendency to favour decoupling at all costs, and TDD zealots will push de-coupling units as an unqualified positive. And if your metric is "good design = easily testable and low coupling" it certainly looks that way. This mindset fails to take into c…

> In my experience, TDD has a natural tendency to favour decoupling at all costs, and TDD zealots will push de-coupling units as an unqualified positive. And if your metric is "good design = easily testable and low coupling" it certainly looks that way.

Depends on the kind of TDD. I feel like TDD as Unit Testing driven design IS a waste of time (I'm talking about unit tests first here). However integration testing first is useful. Let's say I'm writing a library(a http router). The library will obviously have an API that will be consumed by the user, well for me testing will be like writing first how I think the API should look like. For a http router it would be like :

    var router = new Router();
    router.get("/{id}",someController);
    response = httpClientMock.get("/34943");
    expect(response.body).toBe("Hello 34943");

It costs me very little to write that , provided the language has all the facilities to test http servers[1], and check that the result of the operation is what I expect. I don't need to test the code that will parse the path for route variables directly, yet I did write the test first.

1 : that's often the problem , languages care about syntax but don't care about developer experience and the quality of tools around the language. It's not just "a compiler and the rest is your problem",no testing should be entirely part of the language or its standard library. Testing SHOULD be made as easy as possible.

Re: Giving Up on TDD

#79

Post by thesz » Sat, Mar 19, 2016, 9:49 PM UTC

c0achmcguirk wrote: post #18

jblow wrote: post #5

For many paragraphs, I thought this was a parody of TDD defense, but then it turns out it wasn't. His 'defense' of the point is basically: Look, when you do TDD you have to put a lot more work into the tests than you thought! It is not just a simple thing! Okay, fine, but ... Before embarking on TDD, the programmer had a picture in his head of what the costs+benefits of this change would be. Now you are telling him t…

> As usual my advice on this is: look at the people who build things you find highly impressive, and study how they did it. This is much more fruitful than reading the output of people who want to spend all day telling you how to program (which leaves very little time for them to build software that is impressive, i.e. they never even test their own ideas!) "Uncle Bob" Martin has built a lot of impressive things. Mos…

>I've yet to meet an anti-TDD zealot who has actually spent a month developing the art of TDD.

Do you actually apply that reasoning in everything you do? This is highly unproductive approach.

At least one paragraph from [1] this essay by PG is very relevant:

Most people have learned to do a similar sort of filtering on new things they hear about. They don't even start paying attention until they've heard about something ten times. They're perfectly justified: the majority of hot new whatevers do turn out to be a waste of time, and eventually go away. By delaying learning VRML, I avoided having to learn it at all.

[1] http://paulgraham.com/popular.html

Re: Giving Up on TDD

#80

Post by jacques_chester » Sat, Mar 19, 2016, 9:52 PM UTC

ArkyBeagle wrote: post #75

jacques_chester wrote: post #74

Earlier quoted context omitted.

I think it can be rationally traded off, so long as everyone is clear about how to measure and what the decision criteria are. I think that for products with long release latency (ie the kind of shrink-wrapped software Microsoft specialises in), the accounting very much works against TDD on these findings, because the incentives are to front-load features and catch up on bugs with service patches. If you miss your re…

This just confirms my understanding that nobody really understands the cost of defects.

In the way we work at Pivotal, feature stories "earn" velocity, but bugs do not. So if attention is not paid to code quality, velocity will eventually trend downwards.

This is intentional, as it gives you a feedback loop on quality and helps to improve predictions of actual feature progress.

From there you can invert to points/cost per engineer/week.

Of interest, our average points per engineer per week on Cloud Foundry has remained largely steady over time and scale. I'm wary of that metric because it breaks a fundamental concept of measuring velocity (that points are only meaningful inside a single project).