Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why TDD isn't crap

hillelwayne.com

Post reply on HN
Print view View original

141–150 of 171 posts

Re: Why TDD isn't crap

#141

Post by feikname »

hacker_9 wrote: post #58
quicklime wrote: post #32

Earlier quoted context omitted.

I've always thought that the bulk of the value of tests is for the unfortunate person who has to refactor or extend your code years down the track. By that logic, if you write testable, but untested code, then you're still making it difficult for people to refactor later. This applies even if the code is well designed and uncoupled.

You don't even have to think that far ahead - consider some code you wrote in week 1, that was then modified by a colleague in week 2, now you return to the code in week 3 - what does the code do? Any mental model you had at week 1 is now irrelevant as the code has changed in unknown ways. What is the current intent of the code? Do you feel safe modifying it, despite the fact you no longer know how to manually test a…

> Any mental model you had at week 1 is now irrelevant as the code has changed in unknown ways.

Isn't that what good git commit messages are for though? (Or any other source code versioning tool.)

I'm not saying tests aren't useful, but often times I find reading a file diff history is way more useful to understanding it than its tests.

IMO, tests should be there only to provide some level of feeling safe modifying files, as you can easily know if you messed up something.

Re: Why TDD isn't crap

#142

Post by hwayne »

AnimalMuppet wrote: post #131
hwayne wrote: post #90

Earlier quoted context omitted.

Yeah. "Pure" TDD is strict cycles of "Write failing test, write the bare minimum of code that passes, refactor", which most studies suggest isn't much more helpful than the "common practice" TDD of "write tests as you code".

It's not more helpful as tests than "write tests as you code". But "pure" TDD is more helpful as a guide to low-level design. I'm not sure that most studies can quantify that.

My first resource on these things is the absolutely fantastic book "Making Software", which has overviews of a lot of these questions how they're studied. The studies they looked at didn't find a correlation between pure TDD and 'software quality', but they admit the data is pretty scarce.

Re: Why TDD isn't crap

#143

Post by hacker_9 »

feikname wrote: post #141
hacker_9 wrote: post #58

Earlier quoted context omitted.

You don't even have to think that far ahead - consider some code you wrote in week 1, that was then modified by a colleague in week 2, now you return to the code in week 3 - what does the code do? Any mental model you had at week 1 is now irrelevant as the code has changed in unknown ways. What is the current intent of the code? Do you feel safe modifying it, despite the fact you no longer know how to manually test a…

> Any mental model you had at week 1 is now irrelevant as the code has changed in unknown ways. Isn't that what good git commit messages are for though? (Or any other source code versioning tool.) I'm not saying tests aren't useful, but often times I find reading a file diff history is way more useful to understanding it than its tests. IMO, tests should be there only to provide some level of feeling safe modifying f…

It'll definitely show you the changes, but how far back do you start from? and how many files do you do this for? In addition, it doesn't show you how the dataflow changed; you need to model that in your head still.

Tests would also do this for you, but without the mental burden of brain compiling. It's nice to be able to set some breakpoints in code, then start a quick debugging session with the relevant test and see the data flow through the code - you can understand any function usually within 5 minutes.

Re: Why TDD isn't crap

#144

Post by falcolas »

lowbloodsugar wrote: post #97
mattmanser wrote: post #21

Earlier quoted context omitted.

This whole thread is a response to: https://news.ycombinator.com/item?id=15591190 Your points are directly addressed in the pdf. One of his general points being, in practice, the tests become the legacy system instead. And I'd add to that. Given that you've at minimum doubled the code (and doubled the bugs), it seems like a really bad long-term trade off. Also DI does not reduce coupling. I've seen plenty of code wit…

That article is garbage. For example, the claim that an object has trillions of states ignores our ability to classify ranges of values. Sure, an int may contain "four billion states", but for the requirements (which he makes such a big deal about) its highly likely that we can classify the integer into three states: less than zero, zero, greater than zero. As a bank, I might not care how much money you have, only th…

> we can classify the integer into three states: less than zero, zero, greater than zero

Err, not quite. We're running on computers, remember. A comprehensive unit test that doesn't go through every possible integer input should still include:

maximum and minimum integer values for that size

maximum and minimum expected inputs

0, 1, -1

So, there's 7 test cases for a single integral input. And since you bring up banking, let's imagine you're working with a function with two inputs. Since most bugs come about because of the interaction between two variables [0], you'd want to check out each combination. So, 7 possible inputs for the first integer, 7 for the second: 49 test cases.

Why bother? You're working for a bank... imagine facing a client whose balance shot negative because of an overflow error and started accruing "overdraft coverage" fees.

Back to the original point: when was the last time anybody but the AFL tool wrote the "proper" 49+ unit tests for an "add_to_balance(int, int)" style function, when one test would give you 100% coverage for that function?

[0] https://csrc.nist.gov/Projects/Automated-Combinatorial-Testi...

Re: Why TDD isn't crap

#145

Post by marsrover »

JoeNr76 wrote: post #67

I never was a fan of TDD, until I saw this talk by Ian Cooper: https://www.infoq.com/presentations/tdd-original The whole idea of testing functions and/or classes separately means tightly coupling your test code to the implementation of the real code, while you should only care about testing the functionality. Nowadays, I try to write tests that test a unit of functionality. And the tests should only change when the…

Not a word for word quote, but I heard this:

> How many of you have a code base that if you refactored, test would break?

> Most people raise hand

and it made me realize maybe I'm not as stupid as I think I am.

I've been trying to understand how to create unit tests that allow me to refactor for years, and have been completely unsuccessful. The only way I can achieve this is using the "classicist" viewpoint of creating many "unit test" and only isolating the architecturally significant boundaries. That doesn't make me happy either, though.

But, yeah, good to know (or bad to know) that I'm not the only one that struggles with this.

Back to the video I go.

Re: Why TDD isn't crap

#146

Post by Verdex_3 »

lowbloodsugar wrote: post #128
Verdex_3 wrote: post #115

Earlier quoted context omitted.

Although on the other hand, writing your test first with no code and having it fail isn't really telling you much about the test. It just shows that the test handles the least interesting case (the case where there exists no code). You still don't know that your test is correct when you get it passing. You just know that it is correct when no code exists. For example, TestSort(ISorter sorter) { var input = { 1, 2, 3,…

If someone gave me that test, I'd implement the code as "return input;" Forget that you, a human, thinks you know what an ISorter should do. What do the tests demand that you do? Doing TDD, the goal, when coding, is to write the minimal amount of broken code that makes the test pass. If you passed me {4,3,2,1}, then my code would be "return {1,2,3,4};". If you wrote a second test that passed me {6,5,4,3}, my code wou…

And that works fine ... as long as you know all of the cases that will cause the minimal amount of broken code to fail in a test case.

What happens if you're implementing something like a unification algorithm, but you don't know about the occurs check. You'll progressively add test cases that break your code and eventually stop. Until an end user creates a query that contains itself and your algorithm fails to terminate.

The solution to get TDD to work is to know what all of the weird edge cases of your algorithm are. However, I assert that the thing that makes TDD ever work is that the practitioners who are "doing it right" already know all of the weird edge cases of their algorithms. And I assert that if you know the weird edge cases you can drop the test first part of TDD and still get working algorithms.

Random data is interesting because it's not something I've seen anyone mention when they talk about how they do TDD before now. I suppose if you knew nothing about your problem and had some sort of tool progressively feed you intelligently generated random data, that might work for algorithms that have few edge cases or relatively simple edge cases. However, it seems like you would want something more like quick check for cases where you have really esoteric edge cases. And I don't see how you avoid implementing bubble sort (or its equivalent in your domain).

I don't think TDD can't work, but so far nobody has described TDD to me where I have any reason to assume that TDD is actually doing anything worthwhile. It's always sounded like TDD was irrelevant and the real key to success was thinking about the problem carefully. Additionally, it sounds like there are problem domains where anything that TDD might bring to the table would be nullified (ie certain problems that exhibit a certain level of complexity).

Re: Why TDD isn't crap

#147

Post by chewbacha »

michaelmrose wrote: post #109
chewbacha wrote: post #33

Earlier quoted context omitted.

The code is the “test” for your tests ;)

The code and the test are probably to some degree based on the same logical construct. If that logical construct is malformed your tests may pass and your code might not work. Regardless if your code works your tests could still fail, likewise your code could not work and your tests pass so since the tests and code may vary independently if either is buggy I don't see how this can possibly be true.

If the logical construct is flawed, no amount of tests will ever catch it, only user testing would.

The point I’m trying to make is that the verification of correctness is bidirectional. If I’m writing tests around existing code, I rely on the code to test my expectations. If I’m writing new code, I write tests to assert my new assumptions. All automated tests do are assert that I’ve written the same logic twice. Writing it a third time will also decrease the likelihood of transcription error again, but at diminished returns.

Re: Why TDD isn't crap

#148

Post by lowbloodsugar »

falcolas wrote: post #144
lowbloodsugar wrote: post #97

Earlier quoted context omitted.

That article is garbage. For example, the claim that an object has trillions of states ignores our ability to classify ranges of values. Sure, an int may contain "four billion states", but for the requirements (which he makes such a big deal about) its highly likely that we can classify the integer into three states: less than zero, zero, greater than zero. As a bank, I might not care how much money you have, only th…

> we can classify the integer into three states: less than zero, zero, greater than zero Err, not quite. We're running on computers, remember. A comprehensive unit test that doesn't go through every possible integer input should still include: maximum and minimum integer values for that size maximum and minimum expected inputs 0, 1, -1 So, there's 7 test cases for a single integral input. And since you bring up banki…

If you mean "nobody would write 49 tests for a function when they could get away with writing one test and have it show up as 100% code coverage because people are lazy bastards" then I agree, and that's the problem: cargo culting the metrics. Look we have 100% code coverage! We are awesome!

If you mean, "no look seriously, you'd need 49 tests for every method because you're a bank", well then, no, you actually need less than 49 (since some are subsets of the others), and in reality even this would get very tiring so you'd write a currency class that can't overflow, and yes, being a bank we would test every case and certainly use automated test generation while we're at it.

But if you mean that in general, for non-banks, the classes of values is too large to manually write unit tests for, then we disagree. That means that your method is too complex.

Re: Why TDD isn't crap

#149

Post by the_af »

root_axis wrote: post #94
the_af wrote: post #48

Earlier quoted context omitted.

The code is definitely NOT the test of the tests. It may seem that way because (sometimes) when the code breaks (e.g. fails to run at all), the tests also break. But consider this: what if your tests are poorly written and fail to detect bugs? What if they fail because the test code is buggy? Failing to detect bugs is a bug (undesirable behavior/output) of test code. There are some techniques to address this, for exa…

> what if your tests are poorly written This seems like tautological reasoning. Naturally, if you write bad tests then they won't be effective, but that seems like a poor argument against testing. That's like saying "Yes, bypass surgery could save your life, but consider this: what if the surgeon is reckless and kills you on the operating table?". Obviously, there is a minimum expectation that the surgeon will follow…

I missed this before:

> what if the surgeon is reckless and kills you on the operating table?

Excellent question! What's this surgeon's track record? How many patients have died on her operating table? How up to date with current medical research is she? Is she daring and reckless, or does she play it conservatively safe?

Re: Why TDD isn't crap

#150

Post by gvdenbro »

lowbloodsugar wrote: post #95
latch wrote: post #8

Like the author, I subscribe to the less strict view that TDD isn't necessarily about writing the test first, but rather about having test play some part in how the code takes shape. Unlike the author, I absolutely believe that tests are about design. More specifically, they're about identifying coupling so that you can reduce it. The function being "awkward" to use is part of it, but code which is hard to test is al…

How do you test your test if you don't write the test first? If you write the code, and then you write the test, then you haven't tested your test. You have no proof that your test will detect broken code. The only way to prove that your test can detect broken code is to run it against broken code. So you can write the code, then write the test, and then break the code and run the test. Or you could just write the te…

you could test your tests using mutation testing [0]. it's not bulletproof but quite close.

[0] https://en.wikipedia.org/wiki/Mutation_testing

Post reply on HN