Where unit testing fails • Hacker News Board

Re: Where unit testing fails

#31

Post by hmemcpy » Mon, Jul 25, 2011, 1:46 PM UTC

rimantas wrote: post #17

chrislloyd wrote: post #7

I can't help but appreciate the irony of a link titled "Where unit testing fails" leading to a WordPress database connection error.

So you think unit tests would help with that?

Sorry for the late reply.

That's an interesting question - would a unit test prevent this error? To answer this, let's consider what happened: the database didn't respond in an appropriate way, and probably returned an error. I'm not exactly sure what exact error was displayed, but WordPress has caught this situation, and handled it the best it could - displaying a proper error. I would write a unit test for this exact scenario - using a fake (mock) object, simulate the database timeout, and make sure that the error was caught and handled appropriately. Other then that there's no real other way to gracefully recover.

Re: Where unit testing fails

#32

Post by peteretep » Mon, Jul 25, 2011, 1:55 PM UTC

jsdalton wrote: post #21

peteretep wrote: post #11

Earlier quoted context omitted.

It's hocus where contrasted to developer-driven testing, because Test Driven Development - as a development methodology (as opposed to a tool) - espouses that you Write All Your Tests first. Have I had experience (and much value) out of sometimes writing tests for certain problem classes before writing any code? Yes. Changes to existing functionality are often a good candidate. Does TDD as a methodology suggest you s…

I've always thought of testing as simply a way to verify that your code does what is supposed to do. I tend to agree with you that TDD as a development methodology is deeply flawed. I have never found that the act of writing a test case and making it pass has helped me with the actual creative process of solving a problem via code. In fact, I'll frequently dash out a method I'm writing quickly, and then go back and r…

I think you're narrowing in on the distinction I'm trying to make between a methodology and a tool there; writing tests before you implement a piece of functionality is a useful tool to have in your testing toolbox.

Labouring under the idea that Tests Must Come First (and everything I've seen, and everything I /do/ see now suggests that that is the central idea in TDD - you write a test, then you write the code to pass it) without pivoting to see that testing is a useful practice in so much as it helps developers is the wrong approach.

Re: Where unit testing fails

#33

Post by peteretep » Mon, Jul 25, 2011, 1:58 PM UTC

arethuza wrote: post #15

peteretep wrote: post #11

Earlier quoted context omitted.

It's hocus where contrasted to developer-driven testing, because Test Driven Development - as a development methodology (as opposed to a tool) - espouses that you Write All Your Tests first. Have I had experience (and much value) out of sometimes writing tests for certain problem classes before writing any code? Yes. Changes to existing functionality are often a good candidate. Does TDD as a methodology suggest you s…

I don't think I've ever seen a description of TDD that suggests you write all of your tests before writing any production code - rather you iteratively write some tests, have them fail then write the code to make them work and then see the tests pass, repeating that process until you have implemented everything you need.

Sure, but you write /some/ tests first. And that means, if you want to do it meaningfully, that you either need to zoom down in to tiny bits of functionality first in order to be able to write those tests (see the isPalindrome discussion in the article), or you write a test that requires most of the software to be finished, or you cheat and fudge it.

Making tests a central part of the process because they're useful to developers? Awesome. Dictating a workflow to developers that works in some cases as the One True Way: ridiculous.

Re: Where unit testing fails

#34

Post by eru » Mon, Jul 25, 2011, 1:59 PM UTC

arethuza wrote: post #15

peteretep wrote: post #11

Earlier quoted context omitted.

It's hocus where contrasted to developer-driven testing, because Test Driven Development - as a development methodology (as opposed to a tool) - espouses that you Write All Your Tests first. Have I had experience (and much value) out of sometimes writing tests for certain problem classes before writing any code? Yes. Changes to existing functionality are often a good candidate. Does TDD as a methodology suggest you s…

I don't think I've ever seen a description of TDD that suggests you write all of your tests before writing any production code - rather you iteratively write some tests, have them fail then write the code to make them work and then see the tests pass, repeating that process until you have implemented everything you need.

You can work within TDD by just calling all code that comes before the tests a `prototype'.

Re: Where unit testing fails

#35

Post by eru » Mon, Jul 25, 2011, 2:01 PM UTC

colin_jack wrote: post #25

I haven't had time to read them all yet so I was wondering if someone could give me a summary of why the TDD based sudoku approach failed?

The guys approach to the problem wasn't smart enough.

Testing doesn't help you come up with good algorithms. It can help you with not messing up an implementation, though.

Re: Where unit testing fails

#36

Post by peteretep » Mon, Jul 25, 2011, 2:01 PM UTC

Volpe wrote: post #14

peteretep wrote: post #11

Earlier quoted context omitted.

It's hocus where contrasted to developer-driven testing, because Test Driven Development - as a development methodology (as opposed to a tool) - espouses that you Write All Your Tests first. Have I had experience (and much value) out of sometimes writing tests for certain problem classes before writing any code? Yes. Changes to existing functionality are often a good candidate. Does TDD as a methodology suggest you s…

> Writing tests before code works pretty well in some situations. Test Driven Development, as handed down to us mortals by Agile Testing Experts and other assorted shills, is hocus. What? So it works in some situations, but not all. Thus hocus? I'm no 'Agile Testing Expert', but I have worked for the last 2-3 years doing TDD exclusively (as in ALWAYS writing tests first). It isn't hocus. It is effective. I can unders…

> I can understand it doesn't suit all problems, and can potentially be a hinderance sometimes

Then why do it exclusively?

Testing is about helping developers, and recognizing that automated testing is about benefit to developers, rather than cargo-culting a workflow and decreeing that one size fits all.

Writing tests first as a tool to be deployed where it works is "Developer Driven Testing" - focussing on making the developer more productive by choosing the right tool for the job. Generalizing a bunch of testing rules and saying This Is The One True Way Even When It Isn't (I mean seriously, you continued to use it when it was a hindrance?!) - that's not right.

Re: Where unit testing fails

#37

Post by kenjackson » Mon, Jul 25, 2011, 2:07 PM UTC

paganel wrote: post #22

Maybe I'm in a mad mood or somth (it's Monday, after all) but all this "let's write a parallel world, called TDD, to which the actual production code has to comply and praise and give sacrifices"-mantra has gotten a little over the top. Don't get me wrong, I'm sure that all these guys and chicks that live and die by TDD are smart (probably smarter than me), but what they are building is starting to get more and more…

TDD is new abstract concrete factory of super factories. People who get so caught up in an ideal, they lose focus of actually delivering solutions.

Re: Where unit testing fails

#38

Post by damoncali » Mon, Jul 25, 2011, 2:11 PM UTC

Here's my trouble with testing as espoused by proponents of TDD: I bought into the Agile Manifesto. I believe in working code over specifications. How then, am I supposed to write a bunch of tests that will in turn tell me what code to write?

It is not a coincidence that one of the reasons TDD'ers love rspec so much is that the "code reads like documentation" - it's even called r spec.

Perhaps that is ok for mature projects. But I often hear TDD and agile used together as if they weren't in conflict and I get confused. Then again, I've only ever worked on new projects.

Maybe I'm missing something?

Re: Where unit testing fails

#39

Post by berntb » Mon, Jul 25, 2011, 2:19 PM UTC

tychof wrote: post #24

berntb wrote: post #20

Earlier quoted context omitted.

Excuse me for interrupting the budding flame war, but I have a question regarding the subject, which you seem qualified to answer? :-) My experience is the same as (what I believe is) the usual criticism of tests-first. I often need to rethink the API to my functions/methods, sometimes more than once. This goes for both external and internal APIs, so it doesn't help with detailed interface specifications between modu…

One benefit to TDD is that you become a user for your API much more quickly, and so your API doesn't need to change as much. TDD suggests 3 steps: 1. Write one test 2. Write the minimal code to make it pass 3. Refactor the code you've got. Try not to think of tests as immutable. You can throw away tests when they don't provide value. Finally, make sure you're using an IDE with refactoring support. That helps immensel…

OK, thanks to you and zumda.

Interesting TDD argument. I'll buy a book or something and try again when my present hobby project is done and I have time for experimenting.

(IDE refactoring needs a stricter language than I prefer. Also, without an IDE and using my old eyes I have 2 A4 of code + 1 A4 of bash on a 24" monitor...)

Re: Where unit testing fails

#40

Post by misfo » Mon, Jul 25, 2011, 3:07 PM UTC

peteretep wrote: post #32

jsdalton wrote: post #21

Earlier quoted context omitted.

I've always thought of testing as simply a way to verify that your code does what is supposed to do. I tend to agree with you that TDD as a development methodology is deeply flawed. I have never found that the act of writing a test case and making it pass has helped me with the actual creative process of solving a problem via code. In fact, I'll frequently dash out a method I'm writing quickly, and then go back and r…

I think you're narrowing in on the distinction I'm trying to make between a methodology and a tool there; writing tests before you implement a piece of functionality is a useful tool to have in your testing toolbox. Labouring under the idea that Tests Must Come First (and everything I've seen, and everything I /do/ see now suggests that that is the central idea in TDD - you write a test, then you write the code to pa…

I think you've hit the nail on the head, @peteretep.

The problem is that ideologies like TDD try to come up with simple rules for how to develop (i.e. always write tests first). In reality, it's better to use your brain when you're deciding how to approach something (i.e. is this a good situation for writing tests first?)