Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TDD did not live up to expectations

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

311–320 of 450 posts

Re: TDD did not live up to expectations

#311

Post by maruhan2 »

"The tests get in the way. Because my design does not have low coupling, I end up with tests that also do not have low coupling. This means that if I change the behavior of how class works, I often have to fix tests for other classes. Because I don’t have low coupling, I need to use mocks or other tests doubles often. Tests are good to the extent that the tests use the code in precisely the same way the real system uses the code. As soon as I introduce mocks, I now have a test that only works as long as that mock faithfully matches the behavior of the real system. If I have lots of mocks – and since I don’t have low coupling, I need lots of mocks – then I’m going to have cases where the behavior does not match. This will either show up as a broken test, or a missed regression."

Simply comment them out temporarily?

"Design on the fly is a learned skill. If you don’t have the refactoring skills to drive it, it is possible that the design you reach through TDD is going to be worse than if you spent 15 minutes doing up-front design."

I don't quite understand how TDD means you skip up-front design.

Re: TDD did not live up to expectations

#312

Post by jacques_chester »

hinkley wrote: post #65
nostrademons wrote: post #37

TDD failed for economic reasons, not engineering ones. If you look at who were the early TDD proponents, virtually all of them were consultants who were called in to fix failing enterprise projects. When you're in this situation, the requirements are known. You have a single client, so you can largely do what the contract says you'll deliver and expect to get paid, and the previous failing team has already unearthed…

I have a colleague who is not a household name but still fairly widely known in the software process space and he has insisted to me multiple times, for more than ten years now, that this is a west coast thing. Other parts of the US are much less allergic to discipline in development.

> Other parts of the US are much less allergic to discipline in development.

I see TDD as a practice that requires a lot of discipline.

The TDD equivalent of crossing every t and dotting every i requires sustained attention and intention.

Re: TDD did not live up to expectations

#313

Post by i_feel_great »

kazinator wrote: post #296
srcmap wrote: post #235

Earlier quoted context omitted.

TDD always worked for me. All successful projects in my career had full functional regression test coverage. For those successful projects, I made following conclusions: * My personal TDD process are: * Script enable - always run every night, every weekends. Output to html files to see overall green/red in

> TDD always worked for me. > All successful projects in my career had full functional regression test coverage. Regression testing is not the definition of TDD, however. TDD is this crazy religion of writing tests for everything, including the lowest-level helper functions in a module that don't correspond to anything that would be tested in a regression test suite. You write these little tests first (which must be…

> TDD is this crazy religion of writing tests for everything,

Not true in my experience. You should only test code you wrote that can break. You do not test core functionality, the OS underneath etc. You do not test the test themselves.

Re: TDD did not live up to expectations

#314

Post by mlvljr »

plinkplonk wrote: post #199
willvarfar wrote: post #9

No article about TDD, particularly one that shouts out to the respected Ron Jeffries http://ronjeffries.com/ , is complete without mentioning the TDD Sudoku Fiasco :) Ravi has a nice summary: http://ravimohan.blogspot.se/2007/04/learning-from-sudoku-so... Peter Norvig's old-fashioned approach is excellent counterbalance: http://norvig.com/sudoku.html

that 3 line post is the most popular blog post I ever wrote. I got a lot of nasty mail from agile consultants, but it was worth it, heh! Which reminds me, I haven't blogged in years. One day I just stopped. Perhaps it is time to restart.

you do that

Re: TDD did not live up to expectations

#315

Post by nickpsecurity »

charlieflowers wrote: post #294
Joeri wrote: post #275

Earlier quoted context omitted.

They thought "refactoring" was a made-up word that programmers used to cover up dicking around and wasting time. From a non-programmer's perspective, all they see is the programmer spending several hours with the end result being that nothing has visibly changed. The thing is, many refactoring efforts make code worse, not better. Refuctoring is something I've observed many times over, by people who consider themselve…

Did you intend to say "Refuctoring"? It could be a typo or it could be clever wordplay. And it leads to two drastically different interpretations of what you're saying :)

I think I'm going to start using that one. Especially on ports of legacy software from one, crappy language to another crappy language. I'd especially guess that the COBOL to OOP COBOL translations involve a lot of refuctoring. I just gotta remember to give Joeri credit for coming up with it.

Re: TDD did not live up to expectations

#316

Post by nickpsecurity »

smokeyj wrote: post #114
atonse wrote: post #8

In the earlier days of the ruby community, I feel TDD was seen as gospel. And if you dared say that TDD wasn't the way (which I always felt), you'd feel like you were ostracized (update: just rewording to say, you'd worry that it would hurt you in a job search, not that people were mean to you). So I never spoke up. I feel like I was in the TDD-bad closet. I absolutely think _tests_ are useful, but have never found a…

I feel this way about code coverage (need that 100% badge). If I took on a project with low quality and had a limited budget to ensure QA - I'd prioritize E2E testing over unit testing.

Here's you some evidence to use when you bring that up:

https://blog.regehr.org/archives/872

Re: TDD did not live up to expectations

#317

Post by nickpsecurity »

i_feel_great wrote: post #313
kazinator wrote: post #296

Earlier quoted context omitted.

> TDD always worked for me. > All successful projects in my career had full functional regression test coverage. Regression testing is not the definition of TDD, however. TDD is this crazy religion of writing tests for everything, including the lowest-level helper functions in a module that don't correspond to anything that would be tested in a regression test suite. You write these little tests first (which must be…

> TDD is this crazy religion of writing tests for everything, Not true in my experience. You should only test code you wrote that can break. You do not test core functionality, the OS underneath etc. You do not test the test themselves.

"You should only test code you wrote that can break. "

A common source of errors are mismatches between what your dependencies say they'll do and what they'll actually do. There's been huge, costly failures from developers expecting the dependency would honor its documentation or just take a standard approach to error handling. So, either testing them or at least good catch-all error handling + logging should be done.

Re: TDD did not live up to expectations

#318

Post by nickpsecurity »

api wrote: post #223
taeric wrote: post #64

Earlier quoted context omitted.

No. TDD failed for engineering reasons in addition to economic ones. The main failing of TDD is the assumption that you know all of the tests that you will need before you know your software. This is true in bridge building as much as it is in anything else. Until you fully know the domain of what you are building, you cannot possibly know all of the things you need to test for. To that end, if TDD were focused aroun…

I guess another way to put it is that TDD is an attempt to do old school "waterfall" in code form. Way back before anyone really understand how software engineering should be done, people would begin a software product by attempting to pre-specify ahead of time every nuance of the system. They'd write these massive books of requirements. Then they'd get to work coding to these requirements. This rarely worked out wel…

TDD is closer to the Spiral model of iterative development of a product always assumed to be incomplete or broken in some way. Constant improvement instead of perfection.

https://en.wikipedia.org/wiki/Spiral_model

"As such the tests often end up containing bugs, which is the same problem as the classical "when we got going we found that the spec was wrong." Do you code to the bugs in the tests?"

Work in high-assurance field showed it's often easier to assert or check the results of complex software than it is to implement the software itself. Imagine a control system bringing in data, making complex calculations, and deciding whether to accelerate or decelerate. You want to ensure it never pushes past a certain maximum. It's much easier to code something at its output that says (if value >= maximum then StopAcceleratingMaybeNotifySomeone()). The checker is even easier to formally verify if you wanted to go that far. This is one of reasons many proof assistants started building stuff on simple-to-check cores where the generation process is untrusted but the trusted checker is tiny.

Empirically, though, there's confirmation out there in terms of defect detection that tests catch more bugs than they create.

Re: TDD did not live up to expectations

#319

Post by namuol »

The main benefit of TDD:

It strongly encourages you to think of your code as several input/output problems.

When you apply this model of thinking at scale it tends to lead to a much simpler (read: less-complex) codebase.

Re: TDD did not live up to expectations

#320

Post by copperx »

reitanqild wrote: post #213
poikniok wrote: post #200

Earlier quoted context omitted.

What is wrong with autogenerated code?

Nothing. There is nothing wrong with code generation in a production setting with "grown up" engineers who uses version control and knows the pitfalls. The part in parenthesis is just a rant about education and how teachers use notepad and javac to teach Java. So my point is the only thing students shouldn't use in an IDE is code generation.

But going from javac to Maven in CS1 would be insanely difficult; there are many prerequisites before one can use Maven. It may only work in top-tier schools.
Post reply on HN