Live data from Hacker News

Reasons to Avoid Test Driven Development

softwareandi.com

21–30 of 42 posts

Re: Reasons to Avoid Test Driven Development

#21

This article is just irritating sarcasm. There are perfectly good reasons to avoid test-driven development. Like being part of an existing large project, where there are already too many interconnected components. Or working in complex AI algorithms, where separate testing of components is extremely painful, and lots of whole system testing is necessary anyway.

Could you expand on this? Why are too many interconnected components or a complex AI algorithm a valid reason not to test or use test driven development? My view is that in a complex system, only by having confidence in the units can you have confidence in the integrated whole. You might be able to build a house that stands and looks nice, but if the timbers, the bricks, the foundations etc are all flawed in their ow…

Oh, we certainly do massive automated testing. Our release testing script takes currently an 8-core machine about 4 days, and our nightly test runs on about 6 configurations, taking about 2 hours.

However, often the units have properties which are very difficult to specify in isolation, and from experience almost all bugs come from many algorithms reading and writing the same data structure (I'm not thinking of threading issues here, although they are another issue).

Consider something like a sudoku solver (the things I work on are much more complicated than that). Usually writing the function which checks the condition 'a row/column/box are all different' is very simple. The errors come when these functions interact, each effecting each other as they are called.

Certainly, this kind of situation is very unusual. Most people write code which lends itself more naturally to test-driven development, and in those situations I approve of it.

Re: Reasons to Avoid Test Driven Development

#22
The article doesn't seem to recognize there is a distinction between TDD, having unit tests, and having automated tests. Personally, I've used all three approaches, and think each has its place. As nearly as I can tell, anyone who says one approach fits every programming project must not have a very broad experience with different sorts of projects.

Re: Reasons to Avoid Test Driven Development

#23
These are reasons for testing, not TDD. There is no discernable difference between 2 shipped products with identical test coverage where one team used TDD and the other just wrote regression test.

The worst part about TDD is the people trying to shove it down your throat.

Re: Reasons to Avoid Test Driven Development

#24
TDD is awesome, if you have a codebase or a tech stack that makes it nice. TDD will quickly expose bad design through an unpleasant testing process. For example, slow tests, untestable code, or tests that you "always have to rewrite/maintain" are problems that only show up when you try and TDD your code that isn't designed to be testable.

TDD doesn't work well if you don't design your code to be testable because you very quickly become frustrated by the problems TDD uncovers.

1. Slow Tests - If your tests are taking a long time, it's probably because your code isn't written to be tested easily.

For example, if your code is hitting the database a lot and it starts to take minutes or hours to run your test suite, your code has a problem. For example, if your entities/models are coupled to an ORM like ActiveRecord, testing those is going to suck because they probably sholdn't be coupled to an ORM. Many people try to fix this by writing frameworks that make testing their code easier instead of writing better code that isn't tied to an ORM.

Tests should be fast. Most of the time they don't need to hit the DB. Slow tests make TDD suck. Avoid slow tests.

2. Untestable Code - I've run into this a lot and it sucks, especially in languages that lend themselves easily to untestable code (In my experience, that's PHP).

Until you try to test your code, you don't realize how untestable your code is. A lot of things that seem like reasonable design decisions are so untestable, that you end up doing incredibly lazy integration tests because to fix things you have to rewrite whole portions of the system.

I've been there and it sucks. The choice seems to lie between rewriting the whole system on the same platform to be testable one chunk at a time, or to rewrite the system in something that lends itself better to testing.

In either case, the answer is to take the time to learn how to write testable code. To start learning techniques like Dependency Injection. To practice writing portions of your system using TDD to see what works and what doesn't.

3. Maintaining Tests Sucks - Actually, if you don't want to maintain a test suite, don't write one.

At its core, your tests are a specification of how your system should work at a given time. When they start breaking it means your system has likely changed, so either it should be fixed to match the specification, or the specification should be changed to match the appropriate system behavior.

At a fundamental level, tests aren't something you write once, they are something you constantly rewrite to keep them in line with what your system should be doing. Thus, when they are broken, you know that either your specs need to change or something in the code isn't working right.

Test maintenance is a price you pay for wanting higher quality software. It's the same kind of cost you pay in designing a building with blueprints. If people build a building without following the blueprints, it's not going to be the building that was supposed to be built. Tests are a living blueprint of your system. If you aren't willing to treat it that way, TDD isn't going to help you.

TDD isn't for everyone because not everyone is willing to make those kind of investments in code quality and maintainability. Also, TDD doesn't make your code better if you aren't willing to let it make your code better.

Re: Reasons to Avoid Test Driven Development

#25

This article is just irritating sarcasm. There are perfectly good reasons to avoid test-driven development. Like being part of an existing large project, where there are already too many interconnected components. Or working in complex AI algorithms, where separate testing of components is extremely painful, and lots of whole system testing is necessary anyway.

Also, code that controls hardware is very hard to test.

Consider writing tests for the the Linux kernel's device drivers. It would be very difficult because most of the problems have to do with hardware being buggy/underspecified, not with the software being wrong in a testable way.

In order to test a UART driver you have to to simulate the way the UART worked, which means that you wouldn't catch bugs if your understanding of the hardware was wrong. Most of the time spent would be recreating the internal logic of the UART and it wouldn't actually benefit you unless you got it right.

The other option to test the UART is to have a rig set up that where you can run the code on the device and have the serial port looped back or go into another computer that checks the result. But that (by definition) means special hardware is required to run that test, which makes it way less useful as an automated test.

Neither of those situations is amenable to TDD.

Re: Reasons to Avoid Test Driven Development

#26

TDD is awesome, if you have a codebase or a tech stack that makes it nice. TDD will quickly expose bad design through an unpleasant testing process. For example, slow tests, untestable code, or tests that you "always have to rewrite/maintain" are problems that only show up when you try and TDD your code that isn't designed to be testable. TDD doesn't work well if you don't design your code to be testable because you…

> TDD isn't for everyone because not everyone is willing to make those kind of investments in code quality and maintainability.

One thing I've noticed is that code written to be testable tends to suffer from over-abstraction. Sadly, writing nice, compact, easy to read code is somewhat at odds with TDD in my experience.

I personally don't like that I have to make sacrifices to code simplicity just to make it testable. But perhaps I just suck at TDD. :-)

Re: Reasons to Avoid Test Driven Development

#27

TDD is awesome, if you have a codebase or a tech stack that makes it nice. TDD will quickly expose bad design through an unpleasant testing process. For example, slow tests, untestable code, or tests that you "always have to rewrite/maintain" are problems that only show up when you try and TDD your code that isn't designed to be testable. TDD doesn't work well if you don't design your code to be testable because you…

> TDD isn't for everyone because not everyone is willing to make those kind of investments in code quality and maintainability. One thing I've noticed is that code written to be testable tends to suffer from over-abstraction. Sadly, writing nice, compact, easy to read code is somewhat at odds with TDD in my experience. I personally don't like that I have to make sacrifices to code simplicity just to make it testable.…

I totally understand what you mean.

I've gone to great lengths to determine what kind of architecture is needed to truly support TDD and what I've come to realize is that it's not so much about over-abstraction that is the problem, it's that testable code is pluggable, which is mostly at odds with how systems are designed.

For example, standard MVC apps tend to tie models to an ORM in some way shape or form. Think Rails and ActiveRecord and you'll understand what I mean. This makes writing a simple crud app fast, but testing it sucks because you have 2 things tied together - your actual data models and your database layer. Funny that nobody would ever do that with the filesystem, but it's super common with the database.

To make your models clean and easy to test, you need to pull out the ORM bit into something more pluggable.

The same principle applies to writing testable biz logic. A lot of times people don't know where to put it so it ends up in the View, Controller, or Model. All of which are the wrong place usually. Testing that code becomes not awesome because you then have to write a test against what the view outputs, the controller does, or possibly what the model is doing (usually by touching the database).

The answer I've found is to pull your business logic into a separate container that can be tested easily apart from any MVC structure.

None of this means you need to over-abstract anything. It just means you have to figure out the right abstractions and structure for the right layers. None of this is by any means obvious at first.

P.S. I'm working on an architecture framework that simplifies a lot of this to make doing TDD much easier.

Re: Reasons to Avoid Test Driven Development

#28
post #23

These are reasons for testing, not TDD. There is no discernable difference between 2 shipped products with identical test coverage where one team used TDD and the other just wrote regression test. The worst part about TDD is the people trying to shove it down your throat.

I think you've identified one of the weakest parts of the essay. This is about ignoring testing, not TDD.

One way I've convinced people to write unit tests is to remind them that they are already testing. People often write a little main to check the output of a program. This is a test. People often click on a link to validate that some text is present. This is a test. Keep them! Instead of throwing away your stand alone program, put it in the unit test folder. Instead of throwing away that point and click, automate it and put it into the integration tests folder. This alone will get you close to 100% coverage. Now, when you refactor your code and it no longer works the way you verified it would a while ago, you'll get a little red bar or a "FAIL" message. Usually this is because you wanted the behavior to change, so update the test. Occasionally, this is because you introduced unwanted side effects in your code, which is exactly why you want these tests.

TDD is substantially different from what I just described. If the author wants to defend it, that's fine, but let's make sure we understand it's not the only way to get high levels of code coverage.

Re: Reasons to Avoid Test Driven Development

#30
I am a big fan of TDD. I read this post to see if I can get any reasonable arguments against TDD, so I can be better prepared next time I am trying to sell TDD. #10 - 'no clients' and #8 - 'short project' kinda make sense to not practice TDD, but these also mean that there is no real product - you build something and you throw it away after a few days. So there is no need for TDD, testing or anything really. All the other points in the post don't make any sense :)... is it just me?
Post reply on HN