Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Giving up on test-first development

iansommerville.com

Post reply on HN
Print view View original

31–40 of 224 posts

Re: Giving up on test-first development

#31

Post by ebbv »

> Sometimes, the best design is one that’s hard to test

I strongly disagree with this statement. The best design for your program is always the one that's easiest to test; the one that's modular, the one that separates out dependencies, the one where methods are as atomic as possible.

If you find yourself wanting to write code that is hard to test, then you are approaching the problem (and/or the solution) the wrong way.

> The ‘purist’ approach here, of course, is that you design data validation checks so that you never have to process bad data. But the reality is that it’s often hard to specify what ‘correct data’ means

Again I disagree. It should be really clear what correct data means at a low level. If you can't, then you haven't fully fleshed out your design, so yeah it's gonna be impossible for you to test it.

I'm far from a TDD zealot. But on my team we adopted writing unit tests for everything in the last 6 months and it has been night and day. It is incredibly useful and the resulting code is so much better.

I think a lot of the author's problems stem from attempting to do this alone and not having someone else to guide him through how to tackle things that he's having trouble with. Plus, frankly, he seems to just have a defeatist attitude about the whole thing.

Re: Giving up on test-first development

#32

Post by atemerev »

TDD only works if the tests are written correctly.

Tests are not about "code coverage", nor about establishing the exact sequence of things in stone. Tests are about fixing invariants.

When a new project starts, I only know about 10-15% things for sure, and those are exactly which will go in tests, before writing any new code. I don't worry about some things in my code are not yet covered by tests; I don't know yet how they will turn out, so I can't write any meaningful invariants.

In my experience, useful tests are much higher-level than TDD guys prefer. They routinely fix invariants for the entire system / subproject, not assuring coverage of every method in class (some crazy folks are even testing getters and setters — why?)

Re: Giving up on test-first development

#33

Post by crdoconnor »

sago wrote: post #11
ekidd wrote: post #5

From the article: Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests. Interesting. I've often found that the lack of tests leaves me absolutely terri…

It's not a good argument against testing, I agree. Automated testing is a huge boon for software quality (though by no means a panacea). But TDD, in my experience, gives a slightly different dynamic. Because you generated the code being motivated by the tests, there are a lot of unit tests that don't test functional units - tests that are essentially testing implementation decisions. I've seen situations where that b…

>I've seen situations where that big refactoring would involve scrapping or rewriting tests, and that causes its own kind of architectural conservatism.

I call this test concreting:

https://hitchtest.readthedocs.org/en/latest/glossary/test_co...

The way to avoid it is to either be damn sure that the API you've surrounded is tight, clean and unlikely to need to change much or to test at a higher level.

Re: Giving up on test-first development

#34

Post by atemerev »

sago wrote: post #11
ekidd wrote: post #5

From the article: Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests. Interesting. I've often found that the lack of tests leaves me absolutely terri…

It's not a good argument against testing, I agree. Automated testing is a huge boon for software quality (though by no means a panacea). But TDD, in my experience, gives a slightly different dynamic. Because you generated the code being motivated by the tests, there are a lot of unit tests that don't test functional units - tests that are essentially testing implementation decisions. I've seen situations where that b…

> tests that are essentially testing implementation decisions.

There shouldn't be any. Those are not useful at all.

Re: Giving up on test-first development

#35

Post by barrkel »

Test first makes loads of sense for:

* fixing bugs - reproducing the bug in a test case both confirms you're fixing the thing, and acts as a regression test

* defining a protocol - where you need to glue two things together, e.g. a front end UI and a back end controller, or a model shared between different modules

It's a lot weaker for design. Test-first tends to encourage overly open to extension abstractions, because you need to make things visible to tests and make components replaceable by mocks. In the early stages, the weight of updating tests makes the design overly awkward to change. And early on in the process is exactly the wrong time to be creating your general abstractions - that's when you have the least amount of information about what will make for the best abstraction.

You still need to back-fill tests after good abstractions have been chosen, of course. Tests are great; test-first, specifically, isn't always best.

Re: Giving up on test-first development

#36

Post by stinos »

jon-wood wrote: post #15

Over the years I've gone from writing no tests at all, to being a die hard TDD purist, and then out the other side to writing some things with tests first, others with tests after writing the code, and some without any tests at all. In some situations I have clear view of what I need to build, and how that should work. TDD is great in that case - write a test for the expected behaviour, make it pass, refactor, rinse…

Over the years I've gone from writing no tests at all, to being a die hard TDD purist, and then out the other side to writing some things with tests first, others with tests after writing the code, and some without any tests at all.

Hear, hear! Exactly the same here, for the same reasons as you and the OP mention.

And observed fom a distance, it's always the same universal principle: purist behaviour (in the sense of almost religous beliefs that something is a strict rule, sentence starting with Always etc you get the point) in programing or to a further extent, in life, is nearly always wrong, period. No matter what the rule is, you can pretty much always find yourself in a situation where the rule is not the best choice.

Re: Giving up on test-first development

#37

Post by cousin_it »

BonsaiDen wrote: post #20
ekidd wrote: post #5

From the article: Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests. Interesting. I've often found that the lack of tests leaves me absolutely terri…

One good way to the limit breakage in such cases is to solely perform black box tests on the API level. In case of our Node.js based Backends we don't ever write a single classical unit test, instead we have a custom Framework built on top of Mocha which performs tests on the HTTP layer against all of our endpoints. This works remarkable well in practice and allows for large scale refactorings under the hood with lit…

Yeah, I mostly prefer end-to-end tests as well. Though to be fair, they are often slower than unit tests, because you need to start up the whole system. And they are worse at pinpointing problems, though that doesn't seem to be a big deal in practice.

Re: Giving up on test-first development

#38

Post by JulianMorrison »

As a suggestion, QuickCheck type testing frameworks are good for finding bugs relating to unexpected data.

Summary of how they work: you say "this program takes a 32 bit signed int and a string" and the testing framework will throw it a heap of random ints and strings, some of which match the sorts of classic curve balls it might encounter (negative numbers, int max and min values, strings that are very long, empty strings, strings with \0 in them, strings that don't parse as valid unicode, "Robert');DROP TABLE Students; --", and so on.)

Re: Giving up on test-first development

#39

Post by crdoconnor »

mbrock wrote: post #14
nostrademons wrote: post #8

TDD is nuts for code without a client or specification. The whole point of tests is to ensure that the code does what it's supposed to do. When you have neither client nor spec, how are you supposed to know what the code is supposed to do ? There is, IME, a >90% chance that any such code will be ripped out and replaced as you develop a better understanding of the problem domain. I've found it's pretty useful to go ba…

If you don't know what the code is supposed to do, how are you writing it?

Some (quite a lot, actually) pieces of code are essentially experimental - e.g. trying out an API/libary and seeing what it's capable of or trying an approach to solving a particular kind of problem or even trying to see if a particular problem is solvable with a piece of code.

For this kind of coding, TDD makes no sense whatsoever. The 'specs' are as fluid as the code and having confidence in the code isn't that important.

This is entirely different to creating production hardened systems with very clear specs. If you don't do TDD on that, you're an idiot.

Re: Giving up on test-first development

#40

Post by barrkel »

mbrock wrote: post #14
nostrademons wrote: post #8

TDD is nuts for code without a client or specification. The whole point of tests is to ensure that the code does what it's supposed to do. When you have neither client nor spec, how are you supposed to know what the code is supposed to do ? There is, IME, a >90% chance that any such code will be ripped out and replaced as you develop a better understanding of the problem domain. I've found it's pretty useful to go ba…

If you don't know what the code is supposed to do, how are you writing it?

The preference function (or oracle) can return a truth value without you necessarily having insight into its inner workings. People "know it when they see it" without necessarily being able to define it.

So you write the code; expose it to a preference function / oracle, iterate, and hill-climb towards a local optimum. The revealed preferences may give enough information to make a creative leap off the local hill onto a large hill elsewhere - but it may not.

After a solution has been found, then you can write tests.

Post reply on HN