Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Giving up on test-first development

iansommerville.com

Post reply on HN
Print view View original

51–60 of 224 posts

Re: Giving up on test-first development

#51

Post by crdoconnor »

BonsaiDen wrote: post #20
ekidd wrote: post #5

From the article: Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests. Interesting. I've often found that the lack of tests leaves me absolutely terri…

One good way to the limit breakage in such cases is to solely perform black box tests on the API level. In case of our Node.js based Backends we don't ever write a single classical unit test, instead we have a custom Framework built on top of Mocha which performs tests on the HTTP layer against all of our endpoints. This works remarkable well in practice and allows for large scale refactorings under the hood with lit…

That's the main reason I wrote this framework (mainly focused on Django for now, but able to much more than that):

http://hitchtest.com

I took this approach on several different projects, but I figured that a lot of the boilerplate/infrastructural code that you need to actually write these kinds of tests is poor or simply not available.

For example, declaratively starting and running multiple services together, in parallel and at the right time (with service dependencies) and printing their logs out.

Or, mocking the forward passage of time. (click something -> move forward a week -> click something else).

Or, asynchronously 'listening' using epoll for emails on a mock SMTP server that logs out the emails it receives.

Selenium's good for the web interaction stuff, but you need much much more than that to be able to effectively test at this level.

I think this kind of testing would be much more widely used and effective if the tools available were up to scratch.

Re: Giving up on test-first development

#52

Post by BonsaiDen »

cousin_it wrote: post #37
BonsaiDen wrote: post #20

Earlier quoted context omitted.

One good way to the limit breakage in such cases is to solely perform black box tests on the API level. In case of our Node.js based Backends we don't ever write a single classical unit test, instead we have a custom Framework built on top of Mocha which performs tests on the HTTP layer against all of our endpoints. This works remarkable well in practice and allows for large scale refactorings under the hood with lit…

Yeah, I mostly prefer end-to-end tests as well. Though to be fair, they are often slower than unit tests, because you need to start up the whole system. And they are worse at pinpointing problems, though that doesn't seem to be a big deal in practice.

I in part agree with the pinpointing, though in my experience this really boils down to an issue of scope and how much of the data you want to test in each of your tests. E.g. an API returning user data, do you have one test for the whole set of data or one test per field.

For our use case we have some pretty "fancy" deep-equals logic for nested structures and allow to specific fields as "to be ignored" in our test expectations.

When it comes to speed, the most important part is to cut out everything you don't need, for us this means that our testing framework completely throws away the internals of Node.js HTTP layer. We never create a single TCP socket in our tests, which gives an incredible speed up. As a bonus this also allows to test timeouts and low level HTTP errors in > Running "noir:mock" (noir) task > ...................................................... > 1048 passing (10s)

And yes, that really reads 10 seconds and yes I always get a bit bored when I have to run the tests for one of our Django backends, feels like an eternity... :)

Re: Giving up on test-first development

#53

Post by crdoconnor »

barrkel wrote: post #35

Test first makes loads of sense for: * fixing bugs - reproducing the bug in a test case both confirms you're fixing the thing, and acts as a regression test * defining a protocol - where you need to glue two things together, e.g. a front end UI and a back end controller, or a model shared between different modules It's a lot weaker for design. Test-first tends to encourage overly open to extension abstractions, becau…

It's still good, you just need to do it at a higher level.

Re: Giving up on test-first development

#54

Post by mbrock »

crdoconnor wrote: post #39
mbrock wrote: post #14

Earlier quoted context omitted.

If you don't know what the code is supposed to do, how are you writing it?

Some (quite a lot, actually) pieces of code are essentially experimental - e.g. trying out an API/libary and seeing what it's capable of or trying an approach to solving a particular kind of problem or even trying to see if a particular problem is solvable with a piece of code. For this kind of coding, TDD makes no sense whatsoever. The 'specs' are as fluid as the code and having confidence in the code isn't that imp…

I don't agree with "no sense whatsoever." Actually, TDD can be a very pleasant way to do this kind of exploratory programming, precisely because it's oriented around verifying expectations.

Re: Giving up on test-first development

#55

Post by atemerev »

tragic wrote: post #45
atemerev wrote: post #32

TDD only works if the tests are written correctly. Tests are not about "code coverage", nor about establishing the exact sequence of things in stone. Tests are about fixing invariants. When a new project starts, I only know about 10-15% things for sure, and those are exactly which will go in tests, before writing any new code. I don't worry about some things in my code are not yet covered by tests; I don't know yet h…

> some crazy folks are even testing getters and setters — why? Well, I can see the logic if your getters and setters are hiding more activity than simply retrieving/setting the value of a private field, which is the point of having separate getters/setters at all. If you imagine a getFullName() / setFullName(name) pair, for example, that actually reads from/writes to two different private fields for first and last na…

In such case, a correct test should focus on functional invariants, like getFullName() + " " + getLastName() equals to getName() (btw, never do that — names are much more complicated than that. In some countries, there are no first and last names at all; others use multiple name designations; some have meaningful patronyms etc.)

Re: Giving up on test-first development

#56

Post by atemerev »

sago wrote: post #46
atemerev wrote: post #34

Earlier quoted context omitted.

> tests that are essentially testing implementation decisions. There shouldn't be any. Those are not useful at all.

See my reply above. I think your view is either naive, or you understand 'implementation decision' in too narrow a way.

I understand "implementation decisions" in quite well-defined way; only functional invariants should be tested. Your example is a perfect illustration for one.

Additional point is that invariant-based testing can easily be transformed into automated fuzzy tests using Quickcheck and its analogues, and this can find immense amount of otherwise hidden bugs.

Re: Giving up on test-first development

#57

Post by mbrock »

sago wrote: post #27
ncphillips wrote: post #22

Earlier quoted context omitted.

> Because you generated the code being motivated by the tests, there are a lot of unit tests that dot test functional units - tests that are essentially testing implementation decisions. I've seen this happen to myself numerous times as well, but I don't think this is good argument against TDD. If a tests fails because the implementation details change, that tells us that we wrote a bad test, not that TDD is bad. As…

> that tells us that we wrote a bad test, not that TDD is bad I strongly disagree. The only thing that matters to how software works for an end user is its boundary. You simply can't do TDD by only testing the boundary. You have to write a test before each unit of code, so you have to test those internal interfaces, the way units of code work. But that is exactly the stuff that should be allowed to change. It's a mis…

Changing tests is work, yes. Unit tests are precisely about verifying implementations, and maintaining them is the price you pay for having these layers of verification. It's inertia in the way a safety net is a barrier.

Re: Giving up on test-first development

#58

Post by mbrock »

When writing tests is boring, difficult, and tedious, that's a really good time to think hard about the way the program is structured, if you have time for this.

The way to make testing pleasant is to extract more and more behavior into units with clear boundaries... realizing how to do this was a major event in my programming career, and I attribute the insight partly to doing TDD.

I don't agree with some posters who say that TDD encourages overly generalized design. Sure, it encourages some form of dependency injection... but mostly, it just encourages the creation of coherent and loosely coupled units, which is a universally lauded best practice.

Re: Giving up on test-first development

#59

Post by Ensorceled »

organsnyder wrote: post #19
Illniyar wrote: post #10

Earlier quoted context omitted.

To be fair the author was abandoning TDD not tests. He explictly said that he'll add tests after he writes the code.

Perhaps the author is more disciplined than me, but I'd have an extremely hard time actually following through and writing tests after-the-fact. After the code is "done", it's hard to keep working on it—no matter how much the "extra" work is needed. I find tests to be extremely helpful for designing APIs. I write tests to enforce the contract my API is making with its consumers. Therefore, TDD forces me to consider t…

You just need an accountability buddy. I do code reviews for my team and reject any pull request with insufficient test coverage.

Re: Giving up on test-first development

#60

Post by Ensorceled »

ekidd wrote: post #5

From the article: Because you want to ensure that you always pass the majority of tests, you tend to think about this when you change and extend the program. You therefore are more reluctant to make large-scale changes that will lead to the failure of lots of tests. Psychologically, you become conservative to avoid breaking lots of tests. Interesting. I've often found that the lack of tests leaves me absolutely terri…

That's different, I'm against TDD for the same reason the author is. I'm FOR automated testing for exactly the same reasons you are.

Think of it this way: pre-testing makes you afraid to change you mind because you throw out your tests, not post-testing makes you afraid to change your mind because you might break something. If you think changing your mind is good the path forward is clear.

Post reply on HN