Live data from Hacker News

Unit Tests Considered Harmful

shaiyallin.com

41–50 of 76 posts

Re: Unit Tests Considered Harmful

#41
You unit test the pieces, then integrate and system+regression test the product.

Its got nothing to do with how you define unit - just a way to:

1) speed up overall testing and bug fixing

2) get better test coverage of units

3) stop wasting other team members time by testing your unit before saying it's ready for system test

Re: Unit Tests Considered Harmful

#42
> This approach has seen a lot of traction during the past 15 years. [...] And it’s complete and utter rubbish.

I'm a bit tired of hearing this about literally every aspect or approach in programming.

There is generally an appropriate application for almost any tool, design pattern, etc. I see little to no value in making sweeping statements about how "rubbish" something is, especially something like unit testing (regardless of how most would define it).

Re: Unit Tests Considered Harmful

#44
post #39

Earlier quoted context omitted.

Studies on TDD has failed to show a benefit over writing tests after the fact. The only factor that seems to matter is writing tests , the more tests the better. If you can write your code so it's amenable to more testing, or such that it requires fewer tests because it's simpler/has fewer cases to handle, great. If you have a framework that can generate tests (fuzzing, property-based testing), awesome.

I'm pretty convinced that test-first is best seen as a negotiating technique to get the time to write the tests in the first place. "Okay, the code is done, time to write the tests" results in pressure to shorten testing, since if the golden path works then surely everything else works, right? Your management and your sales people then say "You're a great developer, it looks like it works, and we need to deploy that…

I can see that, but only because the dev has arguably already made the mistake of showing sales working features before tests are written. I've always been clear to say when something is a partial mockup to verify that I've understood what they're asking for.

Re: Unit Tests Considered Harmful

#45
Tests - whatever you wish to call them - are an assertion that something will always and forever be true. A unit thus needs to be a unit of code that you are telling all future maintainers that they cannot refactor such that this changes.

If you are writing the list data structure for a new language (if you like LISP you are looking for a better name than car/cdr, while C++ guys are thinking vector without the template mess, and ... all are correct for this discussion) - this will quickly be used by everyone and so it doesn't matter if you can't change the API as this data structure will soon be used everywhere in your code and so a change isn't possible. However most classes/functions are only used in a couple places that they are close to and so it isn't a big deal to change them - thus you shouldn't write tests that force the API.

When you have a large system eventually you need to break it up into things that are smaller just to understand. Those are good places to write tests and mocks as you get flexibility internally to do what you want and you don't really understand what is going on outside. However someone needs to write whole system integration tests as nobody understands what happens between the smaller parts.

Re: Unit Tests Considered Harmful

#46
Yes, this article is correct. I have also been saying this in comments on this site. Automated tests should be on the right level to provide value. On a level that is too low, one ends up with tests that test implementation details or the highly interesting feature that the list type in the standard library of your favorite programming language is still capable of adding elements to itself. On a level that is too high one ends up with fragile tests where there is a lot of state to initialize before anything can be tested. Tests can occur on any level but they should be on a level that makes them interesting, which generally means that they test a property of the system that the customer could recognize as something they value. Most tests should maybe test how between one and five classes work together.

Re: Unit Tests Considered Harmful

#48
post #28

Earlier quoted context omitted.

Agree. The "what about the client" trope is a complete non-argument. If unit tests make code better, the client maybe wants that. The argument should purely be based on if they are helpful.

I'll take it a step further than this. It doesn't matter if the client wants unit tests or not. If the developer has been hired to do the job, it's going to be up to that developer whether or not they feel unit tests will make the code better. If they do, they should do them. If they don't, they shouldn't. Caring about the client's opinions re: unit tests is a little like caring about their opinion re: the interior c…

If it is in the contract and specs (either way) then you do what you are paid to do.

Re: Unit Tests Considered Harmful

#49
post #2

The target audience for unit tests is not the client, it is the developer. Unit tests allow you to change code with more confidence. 100% code coverage is not a useful aim, you should aim for 100% confidence in your code. Unit tests can also function as example code, that can't get out of date, since then the tests will fail. Testing for quality assurance is a different thing, usually called acceptance testing and so…

How do you measure 100% confidence? Pretty sure I have never reached 100% confidence, but I haven't had a metric to track it so I have no data to prove it. The only possible way to reach near 100% is I think by not writing any code, and by not building anything. I do think 100% code coverage goal doesn't mean much, since you can potentially achieve it using 0 assertions. Or you make 1 assertion per test, leaving 95%+…

Confidence can be achieved using https://en.wikipedia.org/wiki/Mutation_testing. this involves running an engine that mutates your code and check if you have at least one assertion failing.

Re: Unit Tests Considered Harmful

#50
post #17

Everyone talking about unit tests, and testing in general as a thing to achieve are missing it entirely. Tests aren't a thing to achieve, and talking about whether unit testing, mocks etc. are better or worse without context is pointless. The thing to achieve is having a code base that 1) works 2) can be changed easily and proved to be still working. Now that we know that we can start deciding "how" we can achieve th…

Very good advice. Additionally, I'd add:

- What sort of testing can be achieved given limited time and resources?

...because nearly all organizations, no matter how quality-oriented they claim to be, will prioritize non-test code delivery over test code, leaving scarce time for test creation before shipment deadlines.

Post reply on HN