Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

11–20 of 348 posts

Re: Multiple assertions are fine in a unit test

#11

Post by TeMPOraL »

littlecranky67 wrote: post #9
jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

Perhaps the author is better off not having heard of it then, and by implication, not having read "Clean Code" in the first place. The book is full of anti-patterns.

Re: Multiple assertions are fine in a unit test

#12

Post by tetha »

wccrawford wrote: post #2

First off, I do put more than 1 assertion in a test. But it definitely leads to situations where you have to investigate why a test failed, instead of it just being obvious. Like the article, I test 1 thing per test, but sometimes that means multiple assertions about the outcome of a test. IMO there's no point in checking that you got a response in 1 test, and then checking the content/result of that response in anot…

IMO, the opposite also has to be considered. I've briefly worked with some code bases that absolutely did 1 assert per test. Essentially you'd have a helper method like "doCreateFooWithoutBarAttribute", and 3-4 tests around that - "check that response code is 400", "check that error message exists", and so on. Changes easily caused 4-5 tests to fail all at once, for example because the POST now returned a 404, but the 404 response also doesn't contain the error message and so on.

This also wasted time, because you always had to look at the tests, and eventually realized that they all failed from the same root cause. And sure, you can use test dependencies if your framework has that and do all manner of things... or you just put the asserts in the same test with a good message.

Re: Multiple assertions are fine in a unit test

#13

Post by devit »

The problem seems to be that assert is implemented as a regular function.

It must be implemented as a macro so that the line number and assertion expressions are printed, allowing to easily identify the failed assertion.

If a language doesn't support such macros and has no ad-hoc mechanism for this case, it should not be used, or if it must the assert function must take a string parameter identifying the assertion.

Re: Multiple assertions are fine in a unit test

#14

Post by Kwpolska »

littlecranky67 wrote: post #9
jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

The book is so full of bad advice I'm not surprised this "rule" comes from there as well.

Re: Multiple assertions are fine in a unit test

#15

Post by jmillikin »

littlecranky67 wrote: post #9
jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

Heard of it, but never read it.

Looking at the Amazon listing and a third-party summary[0] it seems to be the sort of wool-brained code astrology that was popular twenty years ago when people were trying to push "extreme programming" and TDD.

[0] https://gist.github.com/wojteklu/73c6914cc446146b8b533c0988c...

Re: Multiple assertions are fine in a unit test

#16

Post by TeMPOraL »

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure.

How is that even possible in the first place?

The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providing at minimum the expression put in the assertion, verbatim, and precise coordinates of the assertion - i.e. name of the source file + line number.

I've never seen a case where it would be hard to tell which assertion failed. On the contrary, the most common problem I see is knowing which assertion failed, but not how the code got there, because someone helpfully stuffed it into a helper function that gets called by other helper functions in the test suite, and the testing framework doesn't report the call stack. But it's not that big of a deal anyway; the main problem I have with it is that I can't gleam the exact source of failure from CI logs, and have to run the thing myself.

Re: Multiple assertions are fine in a unit test

#17

Post by mollerhoj »

choeger wrote: post #3

What? People really would criticize that code because it has two assertions? How are they ever testing any state changes? And to the author: Your bubble is significantly different from mine. Pretty much every competent developer I've worked with would laugh at you for the idea that the second test case would not be perfectly fine. (But that first iteration would never pass code review either because it does nothing a…

To answer your question: We zealots test for the fact that something changes to some degree. E.g with rubys rspec library:

expect { foo.call() }.to change { bar.value }.by(2)

That is, regardless of the absolute value of bar.value, I expect foo.call() to increment it by 2.

The point of the 1 assertion per test guideline is to end up with tests that are more focused. Giving that you did not seem to think of the above technique, I'd say that this guideline might just have helped you discover a way to write better specs ;-)

Guidelines (that is, not rules) are of course allowed to be broken if you have a good reason to do so. But not knowing about common idioms is not a good reason.

You might argue that the above code is just sugar for 2 assertions, but thats beside the point: The test is more focused, there -appears- to be only one assertion, and thats what matters.

Re: Multiple assertions are fine in a unit test

#18

Post by teddyh »

littlecranky67 wrote: post #9
jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

Where in that book is the rule stated? I ask because I have heard the author explicitly state that multiple assertions are fine (using essentially the same explanation as TrianguloY did in this comment: https://news.ycombinator.com/item?id=33480120).

Re: Multiple assertions are fine in a unit test

#19

Post by jmillikin »

devit wrote: post #13

The problem seems to be that assert is implemented as a regular function. It must be implemented as a macro so that the line number and assertion expressions are printed, allowing to easily identify the failed assertion. If a language doesn't support such macros and has no ad-hoc mechanism for this case, it should not be used, or if it must the assert function must take a string parameter identifying the assertion.

Some languages allow a stack trace to be obtained in normal code, which enables position reporting without macros. Python and Go are good examples.

If you know the file and line of the assertion, plus the values that are being checked, there's not as much need for a stringified version of the expression.

Post reply on HN