Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

61–70 of 348 posts

Re: Multiple assertions are fine in a unit test

#61

Post by magic_hamster »

ramraj07 wrote: post #58
magic_hamster wrote: post #51

Earlier quoted context omitted.

> How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". I envy you for never having seen tests atrocious enough where this is not only possible, but the common case. Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it. Now imagine…

So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

Not at all. It makes sense in some tests. I addressed the part asking how it's even possible to not know what happened.

As for multiple asserts, that is really meaningless. The test case should test one thing. If it requires several asserts that's okay. But having a very long test function with a lot of assertions, is strongly indicating that you're testing more than one thing, and when the test fails it will be harder to know what actually happened.

Re: Multiple assertions are fine in a unit test

#62

Post by [deleted] »

littlecranky67 wrote: post #9
jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

[deleted]

Re: Multiple assertions are fine in a unit test

#63

Post by [deleted] »

ramraj07 wrote: post #58
magic_hamster wrote: post #51

Earlier quoted context omitted.

> How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". I envy you for never having seen tests atrocious enough where this is not only possible, but the common case. Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it. Now imagine…

So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

[deleted]

Re: Multiple assertions are fine in a unit test

#64

Post by MikeDelta »

teddyh wrote: post #18
littlecranky67 wrote: post #9

Earlier quoted context omitted.

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

Where in that book is the rule stated? I ask because I have heard the author explicitly state that multiple assertions are fine (using essentially the same explanation as TrianguloY did in this comment: https://news.ycombinator.com/item?id=33480120 ).

Chapter 9 talks about unit tests and there is a paragraph called 'Single Assert per Test', where he says it is a good concept but that he is not afraid to put more asserts in his tests.

That paragraph is followed by 'Single Concept per Test' where he starts with: 'Perhaps a better rule is that we want to test a single concept per test.'

So, technically he doesn't say it.

Re: Multiple assertions are fine in a unit test

#65

Post by ChrisMarshallNY »

I always have multiple assertions in my unit tests. I test around areas of functionality, as opposed to individual functions. It's a bit arbitrary, but there you have it...

I also use Apple's XCTest, which does a lot more than simple assertions.

If an assertion is thrown, I seldom take the assertion's word for it. I debug trace, and figure out what happened. The assertion is just a flag, to tell me where to look.

Re: Multiple assertions are fine in a unit test

#66

Post by lancebeet »

dd82 wrote: post #49
lancebeet wrote: post #31

Earlier quoted context omitted.

That looks like a case where I would use a parameterized test rather than a for loop inside the test.

this has downsides if you're comparing attributes with a method result and checking whether said attrs match what you expect. Either you run each test N times for N attr comparisons, accepting the cost of setup/teardown, or do a loop and fire off an assert error with text on which comparison failed. Since you already have the object right there, why not do the latter approach?

If the setup/teardown is expensive I would do it in reusable fixtures. The reason I wouldn't choose the latter approach is that it would usually be less convenient in the long run. You'd need to replace your asserts with expects to avoid it throwing on the first error (if this isn't what you want), you'll often need to manually add data to the assertion (as GP did) that you would otherwise get for free, and you'll need to look at the assertion error rather than the test case to know what actually failed. This can be quite inconvenient if you e.g. export your test results in a CI/CD pipeline.

Re: Multiple assertions are fine in a unit test

#67

Post by philliphaydon »

mollerhoj wrote: post #28
philliphaydon wrote: post #21

Earlier quoted context omitted.

That’s because the example test only requires 1 assertion. Any rule that says there should be only 1 assertion ever is stupid.

OP asked how any state change would be tested with a single 'assertion' and I provided an answer. Absolute rules are stupid, but our codebase has just short of 10k tests, and very few have more than one assertion. The only reason I can really see to have more than one assertion would be to avoid having to run the setup/teardown multiple times. However, its usually a desirable goal to write code that require little se…

If there’s 2 tests which are identical but assert 2 different things. It should be a single test with 2 assertions.

Can always refactor to 2 tests if the test setup changes and the assertions begin to differ or become too complex.

Re: Multiple assertions are fine in a unit test

#68

Post by skatanski »

This always rubbed me the wrong way. I think a better way would be to ensure the assertions to be readable, as simple as can be. The worst case of this being broken was the assertions were done in an helper method used in a base class of a test, and navigating to it required multiple hops, it took time to build the context of the test as well. The only downside of multiple assertions is that when the first one fails we won’t know if the subsequent ones are passing.

Re: Multiple assertions are fine in a unit test

#69

Post by TeMPOraL »

jmillikin wrote: post #19
devit wrote: post #13

The problem seems to be that assert is implemented as a regular function. It must be implemented as a macro so that the line number and assertion expressions are printed, allowing to easily identify the failed assertion. If a language doesn't support such macros and has no ad-hoc mechanism for this case, it should not be used, or if it must the assert function must take a string parameter identifying the assertion.

Some languages allow a stack trace to be obtained in normal code, which enables position reporting without macros. Python and Go are good examples. If you know the file and line of the assertion, plus the values that are being checked, there's not as much need for a stringified version of the expression.

> If you know the file and line of the assertion, plus the values that are being checked, there's not as much need for a stringified version of the expression.

It does save time. With the actual condition reproduced, half the time I don't even need to check the source of the failed test to know what went bad and where to fix it. Consider the difference between:

  FAILED: Expected 1, got 4
  In /src/foo/ApiTest.cpp:123
vs.

  FAILED: response.status evaluated to 4
          expected: 1
  In /src/foo/ApiTest.cpp:123
vs.

  FAILED: response.status evaluated to Response::invalidArg (4)
          expected: Response::noData (1)
  In /src/foo/ApiTest.cpp:123
This is also why I insist on adding custom matchers and printers in Google Test for C++. Without it, 90% of the time a failed assertion/expectation just prints "binary objects differ" and spews a couple lines of hexadecimal digits. Adding a custom printer or matcher takes little work, but makes all such failures print meaningful information instead, allowing one to just eyeball the problem from test output (useful particularly with CI logs).

Re: Multiple assertions are fine in a unit test

#70

Post by krona »

mollerhoj wrote: post #17
choeger wrote: post #3

What? People really would criticize that code because it has two assertions? How are they ever testing any state changes? And to the author: Your bubble is significantly different from mine. Pretty much every competent developer I've worked with would laugh at you for the idea that the second test case would not be perfectly fine. (But that first iteration would never pass code review either because it does nothing a…

To answer your question: We zealots test for the fact that something changes to some degree. E.g with rubys rspec library: expect { foo.call() }.to change { bar.value }.by(2) That is, regardless of the absolute value of bar.value, I expect foo.call() to increment it by 2. The point of the 1 assertion per test guideline is to end up with tests that are more focused. Giving that you did not seem to think of the above t…

You tested a postcondition. What about preconditions and invariants, do you have separate tests for those assertions too, or just not bother?
Post reply on HN