Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

21–30 of 348 posts

Re: Multiple assertions are fine in a unit test

#21

Post by philliphaydon »

mollerhoj wrote: post #17
choeger wrote: post #3

What? People really would criticize that code because it has two assertions? How are they ever testing any state changes? And to the author: Your bubble is significantly different from mine. Pretty much every competent developer I've worked with would laugh at you for the idea that the second test case would not be perfectly fine. (But that first iteration would never pass code review either because it does nothing a…

To answer your question: We zealots test for the fact that something changes to some degree. E.g with rubys rspec library: expect { foo.call() }.to change { bar.value }.by(2) That is, regardless of the absolute value of bar.value, I expect foo.call() to increment it by 2. The point of the 1 assertion per test guideline is to end up with tests that are more focused. Giving that you did not seem to think of the above t…

That’s because the example test only requires 1 assertion.

Any rule that says there should be only 1 assertion ever is stupid.

Re: Multiple assertions are fine in a unit test

#22

Post by TrianguloY »

The one assertion per test doesn't mean you need to use only one assertion call but rather that you only need to do one assertion block. Checking everything after a response is considered 1 assertion, no matter how many assert calls you need.

The issue is when you use multiple assertions for multiple logic statements: do > assert > do > assert... In that example imagine that you were also checking that the reservation was successful. That would be considered bad, you should create a different test that checks for that (testCreate + testDelete) and just have the precondition that the delete test has a valid thing to delete (usually added to the database on the setup).

Re: Multiple assertions are fine in a unit test

#23

Post by TeMPOraL »

_lambdalambda wrote: post #6
wccrawford wrote: post #2

First off, I do put more than 1 assertion in a test. But it definitely leads to situations where you have to investigate why a test failed, instead of it just being obvious. Like the article, I test 1 thing per test, but sometimes that means multiple assertions about the outcome of a test. IMO there's no point in checking that you got a response in 1 test, and then checking the content/result of that response in anot…

Even with multiple assertions the test failure reason should be quite clear as most testing frameworks allow to specify a message which is then output in the testing summary. E.g. `assertEqual(actual_return_code, 200, "bad status code")` should lead to output like `FAILED: test_when_delete_user_then_ok (bad status code, expected 200 got 404)`

Even if you don't specify the message, at the very minimum, the output should look like:

  FAILED: test_when_delete_user_then_ok
    Assertion failed: `actual_return_code' expected `200', got `400'.
Note it mentions the actual expression put in the assert. Which makes it almost always uniquely identifiable within the test.

That's the bare minimum I'd expect of a testing framework - if it can't do that, then what's the point of having it? It's probably better to just write your own executable and throw exceptions in conditionals.

What I expect from a testing framework is at least this:

  FAILED: test_when_delete_user_then_ok
    Assertion failed: `actual_return_code' expected `200', got `400'.
    In file: '/src/blah/bleh/blop/RestApiTests.cs:212'.
I.e. to also identify the file and the line containing the failing assertion.

If your testing framework doesn't do that, then again, what's even the point of using it? Throwing an exception or calling language's built-in assert() on a conditional will likely provide at least the file+line.

Re: Multiple assertions are fine in a unit test

#24

Post by mollerhoj »

jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

I believe theres a rubocop that checks for it:

https://docs.rubocop.org/rubocop-minitest/cops_minitest.html...

Re: Multiple assertions are fine in a unit test

#25

Post by pydry »

choeger wrote: post #3

What? People really would criticize that code because it has two assertions? How are they ever testing any state changes? And to the author: Your bubble is significantly different from mine. Pretty much every competent developer I've worked with would laugh at you for the idea that the second test case would not be perfectly fine. (But that first iteration would never pass code review either because it does nothing a…

There's a lot of not very competent people in the industry who cling tightly to dogma.

Testing (especially unit) is an area of tech weirdly with a lot of dogmatism. I think Uncle Bob is the source of some of it.

Re: Multiple assertions are fine in a unit test

#26

Post by philliphaydon »

wccrawford wrote: post #2

First off, I do put more than 1 assertion in a test. But it definitely leads to situations where you have to investigate why a test failed, instead of it just being obvious. Like the article, I test 1 thing per test, but sometimes that means multiple assertions about the outcome of a test. IMO there's no point in checking that you got a response in 1 test, and then checking the content/result of that response in anot…

Maybe it’s different in other languages but in JS and .NET the failed assertion fails and you investigate the failed assertion. You wouldn’t ever have a situation that isn’t obvious.

If an assertion says “expected count to be 5 but got 4” you wouldn’t be looking at the not null check assertion getting confused why it’s not null…

Re: Multiple assertions are fine in a unit test

#27

Post by danuker »

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I use the following pattern for testing regexes:

    expected_positive = [
        'abc',
        'def', 
        ...]
    for text in expected_positive:
        self.assertTrue(matcher(text), f"Failed: {text}")
Before I added the assertion error message, `f"Failed: {text}"`, it was quite difficult to tell WHICH example failed.

Re: Multiple assertions are fine in a unit test

#28

Post by mollerhoj »

philliphaydon wrote: post #21
mollerhoj wrote: post #17

Earlier quoted context omitted.

To answer your question: We zealots test for the fact that something changes to some degree. E.g with rubys rspec library: expect { foo.call() }.to change { bar.value }.by(2) That is, regardless of the absolute value of bar.value, I expect foo.call() to increment it by 2. The point of the 1 assertion per test guideline is to end up with tests that are more focused. Giving that you did not seem to think of the above t…

That’s because the example test only requires 1 assertion. Any rule that says there should be only 1 assertion ever is stupid.

OP asked how any state change would be tested with a single 'assertion' and I provided an answer. Absolute rules are stupid, but our codebase has just short of 10k tests, and very few have more than one assertion.

The only reason I can really see to have more than one assertion would be to avoid having to run the setup/teardown multiple times. However, its usually a desirable goal to write code that require little setup/teardown to test anyways because that comes with other benefits. Again, it might not be practical or even possible, but that goes of almost all programming "rules"..

Re: Multiple assertions are fine in a unit test

#29

Post by asabla »

latch wrote: post #8

I haven't heard of the single-assertion thing in at least 10 years, probably 15. In the early 2000s, when I was starting out and doing .NET, it used to be something you'd hear in the community as a very general guideline, more like "there's something to be said about very focused tests, and too many assertions might be a smell." At the time, I got the impression that the practice had come over from Java and converted…

> I wrote Foundations of Programming for any 2000s .NET developer out there!

Holy moly! Think I still have your book somewhere. So thank you for that.

In my last +10 years of .net development I haven't heard anything about single-assertion.

> Some people are really stuck in the same year of their 10 (or 20, or 30) years of experience.

I think this has manifested even more with the transition into .net core and now .net 5 and beyond. There are so many things changing all the time (not that I complain), which can make it difficult to pick up what's the current mantra for the language and framework.

Re: Multiple assertions are fine in a unit test

#30

Post by profunctor »

danuker wrote: post #27
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I use the following pattern for testing regexes: expected_positive = [ 'abc', 'def', ...] for text in expected_positive: self.assertTrue(matcher(text), f"Failed: {text}") Before I added the assertion error message, `f"Failed: {text}"`, it was quite difficult to tell WHICH example failed.

If you’re using pytest you just paramaterize the tests and it tells you the exact failing case. Seems to be a basic feature I would be surprised to know doesn’t exist across almost all commonly used frameworks.
Post reply on HN