Multiple assertions are fine in a unit test • Hacker News Board

Re: Multiple assertions are fine in a unit test

#171

Post by beezlewax » Sat, Nov 05, 2022, 2:19 PM UTC

torginus wrote: post #89

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I remember writing a small .NET test library for that exact problem - You could pass in a lambda with a complex condition, and it evaluated every piece of the expression separately and pretty printed what part of the condition failed. So essentially you could write Assert(()=>width>0 && x + width And you would get: Assertion failed: x is 1500 width is 600 screenWidth is 1920 It used Expression to do the magic. Amazin…

Is there something like this available for javascript?

Re: Multiple assertions are fine in a unit test

#172

Post by hgomersall » Sat, Nov 05, 2022, 2:22 PM UTC

dicroce wrote: post #113

also, only testing public interfaces is perfectly fine and may actually be preferable as it leaves you free to refactor internals freely without breaking tests. tbh unit testing is a balancing act between reaping code quality benefits and bogging yourself down with too much testing updating.

This is an interesting point IMO. We tend to focus at the API level far more, and implicitly test inner functionality (that is, the inner functionality must be correct for the outer tests to pass). Sometimes testing inner functionality explicitly is required when the outer tests are not complete, or when behaviour is defined by the inner code. We also as far as possible use defensive techniques and extensive type constraints (which is a joy in Rust).

I'm constantly thinking where we need to put tests though, and in still not fully convinced I get it right. My rule of thumb is that each test should map to a specification point, and that spec is a necessary documentation line for the test.

Re: Multiple assertions are fine in a unit test

#173

Post by choeger » Sat, Nov 05, 2022, 2:25 PM UTC

mollerhoj wrote: post #17

choeger wrote: post #3

What? People really would criticize that code because it has two assertions? How are they ever testing any state changes? And to the author: Your bubble is significantly different from mine. Pretty much every competent developer I've worked with would laugh at you for the idea that the second test case would not be perfectly fine. (But that first iteration would never pass code review either because it does nothing a…

To answer your question: We zealots test for the fact that something changes to some degree. E.g with rubys rspec library: expect { foo.call() }.to change { bar.value }.by(2) That is, regardless of the absolute value of bar.value, I expect foo.call() to increment it by 2. The point of the 1 assertion per test guideline is to end up with tests that are more focused. Giving that you did not seem to think of the above t…

I think you forgot at least one valid assertion and implied another one:

foo.call() might have a return value.

Also, the whole story invocation shouldn't throw an exception, if your language has them. This assertion is often implied (and that's fine), but it's still there.

Finally the test case is a little bit stupid, because very seldom code doesn't have any input that changes the behavior/result. So your assertion would usually involve that input.

If you follow that though consequently, you end up with property-based tests very soon. But property-based tests should have as many assertions as possible for a single point of data. Say you test addition. When writing property-based tests you would end up with three specifications: one for one number, testing the identity element and the relationship to increments. Another one for two numbers, testing commutativity and inversion via subtraction, and one for three numbers, testing associativity. In every case it would be very weird to not have all n-ary assertions for the addition operation in the same spot.

Re: Multiple assertions are fine in a unit test

#174

Post by mrits » Sat, Nov 05, 2022, 2:26 PM UTC

magic_hamster wrote: post #51

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

> How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". I envy you for never having seen tests atrocious enough where this is not only possible, but the common case. Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it. Now imagine…

22 assertions in a test is a lot better than 22 separate tests that fail for the same reason.

Re: Multiple assertions are fine in a unit test

#175

Post by jcelerier » Sat, Nov 05, 2022, 2:27 PM UTC

magic_hamster wrote: post #61

ramraj07 wrote: post #58

Earlier quoted context omitted.

So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

Not at all. It makes sense in some tests. I addressed the part asking how it's even possible to not know what happened. As for multiple asserts, that is really meaningless. The test case should test one thing. If it requires several asserts that's okay. But having a very long test function with a lot of assertions, is strongly indicating that you're testing more than one thing, and when the test fails it will be hard…

Why ? In c++ if I have a test case with

    REQUIRES(v1::foo(0) == v2::foo(0));
    REQUIRES(v1::foo(1) == v2::foo(1));

And the second assert fails the error message will tell me exactly that, the line, and the value of both function calls if they are printable. What more do you want to know "what actually happened"?

Re: Multiple assertions are fine in a unit test

#176

Post by mrhands556 » Sat, Nov 05, 2022, 2:27 PM UTC

> You may be trying to simulate a ‘session’ where a client performs many steps in order to achieve a goal. As Gerard Meszaros writes regarding the test smell, this is appropriate for manual tests, but rarely for automated tests.

Integration tests are typically easier to write / maintain and thus are more valuable than small unit tests. Don’t know why the entire premise argues against that.

Re: Multiple assertions are fine in a unit test

#177

Post by TrianguloY » Sat, Nov 05, 2022, 2:27 PM UTC

superjan wrote: post #122

TrianguloY wrote: post #22

The one assertion per test doesn't mean you need to use only one assertion call but rather that you only need to do one assertion block. Checking everything after a response is considered 1 assertion, no matter how many assert calls you need. The issue is when you use multiple assertions for multiple logic statements: do > assert > do > assert... In that example imagine that you were also checking that the reservatio…

I think the issue is that you’ll always have one of those teammates who see this as an excuse to test the entire happy flow and all its effects in a single test case. I think what you want is reasonable, but how do you agree when it is no longer reasonable?

If you logic depends on that happy path, make a test for it. But as I explained in another comment that test should not justify the lack of individual feature tests, which should not only test the happy path but other corner cases too.

On my company we developers usually create white-box unitary/feature tests (we know how it was implemented, so we check components knowing that). But then we have an independent QA team that creates and run black-box flow tests (they don't know how it was implemented, only what it should do and interact)

Re: Multiple assertions are fine in a unit test

#178

Post by marcosdumay » Sat, Nov 05, 2022, 2:27 PM UTC

ronnier wrote: post #152

jeremyjh wrote: post #143

Earlier quoted context omitted.

Tests are not tech debt. You could have bad, brittle tests that you could consider debt but just having tests isn’t debt. Debt implies there is something you could do about it in the future to pay it down, which isn’t the case for a good test suite.

It’s debt. When you can’t add new features quickly because you have nightmarish tests to fix and you spend more time on the tests than the product, I’d say it’s debt. Especially with the insane mocking setups.

Tests can carry tech debit, just like any code. They certainly are not identified by it.

Tests are one of the ways you have to ensure your code is correct. Consequently, they are business-oriented code that exist to support your program usage, and subject to its requirements. How much assurance you need is completely defined by those requirements. (But how you achieve that assurance isn't, and tests are only one of the possible tools for that.)

Re: Multiple assertions are fine in a unit test

#179

Post by ascendantlogic » Sat, Nov 05, 2022, 2:28 PM UTC

I've always operated on the principal that each test should be testing one logical concept. That usually translates into one concrete assertion in the test code itself, but if you need to assert multiple concrete conditions to test the logical concept it's not a bad thing. At the end of the day tests are just there to make you more confident in code changes and they are a tax you have to pay for that comfort. However you arrive at "I feel comfortable making changes", go with it.

Re: Multiple assertions are fine in a unit test

#180

Post by marcosdumay » Sat, Nov 05, 2022, 2:33 PM UTC

smhoff256 wrote: post #149

The reason for one assert pet test is that it forces you to test one thing at a time. It sets your mind in the mode of single responsibility.

And what is the gain from testing a single thing at a time?

On a specific case, what is the gain from not testing the pre and post-conditions of your important test?