Multiple assertions are fine in a unit test • Hacker News Board

Re: Multiple assertions are fine in a unit test

#121

Post by simplotek » Sat, Nov 05, 2022, 1:10 PM UTC

ramraj07 wrote: post #58

magic_hamster wrote: post #51

Earlier quoted context omitted.

> How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". I envy you for never having seen tests atrocious enough where this is not only possible, but the common case. Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it. Now imagine…

So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

> So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

You're falling prey to slippery slope fallacy, which at best is specious reasoning.

The rationale is easy to understand. Running 100 assertions in a single test renders tests unusable. Running 10 assertions suffers from the same problem. Test sets are user-friendly if they dump a single specific error message for a single specific failed assertion, thus allowing developers to quickly pinpoint root causes by simply glancing through the test logs.

Arguing whether two or three or five assertions should be banned misses the whole point and completely ignores the root cause that led to this guideline.

Re: Multiple assertions are fine in a unit test

#122

Post by superjan » Sat, Nov 05, 2022, 1:11 PM UTC

TrianguloY wrote: post #22

The one assertion per test doesn't mean you need to use only one assertion call but rather that you only need to do one assertion block. Checking everything after a response is considered 1 assertion, no matter how many assert calls you need. The issue is when you use multiple assertions for multiple logic statements: do > assert > do > assert... In that example imagine that you were also checking that the reservatio…

I think the issue is that you’ll always have one of those teammates who see this as an excuse to test the entire happy flow and all its effects in a single test case. I think what you want is reasonable, but how do you agree when it is no longer reasonable?

Re: Multiple assertions are fine in a unit test

#123

Post by [deleted] » Sat, Nov 05, 2022, 1:11 PM UTC

danuker wrote: post #52

olex wrote: post #42

Earlier quoted context omitted.

Hm. I think my main issue there is not the speed, but rather seeing the whole picture at once. You mentioned you use this pattern to test regular expressions; say you modify the regexp in question with some new feature requirement, and now the very first of a dozen test inputs fails. You fix it, but then each one of the following keeps failing, and you can only find an elegant solution that works for all of them afte…

In my experience, from doing some TDD Katas[0] and timing myself, I found coding slower and more difficult when focusing on multiple examples at once. I usually even comment out all the failing tests but the first one, after translating a bunch of specifications into tests, so I see the "green" when an example starts working. Maybe it would be easier to grok multiple regex examples than algorithmic ones, but at least…

[deleted]

Re: Multiple assertions are fine in a unit test

#124

Post by BlargMcLarg » Sat, Nov 05, 2022, 1:12 PM UTC

cultofmetatron wrote: post #115

I didn't know you were only supposed to put one assertion per unit test. some of mine have 5-10 assertions. why?. because I don't have time to write tests all day but I recognize there are tests. As long as it catches the bugs before it goes to prod and can be modified easily, who cares?

Unfortunately and from experience, far more seem to care about the how than the what when it comes to testing. This comment section alone is pretty telling: people spending their Saturday to write dogmatic rules.

Re: Multiple assertions are fine in a unit test

#125

Post by drewcoo » Sat, Nov 05, 2022, 1:16 PM UTC

choeger wrote: post #3

What? People really would criticize that code because it has two assertions? How are they ever testing any state changes? And to the author: Your bubble is significantly different from mine. Pretty much every competent developer I've worked with would laugh at you for the idea that the second test case would not be perfectly fine. (But that first iteration would never pass code review either because it does nothing a…

> But that first iteration would never pass code review either because it does nothing and thus is a waste of effort.)

That first iteration would not be subject to code review. The author is using TDD.

https://en.wikipedia.org/wiki/Test-driven_development

Re: Multiple assertions are fine in a unit test

#126

Post by andrewallbright » Sat, Nov 05, 2022, 1:18 PM UTC

I've spent some time testing and thinking about testing. I think the universal guidance that should be followed is this...

Do whatever you need/want to that makes your software successful. The rules are meant to be broken if it means you shipping.

Re: Multiple assertions are fine in a unit test

#127

Post by drewcoo » Sat, Nov 05, 2022, 1:19 PM UTC

warrenmiller wrote: post #112

lodovic wrote: post #4

Wait - people are doing real http calls in unit tests, over a network, and complaining about multiple asserts in the test code?

They can be mocked surely?

It's not nice to mock people "doing real http calls in unit tests," even if they deserve it.

Re: Multiple assertions are fine in a unit test

#128

Post by throwaway0asd » Sat, Nov 05, 2022, 1:19 PM UTC

My learning from test automation:

* The primary goal of test automation is to prevent regression. A secondary goal can be performance tuning your product.

* Tests are tech debt, so don’t waste time with any kind of testing that doesn’t immediately save you time in the near term.

* Don’t waste your energy testing code unless you have an extremely good reason. Test the product and let me the product prove the quality of your code. Code is better tested with various forms of static analysis.

* The speed with which an entire test campaign executes determines, more than all other factors combined, when and who executes the tests. If the test campaign takes hours nobody will touch it. Too painful. If it takes 10-30 minutes only your QA will touch it. When it takes less than 30 seconds to execute against a major percentage of all your business cases everybody will execute it several times a day.

Re: Multiple assertions are fine in a unit test

#129

Post by rhdunn » Sat, Nov 05, 2022, 1:19 PM UTC

I view it more as "only test one operation per unit test". If that needs multiple asserts (status code, response content, response mime type, etc.) to verify, that is fine.

IIUC, the guideline is so that when a test fails you know what the issue is. Therefore, if you are testing more than one condition (missing parameter, invalid value, negative number, etc.) it is harder to tell which of those conditions is failing, whereas if you have each condition as a separate test it is clear which is causing the failure.

Separate tests also means that the other tests run, so you don't have any hidden failures. You'll also get hidden failures if using multiple assertions for the condition, so will need to re-run the tests multiple times to pick up and fix all the failures. If you are happy with that (e.g. your build-test cycle is fast) then having multiple assertions if fine.

Ultimately, structure your tests in a way that best conveys what is being tested and the conditions it is being tested under (e.g. partition class, failure condition, or logic/input variant).

Re: Multiple assertions are fine in a unit test

#130

Post by raverbashing » Sat, Nov 05, 2022, 1:20 PM UTC

mollerhoj wrote: post #24

jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

I believe theres a rubocop that checks for it: https://docs.rubocop.org/rubocop-minitest/cops_minitest.html...

Nothing like machine-enforced nitpicking of the worse kind

But even this rule/plugin has a default value of 3 which is a more sane value.