It's not that there's anything inherently wrong with multiple assertions in a unit test, it's that it's often a smell you're testing multiple behaviours in a single test - which is the thing you should avoid.
Multiple assertions are fine in a unit test
271–280 of 348 posts
Re: Multiple assertions are fine in a unit test
#272Earlier quoted context omitted.
Tests can carry tech debit, just like any code. They certainly are not identified by it. Tests are one of the ways you have to ensure your code is correct. Consequently, they are business-oriented code that exist to support your program usage, and subject to its requirements. How much assurance you need is completely defined by those requirements. (But how you achieve that assurance isn't, and tests are only one of t…
They are still debt since they don't directly contribute to product value: you can delete all your tests and your software will keep functioning. It doesn't mean it's a debt worth taking, though IME most companies are either taking way too much or way to little. Not treating tests as debt typically leads to over-testing, and it is way worse than under-testing. Also, what you're talking about (business-oriented requir…
If you can simply delete it that’s not debt, it’s just cost. My bank doesn’t let me delete my loan obligation and still live in my house.
Re: Multiple assertions are fine in a unit test
#273> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…
> I've never seen a case where it would be hard to tell which assertion failed. There are a set of unit testing frameworks that do everything they can to hide test output (junit), or vomit multiple screens of binary control code emoji soup to stdout (ginkgo), or just hide the actual stdout behind an authwall in a uuid named s3 object (code build). Sadly, the people with the strongest opinions about using a "proper" u…
> There are a set of unit testing frameworks that do everything they can to hide test output (junit), or vomit multiple screens of binary control code emoji soup to stdout (ginkgo), or just hide the actual stdout behind an authwall in a uuid named s3 object (code build).
The test runner in VS2019 does this, too and it's incredibly frustrating. I get to see debug output about DLLs loading and unloading (almost never useful), but not the test's stdout and stderr (always useful). Brilliant. At least their command line tool does it right.
Re: Multiple assertions are fine in a unit test
#274Earlier quoted context omitted.
> The unit tests is a statement that you will never refactor across this line, and that eliminates a lot of flexibility that I want. I certainly don't see it as that. I see it as "this is the smallest thing I _can_ test usefully". Mind you, those do tend to correlate, but they're not the same thing.
> this is the smallest thing I _can_ test usefully Then you're testing useless things. Usefulness is when different parts of a program work together as a coherent whole. Testing DB access layer and service layer separately (as units are often defined) has no meaning (but is often enforced). Queue in memes about "unit tests with 100% code coverage, no integration tests" https://mobile.twitter.com/thepracticaldev/statu…
>> this is the smallest thing I _can_ test usefully
> Then you're testing useless things.
We'll have to agree to disagree then.
> Testing DB access layer and service layer separately (as units are often defined)
Not at all. For me, a unit is a small part of a layer; one method. Testing the various parts in one system/layer is another type of test. Testing that different systems work together is yet another.
I tend to think in terms of the following
- Unit test = my code works
- Functional test = my design works
- Integration test = my code is using your 3rd party stuff correctly (databases, etc)
- Factory Acceptance Test = my system works
- Site Acceptance Test = your code sucks, this totally isn't what I asked for!?!
The "my code works" part is the smallest piece possible. Think "the sorting function" of a library that can return it's results sorted in a specific order.
Re: Multiple assertions are fine in a unit test
#275This always rubbed me the wrong way. I think a better way would be to ensure the assertions to be readable, as simple as can be. The worst case of this being broken was the assertions were done in an helper method used in a base class of a test, and navigating to it required multiple hops, it took time to build the context of the test as well. The only downside of multiple assertions is that when the first one fails…
Totally agreed, I even think duplication is totally fine in tests if it brings more readability. You should be able to read the test to verify its correctness (after all you don't test the test code), and multiple helper functions and levels of abstraction hinder this goal.
Re: Multiple assertions are fine in a unit test
#276Earlier quoted context omitted.
Tests are not tech debt. You could have bad, brittle tests that you could consider debt but just having tests isn’t debt. Debt implies there is something you could do about it in the future to pay it down, which isn’t the case for a good test suite.
It’s debt. When you can’t add new features quickly because you have nightmarish tests to fix and you spend more time on the tests than the product, I’d say it’s debt. Especially with the insane mocking setups.
I think that's projection.
Your tests being a nightmare doesn't imply my tests are a nightmare.
Re: Multiple assertions are fine in a unit test
#277Are people so entrenched in dogmas that we need “Stop doing X” type articles every few months?
Unfortunately, yes. People with good judgement are difficult to find.
Re: Multiple assertions are fine in a unit test
#278Earlier quoted context omitted.
> this is the smallest thing I _can_ test usefully Then you're testing useless things. Usefulness is when different parts of a program work together as a coherent whole. Testing DB access layer and service layer separately (as units are often defined) has no meaning (but is often enforced). Queue in memes about "unit tests with 100% code coverage, no integration tests" https://mobile.twitter.com/thepracticaldev/statu…
>> this is the smallest thing I _can_ test usefully > Then you're testing useless things. We'll have to agree to disagree then. > Testing DB access layer and service layer separately (as units are often defined) Not at all. For me, a unit is a small part of a layer; one method. Testing the various parts in one system/layer is another type of test. Testing that different systems work together is yet another. I tend to…
And the only actual useful tests are functional (depending on how you write them) and above.
If those fail, it means that neither your design nor your code works.
The absolute vast majority of unit tests are meaningless because you just repeat them again in the higher level tests.
Re: Multiple assertions are fine in a unit test
#279Earlier quoted context omitted.
So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?
Not at all. It makes sense in some tests. I addressed the part asking how it's even possible to not know what happened. As for multiple asserts, that is really meaningless. The test case should test one thing. If it requires several asserts that's okay. But having a very long test function with a lot of assertions, is strongly indicating that you're testing more than one thing, and when the test fails it will be hard…
> when the test fails it will be harder to know what actually happened
This should not ever be possible in any semi-sane test environment.
One could in theory write a single test function with thousands of asserts for all kinds of conditions and it still should be 100% obvious which one failed when something fails. Not that I'd suggest going to that extreme either, but it illustrates that it'll work fine.
Re: Multiple assertions are fine in a unit test
#280Earlier quoted context omitted.
Nice framework. Also once you allow more than one assertion, there is no need for top-level && in assertions (making them simpler tests).
While there is no strict need , sometimes assertions logically belong together.
You definitely want x || y, x ^^ y, and x implies y.