Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

51–60 of 348 posts

Re: Multiple assertions are fine in a unit test

#51

Post by magic_hamster »

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

> How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!".

I envy you for never having seen tests atrocious enough where this is not only possible, but the common case.

Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it.

Now imagine this barebones use of assertions in tests which are entirely too long, not isolating the test cases properly, or even completely irrelevant to what's (supposedly) being tested!

If that's not enough, imagine this nightmare failing not after it has been written, but, let's say 18 months later, while being part of a massive test suite running for a while. All you have is a the name of the test that failed, you look into it to find a 630 lines long test "case" with 22 nondescript assertions along the way. You might know which line failed the test, but not always. And of course debugging the test function line by line doesn't work because the test depends on intricate timing for some reason. The person who wrote this might not be around and now this is your dragon to slay.

I think I should stop here before triggering myself any further. Therapy is expensive.

Re: Multiple assertions are fine in a unit test

#52

Post by danuker »

olex wrote: post #42
danuker wrote: post #37

Earlier quoted context omitted.

> your example stops on the first one that fails, and does not evaluate the others after that. I don't think this is a big problem; trying to focus on multiple examples at once is difficult. It might be a problem if tests are slow and you are forced to work on all of them at once. But in that case I'd try to make the tests faster (getting rid of network requests, disk/DB access by faking them away or hoisting to the…

Hm. I think my main issue there is not the speed, but rather seeing the whole picture at once. You mentioned you use this pattern to test regular expressions; say you modify the regexp in question with some new feature requirement, and now the very first of a dozen test inputs fails. You fix it, but then each one of the following keeps failing, and you can only find an elegant solution that works for all of them afte…

In my experience, from doing some TDD Katas[0] and timing myself, I found coding slower and more difficult when focusing on multiple examples at once.

I usually even comment out all the failing tests but the first one, after translating a bunch of specifications into tests, so I see the "green" when an example starts working.

Maybe it would be easier to grok multiple regex examples than algorithmic ones, but at least for myself, I am skeptical, and I prefer taking them one at a time.

[0] - https://kata-log.rocks/tdd

Re: Multiple assertions are fine in a unit test

#53

Post by pydry »

matthews2 wrote: post #36
littlecranky67 wrote: post #9

Earlier quoted context omitted.

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

Are there any "rules" in Clean Code that don't need to be disregarded and burned? https://qntm.org/clean

There's plenty of sensible advice in there it's just that he argues for the sensible stuff and the idiotic stuff with equal levels of conviction and if you are junior you aren't going to be able to distinguish them.

It would be easier if it were all terrible advice.

Re: Multiple assertions are fine in a unit test

#54

Post by dd82 »

mollerhoj wrote: post #28
philliphaydon wrote: post #21

Earlier quoted context omitted.

That’s because the example test only requires 1 assertion. Any rule that says there should be only 1 assertion ever is stupid.

OP asked how any state change would be tested with a single 'assertion' and I provided an answer. Absolute rules are stupid, but our codebase has just short of 10k tests, and very few have more than one assertion. The only reason I can really see to have more than one assertion would be to avoid having to run the setup/teardown multiple times. However, its usually a desirable goal to write code that require little se…

one assert per test seems... as you said, indicative of zealotry. if you already have your object there, why not test for the changes you expect?

So you have one test that indicates that a log error is outut. then another that tests that the property X in the return from the error is what you expect. then another test to determine that propery Y in return is what you expect?

that to me is wasteful, unclear, bloated. About the only useful result I can see that is it allows bragging about how many tests a project has.

Re: Multiple assertions are fine in a unit test

#55

Post by dd82 »

olex wrote: post #50
dd82 wrote: post #48

Earlier quoted context omitted.

not really. one thing this is useful for is extracting out various attributes in an object when you really don't want to compare the entire thing. Or comparing dict attributes, and figuring which one is the incorrect one. for example, expected_results = {...} actual_obj = some_intance.method_call(...) for key, val in expected_results.items(): assert getattr(actual_obj, key) == val, f"Mismatch for {key} attribute" You…

Ok, in this case a parametrized test is not the best approach, I agree. But I would still want to avoid the for-each and "failing fast". One approach would be to gather the required attributes in an array or a struct of some sort, and then do a single assert comparison with an expected value, showing all the differences at once. However, this requires the assertion framework to be able to make such a comparison and r…

Right, and not many actually do. with python and pytest, you could leverage difflib, but that's an additional thing that adds unnecessary complexity. My approach is simple enough, good enough, and doesn't require additional fudging around with the basics of the language's test libs.

also,

>your example stops on the first one that fails, and does not evaluate the others after that.

I would argue this is desirable behavior. there are soft checks, ie, https://pypi.org/project/pytest-check/, that basically replace assertions as raised exceptions and do your approach. But I do want my tests to raise errors at the point of failure when a change occurs. If there's alot of changes occurring, that raises larger questions of "why" and "is the way we're executing this change a good one"?

Re: Multiple assertions are fine in a unit test

#56

Post by [deleted] »

danuker wrote: post #27
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I use the following pattern for testing regexes: expected_positive = [ 'abc', 'def', ...] for text in expected_positive: self.assertTrue(matcher(text), f"Failed: {text}") Before I added the assertion error message, `f"Failed: {text}"`, it was quite difficult to tell WHICH example failed.

[deleted]

Re: Multiple assertions are fine in a unit test

#58

Post by ramraj07 »

magic_hamster wrote: post #51
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

> How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". I envy you for never having seen tests atrocious enough where this is not only possible, but the common case. Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it. Now imagine…

So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

Re: Multiple assertions are fine in a unit test

#59

Post by oweiler »

lifeisstillgood wrote: post #38

Things I find awkward with unit test (and it might just be me) is Inwant to write a test like def testLotsOfWaysTofail(): d = {"Handle Null": (None, foobar), "Don't allow under 13 to do it": (11, foobar), "or old age pensioner": (77,wobble)} for ... generate a unit test dynamically here I have built metaclasses, I have tried many different options. I am sure there is a near solution. But I never seem to have it right

What you are looking for is called property based testing

Re: Multiple assertions are fine in a unit test

#60

Post by SideburnsOfDoom »

This is correct - multiple asserts are OK, but there are still good guidelines:

A good unit test has the phases Arrange, Act, Assert, end of unit test.

You can use multiple assert statements in the "assert" phase, to check the specifics of the single logical outcome of the test.

In fact, once I see the same group of asserts used 3 or more times, I usually extract a helper method, e.g. "AssertCacheIsPopulated" or "AssertHttpResponseIsSuccessContainingOrder" these might have method bodies that contain multiple assert statements, but the question of is this a "single assert" or not, is a matter of perspective and not all that important.

The thing to look out for is - does the test both assert that e.g. the response is an order, and that the cache is populated? Those should likely be separate tests as they are logically distinct outcomes.

The test ends after the asserts - You do not follow up the asserts with a second action. That should be a different test.

Post reply on HN