Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

131–140 of 348 posts

Re: Multiple assertions are fine in a unit test

#131

Post by drewcoo »

z9znz wrote: post #98

Testing is hard when the code it tests is OOP, has mutations everywhere, and has big functions that do too many things. It's practically impossible to thoroughly test such code with one assertion per test; it would mean having dozens of tests just for one object method. Correspondingly, the fixtures/factories/setup for tests would balloon in number and complexity as well to be able to setup the exact circumstance bei…

> Testing is hard when the code it tests is OOP, has mutations everywhere, and has big functions that do too many things.

Yes, that means the software being tested is not very testable. And should probably be refactored.

https://en.wikipedia.org/wiki/Testability

Re: Multiple assertions are fine in a unit test

#132

Post by BlargMcLarg »

simplotek wrote: post #121
ramraj07 wrote: post #58

Earlier quoted context omitted.

So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

> So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test? You're falling prey to slippery slope fallacy, which at best is specious reasoning. The rationale is easy to understand. Running 100 assertions in a single test renders tests unusable. Running 10 assertions suffers from the same problem. Test sets are user-friendly if they dump a single s…

>Test sets are user-friendly if they dump a single specific error message for a single specific failed assertion, thus allowing developers to quickly pinpoint root causes by simply glancing through the test logs.

As if this actually happens in practice, regardless of multiple or single asserts. Anything that isn't non-trivial will at most tell you what doesn't work, but it won't tell you why it doesn't work. Maybe allowing an educated guess when multiple tests fail to function.

You want test sets to be user friendly? Start at taking down all this dogmatism and listening to the people as to why they dislike writing tests. We're pushing 'guidelines' (really more like rules) while individuals think to themselves 'F this, Jake's going to complain about something trivial again, and we know these tests do jack-all because our code is a mess and doing anything beyond this simple algorithm is a hell in a handbasket".

These discussions are beyond useless when all people do is talk while doing zero to actually tackle the issues of the majority not willing to write tests. "Laziness" is a cop-out.

Re: Multiple assertions are fine in a unit test

#133

Post by TeMPOraL »

Macha wrote: post #99
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I've noticed the opposite in a Java codebase I work in. Tests where the test is assertEquals(toJson(someObject), giantJsonBlobFromADifferentFile). Of course the test runner has no idea about formatting strings that happen to be json, so I end up having to copy these out into an editor, formatting them and eyeballing the difference, or for even larger ones having to save them out to files and diff them. And of course…

I had similar case recently, in C++. I ended up spending a few hours writing a simple JSON differ - a bit of code that would parse two strings into a DOM object graph using a rapidjson, and then walk down them simultaneously - basically, I implemented operator== which, instead of terminating early, recorded every mismatch.

Then, I packaged it into a Google Test matcher, and from now on, the problem you describe is gone. I write:

  EXPECT_THAT(someObject, IsEqAsJSON(someBlobFromADifferentFile));
and if it fails, I get output like this:

  Expected someObject to be structurally equivalent to someBlobFromADifferentFile; it is not;
   - #/object/key - missing in expected, found in actual
   - #/object/key2 - expected string, actual is integer
   - #/object/key3/array1 - array lengths differ; expected: 3, actual: 42
   - #/object/key4/array1/0/key3 - expected "foo" [string], actual "bar" [string]
Etc.

It was a rather simple exercise, and the payoff is immense. I think it's really important for programmers to learn to help themselves. If there's something that annoys you repeatedly, you owe it to yourself and others to fix it.

Re: Multiple assertions are fine in a unit test

#134

Post by SeriousM »

I keep it simple by using the AAA pattern:

Arrange: whatever you need for the setup. Act: a single line of code that is under test. Assert: whatever you want to assert, multiple statements allowed and usually required.

We put the 3A in code as comments as boundaries and that works more than perfect for the whole team.

Oh and it's readable!

Re: Multiple assertions are fine in a unit test

#136

Post by Joeboy »

magic_hamster wrote: post #61
ramraj07 wrote: post #58

Earlier quoted context omitted.

So because some idiot somewhere wrote a 100 assertion unit test we should ban anyone from writing even 2 assertions in one test?

Not at all. It makes sense in some tests. I addressed the part asking how it's even possible to not know what happened. As for multiple asserts, that is really meaningless. The test case should test one thing. If it requires several asserts that's okay. But having a very long test function with a lot of assertions, is strongly indicating that you're testing more than one thing, and when the test fails it will be hard…

I guess you might be talking about a different language / environment than I'm used to, but even in the 100 assertion test case you get useful tracebacks in python. Testing lots of things at the same time means strictly speaking you're writing an integration test rather than a unit test, but still I don't see how it's a bad test. It's easy and stops buggy PRs going into production.

The test failures I see that are actually hard to debug are ones where the failures are difficult to reproduce due to random input, tests running in parallel and sharing the same filesystem etc. I don't think I've ever not known what assert was failing (although I guess in theory you could make that happen by catching AssertionError).

Re: Multiple assertions are fine in a unit test

#137

Post by mattmanser »

torginus wrote: post #103
superjan wrote: post #93

Earlier quoted context omitted.

Is this shared somewhere?

Nah, it was some corporate project. But if you are interested, I could rewrite it. It would be a fun weekend project.

It's also worth figuring out for yourself though! They're suprisingly easy to use (look up Expression).

It's incredibly useful once you know how, and encourages you to stop using reflection, passing magic strings as arguments, or having to use nameof().

Re: Multiple assertions are fine in a unit test

#138

Post by TeMPOraL »

masklinn wrote: post #95
jmillikin wrote: post #91

Earlier quoted context omitted.

That's why the first check is a hard assertion (returning on error), and the others are soft (continuing on error). If the list is empty, then the log will contain one error about the length being zero. If the list has one item but it has the wrong properties, the log will contain two errors.

> That's why the first check is a hard assertion (returning on error), and the others are soft (continuing on error). See that's so completely unclear I utterly missed that there were two different calls there. Doesn't exactly help that the functions are the exact same length, and significantly overlap in naming.

That's just a matter of familiarity, though. And if you make a mistake, you'll discover it the first time the test fails - either you'll see too little output, or you'll see the test throw an exception or crash.

Re: Multiple assertions are fine in a unit test

#139

Post by [deleted] »

lifeisstillgood wrote: post #120
_dain_ wrote: post #81

Earlier quoted context omitted.

isn't this just a parametrized test? or do you want to generate the test cases automatically like in property-based testing?

Yes parameterised testing. Just all the packages i have tried seemed to make it awkward to impossible - hence metaclasses

[deleted]

Re: Multiple assertions are fine in a unit test

#140

Post by mananaysiempre »

superjan wrote: post #93
torginus wrote: post #89

Earlier quoted context omitted.

I remember writing a small .NET test library for that exact problem - You could pass in a lambda with a complex condition, and it evaluated every piece of the expression separately and pretty printed what part of the condition failed. So essentially you could write Assert(()=>width>0 && x + width And you would get: Assertion failed: x is 1500 width is 600 screenWidth is 1920 It used Expression to do the magic. Amazin…

Is this shared somewhere?

FWIW, in Python, this is pytest’s entire thing (although it ends up grovelling into bytecode to achieve it).
Post reply on HN