Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

211–220 of 348 posts

Re: Multiple assertions are fine in a unit test

#211

Post by AlphaSite »

mollerhoj wrote: post #17
choeger wrote: post #3

What? People really would criticize that code because it has two assertions? How are they ever testing any state changes? And to the author: Your bubble is significantly different from mine. Pretty much every competent developer I've worked with would laugh at you for the idea that the second test case would not be perfectly fine. (But that first iteration would never pass code review either because it does nothing a…

To answer your question: We zealots test for the fact that something changes to some degree. E.g with rubys rspec library: expect { foo.call() }.to change { bar.value }.by(2) That is, regardless of the absolute value of bar.value, I expect foo.call() to increment it by 2. The point of the 1 assertion per test guideline is to end up with tests that are more focused. Giving that you did not seem to think of the above t…

I think a much better rule of thumb is: “A lot of small unit tests are better than a few big ones”. Same thing, but clearer intent and less rigid.

Re: Multiple assertions are fine in a unit test

#212

Post by mynameisvlad »

magic_hamster wrote: post #193
FartyMcFarter wrote: post #94

Earlier quoted context omitted.

> You might know which line failed the test, but not always. If that's the case, the test framework itself is severely flawed and needs fixing even more than the tests do. There's no excuse to have an assert function that doesn't print out the location of the failure.

Even if the framework is fine, you can see something like an elaborate if-else tree, or even a try-catch block, and after it's all done, there's a condition check with `fail()`. So the point of failure could be manually detached from the actual point of failure. Granted, this is not the way to do things. But it happens anyway.

I mean in your example it’s someone choosing not to use asserts. Which is a problem, don’t get me wrong, but it’s not the problem being talked about here.

The comment thread is about “Assertion Roulette” — having so many assertions you don’t know which went off. Which really seems like a test framework issue more than a test issue.

Re: Multiple assertions are fine in a unit test

#213

Post by shagie »

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

With Junit, you could use assertAll https://junit.org/junit5/docs/5.0.0-M2/api/org/junit/jupiter...

Aside from that, you could do things like

   int failures = 0;
   failures += someAssertion ? 0 : 1;
   failures += anotherAssertion ? 0 : 1;
   failures += yetAnotherAssertion ? 0 : 1;
   assertZero(failures);
With the appropriate logging of each assertion in there.

Consider the situation of "I've got an object and I want to make sure it comes out in JSON correctly"

The "one assertion" way of doing it is to assertEqual the entire json blob to some predefined string. The test fails, you know it broke, but you don't know where.

The multiple assertions approach would tell you where in there it broke and the test fails.

The point is much more one of "test one thing in a test" but testing one thing can have multiple assertions or components to it.

You don't need to have testFirstNameCorrect() and testLastNameCorrect() and so on. You can do testJSONCorrect() and test one thing that has multiple parts to verify its correctness. This becomes easier when you've got the frameworks that support it such as the assertAll("message", () -> assertSomething, () -> assertSomethingElse(), ...)

https://stackoverflow.com/q/40796756

Re: Multiple assertions are fine in a unit test

#214

Post by hedora »

turblety wrote: post #202
hedora wrote: post #191

Earlier quoted context omitted.

> I've never seen a case where it would be hard to tell which assertion failed. There are a set of unit testing frameworks that do everything they can to hide test output (junit), or vomit multiple screens of binary control code emoji soup to stdout (ginkgo), or just hide the actual stdout behind an authwall in a uuid named s3 object (code build). Sadly, the people with the strongest opinions about using a "proper" u…

Yup, it's why I built `just-tap` [1] which trys to minimise as much magic that a lot of these frameworks try to "help" you with. 1. https://github.com/markwylde/just-tap

Here are a few mistakes I've seen in other frameworks:

- Make it possible to disable timeouts. Otherwise, people will need a different runner for integration, long running (e.g., find slow leaks), and benchmark tests. At that point, your runner is automatically just tech debt.

- It is probably possible to nest before and afters, and to have more than one nesting per process, either from multiple suites, or due to class inheritance, etc. Now, you have a tree of hooks. Document whether it is walked in breadth first or depth first order, then never change the decision (or disallow having trees of hooks, either by detecting them at runtime, or by picking a hook registration mechanism that makes them inexpressible).

Re: Multiple assertions are fine in a unit test

#216

Post by kevincox »

nsoonhui wrote: post #142
drewcoo wrote: post #127

Earlier quoted context omitted.

It's not nice to mock people "doing real http calls in unit tests," even if they deserve it.

For mocking, the parent comment means "swapping out the external calls with dummy calls", not "laughing at the developer"

I'm 98% sure that they understood that and it was a joke.

Re: Multiple assertions are fine in a unit test

#217

Post by catlifeonmars »

shagie wrote: post #213
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

With Junit, you could use assertAll https://junit.org/junit5/docs/5.0.0-M2/api/org/junit/jupiter... Aside from that, you could do things like int failures = 0; failures += someAssertion ? 0 : 1; failures += anotherAssertion ? 0 : 1; failures += yetAnotherAssertion ? 0 : 1; assertZero(failures); With the appropriate logging of each assertion in there. Consider the situation of "I've got an object and I want to make su…

For the JSON example, some testing libraries (jest) will output a unified diff on assertion failure.

Re: Multiple assertions are fine in a unit test

#218

Post by eurasiantiger »

beezlewax wrote: post #171
torginus wrote: post #89

Earlier quoted context omitted.

I remember writing a small .NET test library for that exact problem - You could pass in a lambda with a complex condition, and it evaluated every piece of the expression separately and pretty printed what part of the condition failed. So essentially you could write Assert(()=>width>0 && x + width And you would get: Assertion failed: x is 1500 width is 600 screenWidth is 1920 It used Expression to do the magic. Amazin…

Is there something like this available for javascript?

You could parse the failing expression using babel and get an AST back.

Re: Multiple assertions are fine in a unit test

#219

Post by 8n4vidtmkvmk »

i need to send this to all my coworkers. i wrote some tests kind of similar to this, except i had a bunch of setup code then asserted something was true, changed 1 thing and then verified it became false so that it was super clear that it was the 1 thing that causes it to become false.

they were like no, copy paste the entire setup and change the field to false in the new setup.

im like how are you supposed to tell which of the dozen conditions triggered it now? you have to diff the 2 tests in your head and hope they don't get out of sync? ridiculous

Re: Multiple assertions are fine in a unit test

#220

Post by hedora »

autarch wrote: post #206
turblety wrote: post #202

Earlier quoted context omitted.

Yup, it's why I built `just-tap` [1] which trys to minimise as much magic that a lot of these frameworks try to "help" you with. 1. https://github.com/markwylde/just-tap

TAP is better than some things, but it has some serious issues that I wrote about on my blog a while back - https://blog.urth.org/2017/01/21/tap-is-great-except-when-it... Basically it's nearly impossible to fully parse it correctly.

Is test2 a flag for TAP?

If you have to pick one or the other, then you're breaking the common flow (human debugging code before pushing) so that management can have better reports.

The right solution would be to add a environment variable or CLI parameter that told tap to produce machine readable output, preferably with a separate tool that could convert the machine readable junk to whatever TAP currently writes to stdout/stderr.

Post reply on HN