Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

281–290 of 348 posts

Re: Multiple assertions are fine in a unit test

#281

Post by sdenton4 »

teddyh wrote: post #265
sdenton4 wrote: post #232

Earlier quoted context omitted.

Michael Pollan was right: Write tests, not too many, mostly integration.

Contrary viewpoint: Integrated Tests Are A Scam (J.B. Rainsberger): https://www.youtube.com/watch?v=fhFa4tkFUFw

Watched a bit of this... It's typical test-driven zealotry; the main criticism of integration tests seems to be that they don't force your hand in system design in the way that unit tests do? Which seems very silly, but then, I'm not a person who goes to conferences about testing philosophy.

Re: Multiple assertions are fine in a unit test

#282

Post by regularfry »

zmj wrote: post #264
jasonhansel wrote: post #257

Earlier quoted context omitted.

I'd add one more: clearly document what determines the order in which tests are run. On the one hand, running tests in any order should produce the same result, and would in any decent test suite. On the other hand, if the order is random or nondeterministic, it's really annoying when 2% of PRs randomly fail CI, not because of any change in the code, but because CI happened to run unrelated tests in an unexpected ord…

I’ll disagree with this. Every time I’ve seen that, the interference between tests was also possible between requests in production. I’d rather my test framework give me a 2% chance of noticing the bug than 0%.

What's annoying is not being able to reproduce the 2% cases so you can't fix it even when you've noticed them. Sensible test tools give you the random seed they used to order the tests so you can reproduce the sequence.

Re: Multiple assertions are fine in a unit test

#283

Post by ch4s3 »

hedora wrote: post #191
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

> I've never seen a case where it would be hard to tell which assertion failed. There are a set of unit testing frameworks that do everything they can to hide test output (junit), or vomit multiple screens of binary control code emoji soup to stdout (ginkgo), or just hide the actual stdout behind an authwall in a uuid named s3 object (code build). Sadly, the people with the strongest opinions about using a "proper" u…

Junit is especially bad about this. I often wonder how many of these maxims are from people using substandard Java tools and confusing their workarounds with deeper insights.

Re: Multiple assertions are fine in a unit test

#284

Post by Phrodo_00 »

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I think the advantage of having an assertion per test is that it makes sure that all of your assertions are executed. In a lot of test frameworks (that use exceptions for assertions for example) the first assertion fail will stop the test.

That doesn't mean you have to duplicate code, you can deal with it in other ways. In Junit I like to use @TestFctory [1] where I'll write most of the test in the Factory and then each assertion will be a Test the factory creates, and since they're lambdas they have access to the TestFactoy closure.

[1] https://junit.org/junit5/docs/current/user-guide/#writing-te...

Re: Multiple assertions are fine in a unit test

#286

Post by zdragnar »

HereBeBeasties wrote: post #114
torginus wrote: post #97

Earlier quoted context omitted.

xUnit is terrible. It has a horrible culture, the author seems to have a god complex. It is overly complex and opinionated in the worst way possible. Many times I searched for 'how do I do something with xUnit' and found a github issue with people struggling with the same thing, and the author flat out refusing to incorporate the feature as it was against his principles. Other times I found that I needed to do was ov…

NUnit isn't much better. You'd think that they would have good test coverage and therefore high confidence to make changes, especially to fix actual bugs, but I gave up trying to get patches in because the core devs seem so afraid of breaking anything, even when it's an obviously-isolated private, hidden bug fix or performance improvement. "We can't land this tiny fix because we have a release planned within three mo…

How ironic, a unit test framework is so fragile and poorly covered by tests they are not able to determine if a change would cause any regressions.

Re: Multiple assertions are fine in a unit test

#287

Post by edgyquant »

Phrodo_00 wrote: post #284
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I think the advantage of having an assertion per test is that it makes sure that all of your assertions are executed. In a lot of test frameworks (that use exceptions for assertions for example) the first assertion fail will stop the test. That doesn't mean you have to duplicate code, you can deal with it in other ways. In Junit I like to use @TestFctory [1] where I'll write most of the test in the Factory and then e…

This is a feature. I want it to fail fast so I can see the first issue not crawl the stack for error logs.

Re: Multiple assertions are fine in a unit test

#288

Post by ImPleadThe5th »

alkonaut wrote: post #116

An assert message says what went wrong, and on which code line. How on earth does it help to make just one? The arrange part might take seconds for a nontrivial test and that would need to be duplicated both in code and execution time to make two asserts. If you painstakingly craft a scenario where you create a rectangle of a specific expected size why wouldn’t it be acceptable to assert both the width and height of…

The way it's been explained to me is that because one assert failing blocks the other assertions from running you don't get a "full" picture of what went wrong.

So instead of:

- error W doesn't equal 20

Fix that

Run test again

- error H doesn't equal 10

Fix that

Run test again

It's - Error Width doesn't equal 20

- Error Height doesn't equal 10

Fix both

Run test

I think the time savings are negligible though. And it makes testing even more tedious, as if people needed any additional reasons to avoid writing tests.

Re: Multiple assertions are fine in a unit test

#289

Post by switchbak »

__alexs wrote: post #204
bluGill wrote: post #181

Earlier quoted context omitted.

A unit tests tests one unit. And integration tests covers more than one unit. I think everyone agrees with that, but nobody has defined unit. The longer I program the more I am convinced that the larger your unit the better. The unit tests is a statement that you will never refactor across this line, and that eliminates a lot of flexibility that I want. It turns out that debugging failed integration tests is easy,the…

I recently went to the effort of trying to work out where the term unit test came from in some desperate effort to find what a unit was meant to be. After much googling and buying or ancient text books I hit a dead end. At this point I think "unit" is just noise that confuses people into making distinctions that don't exist.

As I recall the TDD mailing list has some background on the use of the word "unit", it goes WAY back, I believe it goes back to the mainframe/ punch card era. Regardless, I think it roughly translates to C's notion of the unit of compilation.

Which is obviously not what people really mean these days, but the phrase stuck. The early Xp'ers even found it an issue back then.

For a while people tried to push the term "micro tests", but that didn't really take off.

I agree with Gerard Mezaros and Martin Fowler and typically follow their (very mainstream) definitions on this stuff. Integration and functional testing have their own ambiguities too, it's definitely a frustrating situation to not have solidly defined foundational terms.

Re: Multiple assertions are fine in a unit test

#290

Post by switchbak »

sdenton4 wrote: post #281
teddyh wrote: post #265

Earlier quoted context omitted.

Contrary viewpoint: Integrated Tests Are A Scam (J.B. Rainsberger): https://www.youtube.com/watch?v=fhFa4tkFUFw

Watched a bit of this... It's typical test-driven zealotry; the main criticism of integration tests seems to be that they don't force your hand in system design in the way that unit tests do? Which seems very silly, but then, I'm not a person who goes to conferences about testing philosophy.

Did you miss his follow-up? "Integration tests are a scam is a scam". For real. I like J.B., but I think he muddies the water too much and overall understanding suffers.
Post reply on HN