Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

151–160 of 348 posts

Re: Multiple assertions are fine in a unit test

#151

Post by HereBeBeasties »

masklinn wrote: post #95
jmillikin wrote: post #91

Earlier quoted context omitted.

That's why the first check is a hard assertion (returning on error), and the others are soft (continuing on error). If the list is empty, then the log will contain one error about the length being zero. If the list has one item but it has the wrong properties, the log will contain two errors.

> That's why the first check is a hard assertion (returning on error), and the others are soft (continuing on error). See that's so completely unclear I utterly missed that there were two different calls there. Doesn't exactly help that the functions are the exact same length, and significantly overlap in naming.

I bet if it were in monospace and you were reading code in context rather than on HN you'd have noticed.

That said, NUnit has much better syntax for this, where you put parallel/multiple assertions like this in an explicit block together: https://docs.nunit.org/articles/nunit/writing-tests/assertio...

Much less boilerplate duplication in the actual test framework, too.

Re: Multiple assertions are fine in a unit test

#152

Post by ronnier »

jeremyjh wrote: post #143
throwaway0asd wrote: post #128

My learning from test automation: * The primary goal of test automation is to prevent regression. A secondary goal can be performance tuning your product. * Tests are tech debt, so don’t waste time with any kind of testing that doesn’t immediately save you time in the near term. * Don’t waste your energy testing code unless you have an extremely good reason. Test the product and let me the product prove the quality o…

Tests are not tech debt. You could have bad, brittle tests that you could consider debt but just having tests isn’t debt. Debt implies there is something you could do about it in the future to pay it down, which isn’t the case for a good test suite.

It’s debt. When you can’t add new features quickly because you have nightmarish tests to fix and you spend more time on the tests than the product, I’d say it’s debt. Especially with the insane mocking setups.

Re: Multiple assertions are fine in a unit test

#153

Post by RHSeeger »

Joeboy wrote: post #136
magic_hamster wrote: post #61

Earlier quoted context omitted.

Not at all. It makes sense in some tests. I addressed the part asking how it's even possible to not know what happened. As for multiple asserts, that is really meaningless. The test case should test one thing. If it requires several asserts that's okay. But having a very long test function with a lot of assertions, is strongly indicating that you're testing more than one thing, and when the test fails it will be hard…

I guess you might be talking about a different language / environment than I'm used to, but even in the 100 assertion test case you get useful tracebacks in python. Testing lots of things at the same time means strictly speaking you're writing an integration test rather than a unit test, but still I don't see how it's a bad test . It's easy and stops buggy PRs going into production. The test failures I see that are a…

> Testing lots of things at the same time means strictly speaking you're writing an integration test rather than a unit test

There's nothing wrong with integration tests, but they're not unit tests. It's fine to have both, but the requirements for a good unit test and those for a good integration test diverge. The title of this post, at least, was specific to unit tests.

Re: Multiple assertions are fine in a unit test

#154

Post by codenesium »

torginus wrote: post #97
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

xUnit is terrible. It has a horrible culture, the author seems to have a god complex. It is overly complex and opinionated in the worst way possible. Many times I searched for 'how do I do something with xUnit' and found a github issue with people struggling with the same thing, and the author flat out refusing to incorporate the feature as it was against his principles. Other times I found that I needed to do was ov…

I'm curious what things you're trying to do that requires you to overload xunit classes? We use xunit for everything and haven't found any gaps so far.

Re: Multiple assertions are fine in a unit test

#155

Post by jeremyjh »

dd82 wrote: post #54
mollerhoj wrote: post #28

Earlier quoted context omitted.

OP asked how any state change would be tested with a single 'assertion' and I provided an answer. Absolute rules are stupid, but our codebase has just short of 10k tests, and very few have more than one assertion. The only reason I can really see to have more than one assertion would be to avoid having to run the setup/teardown multiple times. However, its usually a desirable goal to write code that require little se…

one assert per test seems... as you said, indicative of zealotry. if you already have your object there, why not test for the changes you expect? So you have one test that indicates that a log error is outut. then another that tests that the property X in the return from the error is what you expect. then another test to determine that propery Y in return is what you expect? that to me is wasteful, unclear, bloated.…

If two tests call the same method with the same setup and arguments just to assert two different outcomes I would suggest that is the code smell.

Re: Multiple assertions are fine in a unit test

#156

Post by masklinn »

HereBeBeasties wrote: post #151
masklinn wrote: post #95

Earlier quoted context omitted.

> That's why the first check is a hard assertion (returning on error), and the others are soft (continuing on error). See that's so completely unclear I utterly missed that there were two different calls there. Doesn't exactly help that the functions are the exact same length, and significantly overlap in naming.

I bet if it were in monospace and you were reading code in context rather than on HN you'd have noticed. That said, NUnit has much better syntax for this, where you put parallel/multiple assertions like this in an explicit block together: https://docs.nunit.org/articles/nunit/writing-tests/assertio... Much less boilerplate duplication in the actual test framework, too.

> I bet if it were in monospace

It is.

> and you were reading code in context rather than on HN you'd have noticed.

(X) doubt

Re: Multiple assertions are fine in a unit test

#157

Post by ericvsmith »

danuker wrote: post #33
profunctor wrote: post #30

Earlier quoted context omitted.

If you’re using pytest you just paramaterize the tests and it tells you the exact failing case. Seems to be a basic feature I would be surprised to know doesn’t exist across almost all commonly used frameworks.

We are using unittest. But thanks for bringing it up, it seems it also has "subtest" support, which might be easier than interpolating the error message in some cases: https://docs.python.org/3/library/unittest.html#distinguishi...

Came here to mention unittest.subtest. It's for exactly this case.

Re: Multiple assertions are fine in a unit test

#158

Post by ckp95 »

lifeisstillgood wrote: post #120
_dain_ wrote: post #81

Earlier quoted context omitted.

isn't this just a parametrized test? or do you want to generate the test cases automatically like in property-based testing?

Yes parameterised testing. Just all the packages i have tried seemed to make it awkward to impossible - hence metaclasses

Does this do what you want?

https://github.com/ckp95/pytest-parametrize-cases

    @parametrize_cases(
        Case("handle null", age=None, x="foobar"),
        Case("don't allow under 13s", age=11, x="foobar"),
        Case("or old age pension", age=77, x="wobble"),
        ... # as many as you want
    )
    def test_lots_of_ways_to_fail(age, x):
        with pytest.raises(ValueError):
            function_under_test(age, x)

Re: Multiple assertions are fine in a unit test

#159

Post by jeremyjh »

ronnier wrote: post #152
jeremyjh wrote: post #143

Earlier quoted context omitted.

Tests are not tech debt. You could have bad, brittle tests that you could consider debt but just having tests isn’t debt. Debt implies there is something you could do about it in the future to pay it down, which isn’t the case for a good test suite.

It’s debt. When you can’t add new features quickly because you have nightmarish tests to fix and you spend more time on the tests than the product, I’d say it’s debt. Especially with the insane mocking setups.

Yes, those are the bad tests I was referring to. NOT having tests greatly increases the debt burden of your production code because you cannot refactor with any confidence and so you simply won’t.

Re: Multiple assertions are fine in a unit test

#160

Post by gorjusborg »

The trouble is that the type of developer that blindly follows this sort of shaming/advice is exactly the type that blindly followed someone's rule of thumb to use one assertion per test.

There are no hard and fast rules.

I agree with keeping the number of assertions low, but it isn't the number that matters. Keeping the number of assertions low helps prevent the 'testItWorks()' syndrome.

Oh, it broke, I guess 'it does not work' time to read a 2000 line test written 5 years ago.

Post reply on HN