Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

31–40 of 348 posts

Re: Multiple assertions are fine in a unit test

#31

Post by lancebeet »

danuker wrote: post #27
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I use the following pattern for testing regexes: expected_positive = [ 'abc', 'def', ...] for text in expected_positive: self.assertTrue(matcher(text), f"Failed: {text}") Before I added the assertion error message, `f"Failed: {text}"`, it was quite difficult to tell WHICH example failed.

That looks like a case where I would use a parameterized test rather than a for loop inside the test.

Re: Multiple assertions are fine in a unit test

#32

Post by olex »

danuker wrote: post #27
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I use the following pattern for testing regexes: expected_positive = [ 'abc', 'def', ...] for text in expected_positive: self.assertTrue(matcher(text), f"Failed: {text}") Before I added the assertion error message, `f"Failed: {text}"`, it was quite difficult to tell WHICH example failed.

This seems like a test design issue to me. Best practice is to avoid for-each loops with assertions within tests - using parametrized tests and feeding the looped values as input is almost always a better option. Figuring out which one failed and why is one advantage it gives you in comparison. Another one is that all inputs will always be tested - your example stops on the first one that fails, and does not evaluate the others after that.

Re: Multiple assertions are fine in a unit test

#33

Post by danuker »

profunctor wrote: post #30
danuker wrote: post #27

Earlier quoted context omitted.

I use the following pattern for testing regexes: expected_positive = [ 'abc', 'def', ...] for text in expected_positive: self.assertTrue(matcher(text), f"Failed: {text}") Before I added the assertion error message, `f"Failed: {text}"`, it was quite difficult to tell WHICH example failed.

If you’re using pytest you just paramaterize the tests and it tells you the exact failing case. Seems to be a basic feature I would be surprised to know doesn’t exist across almost all commonly used frameworks.

We are using unittest.

But thanks for bringing it up, it seems it also has "subtest" support, which might be easier than interpolating the error message in some cases:

https://docs.python.org/3/library/unittest.html#distinguishi...

Re: Multiple assertions are fine in a unit test

#34

Post by spiffytech »

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I've seen several test frameworks that don't abort a test case after the first failed assertion.

When you get many tests each emitting multiple failures because one basic thing broke, the output gets hard to sort through. It's easier when the failures are all eager.

Re: Multiple assertions are fine in a unit test

#36

Post by matthews2 »

littlecranky67 wrote: post #9
jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

> Is it weird that not only have I never heard of the "rule" this post argues against This "rule" is known mostly because it is featured in the "Clean Code" book by Robert C. Martin (Uncle Bob). You should have heard of it ;)

Are there any "rules" in Clean Code that don't need to be disregarded and burned? https://qntm.org/clean

Re: Multiple assertions are fine in a unit test

#37

Post by danuker »

olex wrote: post #32
danuker wrote: post #27

Earlier quoted context omitted.

I use the following pattern for testing regexes: expected_positive = [ 'abc', 'def', ...] for text in expected_positive: self.assertTrue(matcher(text), f"Failed: {text}") Before I added the assertion error message, `f"Failed: {text}"`, it was quite difficult to tell WHICH example failed.

This seems like a test design issue to me. Best practice is to avoid for-each loops with assertions within tests - using parametrized tests and feeding the looped values as input is almost always a better option. Figuring out which one failed and why is one advantage it gives you in comparison. Another one is that all inputs will always be tested - your example stops on the first one that fails, and does not evaluate…

> your example stops on the first one that fails, and does not evaluate the others after that.

I don't think this is a big problem; trying to focus on multiple examples at once is difficult.

It might be a problem if tests are slow and you are forced to work on all of them at once.

But in that case I'd try to make the tests faster (getting rid of network requests, disk/DB access by faking them away or hoisting to the caller).

Re: Multiple assertions are fine in a unit test

#38

Post by lifeisstillgood »

Things I find awkward with unit test (and it might just be me) is Inwant to write a test like

def testLotsOfWaysTofail(): d = {"Handle Null": (None, foobar), "Don't allow under 13 to do it": (11, foobar), "or old age pensioner": (77,wobble)} for ... generate a unit test dynamically here

I have built metaclasses, I have tried many different options. I am sure there is a near solution.

But I never seem to have it right

Re: Multiple assertions are fine in a unit test

#39

Post by christophilus »

TrianguloY wrote: post #22

The one assertion per test doesn't mean you need to use only one assertion call but rather that you only need to do one assertion block. Checking everything after a response is considered 1 assertion, no matter how many assert calls you need. The issue is when you use multiple assertions for multiple logic statements: do > assert > do > assert... In that example imagine that you were also checking that the reservatio…

Sometimes, it takes a lot less code to test a particular code by doing it in a long test with multiple assertions.

Migrate up -> assert ok -> rollback 1 -> assert ok -> rollback 2 -> assert ok

I don’t see much benefit to breaking it up, and you’re testing state changes between each transition, so the entire test is useful and simpler, shorter, and clearer than the alternative.

Re: Multiple assertions are fine in a unit test

#40

Post by codeflo »

spiffytech wrote: post #34
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I've seen several test frameworks that don't abort a test case after the first failed assertion. When you get many tests each emitting multiple failures because one basic thing broke, the output gets hard to sort through. It's easier when the failures are all eager.

Which ones? I’ve used at least a dozen at this point, across C++, C#, JavaScript, Rust — and all of them throw (the equivalent of) exceptions on assertion failures.
Post reply on HN