Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

161–170 of 348 posts

Re: Multiple assertions are fine in a unit test

#161

Post by commandlinefan »

throwaway0asd wrote: post #128

My learning from test automation: * The primary goal of test automation is to prevent regression. A secondary goal can be performance tuning your product. * Tests are tech debt, so don’t waste time with any kind of testing that doesn’t immediately save you time in the near term. * Don’t waste your energy testing code unless you have an extremely good reason. Test the product and let me the product prove the quality o…

* tests are a handy way to run one single function in isolation without having to spin up an instance and navigate the UI, too.

Re: Multiple assertions are fine in a unit test

#162

Post by turtleyacht »

Use mutation testing as well, if it's available for your language. This evaluates the quality of your test suite by automatically switching up your app logic and running the same tests.

Nothing like removing one line, breezing through code review, and bringing down production. "But all tests passed!"

Re: Multiple assertions are fine in a unit test

#163

Post by ljm »

alkonaut wrote: post #116

An assert message says what went wrong, and on which code line. How on earth does it help to make just one? The arrange part might take seconds for a nontrivial test and that would need to be duplicated both in code and execution time to make two asserts. If you painstakingly craft a scenario where you create a rectangle of a specific expected size why wouldn’t it be acceptable to assert both the width and height of…

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’.

The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather than 2 and a half hours. It’s monumentally wasteful and the tests aren’t any easier to follow because of it.

The suite doesn’t have to be that slow, but it’s inevitable when every single assertion requires application state to be rebuilt from scratch, even when no state is expected to change between assertions, especially when you’re just doing assertions like ‘assert http status is 201’ and ‘assert response body is someJson’.

Re: Multiple assertions are fine in a unit test

#164

Post by commandlinefan »

wccrawford wrote: post #2

First off, I do put more than 1 assertion in a test. But it definitely leads to situations where you have to investigate why a test failed, instead of it just being obvious. Like the article, I test 1 thing per test, but sometimes that means multiple assertions about the outcome of a test. IMO there's no point in checking that you got a response in 1 test, and then checking the content/result of that response in anot…

> have to investigate why a test failed

Still better than investigating why the whole system failed, though.

Re: Multiple assertions are fine in a unit test

#165

Post by opminion »

superjan wrote: post #93
torginus wrote: post #89

Earlier quoted context omitted.

I remember writing a small .NET test library for that exact problem - You could pass in a lambda with a complex condition, and it evaluated every piece of the expression separately and pretty printed what part of the condition failed. So essentially you could write Assert(()=>width>0 && x + width And you would get: Assertion failed: x is 1500 width is 600 screenWidth is 1920 It used Expression to do the magic. Amazin…

Is this shared somewhere?

https://stackoverflow.com/a/700325 to write out the internal structure of a LambdaExpression.

If you just want the assignments then it's simpler:

Add to the Evaluate method a test for MemberExpression and then:

The variable name is:

((MemberExpression)expr).Member.Name

The value is:

Expression.Lambda(expr).Compile().DynamicInvoke()

Re: Multiple assertions are fine in a unit test

#166

Post by commandlinefan »

z9znz wrote: post #98

Testing is hard when the code it tests is OOP, has mutations everywhere, and has big functions that do too many things. It's practically impossible to thoroughly test such code with one assertion per test; it would mean having dozens of tests just for one object method. Correspondingly, the fixtures/factories/setup for tests would balloon in number and complexity as well to be able to setup the exact circumstance bei…

> when the code it tests is OOP

In my experience when code isn’t OOP, that means all static functions with static (I.e. global) data which isn’t hard to test, it’s actually impossible because you can’t mock out the static data.

Re: Multiple assertions are fine in a unit test

#167

Post by mollerhoj »

mannykannot wrote: post #107
mollerhoj wrote: post #28

Earlier quoted context omitted.

OP asked how any state change would be tested with a single 'assertion' and I provided an answer. Absolute rules are stupid, but our codebase has just short of 10k tests, and very few have more than one assertion. The only reason I can really see to have more than one assertion would be to avoid having to run the setup/teardown multiple times. However, its usually a desirable goal to write code that require little se…

The amount of setup and teardown necessary to test something is a property of the system under test. It is not susceptible to one's opinion as to how things should be.

There are usually different ways to design a system. Its often the case that designing the system such that it is easy to test (with little setup/teardown) has other benefits too. E.g. It often indicates low coupling and a more simple design.

That being said, there can of course me other tradeoffs e.g. performance and even cases where simple test setups are downright impossible.

Re: Multiple assertions are fine in a unit test

#168

Post by nerdponx »

torginus wrote: post #89
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

I remember writing a small .NET test library for that exact problem - You could pass in a lambda with a complex condition, and it evaluated every piece of the expression separately and pretty printed what part of the condition failed. So essentially you could write Assert(()=>width>0 && x + width And you would get: Assertion failed: x is 1500 width is 600 screenWidth is 1920 It used Expression to do the magic. Amazin…

This is what the Python test framework Pytest does, among many other similar useful and magical things. I believe that the Python developer ecosystem as a whole would be substantially less productive without it.

Re: Multiple assertions are fine in a unit test

#169

Post by TrianguloY »

christophilus wrote: post #39
TrianguloY wrote: post #22

The one assertion per test doesn't mean you need to use only one assertion call but rather that you only need to do one assertion block. Checking everything after a response is considered 1 assertion, no matter how many assert calls you need. The issue is when you use multiple assertions for multiple logic statements: do > assert > do > assert... In that example imagine that you were also checking that the reservatio…

Sometimes, it takes a lot less code to test a particular code by doing it in a long test with multiple assertions. Migrate up -> assert ok -> rollback 1 -> assert ok -> rollback 2 -> assert ok I don’t see much benefit to breaking it up, and you’re testing state changes between each transition, so the entire test is useful and simpler, shorter, and clearer than the alternative.

That test is asserting a flow, not features. How can you be sure that the second rollback fails because that rollback code is wrong and not because the previous two functions made some unexpected changes? Or rather, how can you be sure that the second rollback is ok if you dont know the state it was run from? Maybe the migration set an unexpected flag that made the rollbacks pass that test without working properly.

This is also the reason why tests should be run in arbitrary order, to avoid unexpected interactions due to order.

Flow tests can be useful in some situations, but they should never replace individual feature tests.

Re: Multiple assertions are fine in a unit test

#170

Post by marcosdumay »

magic_hamster wrote: post #51
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

> How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". I envy you for never having seen tests atrocious enough where this is not only possible, but the common case. Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it. Now imagine…

> Depending on language, framework and obviously usage, assertions might not be as informative as providing the basic functionality of failing the test - and that's it.

Well, avoiding ecosystems where people act dumb is a sure way to improve one's life. For a start, you won't need to do stupid things in reaction of your tools.

Yes, it's not always possible. But the practices you create for surviving it are part of the dumb ecosystem survival kit, not part of any best practices BOK.

Post reply on HN