Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

201–210 of 348 posts

Re: Multiple assertions are fine in a unit test

#201

Post by charrondev »

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

The majority of the testing I’ve written have been jest tests and PHPUnit tests, and note PHPUnit is my favourite. It’s easy to built up custom assertions, and all of the in built assertions have the ability to provide an additional failure message during a failure.

Assertions throw an exception and the test runner catches them along with any exceptions thrown by the code in test, marks the test as a failure, and reports the given error message and a full stack trace.

Re: Multiple assertions are fine in a unit test

#202

Post by turblety »

hedora wrote: post #191
TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

> I've never seen a case where it would be hard to tell which assertion failed. There are a set of unit testing frameworks that do everything they can to hide test output (junit), or vomit multiple screens of binary control code emoji soup to stdout (ginkgo), or just hide the actual stdout behind an authwall in a uuid named s3 object (code build). Sadly, the people with the strongest opinions about using a "proper" u…

Yup, it's why I built `just-tap` [1] which trys to minimise as much magic that a lot of these frameworks try to "help" you with.

1. https://github.com/markwylde/just-tap

Re: Multiple assertions are fine in a unit test

#203

Post by hamandcheese »

ljm wrote: post #163
alkonaut wrote: post #116

An assert message says what went wrong, and on which code line. How on earth does it help to make just one? The arrange part might take seconds for a nontrivial test and that would need to be duplicated both in code and execution time to make two asserts. If you painstakingly craft a scenario where you create a rectangle of a specific expected size why wouldn’t it be acceptable to assert both the width and height of…

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather…

> I’ve seen people take a dogmatic approach to this in Ruby

I came up in ruby, heard this, and quickly decided it was stupid.

Re: Multiple assertions are fine in a unit test

#204

Post by __alexs »

bluGill wrote: post #181
RHSeeger wrote: post #153

Earlier quoted context omitted.

> Testing lots of things at the same time means strictly speaking you're writing an integration test rather than a unit test There's nothing wrong with integration tests, but they're not unit tests. It's fine to have both, but the requirements for a good unit test and those for a good integration test diverge. The title of this post, at least, was specific to unit tests.

A unit tests tests one unit. And integration tests covers more than one unit. I think everyone agrees with that, but nobody has defined unit. The longer I program the more I am convinced that the larger your unit the better. The unit tests is a statement that you will never refactor across this line, and that eliminates a lot of flexibility that I want. It turns out that debugging failed integration tests is easy,the…

I recently went to the effort of trying to work out where the term unit test came from in some desperate effort to find what a unit was meant to be.

After much googling and buying or ancient text books I hit a dead end. At this point I think "unit" is just noise that confuses people into making distinctions that don't exist.

Re: Multiple assertions are fine in a unit test

#206

Post by autarch »

turblety wrote: post #202
hedora wrote: post #191

Earlier quoted context omitted.

> I've never seen a case where it would be hard to tell which assertion failed. There are a set of unit testing frameworks that do everything they can to hide test output (junit), or vomit multiple screens of binary control code emoji soup to stdout (ginkgo), or just hide the actual stdout behind an authwall in a uuid named s3 object (code build). Sadly, the people with the strongest opinions about using a "proper" u…

Yup, it's why I built `just-tap` [1] which trys to minimise as much magic that a lot of these frameworks try to "help" you with. 1. https://github.com/markwylde/just-tap

TAP is better than some things, but it has some serious issues that I wrote about on my blog a while back - https://blog.urth.org/2017/01/21/tap-is-great-except-when-it...

Basically it's nearly impossible to fully parse it correctly.

Re: Multiple assertions are fine in a unit test

#207

Post by JackFr »

bluGill wrote: post #181
RHSeeger wrote: post #153

Earlier quoted context omitted.

> Testing lots of things at the same time means strictly speaking you're writing an integration test rather than a unit test There's nothing wrong with integration tests, but they're not unit tests. It's fine to have both, but the requirements for a good unit test and those for a good integration test diverge. The title of this post, at least, was specific to unit tests.

A unit tests tests one unit. And integration tests covers more than one unit. I think everyone agrees with that, but nobody has defined unit. The longer I program the more I am convinced that the larger your unit the better. The unit tests is a statement that you will never refactor across this line, and that eliminates a lot of flexibility that I want. It turns out that debugging failed integration tests is easy,the…

> It turns out that debugging failed integration tests is easy,the bug is in the last thing you changed. Sure the test covers hundreds of lines, but you only changed one.

That’s not true.

A correct change might expose an existing bug which hadn’t been tested or expose flaky behavior which existed but hadn’t been exercised. In both cases the solution is not to revert the correct change, but to fix the buggy behavior.

Re: Multiple assertions are fine in a unit test

#208

Post by 0x445442 »

A test should have as many assertions as needed to test an interface. If you find you’re needing a lot of assertions to do that then I suspect either the interface under test is too large or you’re testing multiple stack frames. In my experience, it’s usually the latter; I call it accidental testing. Those delegate calls should have their own tests.

Re: Multiple assertions are fine in a unit test

#209

Post by xhxivuvy »

TeMPOraL wrote: post #16

> The excellent book xUnit Test Patterns describes a test smell named Assertion Roulette. It describes situations where it may be difficult to determine exactly which assertion caused a test failure. How is that even possible in the first place? The entire job of an assertion is to wave a flag saying "here! condition failed!". In programming languages and test frameworks I worked with, this typically includes providi…

It's not about it being hard to tell which assertion failed. It's about being hard to tell what the cause of the failure was.

When every test calls ->run_base_tests() before running it's own assertion sometimes things fail before you get to the root cause assertion.

The other problem of stacking assertions is that you'll see the first failure only. There may be more failures that give you a better picture of what's happening.

Having each assertion fail separately gives you a clearer picture of what's going wrong.

Fwiw, the book doesn't suggest what the reader is saying. It says what I've said above more or less.

You don't need to always stick to the rule but it generally does improve things to the point I now roll my eyes when I come across tests with stacked assertions and lots of test harness code that runs it's own assertions, I just know I'm in for a fun time.

Re: Multiple assertions are fine in a unit test

#210

Post by cratermoon »

The Go idiom is to use table-driven tests[1]. It's still an evolving practice, so you'll see different variants, but the essence is that you have a slice of your inputs and expected outputs, iterate through the slice, and run the assert(s) on each element.

    var flagtests = []struct {
     in  string
     out string
    }{
     {"%a", "[%a]"},
     {"%-a", "[%-a]"},
     {"%+a", "[%+a]"},
     // additional cases elided
     {"%-1.2abc", "[%-1.2a]bc"},
    }

    func TestFlagParser(t *testing.T) {
     var flagprinter flagPrinter
     for _, tt := range flagtests {
      t.Run(tt.in, func(t *testing.T) {
       s := Sprintf(tt.in, &flagprinter)
       if s != tt.out {
        t.Errorf("got %q, want %q", s, tt.out)
       }
      })
     }
    }
Sometimes there will be an additional field in the test cases to give it a name or description, in which case the assertion will look something like:

    t.Fatalf("%s: expected: %v, got: %v", tt.name, tt.out, got)
Another evolving practice is to use a map instead of a slice, with the map key being the name or description of the test case. This is nice because in Go, order is not specified in iterating over a map, so each time the test runs the cases will run in a different order, which can reveal any order-dependency in the tests.

1 https://dave.cheney.net/2019/05/07/prefer-table-driven-tests

Post reply on HN