Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

301–310 of 348 posts

Re: Multiple assertions are fine in a unit test

#301

Post by Macha »

david_allison wrote: post #270
Macha wrote: post #99

Earlier quoted context omitted.

I've noticed the opposite in a Java codebase I work in. Tests where the test is assertEquals(toJson(someObject), giantJsonBlobFromADifferentFile). Of course the test runner has no idea about formatting strings that happen to be json, so I end up having to copy these out into an editor, formatting them and eyeballing the difference, or for even larger ones having to save them out to files and diff them. And of course…

This can be improved, it'd be worth Googling for a better solution than what you have. https://github.com/skyscreamer/JSONassert seems decent. but it can be done from scratch in a few hours (I'd recommend this if you have 'standardized' fields which you may want to ignore): Move to a matcher library for assertions (Hamcrest is decent), and abstract `toJSON` into the a matcher, rather on the input. This would change t…

So the same team does actually use JSONAssert in some places, but it can still provide unhelpful comparisons.

I'd actually rather people just assert on the fields they need, but it's a larger team than I can push that on.

Re: Multiple assertions are fine in a unit test

#302

Post by tester756 »

lodovic wrote: post #4

Wait - people are doing real http calls in unit tests, over a network, and complaining about multiple asserts in the test code?

define "real"

yes, we perform "somewhat real tests" - tests start the app, fake db, and call HTTP APIs

it's really decent

Re: Multiple assertions are fine in a unit test

#303

Post by ljm »

mollerhoj wrote: post #247
ljm wrote: post #163

Earlier quoted context omitted.

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather…

Yes, you got us rubyists there. :-( Its the unfortunate result of trying to avoid premature optimization and strive for clarity instead. Something thats usually sound advice. Enginnering decisions have tradeoffs. When the testsuite becomes too slow, it might be time to reconsider those tradeoffs. Usually though, I find that to road to fast tests is to reduce/remove slow things (almost always some form of IO) not to c…

I think it’s a sound strategy more often than not, it’s just that RSpec’s DSL can make those trade-offs unclear, especially if you use Rubocop and follow its default RSpec rules.

It just so happens that your tests become IO bound because those small tests in aggregate hammer your DB and the network purely to set up state. So if you only do it once by being more deliberate with your tests, you’re in a better place.

Re: Multiple assertions are fine in a unit test

#304

Post by dalmo3 »

beezlewax wrote: post #171
torginus wrote: post #89

Earlier quoted context omitted.

I remember writing a small .NET test library for that exact problem - You could pass in a lambda with a complex condition, and it evaluated every piece of the expression separately and pretty printed what part of the condition failed. So essentially you could write Assert(()=>width>0 && x + width And you would get: Assertion failed: x is 1500 width is 600 screenWidth is 1920 It used Expression to do the magic. Amazin…

Is there something like this available for javascript?

It might not be the general solution you're looking for, but idiomatic Jest does the job for me.

   expect(width).toBeGreaterThan(0); expect(x+width).toBeLessThan(screenWidth);
When an assertion fails it will tell you: "Expected: X; Received: Y"

Re: Multiple assertions are fine in a unit test

#305

Post by Rexxar »

hbrn wrote: post #234
marcosdumay wrote: post #178

Earlier quoted context omitted.

Tests can carry tech debit, just like any code. They certainly are not identified by it. Tests are one of the ways you have to ensure your code is correct. Consequently, they are business-oriented code that exist to support your program usage, and subject to its requirements. How much assurance you need is completely defined by those requirements. (But how you achieve that assurance isn't, and tests are only one of t…

They are still debt since they don't directly contribute to product value: you can delete all your tests and your software will keep functioning. It doesn't mean it's a debt worth taking, though IME most companies are either taking way too much or way to little. Not treating tests as debt typically leads to over-testing, and it is way worse than under-testing. Also, what you're talking about (business-oriented requir…

You can also delete the source code after compiling it and your software will keep functioning. Does it mean that code don't directly contribute to product value ?

Re: Multiple assertions are fine in a unit test

#306

Post by OJFord »

ljm wrote: post #163
alkonaut wrote: post #116

An assert message says what went wrong, and on which code line. How on earth does it help to make just one? The arrange part might take seconds for a nontrivial test and that would need to be duplicated both in code and execution time to make two asserts. If you painstakingly craft a scenario where you create a rectangle of a specific expected size why wouldn’t it be acceptable to assert both the width and height of…

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather…

> I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’.

I can't speak for Ruby, but what I would call 'clean' and happily dogmatise is that assertions should come at the end, after setup and exercise.

I don't care how many there are, but they come last. I really hate tests that look like:

    setup()
    
    exercise(but_not_for_the_last_time)

    assert state.currently = blah

    state.do_other()
    something(state)

    assert state.now = blegh
And so on. It stinks of an integration test forced into a unit testing framework.

I like them to look like:

    foo = FooFactory(prop="whatever")

    result = do(foo)

    assert result == "bar"
I.e. some setup, something clearly under test, and then the assertion(s) checking the result.

Re: Multiple assertions are fine in a unit test

#307

Post by ckastner »

kragen wrote: post #297
ckastner wrote: post #241

Earlier quoted context omitted.

> One of those people liked to write 600 line cron jobs to do critical business functions. I was a long-time maintainer of Debian's cron, a fork of Vixie cron (all cron implementations I'm aware of are forks of Vixie cron, or its successor, ISC cron). There are a ton of reasons why I wouldn't do this, the primary one being is that cron really just executes jobs, period. It doesn't serialize them, it doesn't check for…

Is Dillon cron a fork of Vixie cron?

Hadn't heard of it before, and it appears not to be.

There indeed exist a few non-Vixie-cron-derivative implementations but as far as I'm aware, all major Linux and BSD distributions use a Vixie cron derivative.

Edit: I see now where I caused confusion. In my original post, I should have said all default cron implementations.

Re: Multiple assertions are fine in a unit test

#308

Post by ethbr0 »

RHSeeger wrote: post #150
ethbr0 wrote: post #141

Earlier quoted context omitted.

> ... where this is not only possible, but the common case. Couldn't not read that in Peter Sellers' voice https://m.youtube.com/watch?v=2yfXgu37iyI&t=2m36s > ... to find a 630 lines long test "case" with 22 nondescript assertions along the way. This is where tech team managers are abrogating their responsibility and job. It's the job of the organization to set policy standards to outlaw things like this. It's the jo…

> It's the job of the developer to cut as many corners of those policies as possible to ship code ASAP. I can't tell if this is supposed to be humor, or if you actually believe it. It's certainly not my job as a developer to ship worse code so that I can release it ASAP. Rather, it's my job to push back against ASAP where it conflicts with writing better code.

You are not most developers.

And furthermore, you are not the developer most non-tech companies want.

Those sorts of companies want to lock the door to the development section, occasionally slide policy from memos under the door, and get software projects delivered on time, without wasting any more thought on how the sausage gets made.

Re: Multiple assertions are fine in a unit test

#309

Post by ljm »

OJFord wrote: post #306
ljm wrote: post #163

Earlier quoted context omitted.

I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. The part that is glossed over is that the test suite takes several hours to run on your machine, so you delegate it to a CI pipeline and then fork out for parallel execution (pun intended) and complex layers of caching so your suite takes 15 minutes rather…

> I’ve seen people take a dogmatic approach to this in Ruby without really applying any critical thought, because one assertion per test means your test is ‘clean’. I can't speak for Ruby, but what I would call 'clean' and happily dogmatise is that assertions should come at the end, after setup and exercise. I don't care how many there are, but they come last. I really hate tests that look like: setup() exercise(but_…

I think even with integration tests they should still be treated similarly - at the end of the day you are setting expectations on an output given a certain input, there’s just a lot more going on in between.

There’s no avoiding it though when you want something end-to-end, or a synthetic test. You’re piling up a whole succession of stateful actions and if you tested them in isolation you would fail to capture bugs that depend on state. In that sense, better to run a ‘signup, authenticate and onboard’ flow in one test instead of breaking it down.

Re: Multiple assertions are fine in a unit test

#310

Post by saghm »

jasonhansel wrote: post #257
hedora wrote: post #214

Earlier quoted context omitted.

Here are a few mistakes I've seen in other frameworks: - Make it possible to disable timeouts. Otherwise, people will need a different runner for integration, long running (e.g., find slow leaks), and benchmark tests. At that point, your runner is automatically just tech debt. - It is probably possible to nest before and afters, and to have more than one nesting per process, either from multiple suites, or due to cla…

I'd add one more: clearly document what determines the order in which tests are run. On the one hand, running tests in any order should produce the same result, and would in any decent test suite. On the other hand, if the order is random or nondeterministic, it's really annoying when 2% of PRs randomly fail CI, not because of any change in the code, but because CI happened to run unrelated tests in an unexpected ord…

The Ruby `minitest` API used to have a way to disable non-deterministic test ordering, but it was intentionally named in a condescending way: https://www.rubydoc.info/gems/minitest/Minitest%2FTest.i_suc...!

I sometimes run into issues not so much due to order dependencies specifically, but due to tests running in parallel sometimes causing failures due to races. It's almost always been way more work to convert a fully serial test suite into a parallel one than it is to just write it that way from the start, so I think there's some merit in having test frameworks default to non-deterministic ordering (or parallel execution if that's feasible) with the ability to disable that and run things serially. I'm not dogmatic enough to think that fully parallel/random order tests are the right choice for every possible use case, but I think there's value in having people first run into the ordering/race issues they're introducing before deciding to run things fully serially so that they hopefully will consider the potential future work needed if they ever decide to reverse that decision.

Post reply on HN