Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

251–260 of 348 posts

Re: Multiple assertions are fine in a unit test

#251

Post by dmitriid »

RHSeeger wrote: post #199
bluGill wrote: post #181

Earlier quoted context omitted.

A unit tests tests one unit. And integration tests covers more than one unit. I think everyone agrees with that, but nobody has defined unit. The longer I program the more I am convinced that the larger your unit the better. The unit tests is a statement that you will never refactor across this line, and that eliminates a lot of flexibility that I want. It turns out that debugging failed integration tests is easy,the…

> The unit tests is a statement that you will never refactor across this line, and that eliminates a lot of flexibility that I want. I certainly don't see it as that. I see it as "this is the smallest thing I _can_ test usefully". Mind you, those do tend to correlate, but they're not the same thing.

> this is the smallest thing I _can_ test usefully

Then you're testing useless things.

Usefulness is when different parts of a program work together as a coherent whole. Testing DB access layer and service layer separately (as units are often defined) has no meaning (but is often enforced).

Queue in memes about "unit tests with 100% code coverage, no integration tests" https://mobile.twitter.com/thepracticaldev/status/6876720861...

Re: Multiple assertions are fine in a unit test

#252

Post by seadan83 »

bazoom42 wrote: post #182

Unit tests are one of the many great ideas in our industry which have been undermined by people treating it is a set of rituals rather than a tool.

I wonder how much of this is the journeyman problem (aka, the expert beginner)

I believe writing test code is its own skill. Hence, like a coder learning SRP and dogmatically applying it, so does a person that is forced to write unit tests without deep understanding. (And of course, bad abstractions are worse than code duplication)

I think it's very possible to have a developer with 10 yes experience but effectively only 2 years experience building automated test suites. (Particularly if they came from a time before automated testing, or if the testing and automated tests were someone else's job)

Re: Multiple assertions are fine in a unit test

#253

Post by mollerhoj »

tinyspacewizard wrote: post #118
mollerhoj wrote: post #20

Earlier quoted context omitted.

Hopefully the responses are stored locally and replayed on subsequent runs.

Still a bit flaky. In an OOP language mocking is appropriate or in an FP language defunctionalization.

Heh yeah but it can be used to write tests that check your assumptions about a 3 party api. Granted, It'll only fail once the test are rerun without the cache, but it can still be a valuable technique. It can be valuable to have a testsuite that a) helps check assumptions when implementing the connection and b) helps locate what part of it later starts to behave unexpectedly.

Re: Multiple assertions are fine in a unit test

#254

Post by seadan83 »

christophilus wrote: post #39
TrianguloY wrote: post #22

The one assertion per test doesn't mean you need to use only one assertion call but rather that you only need to do one assertion block. Checking everything after a response is considered 1 assertion, no matter how many assert calls you need. The issue is when you use multiple assertions for multiple logic statements: do > assert > do > assert... In that example imagine that you were also checking that the reservatio…

Sometimes, it takes a lot less code to test a particular code by doing it in a long test with multiple assertions. Migrate up -> assert ok -> rollback 1 -> assert ok -> rollback 2 -> assert ok I don’t see much benefit to breaking it up, and you’re testing state changes between each transition, so the entire test is useful and simpler, shorter, and clearer than the alternative.

This can be a path where things do go bad. Let's say thus test pattern is a success and then is replicated for many tests. Now, the schema or migration changes. A small change there now breaks the entire test suite. At this point the number of failing tests only indicates how many hours you will be fixing assertions.

Another failure mode is when test scaffolding builds up. Imagine that migrate up part becoming multiple schemas, or services. It then fails, now finding exactly where to fix the test scaffolding becomes a multi-hit exercise.

I'm not saying the example is bad, but it can put you on a path where if you constantly build on top of it, it can bad (eg, developers that don't care for tests nor test code quality, or just want to go home, and they just add a few assertions, add some scaffolding, copy-paste it all and mutate some assertions for a different table & rinse-wash-repeat across 4 people, 40 hours a week for 3 years...)

Re: Multiple assertions are fine in a unit test

#255

Post by mollerhoj »

scruple wrote: post #187
mollerhoj wrote: post #28

Earlier quoted context omitted.

OP asked how any state change would be tested with a single 'assertion' and I provided an answer. Absolute rules are stupid, but our codebase has just short of 10k tests, and very few have more than one assertion. The only reason I can really see to have more than one assertion would be to avoid having to run the setup/teardown multiple times. However, its usually a desirable goal to write code that require little se…

Interesting. Our (Rails) codebase is around 25,000 tests and less than half have a single assertion. Personally, there's some calculus in my head when I'm writing a test that determines if/when the scenario I'm testing needs multiple assertions.

rspec or minitest? ;-) Could rspecs 'expect change' idiom be the difference?

I find that reducing assertions per spec where I can a good guideline. E.g. combining expect(foo['a']).to eq(1) and expect(foo['b']).to eq(2) into expect(foo).to include('a' => 1, 'b' => 2) yields better error messages.

Re: Multiple assertions are fine in a unit test

#257

Post by jasonhansel »

hedora wrote: post #214
turblety wrote: post #202

Earlier quoted context omitted.

Yup, it's why I built `just-tap` [1] which trys to minimise as much magic that a lot of these frameworks try to "help" you with. 1. https://github.com/markwylde/just-tap

Here are a few mistakes I've seen in other frameworks: - Make it possible to disable timeouts. Otherwise, people will need a different runner for integration, long running (e.g., find slow leaks), and benchmark tests. At that point, your runner is automatically just tech debt. - It is probably possible to nest before and afters, and to have more than one nesting per process, either from multiple suites, or due to cla…

I'd add one more: clearly document what determines the order in which tests are run.

On the one hand, running tests in any order should produce the same result, and would in any decent test suite.

On the other hand, if the order is random or nondeterministic, it's really annoying when 2% of PRs randomly fail CI, not because of any change in the code, but because CI happened to run unrelated tests in an unexpected order.

Re: Multiple assertions are fine in a unit test

#258

Post by [deleted] »

jmillikin wrote: post #5

Is it weird that not only have I never heard of the "rule" this post argues against, but I can't even conceive of a code structure where it would make sense? How would a test suite with one assertion per test work? Do you have all the test logic in a shared fixture and then dozens of single-assertion tests? And does that rule completely rule out the common testing pattern of a "golden checkpoint"? I tried googling fo…

[deleted]

Re: Multiple assertions are fine in a unit test

#259

Post by ziml77 »

It took me a long time to feel okay with all the times I broke the single assertion “rule” after reading Clean Code. In fact I only recently stopped feeling bad at all when I went to reimplement a bunch of abseil’s flat hash map tests in C#. All of the tests assert multiple things. If a project as big as that can have multiple asserts on a basic data structure then I can too

Re: Multiple assertions are fine in a unit test

#260

Post by klysm »

musingsole wrote: post #117
pydry wrote: post #25

Earlier quoted context omitted.

There's a lot of not very competent people in the industry who cling tightly to dogma. Testing (especially unit) is an area of tech weirdly with a lot of dogmatism. I think Uncle Bob is the source of some of it.

I'm convinced if you read Uncle Bob carefully and follow all his suggestions... you'll have completely incapacitated whatever organization you infiltrated.

Then you need to hire consultants to come fix it!
Post reply on HN