Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multiple assertions are fine in a unit test

stackoverflow.blog

Post reply on HN
Print view View original

311–320 of 348 posts

Re: Multiple assertions are fine in a unit test

#311

Post by commandlinefan »

z9znz wrote: post #197
commandlinefan wrote: post #166

Earlier quoted context omitted.

> when the code it tests is OOP In my experience when code isn’t OOP, that means all static functions with static (I.e. global) data which isn’t hard to test, it’s actually impossible because you can’t mock out the static data.

I didn't downvote you, but I have a hard time either understanding your meaning or imagining the scenario you describe. Can you give an example? OOP functions are usually harder to test because they expect complete objects as arguments, and that tends to require a lot more mocking or fixtures/factories to setup for the test. FP functions typically operate on less complex and more open data structures. You just constr…

> FP functions

Pure functional, yeah, absolutely - that’s not what I usually see though. I see procedural/iterative static functions that connect to static data that connect to live databases and immediately start caching its contents locally.

Re: Multiple assertions are fine in a unit test

#312

Post by kragen »

ckastner wrote: post #307
kragen wrote: post #297

Earlier quoted context omitted.

Is Dillon cron a fork of Vixie cron?

Hadn't heard of it before, and it appears not to be. There indeed exist a few non-Vixie-cron-derivative implementations but as far as I'm aware, all major Linux and BSD distributions use a Vixie cron derivative. Edit: I see now where I caused confusion. In my original post, I should have said all default cron implementations.

I thought Dillon cron was the default cron in Slackware? Hard to be a more major Linux distribution than Slackware, in terms of historical impact if not current popularity.

Re: Multiple assertions are fine in a unit test

#313

Post by RHSeeger »

dmitriid wrote: post #278
RHSeeger wrote: post #274

Earlier quoted context omitted.

>> this is the smallest thing I _can_ test usefully > Then you're testing useless things. We'll have to agree to disagree then. > Testing DB access layer and service layer separately (as units are often defined) Not at all. For me, a unit is a small part of a layer; one method. Testing the various parts in one system/layer is another type of test. Testing that different systems work together is yet another. I tend to…

And the only actual useful tests are functional (depending on how you write them) and above. If those fail, it means that neither your design nor your code works. The absolute vast majority of unit tests are meaningless because you just repeat them again in the higher level tests.

That seems like a silly opinion to me. I use unit tests to make sure that individual units work like I expect them to. And I use them to test edge cases that can be tested separately from their caller. If I had to test all the use cases for each function, all combined together, there number of tests would grow by the multiplication of the partitions of each one, N x M x O x P, ... rather than the sum, plus a much smaller set of tests for how they work together (N + M + O + P + N_M + M_O + O_P, etc). It's much simpler to thoroughly test each unit. Then test how they work together.

Re: Multiple assertions are fine in a unit test

#314

Post by flavius29663 »

alkonaut wrote: post #263
flavius29663 wrote: post #190

Earlier quoted context omitted.

That can be considered one logical assertion though. You're asserting the size of the rectangle. You can even extract an assert helper function AssertRectangleSize(20,10)

Exactly. But if I assert N properties of an object is that then one assert logically for any N? At what point does it stop? Applying any rule dogmatically is often bad, and this is no exception. The is that we don’t like lacking rules. Especially it goes to hell when people start adding code analysis to enforce it, and then developers start writing poor code that passes the analysis. One assert imo isn’t even a good…

It shouldn't be dogmatic, but I think it should be something to think about when writing the test or reviewing one. A test should be single responsibility too, in order for it to not be brittle.

I disagree, it can be a good starting point for most cases. You should be able to condense your test in 3 steps (arrange, act, assert) each one a single line, and even if you do not do it because setup is too complicated, it's not worth it for that single test, you assert more things etc., I think the mental exercise to try to think: "can this be made in a 3 line test" is invaluable in writing maintainable tests.

> is that then one assert logically for any N? At what point does it stop?

This is one of the hard things about good tests, they are a little bit of art too. Maybe you can apply the single responsibility like I said before: the test should change for one reason only. By one reason I mean one "person/role": it should change if the CFO of our clients want something differet, or if Mark from IT wants some change.

I am not stressing or enforcing single asserts too much, I feel like tests allow a little bit of leeway in many ways, as long as the decision enhances expressiveness. If the extra lines are not making the test clearer, if a single assert would be clearer for the story that the test is telling, then it should go into a single assert. If I can break the story into multiple stories that still make sense, then I do that, such that each story has it's own strong storyline.

Re: Multiple assertions are fine in a unit test

#315

Post by david_allison »

Macha wrote: post #301
david_allison wrote: post #270

Earlier quoted context omitted.

This can be improved, it'd be worth Googling for a better solution than what you have. https://github.com/skyscreamer/JSONassert seems decent. but it can be done from scratch in a few hours (I'd recommend this if you have 'standardized' fields which you may want to ignore): Move to a matcher library for assertions (Hamcrest is decent), and abstract `toJSON` into the a matcher, rather on the input. This would change t…

So the same team does actually use JSONAssert in some places, but it can still provide unhelpful comparisons. I'd actually rather people just assert on the fields they need, but it's a larger team than I can push that on.

From a "legacy code" perspective, you're better off picking an 'easy win' (Hamcrest). Initially, you're not going to convince a team to change their testing habits if it causes them pain. Your goal is to push a testing methodology which moves closer to the 'ideal' which saves them time.

Hamcrest is a drop-in replacement for `assertEquals`, and provides obvious benefits. Politically, it's easy to convince developers onboard once you show them:

* You just need to change the syntax of an assertion - no thought required

* You (Macha) will take responsibility for improving the formatting of the output, and developers have someone to reach out to to improve their assertions.

From this: you'll get a very small subset of missionaries who will understand the direction that you're pushing the test code in, and will support your efforts (by writing their own matchers and evangelising).

The larger subset of the developer population won't particularly care, but will see the improved output from what you're proposing, and will realise that it's a single line of code to change to reap the benefits.

EDIT: I've added a lint rule into a codebase to guide developers away from `assertEquals()`. Obviously this could backfire, and don't burn your political capital on this issue.

Re: Multiple assertions are fine in a unit test

#316

Post by dmitriid »

RHSeeger wrote: post #313
dmitriid wrote: post #278

Earlier quoted context omitted.

And the only actual useful tests are functional (depending on how you write them) and above. If those fail, it means that neither your design nor your code works. The absolute vast majority of unit tests are meaningless because you just repeat them again in the higher level tests.

That seems like a silly opinion to me. I use unit tests to make sure that individual units work like I expect them to. And I use them to test edge cases that can be tested separately from their caller. If I had to test all the use cases for each function, all combined together, there number of tests would grow by the multiplication of the partitions of each one, N x M x O x P, ... rather than the sum, plus a much sma…

> If I had to test all the use cases for each function, all combined together, there number of tests would grow by the multiplication of the partitions of each one

Why would they? Do these edge cases not appear when the caller is invoked? Do you not test these edge cases and the behavior when the caller is invoked?

As an example: you tested that your db layer doesn't fail when getting certain data and returns response X (or throws exception Y). But your service layer has no idea what to do with this, and so simply fails or falls back to some generic handler.

Does this represent how the app should behave? No. You have to write a functional or an integration test for that exact same data to test that the response is correct. So why write the same thing twice (or more)?

You can see this with Twitter: the backend always returns a proper error description for any situation (e.g. "File too large", or "Video aspect ratio is incorrect"). However, all you see is "Something went wrong, try again later".

> It's much simpler to thoroughly test each unit. Then test how they work together.

Me, telling you: test how they work together, unit tests are usually useless

You: no, this increases the number of tests. Instead, you have to... write at least double the amount of tests: first for the units, and then test the exact same scenarios for the combination of units.

----

Edit: what I'm writing is especially true for typical microservices. It's harder for monoliths, GUI apps etc. But even there: if you write a test for a unit, but then need to write the exact same test for the exact same scenarios to test a combination of units, then those unit tests are useless.

Re: Multiple assertions are fine in a unit test

#317

Post by Too »

ronnier wrote: post #152
jeremyjh wrote: post #143

Earlier quoted context omitted.

Tests are not tech debt. You could have bad, brittle tests that you could consider debt but just having tests isn’t debt. Debt implies there is something you could do about it in the future to pay it down, which isn’t the case for a good test suite.

It’s debt. When you can’t add new features quickly because you have nightmarish tests to fix and you spend more time on the tests than the product, I’d say it’s debt. Especially with the insane mocking setups.

NOT having tests is dept. When you can’t fearlessly add features quickly because you introduce regressions in another end of the product that you didn’t think of, and later have to spend all your time on firefighting because it got deployed to prod.

If your mocks and tests are in the way when introducing features or refactoring, they are likely not on the right level. Too much unit testing of moving internals, rather than public apis, usually being one of the culprits.

Re: Multiple assertions are fine in a unit test

#318

Post by ckastner »

kragen wrote: post #312
ckastner wrote: post #307

Earlier quoted context omitted.

Hadn't heard of it before, and it appears not to be. There indeed exist a few non-Vixie-cron-derivative implementations but as far as I'm aware, all major Linux and BSD distributions use a Vixie cron derivative. Edit: I see now where I caused confusion. In my original post, I should have said all default cron implementations.

I thought Dillon cron was the default cron in Slackware? Hard to be a more major Linux distribution than Slackware, in terms of historical impact if not current popularity.

Could be. Slackware is a popular name, but I would call it "niche" rather than a major distribution. Just my personal view, obviously.

Re: Multiple assertions are fine in a unit test

#319

Post by codeflo »

jmillikin wrote: post #77
codeflo wrote: post #40

Earlier quoted context omitted.

Which ones? I’ve used at least a dozen at this point, across C++, C#, JavaScript, Rust — and all of them throw (the equivalent of) exceptions on assertion failures.

GoogleTest (C++) and Go's built-in testing framework both support non-fatal assertions. They're used for code like this: assert_eq(list.len(), 1) expect_eq(list[0].username, "jdoe") expect_eq(list[0].uid, 1000) The idea being that if multiple properties are incorrect, then all of them will be printed out to the test log.

GoogleTest was the one we used. I forgot, but now that you mention it, I remember the expect variations. We had decided against them. It’s a confusing feature in my opinion. If that’s what people mean by “multiple assertions”, then I at least understand where there coming from.

Re: Multiple assertions are fine in a unit test

#320

Post by branko_d »

Izkata wrote: post #295
__alexs wrote: post #204

Earlier quoted context omitted.

I recently went to the effort of trying to work out where the term unit test came from in some desperate effort to find what a unit was meant to be. After much googling and buying or ancient text books I hit a dead end. At this point I think "unit" is just noise that confuses people into making distinctions that don't exist.

IIRC the "unit" in "unit test" was meant to mean "semantic unit" ("the access module", for example, should be distinct with a well-defined interface that all the tests go through), but very quickly turned into "syntactic units" ("a single function", for example, where the "well-defined interface" ends up just being function arguments/return value) because most people didn't understand what the original proponents mea…

I have a Web API which calls a DB API which calls a stored procedure which executes several SQL statements.

The Web API has a well-defined and documented interface. Is it a “unit”?

Post reply on HN