Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to test without mocking

amazingcto.com

Post reply on HN
Print view View original

121–130 of 225 posts

Re: How to test without mocking

#121

Post by danielovichdk »

andoando wrote: post #60
whoisthemachine wrote: post #15

> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.

There's a few issues with this IMO: 1. Changes often require changing the functionality of a component, which means many of the current unit tests are bunk and need to be updated. Changes that are simply refactoring but should retain the same behavior, need to update/rewrite the tests, in which case again often requires significant refactoring of the existing tests. 2. Small isolated changes usually require testing e…

Yes changing contracts implies updating tests. They should.

Refactoring under the same contract should not lead to refactoring of tests. Unless of course you introduce a new dependency you have to mock ? That's just one example.

If your code changes a lot it has nothing to do with tests being hard to change. It has to do with the code it tests changes too often. Poor contracts perhaps.

And just like the parent comment. Tests are not about finding or solving bugs, they are about regressions and making sure your contracts are correctly implemented.

Re: How to test without mocking

#122

Post by Pxtl »

solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

The problem with the CRUD situations is I haven't seen a database that resets to precondition states quickly.

Re: How to test without mocking

#123

Post by myfavoritetings »

dgb23 wrote: post #89
time0ut wrote: post #47

Maybe I am missing something, but how else would I test various exception handling paths? There is a whole world of errors that can occur during IO. What happens if I get a 500 from that web service call? How does my code handle a timeout? What if the file isn't found? It is often only possible to simulate these scenarios using a mock or similar. These are also code paths you really want to understand.

Put a small data interface around your IO, have it return DATA | NOT_FOUND etc. Then your tests don't need behavioral mocks or DI, they just need the different shapes of data and you test your own code instead of whatever your IO dependency is or some simulation thereof.

isn't that just mocking with extra steps?

Re: How to test without mocking

#124

Post by chillpenguin »

xienze wrote: post #105
arcticbull wrote: post #14

Earlier quoted context omitted.

When you write tests with mocks you almost always at some point end up with tests that test your mocks lol, and tests that test that you wrote the tests you think you wrote -- not the software itself. I’ve never been thrilled by tests that rely on mocking — it usually means you need to re-express your module interface boundary. Mocks for me fall into the class of software I affectionately call “load-bearing paint.” I…

> It’s basically universally the wrong tool for any given job but that really doesn’t stop people. I find mocks useful for testing conditions that are on the periphery and would be a decent amount of trouble to set up. For instance, if I have a REST controller that has a catch all for exceptions that maps everything to a 500 response, I want a test that will cause the DAO layer to throw an exception and test that the…

I agree. I will mock 3rd party APIs sometimes so I can test that my system correctly handles failures. For example, what if I get a 500 response from the API? With my mock I can easily make that happen. If I was using the actual API, I would have no way of forcing a 500 to happen.

Re: How to test without mocking

#125

Post by w10-1 »

How can one write an article about testing that doesn't even mention the invariants you're trying to validate (by construction or testing)? That's the minimum context for addressing any QA solution.

The GoF pattern book did list patterns, but it primarily argued for a simple language about patterns: context, problem, solution, limitations. It's clear.

The blog-o-sphere recipe of click-bait, straw-man, glib advice designed not to guide practice but to project authority (and promise career advancement) is the exact opposite, because it obfuscates.

The point of writing is to give people tools they can apply in their proximal situation.

Are you really testing if your solutions start by refactoring the code to be more testable? That's more like design if not architecture -- excellent, but well beyond scope (and clearly in the CTO's bailiwick).

And as for mocks: they're typically designed to represent subsystems at integration points (not responses to functions or IO/persistence subsystems). How hard is that to say?

The CTO's way is not to win the argument but to lead organizations by teaching applicable principles, providing guardrails, and motivating people to do the right thing.

Sorry to be exasperated and formulaic, but I think we can do better.

Re: How to test without mocking

#126

Post by drewcoo »

switchbak wrote: post #112
chii wrote: post #107

Earlier quoted context omitted.

> because of mock-reality mismatches. you also need to test the mocks against the real thing separately.

I would prefer that we all try to use this language consistently: https://www.martinfowler.com/articles/mocksArentStubs.html What you're describing sounds like a fake to me.

Can't tell from the context that's here.

It's important to have a contract testing layer in place to make sure your test doubles are still behaving like the real thing, though.

Re: How to test without mocking

#127

Post by layer8 »

mehagar wrote: post #118
layer8 wrote: post #95

Earlier quoted context omitted.

No, static typing proves correctness (with respect to the types), which unit testing doesn’t do.

Unit testing proves correctness in regard to the test written (not necessarily the correctness of the application itself). They're similar in that they are both typically fast to run, and that they check an aspect of the program for correctness.

They typically can only prove correctness for specific input data, and then there’s often still some runtime or environment-dependent chance involved which may cause some fraction of the invocations to fail. Is it correct or not if a single invocation succeeds? How can you be sure?

Re: How to test without mocking

#128

Post by graypegg »

sethammons wrote: post #106
graypegg wrote: post #7

Tests are a tool for you, the developer. They have good effects for other people, but developers are the people that directly interact with them. When something fails, it's a developer that has to figure out what change they wrote introduced a regression. They're just tools, not some magic incantation that protects you from bugs. I think the author might be conflating good tests with good enough tests. If IOService i…

> E2E tests are harder to write and interpret when something goes wrong. If the test is hard to debug when it goes wrong, then I assume the system is hard to debug when something goes wrong. Investing in making that debugging easy/easier unlocks more productivity. Of course it matters on how often bugs show up, how often the system changes, the risks of system failure on the business, etc. it may not be worth the pro…

I think it's always going to be harder to debug 1 thing, versus everything, regardless of how a system is built. If you're not mocking anything, then anything could have gone wrong anywhere.

But also, if you're able to fix things effectively from E2E test results due to a focus on debug-ability, then that's great! I think it's just the framing of the article I have trouble with. It's not an all or nothing thing. It's whatever effectively helps the devs involved understand and fix regressions. I haven't seen a case where going all in on E2E tests has made that easier, but I haven't worked everywhere!

Re: How to test without mocking

#129

Post by randomdata »

postalrat wrote: post #104
randomdata wrote: post #87

Earlier quoted context omitted.

Nothing is left undefined. Absent of documentation, most likely something will end up defined by inference. Which is not a good place to be as a developer as you have lost the nuance that went into it originally.

Then how do you separate what is defined and can't change to what can be changed? If everything is defined then nothing can change.

You don’t change what is already defined (even if only by inference). Change is only by amendment. Will you successfully amend the changes without also changing what was previously defined if all you have is inference to go on? Probably not.

That’s assuming change is even necessary. Oftentimes you only need to modify the implementation, which doesn’t change what is defined. A change in implementation has no impact on the outside, at least as long as you have properly covered your bases, which should you should be able to do as long as you have proper documentation. Without documentation, good luck to you.

Re: How to test without mocking

#130

Post by jweir »

I tell my developers that each mock you use costs you $100. Maybe it is worth it, but probably not.

No I don't really charge em – but it gets the idea across that mocks have costs that you don't always see up front.

Post reply on HN