Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database mocks are not worth it

shayon.dev

Post reply on HN
Print view View original

161–170 of 268 posts

Re: Database mocks are not worth it

#161

Post by bcrosby95 »

ninetyninenine wrote: post #126

Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell). Mocks come from a place where logic is heavily intertwined with IO. It means your code is so coupled that you can’t test logic without touching IO. IO should be a dumb layer as much as possible. It should b…

> IO should be a dumb layer as much as possible. It should be Extremely general and as simple as fetching and requesting in the stupidest way possible. Then everything else should be covered by unit tests while IO is fundamentally not unit testable.

Another way to put this is to ignore all the cool problems your database can solve for you and hand roll it instead. This also is a recipe for riddling your codebase with data races.

You're trading fairly easy problems for very hard problems. No thanks.

When running tests, optimize your database configuration for running tests against it, and make it possible to run those tests in parallel.

Re: Database mocks are not worth it

#162

Post by nerdponx »

jokesterfr wrote: post #125

We do both at work (~300k sloc, >70% unit test coverage). There are unit tests with mocks on repository calls, and integration testing with a fresh pg database bootstraped with docker and .sql seeds. One is fast to code, one gives more realistic feedbacks on real life execution. Both solve issues. This is not the end of the game tho, as performance and back pressure issues arises whith a successful project... Testing…

Mocking out a repository is IMO a lot less bad than mocking out an actual database connection. That's probably one of the biggest arguments in favor of the repository pattern: the ability to replace the repository with a test double of some kind.

Re: Database mocks are not worth it

#163

Post by marcosdumay »

jcgrillo wrote: post #102
nrr wrote: post #95

I think it's probably worth mentioning that the principal concern for tests should be proving out the application's logic, and unless you're really leaning on your database to be, e.g., a source of type and invariant enforcement for your data, any sort of database-specific testing can be deferred to integration and UAT. I use both the mocked and real database approaches illustrated here because they ultimately focus…

The database is often the thing that enforces the most critical application invariants, and is the primary source of errors when those invariants are violated. For example, "tenant IDs are unique" or "updates to the foobars are strictly serializable". The only thing enforcing these invariants in production is the interplay between your database schema and the queries you execute against it. So unless you exercise the…

There's something very, very wrong in the way we write programs nowadays.

Because yeah, the database is you main source of invariants. But there is no good reason for you application environment not to query the invariants from there and test or prove your code around them.

We do DRY very badly, and the most vocal proponents are the worst... But I don't think this is a good example of the principle failing.

Re: Database mocks are not worth it

#164

Post by pitah1 »

senex wrote: post #4

Does anyone have experience making tests against real databases fast? I resonate with the sentiment of this article, but have struggled to find an alternative that’s fast enough as the test suite grows, isn’t flakey in CI, and is able to share the production schema definition for relevant relations. I’d love to hear more from anyone that’s solved for some of these constraints!

I've taken a stab at making a solution for it via https://github.com/data-catering/data-caterer. It focuses on making integration tests easier by generating data across batch and real-time data sources, whilst maintaining any relationships across the datasets. You can automatically set it to pick up the schema definition from the metadata in your database to generate data for it. Once your app/job/data consumer(s) use the data, you can run data validations to ensure it runs as expected. Then you can clean up the data at the end (including data pushed to downstream data sources) if run in a shared test environment or locally. All of this runs within 60 seconds.

It also gives you the option of running other types of tests such as load/performance/stress testing via generating larger amounts of data.

Re: Database mocks are not worth it

#165

Post by withinboredom »

9rx wrote: post #121
withinboredom wrote: post #23

Earlier quoted context omitted.

Mocks are great for testing specific code paths, but if you need to do that, there is usually a better way of doing it. Mocks hide contractual incongruities. For example, a function that returns either "fizz" or "buzz" that gets mocked out. If it gets changed to also return "bar", and you forget to update the mock, you've got a time-bomb in your code.

> If it gets changed to also return "bar", and you forget to update the mock, you've got a time-bomb in your code. In that case the problem isn't so much that you forgot to update your mock as much as the changed function no longer satisfies the interface. Ensuring that an implementation properly satisfies the interface is not within the scope of tests where mocking would be useful.

> Ensuring that an implementation properly satisfies the interface is not within the scope of tests where mocking would be useful.

As I mentioned, if you need mocks to test a code path, you are probably "doing it wrong" as there are much better ways to do that. Such as refactoring the code so you can test just that code path. Mocks are a code smell, IMHO. In other words, if you need mocks, then the interface/contract is a part of your code -- and you should test it.

Re: Database mocks are not worth it

#166

Post by ninetyninenine »

throwawaymaths wrote: post #145
ninetyninenine wrote: post #126

Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell). Mocks come from a place where logic is heavily intertwined with IO. It means your code is so coupled that you can’t test logic without touching IO. IO should be a dumb layer as much as possible. It should b…

how do you test the imperative shell except with mocks? Suppose you're calling an external API

The imperative shell is untestable with unit tests. You don’t test it this way period.

Think about it. This IO layer should be so thin that all you’re doing is fetching and getting so this layer does a direct external call. When you mock this entire thing is basically what is mocked.

So if you mock or don’t mock if you write your code with the pattern of functional core and imperative shell the imperative shell is what is mocked and is never tested via unit tests regardless. You cannot test this.

Another way to think about it, is like this. Write your code in a highly decoupled way such that all logic or transformations done in your code can operate independently from things you have to mock. Then for the things you have to mock, what’s the point of mocking them in the first place when your logic can be tested independently from the mocks?

What’s the point of unit testing code that touches IO and is supposed to be mocked? You’d just be writing mocks and testing mocks.

Don’t write functions like this:

fetchNumberFromIOAndAdd2

Do write two functions like this:

  fetchNumberFromIO
  addTwo(x)
The later function can be unit tested. The former can’t be unit tested and there’s no point in unit testing it. The former function is what lives in the imperative shell.

What you see in a lot of books are authors promoting patterns like this:

   addTwo(database object)
This kind of pattern forces you mock by coupling IO with logic.

Re: Database mocks are not worth it

#167

Post by ninetyninenine »

tomjakubowski wrote: post #157
throwawaymaths wrote: post #145

Earlier quoted context omitted.

how do you test the imperative shell except with mocks? Suppose you're calling an external API

The jargon around all this stuff is terrible. What I would probably write is, like, a fake version of that API, one which runs out of process from the library or test, and which can be "programmed" from the test set up -- "expect these requests, serve these responses", etc. Is that a mock? I don't know. It sure does resemble one, but it doesn't use dependency injection or other runtime tricks which I'd associate with…

The jargon is terrible because I think you misunderstood my point. You don’t mock period is my philosophy but your talking about mocking an entire api here.

Re: Database mocks are not worth it

#168

Post by breckenedge »

lopatin wrote: post #117

Some valid points here. I’m awaiting a response post in 2 days called “Database mocks are worth it” which will also have some valid points. Perhaps I’m a little burned out by tech blogging lately.

It’s healthy to be skeptical of patterns you read about. Don’t lose that. It’s worse to cargo-cult them.

Re: Database mocks are not worth it

#169

Post by BoiledCabbage »

tomjakubowski wrote: post #157
throwawaymaths wrote: post #145

Earlier quoted context omitted.

how do you test the imperative shell except with mocks? Suppose you're calling an external API

The jargon around all this stuff is terrible. What I would probably write is, like, a fake version of that API, one which runs out of process from the library or test, and which can be "programmed" from the test set up -- "expect these requests, serve these responses", etc. Is that a mock? I don't know. It sure does resemble one, but it doesn't use dependency injection or other runtime tricks which I'd associate with…

Yes that still a mock. And thr point is to structure your code in such a way that there is little to no value left in testing that API call independently, because all of the related logic has been tested in a unit test.

Once you've done that, you can just run a simple integration or end-to-end test to cover that API. You never need to manually mock it (like you suggested), or use a mocking tool.

Re: Database mocks are not worth it

#170

Post by hinkley »

blitzar wrote: post #158
throwawaymaths wrote: post #145

Earlier quoted context omitted.

how do you test the imperative shell except with mocks? Suppose you're calling an external API

How do you test that the external API returns are correct?

That’s not a unit test. That’s not even close to a unit test.
Post reply on HN