Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database mocks are not worth it

shayon.dev

Post reply on HN
Print view View original

71–80 of 268 posts

Re: Database mocks are not worth it

#71

Post by bccdee »

diddid wrote: post #49

The whole point of the mocking the database is to not test the database! If you need to test the database then test the database! Just like mocking an API can hide hidden issues with the API… which is again the exact point of mocking the API. This article should really be named “mocking your database isn’t testing your database” which seems like it should be obvious.

The point of mocking the database is to avoid the hassle of spinning up a database to test against. If we ignore that, why not test our use of the database? Are we disinterested in the schema and query errors we might catch?

Unit testing is useful because (a) it's easier to test all the code paths in a unit when that unit is isolated, and (b) unit tests don't need to be modified in response to changes in unrelated modules. But (a) is irrelevant since we can easily put dummy data in the DB, and (b) isn't a concern since the relevant DB schemas and queries are tightly coupled to the module in question anyway.

Re: Database mocks are not worth it

#72

Post by Volundr »

mrkeen wrote: post #18

Two points: 1) The subtle issues with constraint violations, default values and indexes are part of the RDBMS, not the testing strategy. The article suggests stepping on these rakes in test so that you hopefully don't step on them in prod. Another way to avoid them in prod (and to also regain the speed/reliability/simplicity of your test suite) is to not use an RDBMS in prod. 2) If you want to persist something, ask…

> constraint violations

You call these rakes and suggest removing them from prod, I call these vital safeguards. Defining constraints at the database layer gives me confidence that these will truly be invariants, and I won't one day discover that I've got millions of rows in my database missing critical data, with no way of knowing what should be there. Been there, done that, no desire to revisit it.

Re: Database mocks are not worth it

#73

Post by isbvhodnvemrwvn »

DanielHB wrote: post #27
btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

The problem with this approach is that SQLite dialect is not the same as most production setups. Even if you use an ORM you often have manual queries or you are using a feature that is only supported in some databases (like geospatial queries)

And sqlite is extremely lax by default (and even non-defaults are not great).

Re: Database mocks are not worth it

#74

Post by mrkeen »

fcmgr wrote: post #68

I've had some good experience with a mix of those approaches, maybe not using mocks per se, but an "in-memory database implementation" (just a wrapper around the hash map that implements the same behaviors as a repository that deals with a real database) on one hand, and testcontainers on the other. (Still, using an in-memory db is way better than mocking, the tests are not coupled to the implementation details/the u…

My take is that your business logic shouldn't know about your storage tech. (Like, dependency inversion 101 right?)

> Still, using an in-memory db is way better than mocking, the tests are not coupled to the implementation details/the underlying model.

Isn't this backwards? The fact that you've backed your storage with a HashMap (which is 100% what I shoot for too) means your service-under-test cannot know if it's talking to an SQL database.

Re: Database mocks are not worth it

#75

Post by samwillis »

ariofrio wrote: post #39
rikthevik wrote: post #16

Earlier quoted context omitted.

> But there should be very few of those if you're using a framework that abstracts away the database. But I really want that database-specific behaviour. :) PostgreSQL does so many amazing things (recursive CTEs, jsonb, etc) that actively make our system better. If there was a fork of Django that optimized for leveraging advanced postgres features, I'd use it.

Does something like PGlite work for your use case? https://pglite.dev/

Unit tests work well with PGlite, at least in the TS/JS world where it natively sits at the moment. You can have a unique Postgres instance for each unit test that's up and running in just a few ms.

It's possible to use PGlite from any language using native Postgres clients and pg-gateway, but you lose some of the nice test DX when it's embed directly in the test code.

I'm hopeful that we can bring PGlite to other platforms, it's being actively worked on.

The other thing I hope we can look at at some point is instant forks of in memory databases, it would make it possible to setup a test db once and then reset it to a known state for each test.

(I work on PGlite)

Re: Database mocks are not worth it

#76

Post by wesselbindt »

bccdee wrote: post #71
diddid wrote: post #49

The whole point of the mocking the database is to not test the database! If you need to test the database then test the database! Just like mocking an API can hide hidden issues with the API… which is again the exact point of mocking the API. This article should really be named “mocking your database isn’t testing your database” which seems like it should be obvious.

The point of mocking the database is to avoid the hassle of spinning up a database to test against. If we ignore that, why not test our use of the database? Are we disinterested in the schema and query errors we might catch? Unit testing is useful because (a) it's easier to test all the code paths in a unit when that unit is isolated, and (b) unit tests don't need to be modified in response to changes in unrelated mo…

> why not test our use of the database

Primarily slowness and brittleness. Tests with side effects are much more likely to be flaky, and flaky tests are the worst. Especially if they're slow and hence hard to debug.

Of course, you do test your use of the database, but in a much smaller test suite. For the business logic tests you just spin up a copy of your app with fake adapters, and you're done. Quick, deterministic tests.

Re: Database mocks are not worth it

#77

Post by p0w3n3d »

of course one can do the query testing separated, but I have run once or twice into problems covered by hibernate, and showing up on sql dialect on real database. Hence as always the answer is: 'it depends'

Re: Database mocks are not worth it

#78

Post by brightball »

andy_ppp wrote: post #13

As per usual Elixir does this correctly and even has a setup that allows all tests to run against the database in a pristine way in parallel: https://hexdocs.pm/ecto_sql/Ecto.Adapters.SQL.Sandbox.html

Yea, I've been working on a side project for a while and I keep building it with Rails because there's so much about Rails that speeds up this particular type of project...

But the testing setup in Elixir is just exemplary compared to everything else I've worked with. I fight myself daily on just rebuilding the entire project in Elixir.

Re: Database mocks are not worth it

#79

Post by hamandcheese »

My #1 rule: tests should "just work". A new contributor should not have to run any setup, just a single test command. I don't care what kind of tests they are or what you prefer to call them as long as they work (in CI AND locally).

The vast majority of projects fail at this, often because some external dependency like a database than needs some manual set up.

To me, mocking out the db is a relatively cheap way to achieve rule #1. Yes, it has pitfalls, but I am unwilling to compromise on rule #1 so it's purely a question of cost vs fully automating db setup and teardown.

Re: Database mocks are not worth it

#80

Post by jmmv »

btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

Came here to say this: I’ve found that using SQLite for tests is a good sweet spot, and maintaining dual DB implementations ensures my business logic remains generic. But there is no replacement for running the tests agains the real database periodically (maybe only in CI for example).

I wrote about this and some more in https://jmmv.dev/2023/07/unit-testing-a-web-service.html

Post reply on HN