Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database mocks are not worth it

shayon.dev

Post reply on HN
Print view View original

11–20 of 268 posts

Re: Database mocks are not worth it

#11

Post by pdimitar »

senex wrote: post #4

Does anyone have experience making tests against real databases fast? I resonate with the sentiment of this article, but have struggled to find an alternative that’s fast enough as the test suite grows, isn’t flakey in CI, and is able to share the production schema definition for relevant relations. I’d love to hear more from anyone that’s solved for some of these constraints!

On my machine -- which is quite middle range at this point, not even high end -- I get by just fine up to 3000 tests, with Elixir at least. When I was at that contract the ~3200 tests ran in something like 40 seconds.

What kinds of troubles do you have with using a real DB for testing?

Re: Database mocks are not worth it

#12

Post by shayonj »

senex wrote: post #4

Does anyone have experience making tests against real databases fast? I resonate with the sentiment of this article, but have struggled to find an alternative that’s fast enough as the test suite grows, isn’t flakey in CI, and is able to share the production schema definition for relevant relations. I’d love to hear more from anyone that’s solved for some of these constraints!

Doesn’t directly answer your question but at least in Postgres I am curious about UNLOGGED mode and see if it results in faster specs. Trade off being, crash recovery doesn’t work but that’s fine in CI.

There is also something to be said about keeping database transactions atomic (no 3rd party network calls, etc) to keep flakey specs to none. I have some ad hoc thoughts on this, will try to frame it proper in a post.

Re: Database mocks are not worth it

#14

Post by pdimitar »

Deukhoofd wrote: post #8

Yeah, I ran in very much the same issues. Setting up a database for unit tests can be even more of a bother though (especially for CI pipelines, I don't want to have to run a full database there), so I took a middle road, and use my ORM to throw up a temporary in-memory SQLite database that is mostly similar to our actual database. Each unit tests scaffolds its own database, and deletes it when it's done. That allows…

Why a separate DB for each test? Just have only one DB and each test opens a transaction and then rolls it back when it's done. That way you also achieve isolation of tests so they don't interfere with each other.

Re: Database mocks are not worth it

#15

Post by shayonj »

pdimitar wrote: post #5

I thought this was common knowledge and that it became even easier after Docker became a thing? Mocks are wishful thinking incarnate most of the time, though here and there they are absolutely needed (like 3rd party APIs without sandbox environments, or quite expensive API, or most of the time: both). Just pick a task runner -- I use just[0] -- and make a task that brings up both Docker and your containers, then run…

+1 for this being common knowledge but there is still decent bit mocking that happens IME.

Re: Database mocks are not worth it

#16

Post by rikthevik »

btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

> But there should be very few of those if you're using a framework that abstracts away the database.

But I really want that database-specific behaviour. :) PostgreSQL does so many amazing things (recursive CTEs, jsonb, etc) that actively make our system better. If there was a fork of Django that optimized for leveraging advanced postgres features, I'd use it.

Re: Database mocks are not worth it

#17

Post by pdimitar »

btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

I don't know what kind of magic fairy dust Django is but I've found the differences between SQLite and PostgreSQL too big to be worth it, in at least 3 other programming languages that are not Python.

Sounded good at first but we were quickly overwhelmed with false positives and just opted for Postgres in a VM (this was before Docker was a thing).

Re: Database mocks are not worth it

#18

Post by mrkeen »

Two points:

1) The subtle issues with constraint violations, default values and indexes are part of the RDBMS, not the testing strategy. The article suggests stepping on these rakes in test so that you hopefully don't step on them in prod. Another way to avoid them in prod (and to also regain the speed/reliability/simplicity of your test suite) is to not use an RDBMS in prod.

2) If you want to persist something, ask yourself if what you really have is tabular data. Do you have a lot of nulls and/or default values? Catch-all json fields? You probably have objects, not relations. Postgres may have a lot of wonderful features that relate to relations, and an ORM might have a lot of wonderful features (which appear to save you time). But if you stop requiring your objects to be relations at rest, you don't need the features, which is even faster.

Re: Database mocks are not worth it

#20

Post by chrisandchris »

btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

I don't know...

How if you rely e.g. on CASCADE and foreign keys, which are not on by default kn SQLite? I think then things start getting complicated and testing that layer gets difficult.

Post reply on HN