Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database mocks are not worth it

shayon.dev

Post reply on HN
Print view View original

21–30 of 268 posts

Re: Database mocks are not worth it

#21

Post by karmakurtisaani »

btilly wrote: post #6
javcasas wrote: post #2

For tests, first use a real database. If you can't because the database just doesn't want to (ahem mongo ahem) then use a fake. If you can't use a fake, stop lying and use a fake. Mocks for databases are extremely brittle and complicated.

Ahem MongoDB? I must admit, I've never understood what MongoDB's real use case is supposed to be. Whenever I've looked at it, there has always been a better alternative. When I've had to use it, I've had to debug performance problems that wouldn't have existed with alternative solutions. It sounds cool. But running software isn't about sounding cool. A decade ago, it was really clear. As https://aphyr.com/posts/284-c…

In my company they use it to store a large number of varying sized jsons. You can create indexes to make search queries lighting fast. It's also been extremely stable.

I wasn't involved in setting it up tho, so can't say anything about how difficult it is to work with on the technical side.

Re: Database mocks are not worth it

#22

Post by herpdyderp »

pdimitar wrote: post #17
btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

I don't know what kind of magic fairy dust Django is but I've found the differences between SQLite and PostgreSQL too big to be worth it, in at least 3 other programming languages that are not Python. Sounded good at first but we were quickly overwhelmed with false positives and just opted for Postgres in a VM (this was before Docker was a thing).

I've discovered the same thing: can't use SQLite to mock PostgreSQL. PGLite looks super promising for this, but I haven't tried it yet at scale.

Re: Database mocks are not worth it

#23

Post by withinboredom »

shayonj wrote: post #15
pdimitar wrote: post #5

I thought this was common knowledge and that it became even easier after Docker became a thing? Mocks are wishful thinking incarnate most of the time, though here and there they are absolutely needed (like 3rd party APIs without sandbox environments, or quite expensive API, or most of the time: both). Just pick a task runner -- I use just[0] -- and make a task that brings up both Docker and your containers, then run…

+1 for this being common knowledge but there is still decent bit mocking that happens IME.

Mocks are great for testing specific code paths, but if you need to do that, there is usually a better way of doing it. Mocks hide contractual incongruities. For example, a function that returns either "fizz" or "buzz" that gets mocked out. If it gets changed to also return "bar", and you forget to update the mock, you've got a time-bomb in your code.

Re: Database mocks are not worth it

#24

Post by Deukhoofd »

pdimitar wrote: post #14
Deukhoofd wrote: post #8

Yeah, I ran in very much the same issues. Setting up a database for unit tests can be even more of a bother though (especially for CI pipelines, I don't want to have to run a full database there), so I took a middle road, and use my ORM to throw up a temporary in-memory SQLite database that is mostly similar to our actual database. Each unit tests scaffolds its own database, and deletes it when it's done. That allows…

Why a separate DB for each test? Just have only one DB and each test opens a transaction and then rolls it back when it's done. That way you also achieve isolation of tests so they don't interfere with each other.

The code being tested also uses transactions internally at times, so it'd mean additional complexity in the code being tested to allow for unit testing, which is not great. In my experience throwing up a database including all tables in an in-memory SQLite db is extremely fast, so it's not really a major concern.

Re: Database mocks are not worth it

#25

Post by 3pt14159 »

btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

I'm surprised you say so btilly. I've found the differences in between SQLite and Postgres to be large enough to bring up issues that are hard to find. Timestamp handling, non-trivial indexes, etc.

Re: Database mocks are not worth it

#26

Post by Lvl999Noob »

I faced problems with flaky unit tests that used a common unit test database for the whole project. Since all the tests ran at once, the tests would sometimes fail because of concurrency issues. I never got the time to look too deeply into it.

I wonder if there's a database that works sorta like git, giving one "commit" point at the start of every test and then "branching off" so each test can do its own thing, make transactions, commit them, whatever, and finally be able to roll the whole thing back.

If that can be done then each unit test can use a full database clone with actual valid data and not just test data that was fast enough to set up before the test ran.

Re: Database mocks are not worth it

#27

Post by DanielHB »

btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

The problem with this approach is that SQLite dialect is not the same as most production setups. Even if you use an ORM you often have manual queries or you are using a feature that is only supported in some databases (like geospatial queries)

Re: Database mocks are not worth it

#28

Post by fcmgr »

pdimitar wrote: post #5

I thought this was common knowledge and that it became even easier after Docker became a thing? Mocks are wishful thinking incarnate most of the time, though here and there they are absolutely needed (like 3rd party APIs without sandbox environments, or quite expensive API, or most of the time: both). Just pick a task runner -- I use just[0] -- and make a task that brings up both Docker and your containers, then run…

I've had good experience with testcontainers (https://testcontainers.com/) to do that sort of thing.

Re: Database mocks are not worth it

#29

Post by TobiasBales »

senex wrote: post #4

Does anyone have experience making tests against real databases fast? I resonate with the sentiment of this article, but have struggled to find an alternative that’s fast enough as the test suite grows, isn’t flakey in CI, and is able to share the production schema definition for relevant relations. I’d love to hear more from anyone that’s solved for some of these constraints!

I think one helpful thing is what rails calls transactional tests (https://guides.rubyonrails.org/testing.html#transactions). It basically does the database setup (migrations and seeds) and then executes the tests in a transaction that is rolled back at the end (and thus never committed). This helps with speed and also ensuring that tests don't have as much accidental codependence as they might otherwise.

If you use read replicas in production code this can become tricky though since the transactions don't commit they never become visible to the reader or even different connections to the same database

Re: Database mocks are not worth it

#30

Post by kerblang »

Was dealing with mocking voodoo garbage this morning as a result of fixing a bug. What a horrible mess just to claim a few lines of "test coverage" while ignoring critical db functionality. I wouldn't mind if the mocking frameworks weren't absurdly complex in their own right.

Unfortunately our "integration" tests are even worse, corrupting data all over and flaky as hell.

Post reply on HN