Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database mocks are not worth it

shayon.dev

Post reply on HN
Print view View original

131–140 of 268 posts

Re: Database mocks are not worth it

#131

Post by peterldowns »

avg_dev wrote: post #31
senex wrote: post #4

Does anyone have experience making tests against real databases fast? I resonate with the sentiment of this article, but have struggled to find an alternative that’s fast enough as the test suite grows, isn’t flakey in CI, and is able to share the production schema definition for relevant relations. I’d love to hear more from anyone that’s solved for some of these constraints!

Don't know what language or database you use, but check this out: https://github.com/peterldowns/pgtestdb If you happen to use Postgres, the approach is ultimately portable: it uses Pg database templates (also, regarding perf, the author recommends using a ramdisk and turning off fsync on your test DBs; you'll see this in the project readme). But you’ll have to write the code yourself.

Author here, thanks for linking my project — I hope it's been working well for you!

Re: Database mocks are not worth it

#132

Post by peterldowns »

thom wrote: post #99
avg_dev wrote: post #31

Earlier quoted context omitted.

Don't know what language or database you use, but check this out: https://github.com/peterldowns/pgtestdb If you happen to use Postgres, the approach is ultimately portable: it uses Pg database templates (also, regarding perf, the author recommends using a ramdisk and turning off fsync on your test DBs; you'll see this in the project readme). But you’ll have to write the code yourself.

This is what I do, it has an overhead of about 10-20ms per test and I’ve had zero flakiness. Absolute no brainier from my point of view.

Really glad to hear it's been working for you with zero flakiness! If you ever do run into any trouble, or have any suggestions for improvements, come on over to the github issues page :)

Re: Database mocks are not worth it

#133

Post by peterldowns »

shayonj wrote: post #12
senex wrote: post #4

Does anyone have experience making tests against real databases fast? I resonate with the sentiment of this article, but have struggled to find an alternative that’s fast enough as the test suite grows, isn’t flakey in CI, and is able to share the production schema definition for relevant relations. I’d love to hear more from anyone that’s solved for some of these constraints!

Doesn’t directly answer your question but at least in Postgres I am curious about UNLOGGED mode and see if it results in faster specs. Trade off being, crash recovery doesn’t work but that’s fine in CI. There is also something to be said about keeping database transactions atomic (no 3rd party network calls, etc) to keep flakey specs to none. I have some ad hoc thoughts on this, will try to frame it proper in a post.

Here's what I've found — https://github.com/peterldowns/pgtestdb#how-do-i-make-it-go-...

If you come up with any better options please let me know so I can update this readme!

Re: Database mocks are not worth it

#134

Post by ljm »

ninetyninenine wrote: post #126

Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell). Mocks come from a place where logic is heavily intertwined with IO. It means your code is so coupled that you can’t test logic without touching IO. IO should be a dumb layer as much as possible. It should b…

so what you should do instead is write a load of pure functions and intersperse them with impure functions, and when it comes to testing that, if someone writes a different impure function than the one you wanted, you just fail the test and trigger a post to HN: "don't code with mocks."

Re: Database mocks are not worth it

#135

Post by peterldowns »

jaza wrote: post #96

You should do, and I usually do, both "pure" unit tests (with all I/O - database calls, reading/writing local files, 3rd party API calls, etc - being mocked), and integration tests (with ideally all I/O really happening). More of the former, and less of the latter, in line with the "testing pyramid" approach. There is value in testing "assuming that the database returns 3 rows with Foo IDs and Foo Descriptions, my co…

Well said, agreed. One point from my experience — once you can do "as fast as unit" or "as fast as pure" tests that exercise your database, you can basically stop doing integration tests, because using the real database gives you so much confidence in the correctness of your system.

Re: Database mocks are not worth it

#136

Post by mrkeen »

oasisaimlessly wrote: post #128
mrkeen wrote: post #82

Earlier quoted context omitted.

My coworkers call it "data integrity". At best it's a local maximum for consistency. Right now it's how we reject true facts about the business. Our partner bank tells us a customer deactivated? Our database says "no", someone gets a 500, and we get a support ticket when the customer can't sign up again (because they're still in our system).

If this is a common problem, you should have some sort of "dead letter queue" where events like this can be held until your data model is updated to accept them.

Yep, I keep the dead events around, for when the constraints behave incorrectly.

I also keep the live events around, in case there isn't a constraint exception, but something else goes wrong - upstream/user error causing deletion, or programmer error causing the event to have the wrong effect.

Now I have the events, I can just throw away the schemas and constraints. I mean, if the events and the DB disagree, the DB is wrong.

Re: Database mocks are not worth it

#137

Post by peterldowns »

globular-toast wrote: post #108
nwatson wrote: post #66

Earlier quoted context omitted.

Then just use a database for unit testing as well.

Then you get a slow test suite. It's important to have a fast test suite to be able to do proper test driven development (which I still believe is the most efficient and effective way to write software, in general). Unit tests should be near 100% coverage. That means a lot of tests.

That's not always true — database-backed tests can be extremely fast, see https://github.com/peterldowns/pgtestdb.

Re: Database mocks are not worth it

#138

Post by peterldowns »

brettgriffin wrote: post #70

I'd be really surprised if most applications that rely on databases can't just use a containerized image of their production database in any environment, including CI. In fact, the only example I can really think of is when the production database can't be Dockerized (e.g. a proprietary SaaS database like BigQuery or Snowflake). I'm working on a project that has ~22,000 tests that operate on Docker container of the s…

How are you doing it? Asking because I wrote an open source project (see me ~shilling~ linking it elsewhere in this comment section) for doing this, but it sounds like you've maybe come up with an even faster way.

Re: Database mocks are not worth it

#139

Post by addled »

donjoe wrote: post #92
fcmgr wrote: post #28

Earlier quoted context omitted.

I've had good experience with testcontainers ( https://testcontainers.com/ ) to do that sort of thing.

testcontainers is great. I struggled a bit with testcontainers due to the nature of one container per test which just felt too slow for writing gray/blackbox tests. The startup time for postgres was > 10 seconds. After a bit of experimenting, I am now quite happy with my configuration which allows me to have a snappy, almost instant testing experience. My current setup: - generate a new psql testcontainer _or_ reuse…

If your table setup process starts to get slow like ours, checkout psql TEMPLATE (https://www.postgresql.org/docs/current/manage-ag-templatedb...). Do the setup once to a db with known name, then use it as the template when creating the db for each test.

Re: Database mocks are not worth it

#140

Post by peterldowns »

Lvl999Noob wrote: post #26

I faced problems with flaky unit tests that used a common unit test database for the whole project. Since all the tests ran at once, the tests would sometimes fail because of concurrency issues. I never got the time to look too deeply into it. I wonder if there's a database that works sorta like git, giving one "commit" point at the start of every test and then "branching off" so each test can do its own thing, make…

Yes, with Postgres at least you can do this by using a "template" database — run your migrations on it, then create new databases from that template. It's a very fast operation. For more information, read the Postgres docs:

https://www.postgresql.org/docs/current/manage-ag-templatedb...

Post reply on HN