Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database mocks are not worth it

shayon.dev

Post reply on HN
Print view View original

121–130 of 268 posts

Re: Database mocks are not worth it

#121

Post by 9rx »

withinboredom wrote: post #23
shayonj wrote: post #15

Earlier quoted context omitted.

+1 for this being common knowledge but there is still decent bit mocking that happens IME.

Mocks are great for testing specific code paths, but if you need to do that, there is usually a better way of doing it. Mocks hide contractual incongruities. For example, a function that returns either "fizz" or "buzz" that gets mocked out. If it gets changed to also return "bar", and you forget to update the mock, you've got a time-bomb in your code.

> If it gets changed to also return "bar", and you forget to update the mock, you've got a time-bomb in your code.

In that case the problem isn't so much that you forgot to update your mock as much as the changed function no longer satisfies the interface. Ensuring that an implementation properly satisfies the interface is not within the scope of tests where mocking would be useful.

Re: Database mocks are not worth it

#122

Post by ajuc »

I worked on a 2-layer C++ + PL/SQL system with lots of legacy business logic in PL/SQL that was slowly migrated to 3-layer architecture with Java EJB business logic using Hibernate for database operations.

The Java business code still often had to call the old PL/SQL stored procedures, and the state of the application was serialized on the same database using Hibernate. It made for a pretty delicate db transaction handling - in Java it was mostly declarative EJB3 transactions + some custom AOP code, and in PL/SQL we had strict guidelines of never commiting/rollbacking the transaction globally - instead every stored procedure had a savepoint at the beginning and only rollbacked to that savepoint in case of a problem.

Ensuring the system adhered to these guidlines with mocks would be very hard. Instead we wrote a special test-mode-only AOP code that was wrapping around every java -> PL/SQL call in test mode, and checked if the transactions were correctly handled (it added a programatic savepoint before the call and checked if it's still there after the call). It caught some transaction handling errors that would be VERY hard to find otherways.

It also let us find possible deadlocks during testing.

BTW writing tests for PL/SQL made me love Postgres for its transactional DDL. It's very frustrating to make repeatable tests when you can't rollback DDL changes and have to undo them one by one after the test or re-create the whole database.

Re: Database mocks are not worth it

#124

Post by diggan »

lopatin wrote: post #117

Some valid points here. I’m awaiting a response post in 2 days called “Database mocks are worth it” which will also have some valid points. Perhaps I’m a little burned out by tech blogging lately.

I mean, that's kind of engineering for you, choosing between different tradeoffs. I guess reading blogposts with various points is kind of the point, so you can easier know when to apply what, and what tradeoffs they come with.

Re: Database mocks are not worth it

#125

Post by jokesterfr »

We do both at work (~300k sloc, >70% unit test coverage).

There are unit tests with mocks on repository calls, and integration testing with a fresh pg database bootstraped with docker and .sql seeds.

One is fast to code, one gives more realistic feedbacks on real life execution. Both solve issues.

This is not the end of the game tho, as performance and back pressure issues arises whith a successful project... Testing is a long and interesting process.

Re: Database mocks are not worth it

#126

Post by ninetyninenine »

Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell).

Mocks come from a place where logic is heavily intertwined with IO. It means your code is so coupled that you can’t test logic without touching IO.

IO should be a dumb layer as much as possible. It should be Extremely general and as simple as fetching and requesting in the stupidest way possible. Then everything else should be covered by unit tests while IO is fundamentally not unit testable.

If you have a lot of complex sql statements or logic going on in the IO layer (for performance reasons don’t do this if you can’t help it) it means your IO layer needs to be treated like a logical layer. Make sure you have stored procedures and unit tests written in your database IO language. You code should interface only with stored procedures as IO and those stored procedures should be tested as unit tests in the IO layer.

Re: Database mocks are not worth it

#127

Post by lmm »

rikthevik wrote: post #16
btilly wrote: post #3

I've found that replacing the database with in memory SQLite for tests is a sweet spot. Almost as fast as a mock, catches a lot of database issues. And it's really easy to do if you're using something like Django that makes automatically generating database migrations easy. It won't help you with database specific differences. But there should be very few of those if you're using a framework that abstracts away the d…

> But there should be very few of those if you're using a framework that abstracts away the database. But I really want that database-specific behaviour. :) PostgreSQL does so many amazing things (recursive CTEs, jsonb, etc) that actively make our system better. If there was a fork of Django that optimized for leveraging advanced postgres features, I'd use it.

Embedded/in-memory Postgres for testing is easier than you might think.

Re: Database mocks are not worth it

#128

Post by oasisaimlessly »

mrkeen wrote: post #82
Volundr wrote: post #72

Earlier quoted context omitted.

> constraint violations You call these rakes and suggest removing them from prod, I call these vital safeguards. Defining constraints at the database layer gives me confidence that these will truly be invariants, and I won't one day discover that I've got millions of rows in my database missing critical data, with no way of knowing what should be there. Been there, done that, no desire to revisit it.

My coworkers call it "data integrity". At best it's a local maximum for consistency. Right now it's how we reject true facts about the business. Our partner bank tells us a customer deactivated? Our database says "no", someone gets a 500, and we get a support ticket when the customer can't sign up again (because they're still in our system).

If this is a common problem, you should have some sort of "dead letter queue" where events like this can be held until your data model is updated to accept them.

Re: Database mocks are not worth it

#129

Post by SvenL »

withinboredom wrote: post #23
shayonj wrote: post #15

Earlier quoted context omitted.

+1 for this being common knowledge but there is still decent bit mocking that happens IME.

Mocks are great for testing specific code paths, but if you need to do that, there is usually a better way of doing it. Mocks hide contractual incongruities. For example, a function that returns either "fizz" or "buzz" that gets mocked out. If it gets changed to also return "bar", and you forget to update the mock, you've got a time-bomb in your code.

The person who changes it should write a test to verify his change, right? Ultimately the person should write the test in the same file as the previous tests, chances are high it will be seen there. And even if not, the person doing the change should write a test, so no time bomb there?

And yes, people forget to write tests, sure, but even then it would be a time bomb without mocks.

I‘m not a friend of mocks either, but most examples here are not really an issue with mocks.

Re: Database mocks are not worth it

#130

Post by peterldowns »

dinosaurdynasty wrote: post #100
senex wrote: post #4

Does anyone have experience making tests against real databases fast? I resonate with the sentiment of this article, but have struggled to find an alternative that’s fast enough as the test suite grows, isn’t flakey in CI, and is able to share the production schema definition for relevant relations. I’d love to hear more from anyone that’s solved for some of these constraints!

https://eradman.com/ephemeralpg/ (for Postgres) plus making sure your test suite is parallel works wonders.

This is better than nothing but if you have to run migrations before your database is usable, you want an approach like the one I use in pgtestdb — use one server for all your tests, but give them each different databases by cloning from one that's already set up with all the schemas correctly migrated.

https://github.com/peterldowns/pgtestdb

Post reply on HN