Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Database mocks are not worth it

shayon.dev

Post reply on HN
Print view View original

241–250 of 268 posts

Re: Database mocks are not worth it

#241

Post by withinboredom »

9rx wrote: post #236
withinboredom wrote: post #235

Earlier quoted context omitted.

Heh, when you are working on hardware, you usually build a real test device and yes, actually cause real hardware faults. Mocks or tests will not prepare you for the real thing; as the hardware fault you detect is usually just the surface of the problem. Let's examine a practical example where a disk becomes full. Suddenly, file creation will fail, as will writes, yet, how do you handle that? In isolation, you may mo…

> Heh, when you are working on hardware, you usually build a real test device All software works on hardware. Your web application doesn't need a test device, though. The hardware is already tested. You can treat it as an integration point. But even if you did create a test device for whatever reason, that's a mock! Which you say is to be avoided, and that there are better ways, without sharing what those better ways…

> Which you say is to be avoided, and that there are better ways, without sharing what those better ways are...

That's because it would better fit in a book than an HN comment, not because I don't want to answer. Basically the gist is to write "obviously correct" code that doesn't need to be tested, along with an architecture that lends itself to being testable without mocks.

Most people tend to write an interface and then inject a concrete type that could also be a mock. I've seen tests written this way that need to mock out 20-60 things just to test the one thing they want to test.

In most web frameworks I've worked with, this is mostly unavoidable as most frameworks provide dependency injection that is natural to mock.

If you aren't using a framework that has an opinion on how things should work, mocks can be avoided completely through different techniques, such as test harnesses, replacing dependencies with alternative implementations (such as in-memory queues instead of cloud services, sqlite instead of heavy databases, etc), etc. Sometimes you don't have any choice but to not use mocks, for example, distributed systems usually avoid mocks for certain kinds of tests because they simply can't be emulated very well -- either due to latency, network partitioning, or network failures that are just too numerous to mock out (similar to the disk issue I was referring too earlier). You don't know if a node is down, or the cable got cut, and need to behave appropriately to avoid split-brain scenarios. In these cases, test harnesses that can emulate specific scenarios are much better.

Re: Database mocks are not worth it

#242

Post by hamandcheese »

arp242 wrote: post #186
hamandcheese wrote: post #79

My #1 rule: tests should "just work". A new contributor should not have to run any setup, just a single test command. I don't care what kind of tests they are or what you prefer to call them as long as they work (in CI AND locally). The vast majority of projects fail at this, often because some external dependency like a database than needs some manual set up. To me, mocking out the db is a relatively cheap way to ac…

I agree on the "just work" principle, but stuff like "docker compose" is easy enough to use, and "standard enough". Typing "docker compose up -d" once isn't that much effort. And make sure the connection string is configurable through an environment variable so people can use $something_else if they really want/need to. Practicality beats purity, and all of that. Especially when it saves quite of complexity in the co…

The problem with this attitude (and please don't take this as a personal attack) is once a few people with this attitude come through a codebase and make their "practicality" compromises, the codebase devolves into mediocrity.

For a little more context, I work on dev tools for an org of ~400 engineers. There is always someone trying to do things the quick and dirty way and move on without consideration for how the other 399 people may be impacted.

Re: Database mocks are not worth it

#243

Post by d0mine »

ninetyninenine wrote: post #126

Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell). Mocks come from a place where logic is heavily intertwined with IO. It means your code is so coupled that you can’t test logic without touching IO. IO should be a dumb layer as much as possible. It should b…

Here's one of the ways to test IO https://pypi.org/project/pytest-recording/

It can work very well in practice.

Re: Database mocks are not worth it

#244

Post by diddid »

bccdee wrote: post #71
diddid wrote: post #49

The whole point of the mocking the database is to not test the database! If you need to test the database then test the database! Just like mocking an API can hide hidden issues with the API… which is again the exact point of mocking the API. This article should really be named “mocking your database isn’t testing your database” which seems like it should be obvious.

The point of mocking the database is to avoid the hassle of spinning up a database to test against. If we ignore that, why not test our use of the database? Are we disinterested in the schema and query errors we might catch? Unit testing is useful because (a) it's easier to test all the code paths in a unit when that unit is isolated, and (b) unit tests don't need to be modified in response to changes in unrelated mo…

The point of mocking the DB isn’t to avoid hassle, it’s to not test your dependencies. Technically the calling code might not even know it’s a DB, and it might not even care, or it might be a DB sometimes or an API other times, or even a command line once in awhile. They are only tightly coupled if you write it that way. And yes, we would be disinterested in the schema and query errors we might catch because that’s not the point of the test.

Re: Database mocks are not worth it

#245

Post by hinkley »

bluGill wrote: post #184
hinkley wrote: post #175

Earlier quoted context omitted.

Too many people use fakes and call them mocks. Which makes the conversations quite tortured.

I've beconeea fan of good fakes. Good fakes closely resemble the real service but let me query useful things in tests. Mocks just tell me I called a function without concern for the 10 different ways to doethe thing.

If your mocks are hard to write you may have your fanout too wide and would do better with squaring up your dependency tree a bit. I have a rule of 5 for DI frameworks. If any module needs more than 5 other modules you should start looking to refactor to push dependencies down the stack.

What I see with Fakes too often is that they are difficult to write so people reuse them, end up writing too many tests that only test the fake, and couple a bunch of tests to each other which makes refactoring fraught. Anyone who makes refactoring difficult is an enemy to your project, not an asset.

Mocks make DAMP easier. You don’t care how the other function turns “foo” into “bar”, you just care what happens to this function when it gets “bar” as a response. That’s the surface area.

There’s a lot of people in this response chain who are very obviously conflating “unit tests” with all tests. Unit tests don’t care about integration. That’s integration tests. I liken testing to plumbing. Unit tests are similar to quality control at a pipe factory. You verify that the pipes aren’t damaged prior to assembly. The plumber inspects them before installing, and inspects their work as they bond or solder them together. But at the end of the day your plumber still has to turn on the faucet and make sure water 1) comes out the taps and 2) stays in the pipes everywhere but the sink, otherwise they haven’t done their job.

Unit testing the individual pipes is necessary but insufficient. You have to verify the connections and the end to end system still, but more work up front saves you some embarrassing rework at the end.

Re: Database mocks are not worth it

#246

Post by viccis »

globular-toast wrote: post #112
viccis wrote: post #105

Can anyone here who might know more than me explain the difference between dependency injection and the author's suggested pattern at the end? Is it just the same thing? Seems like that's the most common way I see to isolate external services and stub them for testing.

Dependency injection is where functions explicitly receive abstract dependencies that they need to do their job. This is usually like a abstract repository or factory or something. In statically typed languages you'll see things like AbstractRepository etc. When the function is called it is passed a concrete implementation, like a PostgresRepository, while in testing it might be passed a fake like FakeRepository. Thi…

Makes sense. I do have a working understanding of DI and strongly prefer to use it (especially after having to write some unit tests for a external dependency heavy code base not designed with it earlier in the month), but I was just not quite understanding how those RSpec tests at the bottom were working with the instance_double. But looking at it closer now, there's no providing that stub to a function, it's just being created and used immediately.

It seems like you could use instance_double with DI though, as basically an alternative factory for your dependencies when you want to give a faked implementation to your function.

It's been ages since I've done much Ruby, but I recall them being particularly prone to monkeypatching and similar metaprogramming mocking strategies over DI or inversion of control in generation though...

Re: Database mocks are not worth it

#247

Post by winrid »

LtWorf wrote: post #225
winrid wrote: post #219

Earlier quoted context omitted.

PG and Mongo will vertically scale about the same depending on your queries. They were probably using sharding with tiny instances which is dumb. Also, large documents doesn't really hurt performance with mongo, except maybe with writes, or with large array fields due to replication implication.

Not large documents, but documents where the most common operation is to add a number to a list of numbers.

Yeah that's a bad idea. You should instead just create a doc for each number. Each time you add a number the entire list needs copied to disk and to each secondary, so the cost grows quickly with each write.

Re: Database mocks are not worth it

#248

Post by LtWorf »

winrid wrote: post #247
LtWorf wrote: post #225

Earlier quoted context omitted.

Not large documents, but documents where the most common operation is to add a number to a list of numbers.

Yeah that's a bad idea. You should instead just create a doc for each number. Each time you add a number the entire list needs copied to disk and to each secondary, so the cost grows quickly with each write.

I know, but the "seniors" in that place didn't

Re: Database mocks are not worth it

#249

Post by dinosaurdynasty »

peterldowns wrote: post #130
dinosaurdynasty wrote: post #100

Earlier quoted context omitted.

https://eradman.com/ephemeralpg/ (for Postgres) plus making sure your test suite is parallel works wonders.

This is better than nothing but if you have to run migrations before your database is usable, you want an approach like the one I use in pgtestdb — use one server for all your tests, but give them each different databases by cloning from one that's already set up with all the schemas correctly migrated. https://github.com/peterldowns/pgtestdb

I did something similar with pg_tmp as well. I also had a lot of tests around migrations, so there was a lot of tests that would 1) ask for a DB of version x 2) insert things into the DB, etc 3) migrate to version y 4) ensure the DB is appropriate (note: this was not a SaaS).

Re: Database mocks are not worth it

#250

Post by 9rx »

withinboredom wrote: post #241
9rx wrote: post #236

Earlier quoted context omitted.

> Heh, when you are working on hardware, you usually build a real test device All software works on hardware. Your web application doesn't need a test device, though. The hardware is already tested. You can treat it as an integration point. But even if you did create a test device for whatever reason, that's a mock! Which you say is to be avoided, and that there are better ways, without sharing what those better ways…

> Which you say is to be avoided, and that there are better ways, without sharing what those better ways are... That's because it would better fit in a book than an HN comment, not because I don't want to answer. Basically the gist is to write "obviously correct" code that doesn't need to be tested, along with an architecture that lends itself to being testable without mocks. Most people tend to write an interface an…

> Basically the gist is to write "obviously correct" code that doesn't need to be tested

I don't see how that follows. The purpose of testing is to document the code for the understanding of future developers, not to prove correctness. The only 'correctness' a test proves is that the documentation is true. Which is still incredibly useful, as I am sure you are painfully aware if you have ever dealt with legacy forms of documentation (e.g. plain text files, Word documents, HTML, etc.) that quickly become out of date, but is not a statement about the code itself.

> such as test harnesses, replacing dependencies with alternative implementations (such as in-memory queues instead of cloud services, sqlite instead of heavy databases, etc), etc.

These are all mocks, ultimately. Some desperately try to give them different names, but it is all the same at the end of the day.

Post reply on HN