Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to test without mocking

amazingcto.com

Post reply on HN
Print view View original

111–120 of 225 posts

Re: How to test without mocking

#111

Post by jiehong »

_hl_ wrote: post #17
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…

Great answers below (test containers for example).

However, it’s not always possible.

For example:

- you use oracle db (takes minutes to start, license, hope the containers run on ARM fine, etc.) - sometimes an in memory fake is just much faster, and can be an official db on its own for people to try the product - your storage might be only available through a library by a third party provider that is not available locally.

Re: How to test without mocking

#112

Post by switchbak »

chii wrote: post #107
whack wrote: post #88

Earlier quoted context omitted.

> If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at worst. Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to y…

> because of mock-reality mismatches. you also need to test the mocks against the real thing separately.

I would prefer that we all try to use this language consistently: https://www.martinfowler.com/articles/mocksArentStubs.html

What you're describing sounds like a fake to me.

Re: How to test without mocking

#113

Post by switchbak »

jakjak123 wrote: post #100
_hl_ wrote: post #17

Earlier quoted context omitted.

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…

TestContainers, or just assume there is a postgres running locally. > - maintain this, eg have good, representative testing data lying around? This can be tricky, but usually my advice is to never even being trying to do write seed data in the database unless its very static. It just gets annoying to maintain and will often break. Try to work out a clean way to setup state in your tests using code, and do not rely on…

This is a big one. A term of art for this is "General Fixture", and for xUnit type testing I consider it to be an anti-pattern.

There's times when a big test fixture can provide value, but it's very context dependent and almost never for smaller/micro tests.

Re: How to test without mocking

#114

Post by switchbak »

convolvatron wrote: post #98
boustrophedon wrote: post #64

Earlier quoted context omitted.

I made https://github.com/boustrophedon/pgtemp to solve this for myself

i dont understand why everyone just doesn't do this unless they are working with really large volumes of test data. it literally takes a fraction of a second to mkdir, call pginit, and open a postgres socket. idk if you've solved this, but PG doesn't like to bind to 0, so you have to manage ports. And I've had issues with processes sticking around if the test driver has crashed (I dont currently, but i'm turning off…

My experience exactly - we use a JVM equivalent, and it's extremely fast to start up and reliable to use.

Start it once across a bunch of suites, and have each suite manage its DB state. Done deal.

Re: How to test without mocking

#116

Post by nineplay »

For some reason this article gives me flashbacks to the new CTO who comes in and declares 'micro-services' or 'containers' as the perfect solution for some problem that no one has actually run into. The article's author has had their pain points, but it doesn't mean all mocking is bad everywhere in every use case.

I wrote some code recently that detects cycle errors in objects with inheritance and I mocked the DB calls.

- Did I test for DB failures? No, but that's not the goal of the tests.

- Could I have refactored the code to not rely on DB calls? Yes, but every refactor risks the introduction of more bugs.

- Could I have launched a temporary DB instance and used that instead? Yes, but there's no obvious reason that would have been easier and cleaner than mocking DB calls.

In python it wasn't hard to implement. It was the first time I'd used the mock library so naturally there was learning overhead but that's unavoidable - any solution would have learning overhead.

Re: How to test without mocking

#117

Post by lowbloodsugar »

evnix wrote: post #79
lowbloodsugar wrote: post #68

“Mocks only test the happy path.” This is a problem with the test authors, not mocks. “All the bugs are when talking to an actual database.” Databases have rules that need to be fillowed, and a lot of those can be tested very quickly with mocks. The combined system can have bugs, so don’t only use mocks. Mocks and unit tests are not a substitute for all the other tests you need to do. How this person can claim to be…

He probably meant it takes more effort to create mocks for all the negative cases. In most cases you won't have the time or the bandwidth to do this. Try mocking DB triggers, views, access rules etc in mocks and you will know why most teams don't bother mocking but use the real thing instead. And about the comment about him being a CTO. Well he is a CTO and you?

Then he should have said that. Is not clear communication a requirement for CTO these days?

Everything you are describing is about actually testing the database. A database is a complex server and things like db triggers and store procedures should be tested isolation too. And then you have integration tests too.

My team just found a bug that wasn’t covered in a unit test. We found it in a long running API test. And so we added a unit test for the specific low level miss, and a quick integration test too.

Re: How to test without mocking

#118

Post by mehagar »

layer8 wrote: post #95
mehagar wrote: post #48

Earlier quoted context omitted.

On that topic, static type checking can effectively be seen as a "unit test" that tests as well as documents the expected types for an interface.

No, static typing proves correctness (with respect to the types), which unit testing doesn’t do.

Unit testing proves correctness in regard to the test written (not necessarily the correctness of the application itself). They're similar in that they are both typically fast to run, and that they check an aspect of the program for correctness.

Re: How to test without mocking

#119

Post by pmarreck »

> When you add UI-driven tests (and you should have some),

I disagree. If you want to send your test suite into the toilet, add a headless browser driver and nondeterministic assertions based on it. Most output that becomes UI can be tested; the rest can be checked by a quick QA.

Re: How to test without mocking

#120

Post by AlexErrant »

_hl_ wrote: post #17
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…

Maybe use pglite (which wraps postgres-wasm)? Readme indicates it can run in-memory.

Doesn't solve your testing-data question, but it'll save you from spinning up a new DB.

https://github.com/electric-sql/pglite/

https://github.com/electric-sql/postgres-wasm

Post reply on HN