Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to test without mocking

amazingcto.com

Post reply on HN
Print view View original

151–160 of 225 posts

Re: How to test without mocking

#151

Post by slt2021 »

bunderbunder wrote: post #132

I used to love mocks, once upon a time. Nowadays, though, I internally sigh when I see them. I've come to the opinion that test doubles of any kind should be used as a last resort. They're a very useful tool for hacking testability into legacy code that's not particularly testable. But in a newer codebase they should be treated as a code smell. Code that needs mocks to be tested tends to be code that is overly statef…

fully agree with you, mock is a shoehorn to fit unit tests into stateful monoliths with messed up dependencies that cross several stacks and reused in multiple modules.

with better separation of concerns and separation of compute from IO one should not need mocks.

only unit tests + integration e2e tests

Re: How to test without mocking

#152

Post by sevensor »

If you take the article's advice to move everything that's not IO into pure, testable code (which is good), what's left is code that does IO. What are you even testing when you call such a procedure? At that point, it's mostly calls into other people's code. Maybe that's a good place to draw the line on testing things?

Re: How to test without mocking

#153

Post by mvdtnz »

CuriouslyC wrote: post #110
brushfoot wrote: post #74

Bad article. Some of the advice is good, like decoupling I/O and logic where that makes sense. But the general idea of mocking being an anti-pattern is overreach. This kind of thinking is overly rigid/idealistic: > And with Postgres you can easily copy a test database with a random name from a template for each test. So there is your easy setup. > You need to test reality. Instead of mocking, invest in end-to-end (E2…

I've worked with a lot of pro-mocking engineers, and the time they spent on mocks easily outstripped the time a good build engineer would have spent creating a fast reusable test framework using real databases/dummy services/etc. The mocks won not because they were better or more efficient, but because of lack of deeper engineering skill and cargo culting.

That engineer may have spent a couple of dozen hours on their mock. But the engineers who spent time on a test framework that uses real databases will soak up thousands of developer hours in CI time over the next decade.

Re: How to test without mocking

#154

Post by 0xbadcafebee »

solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

> If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests,

Actually that's wrong too. The production database will be different than the "testing Postgres instance", leading to bugs.

It turns out that whatever testing solution you use, if it's not the actual production instance and you're not using real production data, there will be bugs. Even then there's still bugs.

This is the simple truth: you can't catch all the bugs. Just put in Good Enough testing for what you're doing and what you need, and get on with life. Otherwise you will spend 99% of your time just on testing.

Re: How to test without mocking

#155

Post by fire_lake »

solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

Yes! Docker is a wonderful tool for testing.

Re: How to test without mocking

#156

Post by mvdtnz »

jon_richards wrote: post #16
devit wrote: post #13

Earlier quoted context omitted.

Testing both your code and the message system is exactly what you want, since if the message system is broken in a way that upstream didn't catch, you want to learn about it during testing and not production, if possible.

I’m still mad about the time I was told to mock a payment gateway in tests even though they had a testing environment and then got a slew of bug reports from people whose company names had punctuation (and thus failed the name validation the payment gateway was secretly running).

You should be unit testing components that interact with the payment gateway. This could involve dozens of even hundreds of tests, where the gateway should be mocked. These tests should be fast and reliable. In addition, you should have a small suite of integration/E2E tests against a real test instance of the gateway. These tests may be slow and unreliable (because of the real network involved) but catch those hairy issues you failed to find in unit tests.

Also, when your integration suite (or customer reports) discovers that the payment gateway fails on punctuation, another unit test should be added with a mock that responds the same way, and an E2E test added with punctuation in the input data and a failure expectation.

What makes you so certain you would have included punctuation in the input data if the test had not mocked the gateway?

Re: How to test without mocking

#158

Post by andoando »

danielovichdk wrote: post #121
andoando wrote: post #60

Earlier quoted context omitted.

There's a few issues with this IMO: 1. Changes often require changing the functionality of a component, which means many of the current unit tests are bunk and need to be updated. Changes that are simply refactoring but should retain the same behavior, need to update/rewrite the tests, in which case again often requires significant refactoring of the existing tests. 2. Small isolated changes usually require testing e…

Yes changing contracts implies updating tests. They should. Refactoring under the same contract should not lead to refactoring of tests. Unless of course you introduce a new dependency you have to mock ? That's just one example. If your code changes a lot it has nothing to do with tests being hard to change. It has to do with the code it tests changes too often. Poor contracts perhaps. And just like the parent commen…

If your refactoring includes changes to interfaces, different abstractions, logical changes, business logic, then most of your tests need to be effectively rewritten.

The only part where I see unit tests being useful for refactoring is making changes to the internals of a single unit. Its always been more trouble than its worth for me.

In some cases it makes sense, like testing small units that heavy in logic (function that calculates order prices for example, scientific computing, etc). But unit testing every single piece of code has always seemed dogmatic to me (unit tests are good engineering, write unit tests always everywhere). Everything has tradeoffs and as engineers I think our job is to understand the pros and cons and apply them effectively.

Re: How to test without mocking

#159

Post by thiht »

0xbadcafebee wrote: post #154
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

> If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, Actually that's wrong too. The production database will be different than the "testing Postgres instance", leading to bugs. It turns out that whatever testing solution you use, if it's not the actual production instance and you're not using real production data, there will be b…

> The production database will be different than the "testing Postgres instance", leading to bugs.

It never happened to me to be honest. This reads an argument for "if you can’t do perfect, just do it badly" but it’s nonsense. Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work.

And testing your storage layer against the database is probably the most reliable safety net you can add to an app.

Re: How to test without mocking

#160

Post by micvbang »

_hl_ wrote: post #17
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…

I've been on teams where we've done this (very successfully in my opinion!) by creating helper code that automates creating a separate Postgres schema for each test, running all migrations, then running your test function before tearing it all down again. This all runs on CI/CD and developer machines, no credentials to any actual environments.

A major benefit of doing separate schemas for each test is that you can run them in parallel. In my experience, unless you have a metric ton of migrations to run for each test, the fact that your database tests can now run in parallel makes up (by a lot!) for the time you have to spend running the migrations for each test.

EDIT: usually we also make utilities to generate entities with random values, so that it's easy to make a test that e.g. tests that when you search for 5 entities among a set of 50, you only get the 5 that you know happen to match the search criteria.

Post reply on HN