Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to test without mocking

amazingcto.com

Post reply on HN
Print view View original

141–150 of 225 posts

Re: How to test without mocking

#141

Post by edrenova »

_hl_ wrote: post #17
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…

The ideal experience is that you anonymize prod and sync it locally. Whether it's for testing or debugging, it's the only way to get representative data.

When you write mock data, you almost always write "happy path" data that usually just works. But prod data is messy and chaotic which is really hard to replicate manually.

This is actually exactly what we do at Neosync (https://github.com/nucleuscloud/neosync). We help you anonymize your prod data and then sync it across environments. You can also generate synthetic data as well. We take care of all of the orchestration. And Neosync is open source.

(for transparency: I'm one of the co-founders)

Re: How to test without mocking

#142

Post by looperhacks »

As a Java (mostly Spring) dev, I use mocks a lot to separate different components from each other, if I only want to test one of them. If your code only contains tests that mock other things, you're missing something, as others have pointed out. But just because you have a good coverage of integration testing, doesn't mean that writing isolated unit tests are bad. I find it much easier to diagnose a problem in a tight unit test than in a integration test that covers half the project.

Some criticism to the article:

The "more unit testing" section reminds me of junior devs asking why they can't test private methods in Java. If I'm testing a unit, I want to test the contract it promises (in this case, a method that does some checks and then sends something). That the behavior is split between multiple methods is an implementation detail, and writing tests around that makes changes harder (now I can't refactor the methods without also having to update the tests, even if the contract doesn't change) and it doesn't even test the contract! (There's nothing that makes sure that the mail is actually sent - we could be testing methods that aren't used by anything but the test code)

For the "easier to test IO" section: just don't. Your tests now depend on some in-memory implementation that will behave differently than the real thing. That's just mocking with extra steps, you still don't know whether your application will work. If you want to do io, do the real io

"Separation of logic and IO": this is in general the right thing to do, but the way it's described is weird. First, it does the same as in the "more unit testing" section with the same problems. Then, the code is refactored until it's barely understandable and the article even admits it with the Greenspan quote. In the end, the production code is worse, just to ... Not test whether there's actually some code doing the IO.

I actually think there are some good ideas in there: separating the logic from the IO (and treating them as separate units) is important, not just for better testability, but also for easier refactoring and (if done with care) to be easier to reason about. In the end, you will need both unit and integration tests (and if your system is large enough, e2e tests). Whether you're using mocks for your unit tests or not, doesn't make much of a difference in the grand picture.

Just don't mock stuff in integration or e2e if you absolutely can't prevent it.

Re: How to test without mocking

#143

Post by bunderbunder »

dionian wrote: post #137
bunderbunder wrote: post #132

I used to love mocks, once upon a time. Nowadays, though, I internally sigh when I see them. I've come to the opinion that test doubles of any kind should be used as a last resort. They're a very useful tool for hacking testability into legacy code that's not particularly testable. But in a newer codebase they should be treated as a code smell. Code that needs mocks to be tested tends to be code that is overly statef…

Seconded. I shudder when I think back to the "test driven development" days when I wrote so much throwaway test code. Later, you try to refactor the app and it's another 50% of effort to update all the tests. The solution is to avoid it in the way you described.

I'm curious if you've seen Ian Cooper's excellent talk, "TDD, Where Did It All Go Wrong?" https://www.youtube.com/watch?v=EZ05e7EMOLM

One of his key points is that throwaway test code is only a problem if you neglect to throw it away.

Re: How to test without mocking

#144

Post by d0mine »

solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

To test against real 3rd-party http API from time to time, and to generate mocks automatically for the same tests, you could use VCR https://pypi.org/project/vcrpy/

Re: How to test without mocking

#145

Post by jph00 »

whack wrote: post #88
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

> If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at worst. Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to y…

I have a custom frontend for GitHub using the GitHub API (https://github.com/fastai/ghapi/) and don't use mocks - I test using the real API. I've had very occasional, but not enough to ever cause any real issues.

I don't find mocks for this kind of thing very helpful, because what you're really testing for are things like changes to how an API changes over time -- you need real API calls to see this.

Re: How to test without mocking

#146

Post by d0mine »

UK-Al05 wrote: post #42
mehagar wrote: post #33

Earlier quoted context omitted.

I agree that if you need to write mocks, it's likely that your interfaces are poorly defined. This is one of the claimed benefits of test driven development - writing the tests first forces you to design the code in a way that cleanly separates modules so they can be tested.

You have to mock/fake when modules call dependencies. Your way means you only ever have siblings. With an orchestrator pulling results out of one module and pushing it into another.

Code that uses hexagonal architecture/dependency inversion requires less mocks in their tests.

Re: How to test without mocking

#147

Post by 0xbadcafebee »

Mocks aren't an anti-pattern. Anti-patterns are a "common response to a recurring problem, usually ineffective, risking being highly counterproductive". On the contrary, mocks are a common response to a recurring problem which are often effective and have no greater risk than a great many alternative testing methodologies. They do solve problems and they are useful. But like literally anything else in the universe: it depends, and they don't solve every problem.

You wanna know how to test without mocking? Use any kind of test. Seriously, just make a test. I don't care what kind of test it is, just have one. When you notice a problem your testing doesn't catch, improve your testing. Rinse, repeat. I don't care what kind of 10x rockstar uber-genius you think you are, you're going to be doing this anyway no matter what super amazing testing strategy you come up with, so just start on it now. Are there some ways of testing that are more effective than others? Yes, but it depends. If testing were simple, easy, straightforward and universal we wouldn't be debating how to do it.

(about 99% of the time I'm disappointed in these clickbait blog posts upvoted on HN. they are shallow and brief (it's a blog post, not a book), yet quite often dismissive of perfectly reasonable alternatives, and in the absence of any other information, misleading. it would be better to just describe the problem and how the author solved it, and leave out the clickbaity sweeping generalizations and proclamations)

Re: How to test without mocking

#149

Post by koonsolo »

solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

You could still use stubs instead of mocks in those cases.

Re: How to test without mocking

#150

Post by timsehn »

Pxtl wrote: post #122
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

The problem with the CRUD situations is I haven't seen a database that resets to precondition states quickly.

We built a MySQL-compatible one and are building a Postgres-compatible one :-) Free and open source.

https://github.com/dolthub/dolt

We use the reset functionality to speed up our tests.

https://www.dolthub.com/blog/2022-06-10-enginetest-perf/

Post reply on HN