Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to test without mocking

amazingcto.com

Post reply on HN
Print view View original

131–140 of 225 posts

Re: How to test without mocking

#131

Post by simonw »

whack wrote: post #88
solatic wrote: post #12

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…

> If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at worst. Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to y…

Outside of the payments industry I haven't encountered many sandbox APIs that don't have rate-limits, what are some good ones you've seen of those?

Re: How to test without mocking

#132

Post by bunderbunder »

I used to love mocks, once upon a time. Nowadays, though, I internally sigh when I see them.

I've come to the opinion that test doubles of any kind should be used as a last resort. They're a very useful tool for hacking testability into legacy code that's not particularly testable. But in a newer codebase they should be treated as a code smell. Code that needs mocks to be tested tends to be code that is overly stateful (read: temporally coupled), or that doesn't obey the Law of Demeter, or that does a poor job of pushing I/O to the edge where it belongs. And those are all design elements that make code brittle in ways that mocking can't actually fix; it can only sweep it under the carpet.

Re: How to test without mocking

#134

Post by choeger »

A radical point of view. And as such it is of course wrong ;).

First of all, there are languages where dry-running your code with all parameters mocked is still a valid test run. Python, js, and Perl for instance make it very simple to have a stupid error in the routine that crashes every run.

But more importantly, a unit test usually executes inside the same process as the code. That gives you tremendous introspection capabilities and control over the execution flow. Testing for a specific path or scenario is exactly what you should do there.

Finally, what if not mocks, are in-memory filesystems or databases? They, too, won't show all the behaviors that the real thing will do. And so so test containers or even full dedicated environments. It's all going to be an approximation.

Re: How to test without mocking

#136

Post by mason55 »

arp242 wrote: post #35
_hl_ wrote: post #17

Earlier quoted context omitted.

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…

I feel that trying to maintain "representative testing data" is generally not a good idea; set up the data you want/need in the test instead. Just run PostgreSQL on your local machine, connect to that, setup a new schema for every test (fairly cheap-ish) inside a test database. def Test1: setupdb() obj1 = createObj1() obj2 = createObj2() have = doStuff(obj1, obj2) if have != want: ... def Test1: setupdb() obj = creat…

Yeah the keys to make it all work are

1. It's easy to create the objects you need

2. Your creation functions are well tested so that the rest of your tests can rely on them.

If you have spotty coverage or just poorly defined creation semantics, or it's a bunch of calls to functions all over the place just to set up your test data, then this doesn't work.

But the solution typically isn't "write a bunch of JSON mock test data", it's to solve those problems.

Re: How to test without mocking

#137

Post by dionian »

bunderbunder wrote: post #132

I used to love mocks, once upon a time. Nowadays, though, I internally sigh when I see them. I've come to the opinion that test doubles of any kind should be used as a last resort. They're a very useful tool for hacking testability into legacy code that's not particularly testable. But in a newer codebase they should be treated as a code smell. Code that needs mocks to be tested tends to be code that is overly statef…

Seconded. I shudder when I think back to the "test driven development" days when I wrote so much throwaway test code. Later, you try to refactor the app and it's another 50% of effort to update all the tests. The solution is to avoid it in the way you described.

Re: How to test without mocking

#138

Post by Sohcahtoa82 »

JasonSage wrote: post #32
didiskaka wrote: post #27

Earlier quoted context omitted.

> operate this? Do you spin up a new postgres DB for each unit test? Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". The big thing is to wrap each of your tests in a transaction which gets rolled back after each test. > maintain this, eg have good, representative testing data lying around This is much harder. Maintaining good seed dat…

> Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.

If ANYBODY has quick and easy access to connect to the prod DB, let alone have prod DB creds, you are doing something very wrong.

Re: How to test without mocking

#139

Post by mehagar »

layer8 wrote: post #127
mehagar wrote: post #118

Earlier quoted context omitted.

Unit testing proves correctness in regard to the test written (not necessarily the correctness of the application itself). They're similar in that they are both typically fast to run, and that they check an aspect of the program for correctness.

They typically can only prove correctness for specific input data, and then there’s often still some runtime or environment-dependent chance involved which may cause some fraction of the invocations to fail. Is it correct or not if a single invocation succeeds? How can you be sure?

Unit tests should be independent of the environment they are run in, and pass or fail consistently. Otherwise, it is not a unit test.

Re: How to test without mocking

#140

Post by throwaway894345 »

bunderbunder wrote: post #132

I used to love mocks, once upon a time. Nowadays, though, I internally sigh when I see them. I've come to the opinion that test doubles of any kind should be used as a last resort. They're a very useful tool for hacking testability into legacy code that's not particularly testable. But in a newer codebase they should be treated as a code smell. Code that needs mocks to be tested tends to be code that is overly statef…

I think the answer, like most things, is "it depends". Specifically it depends on the complexity of the thing you're mocking. Mocking a database is a bad idea because there's a ton of complexity incumbent in Postgres that your mocks are masking, so a test that mocks a database isn't actually giving you much confidence that your thing works, but if your interface is a "FooStore" (even one that is backed by a database), you can probably mock that just fine so long as your concrete implementation has "unit tests" with the database in the loop.

Additionally, mocking/faking is often the only way to simulate error conditions. If you are testing a client that calls to a remote service, you will have to handle I/O errors or unexpected responses, and that requires mocking or faking the remote service (or rather, the client side transport stack).

But yeah, I definitely think mocks should be used judiciously, and I _really_ think monkeypatch-based mocking is a travesty (one of the best parts about testing is that it pushes you toward writing maintainable, composable code, and monkey patching removes that incentive--it's also just a lot harder to do correctly).

Post reply on HN