Wow, some great examples in here for how to use mocks wrong. I get the impression the author has just never seen tests that use mocks properly, honestly. The various refactorings contained in here are fine, of course, but I see no reason to call the entire use of mocks an anti-pattern. They're a tool, and they need to be used properly. Let's not throw the baby out with the bath water.
How to test without mocking
91–100 of 225 posts
Re: How to test without mocking
#92Anyone who is overly zealous about anything is always wrong in the end. Including testing.
"Why would people mock everything? Why not stand up a real test db and test on it?" Because the test zealous have explicitly declared that EACH test should be atomic. Yes you can find these people at major tech conferences. Each test should mock its own db, web service, etc. Every single time. And it should do that in no more than a few milliseconds, so that the entire project compiles in no more than 2mins, even for the largest and most complex corporate projects. And these tests should be fully end-to-end, even for complex microservices across complex networking architecture.
Some of you may be rolling on the floor laughing at how naive and time-consuming such a project would be.
We all agree such testing is a noble goal. But you need a team of absolute geniuses who do nothing but write "clever" code all day to get there in any sizeable project.
My organization won't hire or pay those people, no matter what they say about having 100% coverage. We just do the best we can, cheat, and lower the targets as necessary.
Re: How to test without mocking
#93Argument by exaggeration:
For car crash tests, we should always use full humans. A test dummy might have a lot of sensors and be constructed to behave like a human in a crash, but you'll never get the full crash details with a doll.
Notice the problem here? This argument does not consider the costs and risks associated with each approach.
For testing, IO is very expensive. It leads to huge CI setups and testsuites that take multiple hours to run. There is no way around this except using some kind of test double.
Re: How to test without mocking
#94> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.
Regressions are bugs.
Re: How to test without mocking
#95> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.
On that topic, static type checking can effectively be seen as a "unit test" that tests as well as documents the expected types for an interface.
No, static typing proves correctness (with respect to the types), which unit testing doesn’t do.
Re: How to test without mocking
#96[deleted]
Re: How to test without mocking
#97The problem is when IOService has edge cases. When building the mock, does it address the edge cases? When you want to find bugs by testing, the tests need to test the real world. So to work, the mock for IOService needs to model the edge cases of IOService. Does it? Does the developer know they need to model the edge cases, or the mock is not helping with finding bugs? Do you even know the edge cases of IOService? W…
I totally agree, there is a balance between what makes sense to mock, and what needs proper integration tests.
Additionally, just using integration tests does not guarantee that edge cases are covered, and you can just as easily write integration tests for happy path, without thinking about the rest.
Re: How to test without mocking
#98Earlier quoted context omitted.
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…
I made https://github.com/boustrophedon/pgtemp to solve this for myself
i dont understand why everyone just doesn't do this unless they are working with really large volumes of test data. it literally takes a fraction of a second to mkdir, call pginit, and open a postgres socket.
idk if you've solved this, but PG doesn't like to bind to 0, so you have to manage ports. And I've had issues with processes sticking around if the test driver has crashed (I dont currently, but i'm turning off setsid in postgres).
Re: How to test without mocking
#99Anyone who is overly zealous about anything is always wrong in the end. Including testing. "Why would people mock everything? Why not stand up a real test db and test on it?" Because the test zealous have explicitly declared that EACH test should be atomic. Yes you can find these people at major tech conferences. Each test should mock its own db, web service, etc. Every single time. And it should do that in no more t…
Lets not forget how long it would take to spin up an enterprise database, even in memory, there are hundreds (or thousands) of tables. Also there can be multiple databases with their own schema, and each require a fair amount of data in some of those tables just to do anything..
Re: How to test without mocking
#100If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at…
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up…
TestContainers, or just assume there is a postgres running locally.
> - maintain this, eg have good, representative testing data lying around?
This can be tricky, but usually my advice is to never even being trying to do write seed data in the database unless its very static. It just gets annoying to maintain and will often break. Try to work out a clean way to setup state in your tests using code, and do not rely on magic auto increment ids. Some of the more effective ways I have found is to f.ex. have every test create a fresh customer, then the test does work on that customer. Avoid tests assuming that the first object you create will get id == 1, makes it very annoying to maintain.