Stop mocking your system
blog.bitgloss.ro
Stop mocking your system
1–10 of 178 posts
Re: Stop mocking your system
#2Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more.
Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.
Re: Stop mocking your system
#3Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more. Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.
what does unit testing have to do with whether or not you instrument the test with fake responses? those points of contact that you're mocking out at the perimeter, that data will sometimes need to reach a particular function through a collaborator...which you may want to mock?
sometimes the dependency is not a third party, but it may be code that requires a ton of setup (as mentioned in article) that's not worth the cost. it may make sense to just mock at that point to actually test the rest of the business logic in a function. I don't think i'd say "well, that's no longer a unit test!". You can argue that it's a more brittle test, sure.
update: also, i'll be honest that comments like this really rub me the wrong way. This type of dogmatism around what is or isn't unit testing (which is a pretty ill-defined phrase in industry) is something that needs to stop. I think it hurts new practioners in the field who are mislead into black and white thinking.
Re: Stop mocking your system
#4Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more. Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.
Why?
Re: Stop mocking your system
#5One of the most obvious problems with mocking is that the team that develops some code usually also develops the mocks that are used for testing it. So precisely the same misunderstandings will be present in the code as in the mocks. In other words you are not really testing anything.
From my experience most errors are at boundaries between code from different teams. Mocking does not help here.
My favorite form of tests are what I usually call subsystem tests. Try to test as much code as is feasible with each single test.
Usually there are parts of your system that can be expensive to setup or use. For example, creating and filling a real database can be slow. In this case you could use an in memory database. Creating it and filling it with some representative data can be very fast. This database could be used by multiple teams, and is vastly superior to mocking.
A similar approach can be used with other expensive parts like remote procedure calls [0], or input from browsers.
This approach works when you design your system so that it can easily switch between using the real (expensive) resources and the ones that are only used for testing. But that is not very difficult.
[0] Both REST and SOAP are RPC
Re: Stop mocking your system
#6Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more. Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.
[deleted]
Re: Stop mocking your system
#7> But dude, I don’t want to use a real database (or AWS endpoint or rocket launcher) in my tests. Debatable, but fair enough.
Is this debatable? Hermetic tests give you a lot of things for free and I don't really see a reason why you would default to making all of your tests hermetic.
The real thing that this article is touching on is that your tests should test all of the code you write but not code others have written. This sounds wild at first but: do you need to test that Postgres knows how to parse a query? Probably not. I think the postgres team knows how to test and release their database and I don't really need to spend time doing that. Then, the next layer of abstraction: if you use an ORM or some middle layer that abstracts the database do you need to test that the ORM knows how to talk to postgres? In a unit test, likely not. The ORM people have provided you an API and you should use their API to fake/stub/mock/whatever that system so you can focus on testing your business logic. After you have that system built you should then build integration/E2E tests that actually talk to hermetic copies of the real systems. An easy way to do this is to build a troubleshooting cli tool that you can run against your backend services/dbs/etc that can be used in CI against a copy of your backend or in prod to debug configuration issues.
Re: Stop mocking your system
#8The mocks are the symptom. The problem is that your code doesn't restrict side effects in any way. And so you end up with integration tests for everything and setting up a single test requires recreating the universe from scratch and slightly tweaking it on every run.
But that's what our program does, it talks to databases and file systems and HTTP servers!
Sure, and the effect of doing those things is moving data around. What does your program do with the results of these side-effects? Does it parse it? Transform it in any way? Decide whether to run effect A next with the result or effect B? This is the code that, if extracted, can be tested in isolation of databases and HTTP servers. You want as much of your code to be in this place as possible. It's a thousand times easier to test and the tests are thousands of times more reliable because they don't perform any effects.
Mocks have their places but if you can't test your code without mocking out the universe then the problem is that your code is interleaving too many effects with the core logic of the program. The cure is to refactor effects to the edges of your program and run effects in one place in your code. Make the rest just plain, pure code as much as possible.
Re: Stop mocking your system
#9Strongly agree, especially when it comes to things like AWS services. Their APIs and services evolve so quickly that things like local mocking (or emulation for local development) is an anti-pattern.
Where possible, I prefer to utilize short-term, pay-per-use infrastructure for development and testing.
Re: Stop mocking your system
#10This isn't a terribly practical article. I don't disagree with mocks being an "alternate reality". The author is entitled to their opinion on whether this is a good or bad thing. This said...what is the alternative? Integration testing all the way down?
The implication here is to work with stubs over mocks (i.e. I need to work with S3; I would then abstract that to provide a StubObjectStore to replace the S3ObjectStore used by other pieces of my code during tests). Great; I know they work now. But at some point, I need confidence my S3ObjectStore handles everything correctly. Do I give everyone on my team a bucket? Perhaps their own test AWS account? Test it, but only in the pipeline on its way to an intermediate stage? I can't control how AWS writes their SDKs (spoiler alert: they don't stub well), but I need some confidence that I can handle their well-understood behavior that scales. Likewise, I often can't control the libraries and integration points with other systems, and mocking offers a "cheap" (if imperfect) way to emulate behavior.