91–100 of 125 posts
So has Martin Fowler: https://martinfowler.com/articles/mocksArentStubs.html
The TL;DR: Mocks increase coupling and make systems harder to change. Fakes are more versatile.
Earlier quoted context omitted.
But the price break is a requirement that I expect to change. Today it is 10. Tomorrow someone looks at the production process the machines give groups of 11, so the bulk discount should be multiples of 11 (if you order 12 you get 11 at the bulk rate and 1 and the regular price). Then next week management says small orders are too hard to serve, and so the minimum bulk order is 100, but bigger discounts if the total…
If your requirement changes so that the price break is at 11 rather than 10, then I expect the process to go: * Change the (or shared test code) that has the price break specification from 10 to 11 * Run it, see the failures * Update the source code appropriately The updated source code is the same in both our cases. In my case the changed test code is testing the ShoppingCart interface and in yours it's a unit test…
Earlier quoted context omitted.
I prefer to express it as the purpose of tests is to ensure something never changes. I don't care how future maintainer touches the code so long as given this set of inputs this is the output.
People are touching the code because something has to change. Typically in the service of a new business case.
Earlier quoted context omitted.
If your requirement changes so that the price break is at 11 rather than 10, then I expect the process to go: * Change the (or shared test code) that has the price break specification from 10 to 11 * Run it, see the failures * Update the source code appropriately The updated source code is the same in both our cases. In my case the changed test code is testing the ShoppingCart interface and in yours it's a unit test…
I'm saying the discount code tests have the change from 10 to 11. this entire thread isn't about the discount code tests though it is about the shopping cart. There is a lot more to calculating price than just discount codes, tax rates for example. The cart needs to show regular price, the discount, the tax, shipping costs, and then total. Each of those is tested for multiple edge cases, so if we do it all in the sho…
It might affect multiple tests, but all you should have to change is one or two places in some shared code pulled in by those tests. Too many tests have too much repeated code, and so become expensive to maintain, exactly like normal code. Refactor, think about shared functions etc. exactly as you would do with your production code.
Earlier quoted context omitted.
I would argue then that this isn't Unit Testing, it's some other kind of testing. One of the problems I'm dealing with at the moment is exactly this problem - someone's gone and written a whole ton of tests which involve spinning up pretty much everything short of the DB, and now writing a small test that just tests the behaviour of one class is exceedingly difficult.
You're right, it's not unit testing. I call it integration testing, some people call it functional testing. The question is why you want to test that one class? We do have some modules in our (elixir) system that are 'thoroughly' unit tested. These are modules that are algorithmically complicated so testing them in isolation is useful. Most of the code is tested in logical chunks though.
In unit specs you should use mocks/fakes for external behaviour. Then you can simply verify that it's calling the dependency correctly.
Isolation specs are useful because when integration tests fail, they usually give little feedback as to what exactly went wrong in the system.
Also integration specs are expensive and slow to run since they execute everything.
Finally I think mocks/fakes are useful because if you find yourself mocking 20 objects to get a single unit test working, then it's a sign that you need to redesign the code.
Even more important, IMO, regardless of whether you're mocking or faking, is to not mock or fake objects that you own. Only mock/fake truly external dependencies. The nice property you get with this is true tests of integration and interoperability between your modules, that are actually calling into each other. Only at the leaf nodes is anything mocked or faked, and that's when it leaves your ownership.
It's a fantastic comparison of the Detroit school of testing (what you and I believe) vs the London school, which is simply a different mental model about code and testing and mocks especially.
Framing the differences this way make it clearer why some people might disagree with us, and it's not because they're stupid or wrong- they just have a different school of thought on the matter. This doesn't excuse, however, the people who are doing a little bit of both and they don't know why, and they're getting the benefits of neither side of that coin.
Earlier quoted context omitted.
I prefer to express it as the purpose of tests is to ensure something never changes. I don't care how future maintainer touches the code so long as given this set of inputs this is the output.
People are touching the code because something has to change. Typically in the service of a new business case.
The fatal flaw is that you’re really introducing a new class which may have its own behaviour problems. Now your test is implicitly testing the test class. I also prefer to see explicit mocking than just trust that the fake does what it says it will. How do I know without looking whether the fake store method REALLY stores the doc, or whether someone set it up to just return OK? And how do you test error handling - m…
But those are easy to find. This is why double-entry bookkeeping is the accounting standard: it's easy to add incorrectly once. It's much less common to add incorrectly the exact same way twice.
What exactly is the difference between a Mock and a Fake? In one you store any side effecting values in memory with a predefined behavior in a separate class. In the other, you define the predefined behavior in the test class by intercepting calls to external components. Your test coupling to internal implementation is just as high, or higher. Your Fake needs to do exactly what you expect in isolation of the class that depends upon it, or the test result won't work. In addition, without the reflection capability provided by many mocking frameworks, it is impossible to "Fake" implementations of certain library components via `sealed` or `final` modifiers on external library classes. It is also impossible to verify side effects without a method to expose those side effect actions in your Fake, which will force, at the very least, an extended declaration in your test class.
Integration testing is an essential part of the test pyramid. When your required interface for your injected mock/fake changes, your test is going to have to change. Your tests will be just as coupled to your implementation with a fake. The coupling is just hidden in your fake implementation.
In addition, often we are mocking huge interfaces with complex inner machinery that we don't fully understand.
The real issue with mocking and faking is the necessity to mock and fake to isolate tests due to poor interface design in the class under test, usually focused around side-effecting methods.
All side effecting methods must return some sum type that is not void that indicates success or failure of the side-effecting actions. All side effecting actions taken by the code must be apparent at the type level of this success or failure wrapper. Tests must use random, generated input parameters to verify the returns generated are correct. And as always, a type signifies for all X, then Y; while a test signifies for this x in X, then this y in Y. Types are enforced by the compiler or runtime, which (though they may have a bugs themselves) are never wrong as the interpreters of code. What you get is what you get, even if it doesn't match up with what you think a particular type expression should output.
Even with effect encoding and property-based generated testing and compiler/runtime proofs that your program interfaces are correct, it is highly unlikely that a person incapable of designing and implementing a correct implementation would be capable of designing and implementing a correct test for that implementation or type hierarchy. Even with effect encoding at the type level all the way down, integration testing between components against live infrastructure remains the best way of detecting bugs -- all your unit testing and design assumptions are exercised by exhaustive business case testing.
Mathematical proofs derived from the type system are very strong validators of design. But they can only verify that the assumptions and constraints the designer has created are correct. Only verification via integration and user testing can verify that those assumptions and constraints actually meet the business requirements of a given design.
All of this is to say, if you are focusing on using Fakes over Mocks and relying upon tests to ensure correct behavior, eventually you will fail. In other words, bugs will eventually manifest in any system in which automated integration and user testing that verifies exhaustively the business cases requested by the business user is not part of the release process.
Unit tests and types are measures of the correctness of a given design with respect to itself. Unit tests and types are not the measure of the reliability of a given design with respect to the use of software artifacts, and cannot be. Nor can coupled/uncoupled unit tests improve the ease of maintenance of a program. Types and unit tests are a useful design and maintenance tools that can make the intent of a design clearer to the reader of a piece of code, and can help identify times while implementing where you are violating the constraints you have set for yourself. But they cannot make refactoring or interface changes easier to encode without inflicting line churn cost in the tests. If an injection is exposed to the user of a class, it is exposed, and you must deal with it.
Even more important, IMO, regardless of whether you're mocking or faking, is to not mock or fake objects that you own. Only mock/fake truly external dependencies. The nice property you get with this is true tests of integration and interoperability between your modules, that are actually calling into each other. Only at the leaf nodes is anything mocked or faked, and that's when it leaves your ownership.
Yes, integration tests are important too, but you'll want to minimize the number of scenarios you still need to test at that level.