Live data from Hacker News

Prefer Fakes over Mocks

tyrrrz.me

71–80 of 125 posts

Re: Prefer Fakes over Mocks

#71

Even more important, IMO, regardless of whether you're mocking or faking, is to not mock or fake objects that you own. Only mock/fake truly external dependencies. The nice property you get with this is true tests of integration and interoperability between your modules, that are actually calling into each other. Only at the leaf nodes is anything mocked or faked, and that's when it leaves your ownership.

Mocks for your own code, fakes for external dependencies but in both cases only where you can’t rework your most of your code to consume the outputs of dependencies instead of soliciting them. Fixtures trump mocks and fakes.

Having multiple tests fail for the same reason is a waste of effort, either now or in the future (when new features are added or the constraints otherwise change).

Sometimes you have to depend on transitive properties to keep your test sizes down. If a function received the data, and the function has been tested thoroughly, why test it again while testing the callers?

Re: Prefer Fakes over Mocks

#72
post #59

Earlier quoted context omitted.

If what you "spin up" is internal classes/functionality, not external system, I don't understand the problem? The article doesn't argue "this is how you do unit testing", it argues "this is how you do better testing". It's highly debatable whether unit tests are preferable to larger, "component tests".

The problem is that it makes testing every upstream component more difficult to write for, and more fragile. To take the shopping cart example I gave, if I write it the way that I understand you're suggesting, I would have to consider the behavior of all of the ICartDiscountStrategy implementations. For instance, I've written tests for CartDiscountCalculator and ShoppingCart that have 10 different items in the cart f…

> if I write it the way that I understand you're suggesting, I would have to consider the behavior of all of the ICartDiscountStrategy implementations.

No, you would need to consider only the business requirements.

> Some time later, I implement a new AcmeCorpDiscountStrategy which results in a different discount being returned.

Why did you implement that? Is it a business requirement change, that "existing customers should get this different discount"? If yes, then your tests break - but for a good reason. If it's not about existing customers, but new sort of customers, then your tests should be unaffected and you just need to write new tests that target these new sorts of customers and validate that the new discount strategy is applied.

Re: Prefer Fakes over Mocks

#73
post #56

Earlier quoted context omitted.

Um, no. The test needs to know the expected system behaviour, given that dependencies are properly wired & behave as expected. You can argue that "it no longer tests just the ShoppingCart in isolation" and that would be true; but whether that's a downside or an upside is up for debate IMO (and more often than not it really is an upside). [edit] In other words, you don't test that "if discount is 10%, adding item to s…

I would argue then that this isn't Unit Testing, it's some other kind of testing. One of the problems I'm dealing with at the moment is exactly this problem - someone's gone and written a whole ton of tests which involve spinning up pretty much everything short of the DB, and now writing a small test that just tests the behaviour of one class is exceedingly difficult.

I say that it depends on the "unit" you're trying to test. Is it large and complex enough in itself that you need to isolate it from other components (dependencies)?

For your use case, if the shopping cart is simple enough, I'll just choose a simple or frequently used discount calculator and make some unit tests to test the shopping cart functionalities. Then for the rest of discount calculator cases there will be separated unit tests.

The problem with testing almost everything in isolation makes refactoring some small-inner components costly, since every change may break some of unit tests.

Re: Prefer Fakes over Mocks

#74
post #60

Earlier quoted context omitted.

It shouldn't need to know implementation details, but it should need to know expected behaviour. The use of ICartDiscountStrategy and BulkPurchaseDiscountStrategy is an implementation detail, the high level requirement that buying 10 of something should be cheaper than 9 is just specified system behaviour and that is exactly what you should be testing for. I would refer to these tests as "integration tests" although…

> the high level requirement that buying 10 of something should be cheaper than 9 is just specified system behaviour and that is exactly what you should be testing for. But that is an implementation detail. Only the BulkPurchaseDiscountStrategy knows whether it's 9 or 10 that's the trigger. Maybe it varies based on time of day or for certain customers. > I would refer to these tests as "integration tests" although th…

> But that is an implementation detail. Only the BulkPurchaseDiscountStrategy knows whether it's 9 or 10 that's the trigger.

Language is poor, but I think there is consensus on what "implementation detail" means, and I don't think that's it. Whether it's 9 or 10 is customer facing behaviour. It's behaviour that might come from your marketing team or product owner or whatever, but it is outward facing. It is something that someone could write in a word document and call part of your specification.

The implementation detail would be whether that logic lives in BulkPurchaseDiscountStrategy or as one case in a big ShoppingCart class. A user pressing buttons on your UI can see the result of the price break, but they can't see the difference in code organisation.

You can say "price breaks should occur at 9 items" without ever saying "BulkPurchaseDiscountStrategy". In one ideal, all tests would test user facing requirements and not know anything about your code. You could refactor your code to your heart's content and know you the result was still what you wanted. There are downsides to that though because those end to end or system tests are fragile to write and expensive to run. It's much easier to cover all possible code paths with more focused testing. Hence in this case you could test the top interface of ShoppingCart, with all it's assembled dependencies. Not the same as pushing button in a Web Browser, but a lot more convenient to write and run.

Re: Prefer Fakes over Mocks

#75
post #25

Earlier quoted context omitted.

The article addresses this: you need to write tests for the fake as well. I'm not sure I buy this as something that's desirable to have to do, but it should address your concern.

Hmm, if I’m having to write tests for my tests, I feel something has gone wrong in my life ;)

A "fake" isn't a test; it's a real, working implementation of some interface. The only reason we don't use such implementations in production is for some non-functional reason (efficiency, resilience, etc.). For example, if we want a key/value store we might choose Redis for production, but a HashMap is a perfectly good fake (and you can bet that the language/library implementing HashMap has a ton of tests!)

Re: Prefer Fakes over Mocks

#76
post #13

A tangent, and somewhat philosophical: is "The primary purpose of software testing" really "to detect any potential defects in a program before it reaches its intended consumers"? I would argue that whilst that is a commonly held belief, and certainly true for some tests of business logic, the main value and reason for testing is to allow developers to confidently make changes to existing code. In my experience user…

I prefer to express it as the purpose of tests is to ensure something never changes. I don't care how future maintainer touches the code so long as given this set of inputs this is the output.

Re: Prefer Fakes over Mocks

#77

Even more important, IMO, regardless of whether you're mocking or faking, is to not mock or fake objects that you own. Only mock/fake truly external dependencies. The nice property you get with this is true tests of integration and interoperability between your modules, that are actually calling into each other. Only at the leaf nodes is anything mocked or faked, and that's when it leaves your ownership.

Perhaps I misunderstand you. Say I'm writing tests for lets say ShoppingCart. ShoppingCart depends on an implementation of ICartDiscountCalculator (CartDiscountCalculator) CartDiscountCalculator depends on a number of implementations of ICartDiscountStrategy (ChristmasCartDiscountStrategy, VIPCustomerDiscountStrategy, BulkPurchaseDiscountStrategy, etc) Each of those, in turn has their own dependencies, perhaps webser…

You have found one exception: if the item has multiple different implementations - as your discount strategy does - then you should mock (or better yet fake) the implementation. In this case testing with a real doesn't gain you anything.

Re: Prefer Fakes over Mocks

#78

I've tried this approach twice with mixed success. In both cases I wanted to stub out the persistence tier of a Node.js application when test driving the API. I verified the real and fake implementations by running the same tests against them. The first application was quite small, and the process worked well, although it did feel somewhat onerous to implement the fake. However, the second application was more comple…

My hierarchy goes: None > Real > Fake > Mock

The best solution doesn't need anything, e.g. if we have calls to our persistence layer mixed in with calculations, the latter shouldn't be tested with fakes/mocks; instead we should refactor the code so the calculations don't depend on the persistence layer.

If the behaviour of some code depends inextricably on some external system, then our tests should use that system. This avoids unnecessary code/work/duplication, allows tests to exercise more paths through our codebase, exposes problems in our understanding of that system, etc.

If calling an external system in our tests is dangerous, unacceptably slow, costly (if it's some Web service), etc. then we should make a fake implementation to test against. If our code only uses a small part of some large, complicated dependency, we might want to define an interface for the subset that we need, refactor our code to use that, and only have our fake implement that part.

If a fake implementation isn't practical, or would require so much overriding per-test as to be useless on its own, then I might consider using mocks. I would seriously consider whether that code could be refactored to avoid such coupling. I would also never make assertions about irrelevant details, like whether a particular method was called, or what order things are run in.

Re: Prefer Fakes over Mocks

#79
post #20

Earlier quoted context omitted.

You're totally right. But think about a blob storage class that has two methods, ReadOneFile(), ReadMultipleFiles(). You have one test that just mocks ReadOneFile(string fileName) and doesn't mock the ReadMultipleFiles(). You now change the implementation of the System Under Test (SUT) to use once the ReadMultipleFile(string[] fileNames) call instead of three times the ReadOneFile(). Now your test fails, but the impl…

Yes, good point, that’s true. Personally, I know it’s not the ‘right’ ethos, but I see the mock test failure as a bit of a bonus. My change intended to stop calling ReadOne and start calling ReadMultiple. Having to change the test sure tells me I accomplished that! But you’re right that if I’m intending to only test behaviour, the fake lets me do that better. Interesting post, thank you :)

Somewhat tangential to the conversation at hand, but I feel part of the problem here is having two methods i.e. ReadOne/ReadAll. Why not just embrace OOP and go with one method (Read) that takes a specification which itself encapsulates the set of files you want to read? Then it works for none, one, and all files...and makes the argument about Mocking vs Faking somewhat moot; the mock is a simple "for any args return this" and the fake is just as simple.

I guess my point is that if you have to agonise over mock vs fake then really it's a sign that your design is not quite as testable as you might like :)

Re: Prefer Fakes over Mocks

#80
post #74

Earlier quoted context omitted.

> the high level requirement that buying 10 of something should be cheaper than 9 is just specified system behaviour and that is exactly what you should be testing for. But that is an implementation detail. Only the BulkPurchaseDiscountStrategy knows whether it's 9 or 10 that's the trigger. Maybe it varies based on time of day or for certain customers. > I would refer to these tests as "integration tests" although th…

> But that is an implementation detail. Only the BulkPurchaseDiscountStrategy knows whether it's 9 or 10 that's the trigger. Language is poor, but I think there is consensus on what "implementation detail" means, and I don't think that's it. Whether it's 9 or 10 is customer facing behaviour. It's behaviour that might come from your marketing team or product owner or whatever, but it is outward facing. It is something…

But the price break is a requirement that I expect to change. Today it is 10. Tomorrow someone looks at the production process the machines give groups of 11, so the bulk discount should be multiples of 11 (if you order 12 you get 11 at the bulk rate and 1 and the regular price). Then next week management says small orders are too hard to serve, and so the minimum bulk order is 100, but bigger discounts if the total order is a multiple off 11. Then the week after...

Point is that your implementation of the discounts is expected to change. I should not have to change my shopping cart tests at all just because the discount requirement changed - it makes the costs of those tests too high.

Of course this is domain specific. I expect discounts to change all the time. Likewise I expect taxes to change often. In both cases they need to be mocked/faked just to keep my test code sane. there are many other domain specific things that won't change. I'm always going to store the shipping mass in KG, and for some customers convert it to lbs. It is reasonable to hardcode the weight in lbs as that won't change. (future me who has to ship to mars will disagree, but I don't know what the requirements are)

Post reply on HN