Live data from Hacker News

Prefer Fakes over Mocks

tyrrrz.me

121–125 of 125 posts

Re: Prefer Fakes over Mocks

#121

Earlier quoted context omitted.

On the best scenario, your fakes just replace some standard components your have on production with other standard components, and keep everything else equal. On those cases, no they don't need tests. But when you are doing something weirder, they pretty much do. On the best scenario, your mocks are just data that doesn't have timing characteristics, don't react to the system being tested, and are short enough to not…

> Anyway, why do you claim any of that is specific to OOP? There is no need for Mocks or Fakes with pure functions.

And in what paradigm do your programs consist of only pure functions? Certainly not on FP, where IO is explicitly added to a lot of code.

Anyway, there isn't much difference between mocking a unity and passing some static data into your functions and comparing the results with the expected ones.

Re: Prefer Fakes over Mocks

#122

It seems to me that a Fake is really a Mock, and Mocks became this unholy dynamic framework stuff. I'm not a fan of dynamic mocks where the test code has the mocked implementation because it is super brittle. Instead, a "fake" (i.e. a mock as I understood them) is an alternatively implementation of an interface. There are a lot of really good "fakes" out there. Fake file system which lets you simulate disk failures.…

Agreed. This is a naming/semantics/nomenclature issue in the testing world at large. The lines have all been blurred and words have different meanings across different language ecosystems.

The most complete nomenclature I have seen is that used by Marin Fowler (which he attributes to Gerard Meszaros). I like it because it gives a fairly decent taxonomy of test doubles, and thus makes it possible to easily describe some of the common types, and for example, suggest that some other type might be better suited in some situation.

I summarize it in my own words below:

Dummy: Trivial implementation that might do nothing, return some default value, or throw exception if called. These are intended to be used only when an object gets passed but never used.

Fake: Have a working implementation, but takes some shortcut that makes them unsuitable for use in production. These shortcuts can potentially be pretty extreme, like a fake sales tax service that just uses a flat percentage for everything, an image watermarker that just returns the original image, etc. Basically if one could substitute the original implementation for this, still be spin up and use the whole app, without worrying about it crashing/misbehaving (such as hitting some unimplemented case or fixed return data causing map collisions etc), then it is a fake.

Stub: Harded implementation. Is not fully functional, and designed to have correct or usuable responses only for scenarios that the test will care about. (Sometimes designed to take into account a small group of tests).

Spy: A spy is an implementation that records some information about how it is used. It could be a stub or a fake. IT could also be a decorator patten wrapper around some other implementation. (Fowler incorrectly oversimplifies this by defining it to be a stub implementation). The recorded information can later be accessed to verify say the number of emails the code being tested tried to send.

Mock: An object that gets preprogrammed with expectations about how it will be called (and what it should return). Then after running the test, it is asked to confirm if the calls made upon it match the pre-programmed ones. One can pretty clearly see that this is a bit different than the others, in that it contains special verification logic.

-----

Either a spy or a mock is needed when the behavior being tested is not visible as either part of the return value, or as part of the state of some object after the test is completed. Those cases require "behavior verification", as opposed to "state verification". State verification might check some object's propeties, or query the fake database to see if the resulting state is as expected, while behavior verification.

The need for behavior verification can be common in certain "command" style apis. For example, if a dependency's purpose is to send emails, then it is quite possible that there is no state that remains (within the program) to verify if it got called. So the best that is possible is to verify that the dependency got called (or did not get called) as expected.

(Don't take the state vs behavior distinction too literally. Obviously both Mocks and Spies store recoded information as state. The important concept is that the normal implementations would not include this state.)

---

The article in question on which the above is all loosely based is: https://martinfowler.com/articles/mocksArentStubs.html

Re: Prefer Fakes over Mocks

#123

I watch more people overcomplicate their lives when it comes to unit testing. The problem discussed in this article is not one of mock vs. fake its actually about how this individual chose to engineer their application. First I'll introduce some axioms... 1. Mixing unit and integration testing is not ideal. If you are doing unit testing focus on verifying the code paths within the unit. When considering the collabora…

"Making code more testable" by moving the logic into functions you don't test, is not a winning strategy.

This is strawman. If you phrase like this...

"Make your code more testable by moving things that are cumbersome/expensive to test but extremely unlikely to break into functions you don't test. Instead focusing your efforts on making the items that are likely to break easy to test"

Phrased honestly it becomes more interesting discussion.

I have worked on projects that had static analysis and high code coverage standards where code quality was abysmal, the product was barely functional, and development was paralyzed. I have also worked on projects that had none of those things but managed to have happy customers and quality product.

What does this mean?

It means that writing lots of tests is not enough to guarantee good outcomes. 1 great test is an asset, 400 bad tests are a liability. What does guarantee good outcomes is focusing on "equity". Does writing this test improve my equity position? If the answer is no or if you can't qauntitatively demonstrate the value in terms of opportunity cost then the answer is obvious...don't write the test.

The current state of software quality at the development level closely parallels the plot of Moneyball. Consider this quote...

"Okay, people who run ball clubs, they think in terms of buying players. Your goal shouldn’t be to buy players. Your goal should be to buy wins and in order to buy wins, you need to buy runs. You’re trying to replace Johnny Damon. The Boston Red Sox see Johnny Damon and they see a star who’s worth seven and a half million dollars a year. When I see Johnny Damon, what I see is an imperfect understanding of where runs come from” ― Michael Lewis, Moneyball: The Art of Winning an Unfair Game

Many people try to buy quality with an imperfect understanding of where quality comes from.

Re: Prefer Fakes over Mocks

#124
post #71

Earlier quoted context omitted.

Mocks for your own code, fakes for external dependencies but in both cases only where you can’t rework your most of your code to consume the outputs of dependencies instead of soliciting them. Fixtures trump mocks and fakes. Having multiple tests fail for the same reason is a waste of effort, either now or in the future (when new features are added or the constraints otherwise change). Sometimes you have to depend on…

Why would you mock your own code, when you have your own real code right there?

In my case, I'm an embedded dev. So some of my own code is doing memory-mapped I/O to hardware, inline microcontroller ASM, or other stuff that can't physically happen on the system running the unit tests. So mocks (or fakes, it doesn't matter much here) are necessary to avoid running hardware-specific code on the wrong hardware.

Of course I try to make those fakes at as low a level as possible, but there's still a substantial part of my code that can't be unit tested, and needs test programs in the actual firmware to check it.

Re: Prefer Fakes over Mocks

#125
post #96

Earlier quoted context omitted.

While I 100% agree with you on this point, you should take a few minutes and read this: https://medium.com/@adrianbooth/test-driven-development-wars... It's a fantastic comparison of the Detroit school of testing (what you and I believe) vs the London school, which is simply a different mental model about code and testing and mocks especially. Framing the differences this way make it clearer why some people might dis…

What if I like outside-in, black-box testing Because I like to test everything, not just little parts I isolate?

Then you follow the classicist, Detroit-school of testing. Me too.

But if you read the article, you can see that this isn't the only school of thought.

Post reply on HN