Live data from Hacker News

Why a mock doesn’t work

nedbatchelder.com

11–20 of 77 posts

Re: Why a mock doesn’t work

#11
post #5

Earlier quoted context omitted.

> The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. Alternative: have an abstract base class that describes a file system API. Have one implementation that builds on top…

Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to…

>Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain

Or easier - as the abstraction can simplify the interface, localize the logic for directory work, make it easier to port, and so on. That's the reason why most of us use "requests" and not urllib, too, for example, or Yoda and not Java's legacy time mess.

>a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime.

You can write alterative implementations (whether with base class, interface, traits, or what have you) and use them for production code and for testing without any dependency injection.

>Those of us using Python are using it because it is an interpreted, dynamic scripting language.

The abstractions mentioned come from Smalltalk, which arguably is the same or even more dynamic than Python. Having a class/interface/protocol/trait and a test implementation is by no means a static typing/Java thing, or counter to a dynamic language, as implied here...

Re: Why a mock doesn’t work

#13
Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing:

https://martinfowler.com/articles/mocksArentStubs.html

As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings.

Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests it's tedious, constantly updating tests when no semantic behavior was supposed to change.

Also, mocking via module hacks like in the article (and in the JS world) is scary; modules are basically global variables so it's a very coarse grained slice point. Dependency injection is almost always better.

Re: Why a mock doesn’t work

#14
post #13

Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing: https://martinfowler.com/articles/mocksArentStubs.html As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings. Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests…

Even better than "observable state from a to b" is "a transform of value from a to b", no inspection and whitebox needed :D

Re: Why a mock doesn’t work

#15
post #5

Earlier quoted context omitted.

> The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. Alternative: have an abstract base class that describes a file system API. Have one implementation that builds on top…

Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to…

> a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime.

Dependency injection can be as simple as:

    class Printer:
      def run(self):
        print("hello world")
    
    class Caller:
      def __init__(self, printer_class=Printer):
        self.printer = printer_class()

      def doit(self):
        self.printer.run()
In your tests all you have to do is pass a different object to your Caller class. No need to mess with factories, interfaces or Impl suffixes.

If the point is just having simpler testability and making dependencies more explicit, this gets the job done.

And since Python is a dynamic language, you can use this with stdlib classes too, so it solves your first complaint.

I've seen it a lot in Ruby and C# lately.

Re: Why a mock doesn’t work

#16
post #13

Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing: https://martinfowler.com/articles/mocksArentStubs.html As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings. Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests…

Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time.

And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them.

I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can make for pretty descriptive code but is outside of some people’s experience and so they resist it.

Earlier tests have already verified all routes to State A, the next batch of test now takes it as a given. This controls test explosion by utilizing transitivity. A->B B->C implies A->C. Tons of unit tests for A->B and B->C and then you only need a couple of functional tests (say, one negative and positive) test of A->C. You’re just checking the plumbing isn’t broken.

Otherwise, you get Cartesian products. You end up with elaborate, (often custom), mocks that couple all of the tests together in hard to maintain ways. You end up with tests that accidentally test the mocks/fixtures instead of the code.

Some of my current coworkers do this too. I don’t know where this pattern comes from. It’s often easier to replace their two page fixtures with two or three lines of mocks per test. It’s often almost the same amount of code, but so what, each test can be read. Each test can be run individually, and behavioral changes to the code due to urges affect mostly the tests you would expect. Those tests can be fixed, rewritten, or just removed as they disagree with the new requirements.

The absolute worst is fixtures with asynchronous code. Those break constantly, and often invisibly. What’s the point of a fire alarm if the damned thing doesn’t work? It’s almost worse than nothing at all.

Re: Why a mock doesn’t work

#19
post #16
post #13

Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing: https://martinfowler.com/articles/mocksArentStubs.html As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings. Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests…

Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time. And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them. I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can ma…

> Dependency injection leads to elaborate, brittle fixtures

I believe that can happen, but personally (N=1) haven't seen it. If anything, (well-done) DI is supposed to prevent that, b/c fixtures are isolated instead of depending on random global behavior.

That said, I also don't like Spring/Guice/Dagger auto-wiring DI (somewhat guessing that is what caused your headaches), and instead just create an "AppContext" type with all of the applications "singletons" and pass that around.

Granted, it's still global-ish, so maybe I'm cheating, but its nicer IMO than module hacks.

> I use mocks to arrange state A without going through > all the business rules involved in creating state A

I like tests being able to immediately jump to state A, but fwiw don't see why mocks would be needed to do so.

I do agree re avoiding test explosion/transitivity, and a few functional happy/sad plumbing tests, but again seems orthogonal to state/mock.

> You end up with elaborate, (often custom), mocks that > couple all of the tests together in hard to maintain ways

Totally agreed. Another -1 for mocks. :-) I.e. with state-based you should be able to test "end-to-end" (with state-based / in-memory stubs for your input/output data stores/etc.) without any of "oh right, copy/paste these 5 lines of 'when method X return result Y' for ~2-5 some mocked-out calls).

(And, to be pedantic, if you mean something other than 'when method X return result Y' for the term 'mock', then we're probably talking about different things.)

Re: Why a mock doesn’t work

#20
post #2

For a person who does not code very large programs in Python, this looks scary and like something that is hardly "only one way to do it" and that the most elegant solution gives you what you want. How I import things (or the libraries I depend on!) affects how I can write my tests? Really? My own Python scripts are typically single-screen in length. And one-off stuff, where they either work or don't, basically. Is th…

> How I import things (or the libraries I depend on!) affects how I can write my tests? Really?

Well, no. Everything that needs to be understood is explained in the first section, A quick aside about assignment. Everything else is just discussion about logical consequences of this behavior.

If you do an assignment (and "from foo import bar" is an assignment) you're just creating another name for a value (in this case, you're creating a name foo, in your namespace (presumably, but not necessarily a module) that points at whatever the name foo in module bar happened to point at at the moment you created it). If the original (or should I say, one of the previous) name(s) to the value is reassigned that does not affect what your name is pointing at.

This is really all there is to know, and once you've grokked that, everything Python falls in place.

Post reply on HN