Beautiful explanation for something that tripped me up in early days of using mock/patch. Summary:
(1) Variables in Python are names that refer to values.
(2) (For that reason) Mock an object where it is used, not where it is defined.
Why a mock doesn’t work
61–70 of 77 posts
Re: Why a mock doesn’t work
#62Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing: https://martinfowler.com/articles/mocksArentStubs.html As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings. Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests…
Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time. And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them. I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can ma…
> I use mocks to arrange state A without going through all the business rules involved in creating state A.
An issue is that unless the types are really nailed down you can create easily create invalid or incomplete states, or create a state which is correct at one point then becomes invalid / incomplete at an other.
> Earlier tests have already verified all routes to State A
Unless they also assert the complete state at that point and that assertion is tied into the next state[0] then what they've verified is a series of steps.
Seems risky to assume it matches what an unrelated tests starts with, unless you've implemented state restoration, at which point you'd have to add cross-test dependencies such that the result of the tests validating section A can be saved and restore for every test needing a state A to test section B.
[0] that is, ∅ -> A tests end with an identity check for state A (that the system entire has no more and no less than expected) and that A -> B tests start with one, right after they've generated their fake state
Re: Why a mock doesn’t work
#63Earlier quoted context omitted.
Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time. And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them. I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can ma…
I don't see how you decided that DI requires some kind of special fixtures that are more complex than mocks. Mocks are generalized fixtures—just a reusable way to create them. And you can use them with DI. Only the method of delivery is changed, from an implicit backdoor to an explicit argument. DI puts new requirements on production code and tests that aren't related to a specific mock, that much is true. At least i…
That was an empirical statement. Inasmuch as it was a value judgement, it was a condemnation of hand-rolling your own mocks. Which people seem to do with an alarming regularity, and with a fairly consistently appalling degree of skill.
And not junior developers.
Re: Why a mock doesn’t work
#64Earlier quoted context omitted.
Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time. And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them. I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can ma…
> The absolute worst is fixtures with asynchronous code I concur that setting up a test for async code is a complete pain in the arse. It's extra fun when the caller doesn't get a reply, so there's no response that you can assert your as valid or invalid. It's extra, extra fun when the tests in your test suite need to run in parallel without stepping on each other. Here's an example. Say you've got PipelineService123…
Just curious : which language do you work with ?
In my previous work I had to work on a big asynchronous codebase in JavaScript and I had zero issues with asynchronicity at all, so maybe it's an ecosystem/language dependent issue ?
Re: Why a mock doesn’t work
#65The author hasn't a problem with mocks. It has got a problem with monkey patching. You can use a mock along dependency injection, and never run into the problems the author has.
Re: Why a mock doesn’t work
#66Earlier quoted context omitted.
I’ve recently learned that some actually do consider mocking harmful. Where I work right now, there’s a really outdated and unfashionable fight over the benefit of unit testing in general. Existing engineers don’t see value in test driven development, exhaustive testing, unit testing, and mocks/spies are thrown in... and we’re a python shop. I’m utterly confused. I, too, grew concerned after reading–will I be hearing…
I'm a died in the wool TTDer, but I actually think that mocking unnecessarily is usually harmful. I use it as a technique of last resort. I should define my terms before I explain, because many people use the term "mock" to mean things it didn't originally mean. Test objects that are used in place of domain level objects were traditionally known as "fakes". A fake that represented a fixed known value was called a "st…
I agree with basically all of this.
> some people believe very strongly that unit tests should test things in isolation. Secondly people believe that unit testing should be a black box testing strategy. So you should test through your public interfaces only and any collaborators that adheres to the interface contract should work as expected.
I think much of the confusion and talking-past-each-other comes from ambiguous language. I actually agree with all the things in the above quote (isolation, black-box, public-only, relying only on specified interfaces). Where I've differed from co-workers is that I consider the appropriate "unit" to be a feature/piece-of-functionality (e.g. "logging in"), whereas they consider the appropriate "unit" to be a piece of code (e.g. a method or class).
I had a bit of a rant about this at http://chriswarbo.net/blog/2017-11-10-unit_testing_terminolo...
Re: Why a mock doesn’t work
#67Earlier quoted context omitted.
I'm not sure what your objection is. People try to use mocks, and they don't work because they've mocked the wrong name. I explain this in the article. The article is about why their mock didn't work. How is this a misleading title? How is this a title I don't agree with? Are we talking about the same piece and the same title? You seem to be under the impression that I am trying to tell people not to use mocks. Have…
Now you seem to be willfully misunderstanding. He states clearly that your article is specifically about mocking in python . But mocking as a concept need not relate to python at all. Hence the confusion. A better title for your article would be "Why a mock doesn't work in python" edit: I enjoyed the article by the way.
I can see why adding "Python" would help here on Hacker News. The original complaint seemed to go deeper than that, I think because the title here is different than the title on my site.
Re: Why a mock doesn’t work
#68I've wrestled with this problem several times. My conclusion was mocks are just fine, but this is a wart in that there isn't "one way to do it" For the most part I can get by with one rule: always mock the module . with mock.patch('os.listdir'): will always work, even if it doesn't accomplish what you want. with mock.patch('mymodule.os.listdir') will fail if that module does not explicitly import os and instead does…
I don't understand what you mean by "will always work, even if it doesn't accomplish what you want." Mocking os.listdir will be useless if your product code's imports don't match it. How is this "one rule" to use?
The os module ships with python, so the function call mocking os.listdir will always succeed even if the code being tested does not use os.listdir
By mocking mymodule.os.listdir you add a requirement that mymodule actually import the os module and take advantage of mock.patch failing loudly if it does not.
Re: Why a mock doesn’t work
#69Earlier quoted context omitted.
I'm a died in the wool TTDer, but I actually think that mocking unnecessarily is usually harmful. I use it as a technique of last resort. I should define my terms before I explain, because many people use the term "mock" to mean things it didn't originally mean. Test objects that are used in place of domain level objects were traditionally known as "fakes". A fake that represented a fixed known value was called a "st…
I agree with basically all of this. > some people believe very strongly that unit tests should test things in isolation. Secondly people believe that unit testing should be a black box testing strategy. So you should test through your public interfaces only and any collaborators that adheres to the interface contract should work as expected. I think much of the confusion and talking-past-each-other comes from ambiguo…
The talk that you link to: https://www.infoq.com/presentations/tdd-original/
Ian Cooper reminds what was Kent's original proposition on TDD, what misunderstandings occurred along the way,
and suggests a better approach to TDD, one that supports development rather impeding it.Re: Why a mock doesn’t work
#70Earlier quoted context omitted.
I think they are asking a question about Python, not about mocks. I was also very surprised Python does it this way. Normally I would expect "import" statements in a language to just rearrange the namespace and have no other effects at all. The author explains that it doesn't work that way: > “from mod import val” means, import mod, and then do the assignment “val = mod.val”. In other words, I would expect an import…
How is "altering a symbol table" different than "creating a new variable"? Imports have to create a new variable of some sort: their entire point is to provide a name that you can use. "import mod" defines mod, which didn't exist in your module before. "from mod import val" defines val, which didn't exist in your module before.
For a moment, let's talk in general terms, not about how Python chose to do it.
Most languages have scopes. Things may exist in your scope, so you can access them. Or they may exist but in some other scope not visible to you where you currently are. Symbol tables track what is in a scope.
A variable is a name associated with a storage location. The location could be a stack, heap, register, or something way more abstract as long as it behaves as a "place" you can store a value into and read from.
When a variable is created, a name is defined and some storage is arranged for. But that name is also added to a scope. (If it weren't, it would have a name, but nobody could see it.)
There's no reason, once the variable has been added to one scope, that it can't be added to another scope. That is what "import" statements do in a lot of languages.
Python has apparently chosen a very different approach for imports, which is to create a new variable. This isn't necessarily wrong or right, it's just not something I had ever seen a language do before.