Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why a mock doesn’t work

nedbatchelder.com

Post reply on HN
Print view View original

71–77 of 77 posts

Re: Why a mock doesn’t work

#71

Post by stevenhuang »

nedbat wrote: post #67
stevenhuang wrote: post #58

Earlier quoted context omitted.

Now you seem to be willfully misunderstanding. He states clearly that your article is specifically about mocking in python . But mocking as a concept need not relate to python at all. Hence the confusion. A better title for your article would be "Why a mock doesn't work in python" edit: I enjoyed the article by the way.

I can see why adding "Python" would help here on Hacker News. The original complaint seemed to go deeper than that, I think because the title here is different than the title on my site.

That's... not the reason.

Re: Why a mock doesn’t work

#72

Post by mikekchar »

chriswarbo wrote: post #66
mikekchar wrote: post #57

Earlier quoted context omitted.

I'm a died in the wool TTDer, but I actually think that mocking unnecessarily is usually harmful. I use it as a technique of last resort. I should define my terms before I explain, because many people use the term "mock" to mean things it didn't originally mean. Test objects that are used in place of domain level objects were traditionally known as "fakes". A fake that represented a fixed known value was called a "st…

I agree with basically all of this. > some people believe very strongly that unit tests should test things in isolation. Secondly people believe that unit testing should be a black box testing strategy. So you should test through your public interfaces only and any collaborators that adheres to the interface contract should work as expected. I think much of the confusion and talking-past-each-other comes from ambiguo…

I think Michael Feathers explained it the best. He likened unit testing to clamping a piece of woodwork while you are working on it. The bits you are working on need to be in motion because you are working on them. The bits you are not working on them need to be clamped in place -- you don't want those things moving while you are working on some other bit. A "unit" is anything you might want to be clamped in place. It can be a function. It can be an object. It can be a subsystem. You want to unit test at different levels of abstraction so that you can "clamp" those levels of abstraction down.

One of the things I've found people get confused with is that they see unit testing and integration testing as orthogonal. They think a unit test should exercise a small piece of code in isolation and an integration test should test examples of real collaborators. Frequently they mock out all their unit tests and write a few integration tests. Then their unit tests become brittle and annoying and so they delete them, leaving only a few integration tests. This leads a lot of people with the impression that only integration tests are useful. If we can back up and redefine "unit test", then the problem disappears.

I read your "rant". Don't even get me started on BDD :-) Originally people had problems understanding the purpose of TDD because the word "test" had them confused. They would think, "I need to write tests to ensure that this is working". They didn't think about it in terms of clamping the behaviour so that it doesn't change when you are working on another part of the system. For that reason, a lot of people discussed changing the word "test" to something else that truly embodied what TDD was all about. Many people hit on the word "behavior" -- you want to document the current behaviour of your "units" (at different levels of abstraction). Somehow this got totally confused with automated acceptance testing! Now we have things like cucumber (which I don't actually hate, but it accomplishes a completely different goal than TDD!)

What really frustrates me is when I talk to people about this stuff and they think I'm a complete lunatic :-)

Re: Why a mock doesn’t work

#73

Post by PopeDotNinja »

littlestymaar wrote: post #64
PopeDotNinja wrote: post #21

Earlier quoted context omitted.

> The absolute worst is fixtures with asynchronous code I concur that setting up a test for async code is a complete pain in the arse. It's extra fun when the caller doesn't get a reply, so there's no response that you can assert your as valid or invalid. It's extra, extra fun when the tests in your test suite need to run in parallel without stepping on each other. Here's an example. Say you've got PipelineService123…

Just curious : which language do you work with ? In my previous work I had to work on a big asynchronous codebase in JavaScript and I had zero issues with asynchronicity at all, so maybe it's an ecosystem/language dependent issue ?

Elixir. Are you running your tests in parallel in the same process?

Re: Why a mock doesn’t work

#74

Post by arantius »

moron4hire wrote: post #46
stephen wrote: post #13

Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing: https://martinfowler.com/articles/mocksArentStubs.html As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings. Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests…

A situation in which I've used mocking well: testing parsing of data coming over a serial port. I have had several situations to need to parse common data formats coming from hardware over a wire and mocking that data with recorded streams is super helpful for making sure your parser is working right.

It sounds to me like you've got two things: some hardware code, and some data code. You're not testing the hardware code, you are testing the data code.

If you split them into two places, you could get the same effect, with no mocks. And be more clear about what parts are tested and what aren't.

Re: Why a mock doesn’t work

#75

Post by littlestymaar »

PopeDotNinja wrote: post #73
littlestymaar wrote: post #64

Earlier quoted context omitted.

Just curious : which language do you work with ? In my previous work I had to work on a big asynchronous codebase in JavaScript and I had zero issues with asynchronicity at all, so maybe it's an ecosystem/language dependent issue ?

Elixir. Are you running your tests in parallel in the same process?

It depends on what you mean by parallel: nodeJs is single-threaded, so every CPU-bound task runs sequentially. But all the asynchronous tests are run concurrently by the test runner (Mocha[1] in my case).

[1] https://mochajs.org/

Re: Why a mock doesn’t work

#76

Post by moron4hire »

arantius wrote: post #74
moron4hire wrote: post #46

Earlier quoted context omitted.

A situation in which I've used mocking well: testing parsing of data coming over a serial port. I have had several situations to need to parse common data formats coming from hardware over a wire and mocking that data with recorded streams is super helpful for making sure your parser is working right.

It sounds to me like you've got two things: some hardware code, and some data code. You're not testing the hardware code, you are testing the data code. If you split them into two places, you could get the same effect, with no mocks. And be more clear about what parts are tested and what aren't.

That is literally what I said I was doing.

Re: Why a mock doesn’t work

#77

Post by wojciech_bulaty »

Ned's implicit definition of a mock is narrower than the generally accepted one. He actually described a stub created by monkey-patching. A mock allows for call verifications as well.

There are 3 main categories of techniques for managing dependent components used these days:

1. In-process class/method/function mocks or stubs (http://xunitpatterns.com/Mocks,%20Fakes,%20Stubs%20and%20Dum... and https://martinfowler.com/articles/mocksArentStubs.html)

1a. By monkey patching (which is what Ned has demonstrated in his article very well)

2a. By dependency injection

2. Over-the-wire API mocks or stubs (https://en.wikipedia.org/wiki/Comparison_of_API_simulation_t...)

3. Virtual services/simulators (https://en.wikipedia.org/wiki/Comparison_of_API_simulation_t...)

It's worth keeping in mind that all of them are part of a wider group of test doubles: https://www.infoq.com/articles/stubbing-mocking-service-virt...

Other options available for decoupling from test dependencies:

1. In-memory database https://en.wikipedia.org/wiki/In-memory_database

2. Test container https://www.testcontainers.org/

3. Legacy in a box https://www.thoughtworks.com/radar/techniques/legacy-in-a-bo...

Post reply on HN