Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Prefer Fakes over Mocks

tyrrrz.me

Post reply on HN
Print view View original

61–70 of 125 posts

Re: Prefer Fakes over Mocks

#61

Post by throw_m239339 »

andix wrote: post #43
throw_m239339 wrote: post #32

Earlier quoted context omitted.

> Because they don't just implement a part of the interface (just one store method, or just one of two load methods). They try to create a fully functional fake of the original class. So shouldn't fakes have their own tests as well? Mocks don't need tests (at least in C# with a framework) because no new class is ever written. Anyway most of these issues are inherently linked to the nature of OOP and are a direct trad…

Why do fakes need tests, but mocks don't? In one case you create the class on your own, in the other case the mocking framework does that for you on the fly. With both approaches you can do very basic or very complicated things. Or do you have a rule in your company that every class needs a test? Maybe this rule is the problem.

> Why do fakes need tests, but mocks don't?

because no new class or method is created with mocks, it's merely declarative, no new logic is created than needs to be tested. Fakes very much have logic since they are implementations of classes.

> Or do you have a rule in your company that every class needs a test? Maybe this rule is the problem.

No Fakes are, not the fact that every unit should be tested. Fakes are very much a unit, the fact that they are classes is irrelevant.

furthermore:

https://news.ycombinator.com/item?id=24774752

which demonstrates my point. I don't want to have to write tests for tests.

Re: Prefer Fakes over Mocks

#62

Post by chriswarbo »

viraptor wrote: post #44
mathgladiator wrote: post #39

It seems to me that a Fake is really a Mock, and Mocks became this unholy dynamic framework stuff. I'm not a fan of dynamic mocks where the test code has the mocked implementation because it is super brittle. Instead, a "fake" (i.e. a mock as I understood them) is an alternatively implementation of an interface. There are a lot of really good "fakes" out there. Fake file system which lets you simulate disk failures.…

The problem with those alternative implementations is that they turn your unit tests into expensive and complicated integration tests. In a complex system getting a socket into the state you're interested in can be many times harder than the test itself. "Magic" mocks make it cheap and easy.

There's nothing preventing a fake implementation of interface Foo from also having extra functionality, like controlling it's initial state. In the case of sockets, we might have a FakeSocket which implements methods Socket::close, Socket::read, etc. but also provides methods like FakeSocket::appendToBuffer, FakeSocket::withState, etc.

The boundary between such fakes and mocks is obviously fuzzy: as we override more of a fake's behaviour we approach a mock, as we make a mock more functionally-complete we approach a fake.

Mocks (in this sense) are most useful when we want a lot of control over a very simple interaction; e.g. having Socket::read return a particular error state. We don't need a whole fake implementation for that, and indeed it's probably easier to just specify a mock with only those parts (and nothing else).

In my experience such situations are rare, since they tend to result from over-complication in the system under test, which makes it hard to test. Mocks are a useful crutch in those situations, but it's often better to refactor the code such that mocks aren't needed.

For example, whenever we create a mock that returns some canned response, that's a hint that our 'service object' is probably just a (possibly lazy) wrapper around that response. Any code that processes such responses can often be refactored into an ordinary function/method which accepts such response data as an argument. Sending test data into such functions is trivial (just call the function on the test data), and we can delete whatever wrapping/unwrapping boilerplate was previously littering the code.

When our mocks are more complicated that just spitting out canned responses (e.g. there's some temporal dependency between method calls, and correlations between their responses), those are exactly the things that fake implementations are for.

Re: Prefer Fakes over Mocks

#63

Post by paranoidrobot »

virgilp wrote: post #59
paranoidrobot wrote: post #58

Earlier quoted context omitted.

I would argue then that this isn't Unit Testing, it's some other kind of testing. One of the problems I'm dealing with at the moment is exactly this problem - someone's gone and written a whole ton of tests which involve spinning up pretty much everything short of the DB, and now writing a small test that just tests the behaviour of one class is exceedingly difficult.

If what you "spin up" is internal classes/functionality, not external system, I don't understand the problem? The article doesn't argue "this is how you do unit testing", it argues "this is how you do better testing". It's highly debatable whether unit tests are preferable to larger, "component tests".

The problem is that it makes testing every upstream component more difficult to write for, and more fragile.

To take the shopping cart example I gave, if I write it the way that I understand you're suggesting, I would have to consider the behavior of all of the ICartDiscountStrategy implementations.

For instance, I've written tests for CartDiscountCalculator and ShoppingCart that have 10 different items in the cart from Acme Corp Manufacturing.

Some time later, I implement a new AcmeCorpDiscountStrategy which results in a different discount being returned. Now my tests for CartDiscountCalculator and ShoppingCart need to be modified.

I also need to potentially modify everything else up the dependency graph from CartDiscountCalculator - so my internal API that is used by customer service, an API used by mobiles, and the front end website.

Using Mocks/Fakes to isolate the ShoppingCart, it doesn't matter what the strategies are, we only care about how it interacts with ICartDiscountCalculator.

That interaction might change, maybe ShoppingCart uses the CartDiscountCalculator in a different way yeah, that'll break ShoppingCart Unit Tests - but it's that method only, and you're already in there modifying ShoppingCart anyway.

Re: Prefer Fakes over Mocks

#64

Post by hinkley »

tomgp wrote: post #13

A tangent, and somewhat philosophical: is "The primary purpose of software testing" really "to detect any potential defects in a program before it reaches its intended consumers"? I would argue that whilst that is a commonly held belief, and certainly true for some tests of business logic, the main value and reason for testing is to allow developers to confidently make changes to existing code. In my experience user…

I was left stupefied by a senior staff member asserting that the point of tests was full code coverage. I thought I knew why he left such a wake behind him but apparently he had whole dimensions of crazy I had yet to discover.

Re: Prefer Fakes over Mocks

#65

Post by timclark »

paranoidrobot wrote: post #58
virgilp wrote: post #56

Earlier quoted context omitted.

Um, no. The test needs to know the expected system behaviour, given that dependencies are properly wired & behave as expected. You can argue that "it no longer tests just the ShoppingCart in isolation" and that would be true; but whether that's a downside or an upside is up for debate IMO (and more often than not it really is an upside). [edit] In other words, you don't test that "if discount is 10%, adding item to s…

I would argue then that this isn't Unit Testing, it's some other kind of testing. One of the problems I'm dealing with at the moment is exactly this problem - someone's gone and written a whole ton of tests which involve spinning up pretty much everything short of the DB, and now writing a small test that just tests the behaviour of one class is exceedingly difficult.

It depends on how you define your 'unit'; a unit can be as large or as small as you require.

But, I'd suggest if you can't test a single class in isolation easily (and you might need fakes/mocks) then the class probably needs redesigning.

Re: Prefer Fakes over Mocks

#66

Post by hinkley »

rkangel wrote: post #57
tomgp wrote: post #13

A tangent, and somewhat philosophical: is "The primary purpose of software testing" really "to detect any potential defects in a program before it reaches its intended consumers"? I would argue that whilst that is a commonly held belief, and certainly true for some tests of business logic, the main value and reason for testing is to allow developers to confidently make changes to existing code. In my experience user…

Let's look at it a different way and start with the assumption that in all cases we're delivering reliable software, it's just a question of how we get there. The simplest way of doing this is to write code with good practices then do a load of manual system testing to find bugs (system testing including user acceptance testing in this example). This is basically what most people were doing 20 years ago. This can wor…

> start with the assumption that in all cases we're delivering reliable software.

This is like the physics joke about spherical cows.

You cannot assume this. Good software doesn’t just happen, it takes a ton of work, a lot of which is invisible - if you succeed. People might get lucky or cleverly hide problems in version 1 but in a long enough time horizon, the scales fall away and the hot messes generally reveal themselves.

Re: Prefer Fakes over Mocks

#67

Post by paranoidrobot »

rkangel wrote: post #60
paranoidrobot wrote: post #55

Earlier quoted context omitted.

Perhaps I misunderstand you. Say I'm writing tests for lets say ShoppingCart. ShoppingCart depends on an implementation of ICartDiscountCalculator (CartDiscountCalculator) CartDiscountCalculator depends on a number of implementations of ICartDiscountStrategy (ChristmasCartDiscountStrategy, VIPCustomerDiscountStrategy, BulkPurchaseDiscountStrategy, etc) Each of those, in turn has their own dependencies, perhaps webser…

It shouldn't need to know implementation details, but it should need to know expected behaviour. The use of ICartDiscountStrategy and BulkPurchaseDiscountStrategy is an implementation detail, the high level requirement that buying 10 of something should be cheaper than 9 is just specified system behaviour and that is exactly what you should be testing for. I would refer to these tests as "integration tests" although…

> the high level requirement that buying 10 of something should be cheaper than 9 is just specified system behaviour and that is exactly what you should be testing for.

But that is an implementation detail. Only the BulkPurchaseDiscountStrategy knows whether it's 9 or 10 that's the trigger. Maybe it varies based on time of day or for certain customers.

> I would refer to these tests as "integration tests" although the language is poor here.

Agreed, it's a discussion I'm having at my job now, I havn't found particularly great wording. "Functional" testing still doesn't quite work right.

> Integration tests do not replace unit tests, they should just be taking a lot of the load.

Agreed on the first part. Integration tests are valuable, but I think they're more fragile.

Writing them is, IMO/IME, slower and more difficult.

As for the linked tweet, it's the "mostly integration" part that I disagree with.

Re: Prefer Fakes over Mocks

#68

Post by exabrial »

I don't think he really sells the point to me. You can use any tool incorrectly. An application with poor composition conmbined with mocks without verifications leads you down the path he describes. What we should do is only use "mocks to create fakes", which if you read the manuals on Mockito, is what they're pushing you towards anyway.

So in conclusion I would say that a lot of people use mocking frameworks incorrectly and the author is hitting home on that.

Re: Prefer Fakes over Mocks

#69

Post by hinkley »

FartyMcFarter wrote: post #18
tomgp wrote: post #13

A tangent, and somewhat philosophical: is "The primary purpose of software testing" really "to detect any potential defects in a program before it reaches its intended consumers"? I would argue that whilst that is a commonly held belief, and certainly true for some tests of business logic, the main value and reason for testing is to allow developers to confidently make changes to existing code. In my experience user…

But if we didn't care about defects, developers could confidently make changes to existing code without having tests. The end goal is still avoiding defects, helping developers is just the "middle-man".

The interim goal is objective confidence, not Dunning-Kruger confidence. That’s how you go faster.

There’s always someone who’s willing to plow ahead, safeties off, and ignore the warning signs they see and the complaints of their peers. Denial is a hell of a drug (“That bug can’t be in my code, la la la can’t hear you look how fast I’m going!”).

Some people need a little back pressure.

Re: Prefer Fakes over Mocks

#70

Post by chriswarbo »

silveroriole wrote: post #16
andix wrote: post #12

Earlier quoted context omitted.

Mocks have the same problem. You create something new, that can have uninteneded behavior. Even worse, if you are not perfectly familiar with the mocking framework, you maybe won't even notice it. The good thing about (simple) fakes is, that you can see all the code in a very small class. No complicated library, that does something funky in a special case. Mocking can be very useful though, but it can get too complic…

Mocks are essentially just “calling this function on the mock object during this test returns this result”, though, right? At least, the ones I write are. No state, no logic, no real ‘behaviour’. As soon as you introduce a fake that really has an underlying state and maybe some kind of validation logic (assuming that you want tests to make sure you can’t insert nonsense/retrieve non-existent stuff), you’ve introduced…

I agree that a fake implementation would probably be overkill for such situations, but I would also suggest that anything requiring mocks sounds like overkill too.

If a mock object returning specific results from particular functions is enough to satisfy some code's requirements, then I would seriously consider whether that code could take those results as normal function arguments (possibly lazy/thunked).

Post reply on HN