Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Prefer Fakes over Mocks

tyrrrz.me

Post reply on HN
Print view View original

21–30 of 125 posts

Re: Prefer Fakes over Mocks

#21

Post by nickff »

FartyMcFarter wrote: post #18
tomgp wrote: post #13

A tangent, and somewhat philosophical: is "The primary purpose of software testing" really "to detect any potential defects in a program before it reaches its intended consumers"? I would argue that whilst that is a commonly held belief, and certainly true for some tests of business logic, the main value and reason for testing is to allow developers to confidently make changes to existing code. In my experience user…

But if we didn't care about defects, developers could confidently make changes to existing code without having tests. The end goal is still avoiding defects, helping developers is just the "middle-man".

I agree with this view, and would like to add that I see improving the tests available to the developer as really speeding up the OODA loop of development.

Re: Prefer Fakes over Mocks

#22

Post by hackerfromthefu »

andix wrote: post #17
doubletgl wrote: post #6

How are fakes, as described in this article, not also implementation-aware testing?

Because they don't just implement a part of the interface (just one store method, or just one of two load methods). They try to create a fully functional fake of the original class. In this case instead of storing the blobs via a REST interface or a database connection, it just keeps it in memory. So it should mostly behave like the real implementation, but is not suitable for production, as it doesn't persist the da…

>> .. fully functional fake of the original class.

So they are coupled to the implementation of the object in any case!

Re: Prefer Fakes over Mocks

#23

Post by zabil »

andix wrote: post #10

Fakes are great for objects that store data. Like in this example. If I use Entity Framework Core I also love the in-memory provider (or SQLite in-memory), which gives you a very well faked database context for unit tests. In some cases you can create end to end tests, that run and verify the complete business logic, without hitting the database at all. A mixture between unit and integration tests. https://docs.micro…

> If I use Entity Framework Core I also love the in-memory provider (or SQLite in-memory), which gives you a very well faked database context for unit tests.

This approach easy to setup, however I found that it slows down unit tests especially with data migrations and setup before each test (to discard test data from other tests). I've since limited these to test only migrations and in some well thought out integration testing.

Re: Prefer Fakes over Mocks

#24

Post by banthar »

silveroriole wrote: post #16
andix wrote: post #12

Earlier quoted context omitted.

Mocks have the same problem. You create something new, that can have uninteneded behavior. Even worse, if you are not perfectly familiar with the mocking framework, you maybe won't even notice it. The good thing about (simple) fakes is, that you can see all the code in a very small class. No complicated library, that does something funky in a special case. Mocking can be very useful though, but it can get too complic…

Mocks are essentially just “calling this function on the mock object during this test returns this result”, though, right? At least, the ones I write are. No state, no logic, no real ‘behaviour’. As soon as you introduce a fake that really has an underlying state and maybe some kind of validation logic (assuming that you want tests to make sure you can’t insert nonsense/retrieve non-existent stuff), you’ve introduced…

Mocks have to be updated and fixed every time you do any meaningful changes in production code. Good fake can be reused in many tests and will keep behaving like a real thing with minimum maintenance. Much less complex than maintaining hundreds of mocks each implementing different parts of the interface.

Re: Prefer Fakes over Mocks

#25

Post by kelnos »

silveroriole wrote: post #11

The fatal flaw is that you’re really introducing a new class which may have its own behaviour problems. Now your test is implicitly testing the test class. I also prefer to see explicit mocking than just trust that the fake does what it says it will. How do I know without looking whether the fake store method REALLY stores the doc, or whether someone set it up to just return OK? And how do you test error handling - m…

The article addresses this: you need to write tests for the fake as well.

I'm not sure I buy this as something that's desirable to have to do, but it should address your concern.

Re: Prefer Fakes over Mocks

#26

Post by silveroriole »

andix wrote: post #20
silveroriole wrote: post #16

Earlier quoted context omitted.

Mocks are essentially just “calling this function on the mock object during this test returns this result”, though, right? At least, the ones I write are. No state, no logic, no real ‘behaviour’. As soon as you introduce a fake that really has an underlying state and maybe some kind of validation logic (assuming that you want tests to make sure you can’t insert nonsense/retrieve non-existent stuff), you’ve introduced…

You're totally right. But think about a blob storage class that has two methods, ReadOneFile(), ReadMultipleFiles(). You have one test that just mocks ReadOneFile(string fileName) and doesn't mock the ReadMultipleFiles(). You now change the implementation of the System Under Test (SUT) to use once the ReadMultipleFile(string[] fileNames) call instead of three times the ReadOneFile(). Now your test fails, but the impl…

Yes, good point, that’s true. Personally, I know it’s not the ‘right’ ethos, but I see the mock test failure as a bit of a bonus. My change intended to stop calling ReadOne and start calling ReadMultiple. Having to change the test sure tells me I accomplished that! But you’re right that if I’m intending to only test behaviour, the fake lets me do that better. Interesting post, thank you :)

Re: Prefer Fakes over Mocks

#27

Post by silveroriole »

kelnos wrote: post #25
silveroriole wrote: post #11

The fatal flaw is that you’re really introducing a new class which may have its own behaviour problems. Now your test is implicitly testing the test class. I also prefer to see explicit mocking than just trust that the fake does what it says it will. How do I know without looking whether the fake store method REALLY stores the doc, or whether someone set it up to just return OK? And how do you test error handling - m…

The article addresses this: you need to write tests for the fake as well. I'm not sure I buy this as something that's desirable to have to do, but it should address your concern.

Hmm, if I’m having to write tests for my tests, I feel something has gone wrong in my life ;)

Re: Prefer Fakes over Mocks

#28

Post by andix »

doubletgl wrote: post #7
ddek wrote: post #4

Earlier quoted context omitted.

I question a lot of advice like this, and agree with you. I’d generally far rather use NSubstitute than have to write fake classes, but I can see the utility in some cases. My honest response is this: Writing tests should be quick and painless. Maintaining fakes is more painful than maintaining substitutes. Substitutes let me interrogate side effects in great detail. I’m not concerned will performance in unit tests.…

> why should anyone care about someone else’s test architecture, of all things. As long as you neither under- or over-testing, why on earth does it matter? If you have to maintain tests written by other devs you might start to care. Devs apply different patterns to writing tests and show different amounts of discipline doing so. Some produce lots of duplication ("it doesn't matter, it's just a unit test"), others pre…

I've seen unit tests that test around bugs. Once you fix the bug, a lot of tests fail. Then you discover there is no easy fix for the test. Either delete them all or replace them with new ones.

Re: Prefer Fakes over Mocks

#29

Post by scresswell »

I've tried this approach twice with mixed success. In both cases I wanted to stub out the persistence tier of a Node.js application when test driving the API. I verified the real and fake implementations by running the same tests against them. The first application was quite small, and the process worked well, although it did feel somewhat onerous to implement the fake. However, the second application was more complex, calling for some PostgreSQL specific features which were difficult to implement in the fake. In the end I abandoned the approach for the second application, settling on slower, duplication heavy API tests which used the real persistence tier. I'd love to hear how others solve this problem without mocks, which I agree tend towards brittle, tightly coupled tests with a poorer signal to noise ratio.

Re: Prefer Fakes over Mocks

#30

Post by jakevoytko »

silveroriole wrote: post #11

The fatal flaw is that you’re really introducing a new class which may have its own behaviour problems. Now your test is implicitly testing the test class. I also prefer to see explicit mocking than just trust that the fake does what it says it will. How do I know without looking whether the fake store method REALLY stores the doc, or whether someone set it up to just return OK? And how do you test error handling - m…

This isn't a "fatal flaw." It's a tradeoff. You add a little bit of extra code (which is a liability), and in exchange, you get a test double that you can reuse in many places (which is a benefit). This is in opposition to mocks, where you get an automated framework that can substitute calls anywhere in the system (a benefit), but you're completely replicating the behavior in every single test (a liability).

I argue that [0] the liability of mocks are much higher than the fake. With a fake, you could theoretically write tests that asserts that your class is satisfying the interface. There's no way to do this with a mock. You can just make an interface do anything, regardless of whether it makes sense. This is fragile under modification. With a large codebase, you're inevitably going to change the behavior of a class during a refactoring. Are you going to read every test that mocks that class to decide how the mocks should be rewritten? Probably not - the mocks force-overrode the behavior of the class, so your tests will still pass. Why would you look at them again?

> Make a fake for every error scenario?

By the time you're implementing a fake, you have an interface that the fake implements. One way that I've handled this is to make a FailingBlobStorage that takes an error in its constructor and throws it from any calls. If you need even more detailed behavior than that, you can create a mock for that specific test case. You're not married to the fake. It's not going to magically test every scenario. But fakes handle the common case in so many situations that it's actually surprising when you start trying them out.

[0] In fact, I've basically written this same blog post before. https://www.bitlog.com/2019/12/10/simple-software-engineerin...

Post reply on HN