Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop mocking your system

blog.bitgloss.ro

Post reply on HN
Print view View original

21–30 of 178 posts

Re: Stop mocking your system

#21

Post by yepthatsreality »

rad_gruchalski wrote: post #16

I agree with the premise of this post. Instead of mocking, design your components in such a way that they are easy to spin up in tests. Or use containerized versions of services. I started disliking the idea of mocks a few years ago. I was writing a system based on the Play framework. Play framework used to (still does?) come with a dedicated integration testing environment. The problem with it was that the setup of…

I personally use mocks or stubs only in unit tests. Everything else should be live test or recorded network responses but always run through live code. Unless the service really depends on it, I do not reccommend setting up local swarm of services. This breaks down if you shift to distributed micro services and has little benefit for a single service app (ex crud + database app, no reason to test the database in the app tests).

Re: Stop mocking your system

#22

Post by mahidhar »

I've generally had a little bit more success with mocking when I'm hiding that dependency behind my own interface. So for example in Java, instead of trying to mock the AWS provided class, I write my own class (like a facade or repository pattern) which has a very simple interface of a success case and maybe a couple of relevant failure cases. It's calling the AWS library within it. But my mocks are at the level of my facade class which I find easier. The drawback is I'm not sure if there's a good general strategy to test the implementation of that facade. Most of the times the implementation is simple enough that I can do some simple integration tests for the most relevant cases, but there's always a risk that I am missing out some weird edge cases and I don't know how to properly deal with that.

Re: Stop mocking your system

#23

Post by pydry »

agentultra wrote: post #8

The mocks are the symptom. The problem is that your code doesn't restrict side effects in any way. And so you end up with integration tests for everything and setting up a single test requires recreating the universe from scratch and slightly tweaking it on every run. But that's what our program does, it talks to databases and file systems and HTTP servers! Sure, and the effect of doing those things is moving data ar…

>Sure, and the effect of doing those things is moving data around. What does your program do with the results of these side-effects? Does it parse it? Transform it in any way? Decide whether to run effect A next with the result or effect B? This is the code that, if extracted, can be tested in isolation of databases and HTTP servers.

It's very frequent that this "decision making/calculation" code:

* Doesn't do very much.

* Isn't even in the top 5 source of bugs for your app.

* Integration tests caught those bugs just fine anyway.

* The process of extraction balloons the size of the code base (all those yummy extra layers of indirection), sometimes introducing fun bugs.

It's certainly the right thing to do if you have a self contained chunk of complex decision making/calculation code (e.g. a parser or pricing engine).

However, if you do this as a matter of principle (and far too many do) then this advice isn't just wrong, it's dangerously wrong.

Re: Stop mocking your system

#24

Post by jnaddef »

The author seems to believe people either mock everything or don't mock anything. Obviously using mocks for all your tests is a very bad idea, but that's not how things are done generally.

Unit tests allow you to validate a unit's behavior very quickly. If your unit test takes more than 1 second to run it is probably a bad unit test (some would argue 1/100 second max so your whole unit test suite can complete in a few seconds). In unit tests you use mocks not only to keep the test hermetic, but also to keep the execution time as low as possible.

Then you should have integration & e2e tests where you want to mock as little as possible, because you want a behavior as close as production as possible. For those you care less about how long they take. That's because you usually don't run those tests at the same stage as unit tests (development vs release qualification).

The author does not make the distinction between different types of testing, the resulting article is of pretty poor quality imho.

Re: Stop mocking your system

#25

Post by billytetrud »

jnaddef wrote: post #24

The author seems to believe people either mock everything or don't mock anything. Obviously using mocks for all your tests is a very bad idea, but that's not how things are done generally. Unit tests allow you to validate a unit's behavior very quickly. If your unit test takes more than 1 second to run it is probably a bad unit test (some would argue 1/100 second max so your whole unit test suite can complete in a fe…

I've seen a lot of tests where people just mock everything by default without thinking. Smart programmers at a good company. It's an issue that does deserve more recognition. Abuse of mocks is bad for tests.

Re: Stop mocking your system

#26

Post by gregmac »

> QA says: “it doesn’t work”. Dev says: “it must be working, all the tests pass”.

I've never experienced this, and I'm kind of doubtful this is a true reaction. I guess maybe it could happen in a team that is totally dysfunctional, where there's zero trust between QA and developers .. but in that case mocks are not the problem.

The realistic reaction is "huh, guess we are missing some test cases".

Re: Stop mocking your system

#27

Post by ItsMonkk »

Spivak wrote: post #19
ItsMonkk wrote: post #15

Earlier quoted context omitted.

I'm confused as to where any mention of integration tests are? Integration tests can't use Mocks, as then they wouldn't be testing the integration. Unit tests can use Mocks, but as the article and GP and Mark Seemann[0] points out, that is always worse than writing your logic in a pure way, if you can. [0]: https://blog.ploeh.dk/2017/02/02/dependency-rejection/

Because you don’t gain as much as you think when you do this. Before your external external service touches your integration point and then runs some logic. You mock the external service to unit test the internal logic. Then you run integration tests to exercise the integration point. In the new world you have an external service talking to an integration point which then passes the data to your internal logic. You d…

I think I was confused because I naturally do what you say few people do. My external calls all parse into an object that you can then pass into the unit. It's the integration tests job to ensure that the parse works. Then its the unit tests job to ensure that you run enough that you can be reasonably sure that you cover all cases.

Re: Stop mocking your system

#28

Post by pydry »

jnaddef wrote: post #24

The author seems to believe people either mock everything or don't mock anything. Obviously using mocks for all your tests is a very bad idea, but that's not how things are done generally. Unit tests allow you to validate a unit's behavior very quickly. If your unit test takes more than 1 second to run it is probably a bad unit test (some would argue 1/100 second max so your whole unit test suite can complete in a fe…

>The author seems to believe people either mock everything or don't mock anything.

The author is saying that people frequently mock things that it would be more economic to just run because you've got the real thing right there. Building a model for it is an expensive waste that probably won't even match reality anyway and will demand constant maintenance to sync up with reality.

If you're overtly concerned with the speed of your test suite or how fast individual tests run then you're probably the kind of person he's talking about. Overmocking tends to creep in with a speed fetish.

Re: Stop mocking your system

#29

Post by u801e »

I try to adopt the approach of using a functional core with an imperative shell (even if the latter is handled in a OO fashion).

This way, I can unit test the functional core without the need to use mocks or stubs, and rely on integration tests to test the imperative shell.

Re: Stop mocking your system

#30

Post by pydry »

drewcoo wrote: post #11

This blog post has tell us that we shouldn't use mocks but should instead "send data" to the code we are testing. I think this is supposed to mean using dependency injection instead. But the case isn't really made. Instead, with a waving of hands and wild assertions such as that I'm lying to myself, I'm left wondering what I just read. The previous post was on cargo cult programming. It warns that CCG is bad but can'…

>I think this is supposed to mean using dependency injection instead.

I'm pretty sure he means "just use integration tests".

Post reply on HN