Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop mocking your system

blog.bitgloss.ro

Post reply on HN
Print view View original

11–20 of 178 posts

Re: Stop mocking your system

#11

Post by drewcoo »

This blog post has tell us that we shouldn't use mocks but should instead "send data" to the code we are testing. I think this is supposed to mean using dependency injection instead. But the case isn't really made. Instead, with a waving of hands and wild assertions such as that I'm lying to myself, I'm left wondering what I just read.

The previous post was on cargo cult programming. It warns that CCG is bad but can't seem to define it. There is "how to spot" advice but no evidence to show that the advice works (except for a minor appeal to authority).

Yes, English is probably a second language, but the writing has more fundamental problems than that. The author needs to consider what a thesis is and how to support it. And after that, who the audience is. Etc.

Re: Stop mocking your system

#12

Post by Spivak »

agentultra wrote: post #8

The mocks are the symptom. The problem is that your code doesn't restrict side effects in any way. And so you end up with integration tests for everything and setting up a single test requires recreating the universe from scratch and slightly tweaking it on every run. But that's what our program does, it talks to databases and file systems and HTTP servers! Sure, and the effect of doing those things is moving data ar…

But what you’re describing creates tight coupling between your integration points and your internal logic that doesn’t look like tight coupling. You’re not feeding you code real data, you’re only feeding it what you think that external service will provide. And to validate that your tests are correct you need integration tests again and to enforce that your integration point code always produces consistent valid output.

So yeah, the advice to avoid side effects is good but still exercise your integration points with real data and services.

Re: Stop mocking your system

#13

Post by astuyvenberg »

NBJack wrote: post #10

This isn't a terribly practical article. I don't disagree with mocks being an "alternate reality". The author is entitled to their opinion on whether this is a good or bad thing. This said...what is the alternative? Integration testing all the way down? The implication here is to work with stubs over mocks (i.e. I need to work with S3; I would then abstract that to provide a StubObjectStore to replace the S3ObjectSto…

For AWS specifically, I prefer to have an AWS account specifically dedicated to each service + stage. For example, if I have an image service that handles s3 uploads (say Lambda, S3, Cloudfront and API Gateway), then I'd deploy a "test" environment to a dedicated AWS account and run tests against that. Since it's fully serverless, it only costs a few pennies to test (or free).

I try not to develop locally at all anymore. If you're looking for more practical advice, perhaps this will help: https://dev.to/aws-builders/developing-against-the-cloud-55o...

Re: Stop mocking your system

#14

Post by aranchelk »

Ghost of Christmas Future:

I did some contracting work for a company with a 1.5 hour test suite that ran on every deploy, the section with mocked tests only took a couple of minutes — the rest were end-to-end (no mocks).

The worst parts of those non-mocked tests:

* They would interfere with each other, and could not be run in parallel.

* They were subject to real-world variability and were not entirely deterministic.

Management wouldn’t budget fixing or replacing the tests. Bugs still regularly found their way into production; arguments to pare down the tests were vigorously rejected.

My takeaways: If possible, use a language with a strong type system to avoid writing as many tests as possible. Move as much application logic as you can into pure code (so it can be tested in isolation). Observe the test pyramid.

Re: Stop mocking your system

#15

Post by ItsMonkk »

Spivak wrote: post #12
agentultra wrote: post #8

The mocks are the symptom. The problem is that your code doesn't restrict side effects in any way. And so you end up with integration tests for everything and setting up a single test requires recreating the universe from scratch and slightly tweaking it on every run. But that's what our program does, it talks to databases and file systems and HTTP servers! Sure, and the effect of doing those things is moving data ar…

But what you’re describing creates tight coupling between your integration points and your internal logic that doesn’t look like tight coupling. You’re not feeding you code real data, you’re only feeding it what you think that external service will provide. And to validate that your tests are correct you need integration tests again and to enforce that your integration point code always produces consistent valid outp…

I'm confused as to where any mention of integration tests are? Integration tests can't use Mocks, as then they wouldn't be testing the integration. Unit tests can use Mocks, but as the article and GP and Mark Seemann[0] points out, that is always worse than writing your logic in a pure way, if you can.

[0]: https://blog.ploeh.dk/2017/02/02/dependency-rejection/

Re: Stop mocking your system

#16

Post by rad_gruchalski »

I agree with the premise of this post. Instead of mocking, design your components in such a way that they are easy to spin up in tests. Or use containerized versions of services.

I started disliking the idea of mocks a few years ago. I was writing a system based on the Play framework. Play framework used to (still does?) come with a dedicated integration testing environment. The problem with it was that the setup of the testing world slightly differ from the real world. I was bitten a few timer by the real world setup process while the tests were executing flawlessly. In essence, there was no simple way to test the real world construction before deployment to production.

Since then, the only integration tests I accept as real tests are the ones that test the production code path. Database? In containers. Kafka? In containers. etcd? In containers. There are exceptions, though. Proprietary APIs like SQS, or Kinesis, or Firestore are the difficult ones. I usually replace them with hand written memory bound implementations with an abstracted API adhering to some general top level constraints (put / get / consume / publish). This does not prevent errors rooted in the wrong understanding of the design principles of the dependencies, for example, consistency guarantees or delivery ordering, but those are usually possible to cover with additional tests further down the line.

Re: Stop mocking your system

#17

Post by aliasEli »

tuxie_ wrote: post #2

Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more. Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.

The term "unit test" can mean a lot of things (which is very unfortunate).

Does it mean:

- A test of some of the requirements that can be done very fast. (That's my preferred form when involves a large part of the code)

- A test of a small piece of code that makes a lot of assumptions about other code by mocking it. (Not a good idea, but it might work in your organization)

P.S. I did not read your post, while I was writing mine.

Re: Stop mocking your system

#18

Post by ipaddr »

bluepizza wrote: post #4
tuxie_ wrote: post #2

Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more. Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.

Why?

Because he has reached a conclusion. Anything further can be safely ignored if it doesn't fit that conclusion.

Never let someone tell you what unit tests are.

Re: Stop mocking your system

#19

Post by Spivak »

ItsMonkk wrote: post #15
Spivak wrote: post #12

Earlier quoted context omitted.

But what you’re describing creates tight coupling between your integration points and your internal logic that doesn’t look like tight coupling. You’re not feeding you code real data, you’re only feeding it what you think that external service will provide. And to validate that your tests are correct you need integration tests again and to enforce that your integration point code always produces consistent valid outp…

I'm confused as to where any mention of integration tests are? Integration tests can't use Mocks, as then they wouldn't be testing the integration. Unit tests can use Mocks, but as the article and GP and Mark Seemann[0] points out, that is always worse than writing your logic in a pure way, if you can. [0]: https://blog.ploeh.dk/2017/02/02/dependency-rejection/

Because you don’t gain as much as you think when you do this.

Before your external external service touches your integration point and then runs some logic. You mock the external service to unit test the internal logic. Then you run integration tests to exercise the integration point.

In the new world you have an external service talking to an integration point which then passes the data to your internal logic. You don’t need the mocks anymore because you can just call your logic function with your data that would have come from the mock. Great! You run integration tests again to test the integration point. But now you have another linkage at the call site. Your internal logic function now has a dependency on the output of your integration point and that’s an invariant that has to be tested to catch someone modifying one but not the other.

With enough discipline you might be able to make the type system do these kinds of checks for you but IRL very few people do enough to say catch an external string field changing format slightly.

Re: Stop mocking your system

#20

Post by wffurr »

NBJack wrote: post #10

This isn't a terribly practical article. I don't disagree with mocks being an "alternate reality". The author is entitled to their opinion on whether this is a good or bad thing. This said...what is the alternative? Integration testing all the way down? The implication here is to work with stubs over mocks (i.e. I need to work with S3; I would then abstract that to provide a StubObjectStore to replace the S3ObjectSto…

Yes, use stubs. But then also have integration tests.

The point is to have easier to write lower overhead unit tests, and then have a few full fat integration tests that put everything together.

Mocking is a terrible middle ground.

Post reply on HN