Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When to Mock

enterprisecraftsmanship.com

Post reply on HN
Print view View original

31–40 of 82 posts

Re: When to Mock

#31

Post by je42 »

mabbo wrote: post #3

What bothers me about this article is that it doesn't really present alternatives or real-world examples of how to do what the author is saying. > Use real instances of managed dependencies in tests. Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? Because I often want to run my unit tests very frequently as I develop, ensuring I'm still green, etc. And my code ma…

yep. the test pyramid helps figuring when to use what test setup. https://martinfowler.com/bliki/TestPyramid.html

Re: When to Mock

#32

Post by pc86 »

grey-area wrote: post #7
mabbo wrote: post #3

What bothers me about this article is that it doesn't really present alternatives or real-world examples of how to do what the author is saying. > Use real instances of managed dependencies in tests. Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? Because I often want to run my unit tests very frequently as I develop, ensuring I'm still green, etc. And my code ma…

When you're developing, aren't you running a db server with a dev database for that? Just add another separate test db and use that for tests where appropriate. Also, most unit tests don't need a db, it's more useful for integration tests.

If you're using a Repository pattern or similar you'll have a lot of unit tests where a database or db provider is an injected dependency.

Re: When to Mock

#33

Post by sanderjd »

bluejekyll wrote: post #6
mabbo wrote: post #3

What bothers me about this article is that it doesn't really present alternatives or real-world examples of how to do what the author is saying. > Use real instances of managed dependencies in tests. Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? Because I often want to run my unit tests very frequently as I develop, ensuring I'm still green, etc. And my code ma…

Right, the testing pyramid. Your largest set of tests should be a foundation of unit tests that run quickly. Then integration and or functional tests, which is the section this is talking about. They are still necessary, then move up the stack and you get into the end-to-end tests of a fully running system. The reason isn’t only speed but also signal to noise ratio. The further up the testing pyramid you go, the less…

Do people usually mock the database at the integration / functional level? I haven't seen that... I interpreted this to mean the unit test level, which similarly to the thread starter, seems crazy to me.

Re: When to Mock

#34

Post by john-radio »

pgt wrote: post #2

Panic closed the tab when a giant form popup appeared asking for my email. Hate to add noise to the discussion, but I hope the author sees this and understands the market doesn't want this and reduces readership.

ublock origin seems to have stepped on it; I didn't encounter this popup.

Re: When to Mock

#35

Post by tyurok »

doublesCs wrote: post #27
devit wrote: post #20

You should never mock anything. Test against the same dependencies that exist in production. If that's impossible (i.e. you get charged for the backends or you are controlling physical objects), then generalize the program to support alternative backends and frequently test only those that can work in the test environment, using some more ad-hoc methodologies for the others. Also, in general, if you need to change yo…

> Test against the same dependencies that exist in production. Interesting. I said this at my work recently and I got a condescending explanation about how production things are production , we don't touch them. If we need stuff for development, those are dev things. I now think that whether this is or isn't a good idea depends on specifics. Most often than not, I think it makes sense.

That's where "works on my machine" comes from.

The further you steer dev environments from production, the more you'll have these kinds of issues.

Would you mind expanding on their explanation? The way you described doesn't sound like an argument at all.

Re: When to Mock

#36

Post by UK-Al05 »

Most of the reason for 'mocking' databases is because of performance reasons. If you have 8000 tests, having a fake database drastically increases performance.

One trick I use is that I write a in-memory version. I use that in unit tests.

I then write integration tests, that check behaviour of the InMemory and RealVersion are exactly the same. I inject either version into the same tests. They also check I haven't broken any code in the RealVersion which isn't been covered by unit tests mostly because its just external interaction in there.

If you have to verify inter-service interactions, use contract tests.

Re: When to Mock

#37

Post by pc86 »

davedx wrote: post #10
mabbo wrote: post #3

What bothers me about this article is that it doesn't really present alternatives or real-world examples of how to do what the author is saying. > Use real instances of managed dependencies in tests. Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? Because I often want to run my unit tests very frequently as I develop, ensuring I'm still green, etc. And my code ma…

> Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? So what we do where I work is for unit tests, we mock services and repositories, unit tests don't go anywhere near the database. For integration tests, we use an in memory database. BUT! Be careful before you embark down the path of running integration tests against a different database than the one you use when r…

I've yet to have a good testing experience when using anything more than a trivial database if it has EntityFramework on top. The de facto standard seems to be Sqlite for unit tests, which causes some blocking issues if you have the same database name in two different schemas, or composite primary keys, to use my two most recent examples.

Re: When to Mock

#38

Post by pydry »

bluejekyll wrote: post #6
mabbo wrote: post #3

What bothers me about this article is that it doesn't really present alternatives or real-world examples of how to do what the author is saying. > Use real instances of managed dependencies in tests. Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? Because I often want to run my unit tests very frequently as I develop, ensuring I'm still green, etc. And my code ma…

Right, the testing pyramid. Your largest set of tests should be a foundation of unit tests that run quickly. Then integration and or functional tests, which is the section this is talking about. They are still necessary, then move up the stack and you get into the end-to-end tests of a fully running system. The reason isn’t only speed but also signal to noise ratio. The further up the testing pyramid you go, the less…

However, the further down the pyramid you go:

* The more unrealistic the tests generally become (larger % of false positives - tests that fail when they shouldn't - and false negatives - unit tests that simply don't catch bugs at all).

* The less reusable the test infrastructure becomes. Stubbing/mocking individual method calls to the database is an ongoing cost of development whereas building scripts to start the database and shut it down is an investment cost that pays dividends.

On the whole I think the pyramid idea enforces a wrong view that there is a "right" mix of "test levels" across any project. The best mix is determined by the kind of bugs and code you have (integration vs logical) and the kinds of abstractions you need or already have (in general, the worse your abstractions, the higher level you need your tests to be).

A lot of projects are best done with 100% integration tests while others can be done with 100% unit (especially small, self contained, simple-to-interact-with code bases that are 99% about calculations/logical decision making).

Re: When to Mock

#39

Post by devit »

doublesCs wrote: post #27
devit wrote: post #20

You should never mock anything. Test against the same dependencies that exist in production. If that's impossible (i.e. you get charged for the backends or you are controlling physical objects), then generalize the program to support alternative backends and frequently test only those that can work in the test environment, using some more ad-hoc methodologies for the others. Also, in general, if you need to change yo…

> Test against the same dependencies that exist in production. Interesting. I said this at my work recently and I got a condescending explanation about how production things are production , we don't touch them. If we need stuff for development, those are dev things. I now think that whether this is or isn't a good idea depends on specifics. Most often than not, I think it makes sense.

I don't mean testing against the production instance, but rather against an instance configured like the production instance, except for being scaled/sized appropriately for the testing data and workload instead of the production one.

It's also possible to test against live production instances and data itself (mostly useful for performance optimization work and testing), although that's more of an ad-hoc process and some kinds of tests are not possible because the actual data there is arbitrary. Also, those tests, if not used for development, might be better expressed as self-checks done on system start and monitoring systems.

Re: When to Mock

#40

Post by amw-zero »

mabbo wrote: post #3

What bothers me about this article is that it doesn't really present alternatives or real-world examples of how to do what the author is saying. > Use real instances of managed dependencies in tests. Is the author suggesting I should spin up an entire instance of Oracle to run a single 50ms unit test? Because I often want to run my unit tests very frequently as I develop, ensuring I'm still green, etc. And my code ma…

What I got out of the article was that, in unit tests, you should stub the response of a database query. However, you shouldn't directly assert that some database API method is called, because that is one additional level of coupling.

What I don't understand is, a stub to me is almost the same level of coupling as a mock. Your test setup needs to know about the dependency on what you're stubbing. So when you want to change the interface of the dependency or use a different one, or remove the dependency, your tests have to change.

It's not like you can add a query to an operation that didn't have one previously and the result will magically appear without connecting to a real database. So you have to modify the tests to provide the stub. Seems like a logical contradiction to me.

Post reply on HN