Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop mocking your system

blog.bitgloss.ro

Post reply on HN
Print view View original

71–80 of 178 posts

Re: Stop mocking your system

#71

Post by midrus »

aranchelk wrote: post #14

Ghost of Christmas Future: I did some contracting work for a company with a 1.5 hour test suite that ran on every deploy, the section with mocked tests only took a couple of minutes — the rest were end-to-end (no mocks). The worst parts of those non-mocked tests: * They would interfere with each other, and could not be run in parallel. * They were subject to real-world variability and were not entirely deterministic.…

Static typing has nothing to do with testing, or bugs. Static typing is about tooling, autocompletion and being able to follow the code.

Using static typing as a way to reduce the number of tests is just terrible advice in my opinion.

Also, the fact that tests interfere with each other, etc makes me think of just bad tests, not with a problem with the concept of e2e tests itself.

Re: Stop mocking your system

#72

Post by tuxie_ »

catwind7 wrote: post #3
tuxie_ wrote: post #2

Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more. Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.

what does unit testing have to do with whether or not you instrument the test with fake responses? those points of contact that you're mocking out at the perimeter, that data will sometimes need to reach a particular function through a collaborator...which you may want to mock? sometimes the dependency is not a third party, but it may be code that requires a ton of setup (as mentioned in article) that's not worth the…

I'm sorry, I did not intend to offend anyone obviously. Needless to say, this is just my opinion condensed in a sentence (therefore lacking a lot of context, which I should have provided).

I was not aiming to define what a unit test is, more like when it stops being a unit test (which I thought it would be an easier agreement to reach than a definition of what is, but I guess I underestimated the task).

My point was that if you have to mock, for example, a DB call inside your business logic, well you are writing an integration test at that point, whether or not you mock the DB out. If you design your code so that you only have those dependencies at the edge of the system then you get, in my opinion, a much cleaner design and much more testable code.

Too much mocking (and/or more like mocking in the wrong places) is a code smell in my opinion.

Re: Stop mocking your system

#73

Post by arkh »

alecbz wrote: post #51

True "unit" tests are: * faster to run * give you less confidence in the correctness of the system (per time spent writing them) * when they fail, give you more information about where the failure is The more integration-y/e2e-y a test is, the more it strays from this: slower to run, more confidence that the system is correct, less info about where the failures are. I think people have learned to undervalue the prope…

Don't forget a very important thing: unit test are hindering any refactoring. People will resist it because it means they have to rewrite their tests. And if you don't have enough e2e tests, you have nothing to check your refactoring efforts have not broken anything.

Re: Stop mocking your system

#74

Post by tuxie_ »

ipaddr wrote: post #18
bluepizza wrote: post #4

Earlier quoted context omitted.

Why?

Because he has reached a conclusion. Anything further can be safely ignored if it doesn't fit that conclusion. Never let someone tell you what unit tests are.

Hi, I don't really understand who do you refer to when you say that "anything further can be safely ignored". By me? What exactly can be ignored?

Sorry, I would like to answer your comment since it seems that it upset you, and nothing was further from my intention. But I honestly don't understand what you mean.

Re: Stop mocking your system

#75

Post by tuxie_ »

bluepizza wrote: post #4
tuxie_ wrote: post #2

Mocking has a place, which is not unit testing. If you find yourself mocking a dependency in a unit test you are not unit testing any more. Those points of contact with 3rd parties should be clearly defined and encapsulated at the perimeter of your system. Mock at that level. Not when testing business logic.

Why?

Hey bluepizza. As I wrote in response to another comment, I think that too much mocking (and/or mocking in the wrong places) is a code smell.

I was not aiming to define what a unit test is, more like when it stops being a unit test, and I think that if you have to mock, for example, a DB call inside your business logic, well you are writing an integration test at that point, whether or not you mock the DB out. If you design your code so that you only have those dependencies at the edge of the system then you get, in my opinion, a much cleaner design and much more testable code.

Re: Stop mocking your system

#76

Post by devit »

NBJack wrote: post #37
wffurr wrote: post #20

Earlier quoted context omitted.

Yes, use stubs. But then also have integration tests. The point is to have easier to write lower overhead unit tests, and then have a few full fat integration tests that put everything together. Mocking is a terrible middle ground.

That still doesn't address my concern. I cannot test my S3 implementation without either mocks or a very specific emulator of the protocol. AWS happens to be popular enough that some libraries exist to do the latter, but I assert this is the exception rather than the rule for external integrations. You shouldn't be checking in code without at least some unit testing along for the ride, mocks or otherwise. It is indee…

If your external integration has no local alternative, you are getting locked-in to its provider, so you should either not use it, or have an abstraction layer and implement an alternative backend.

Re: Stop mocking your system

#77

Post by lilyball »

Why stop there? Why not delete your test methods too and just test in production?

Mocks are just test code, same as your test functions. And they’re necessary fur unit tests. If the thing you’re testing talks to another component without a mock, it’s now an integration test instead of a unit test.

Unit tests test the API surface of a component. They’re useful for ensuring a component adhere to its documented API contract. It’s also useful for testing how it reacts to edge cases. Integration tests are useful for ensuring components work together, and testing that data flows through the system properly. But integration tests make it harder to test various edge cases.

For example, if my component talks to yours and issues several requests to your component with callbacks, it may be that 99% of the time your component calls the callbacks in the same order. So I can’t really validate in an integration test that my component works properly if the callbacks are involved in a different order. By mocking your component in a unit test, I can test any order I want.

Unit tests also tend to be more deterministic. By mocking, I can eliminate sources of non-determinism from outside my component. As a trivial example, my component might need random numbers, so I might mock the random number generator to return a specific chosen sequence that I know will exercise different code paths.

Really the lesson that should be taught here instead of “don’t use mocks” is “unit tests aren’t sufficient, write integration tests too”.

Re: Stop mocking your system

#78

Post by GrumpyNl »

arkh wrote: post #73
alecbz wrote: post #51

True "unit" tests are: * faster to run * give you less confidence in the correctness of the system (per time spent writing them) * when they fail, give you more information about where the failure is The more integration-y/e2e-y a test is, the more it strays from this: slower to run, more confidence that the system is correct, less info about where the failures are. I think people have learned to undervalue the prope…

Don't forget a very important thing: unit test are hindering any refactoring. People will resist it because it means they have to rewrite their tests. And if you don't have enough e2e tests, you have nothing to check your refactoring efforts have not broken anything.

I have noticed that 80% of the time the unit test fails, the unit test has to be rewritten.

Re: Stop mocking your system

#79

Post by AlphaSite »

EdSharkey wrote: post #40
pydry wrote: post #30

Earlier quoted context omitted.

>I think this is supposed to mean using dependency injection instead. I'm pretty sure he means "just use integration tests".

I think so too. Unfortunately, integration tests are too slow, so the practice doesn't scale if one is trying to TDD. Insinuating integration testing into every user story will lead to friction. Test run times will balloon, cycle times will get extended and resentment will grow for the test suite and the team's testing regime. If your test suites cannot complete quickly (seconds), then they cost more than they're wor…

Why run your whole test suite every time? Chances are you’re only really interested in a small set of them, just let those run continuously.

Re: Stop mocking your system

#80

Post by wffurr »

NBJack wrote: post #37
wffurr wrote: post #20

Earlier quoted context omitted.

Yes, use stubs. But then also have integration tests. The point is to have easier to write lower overhead unit tests, and then have a few full fat integration tests that put everything together. Mocking is a terrible middle ground.

That still doesn't address my concern. I cannot test my S3 implementation without either mocks or a very specific emulator of the protocol. AWS happens to be popular enough that some libraries exist to do the latter, but I assert this is the exception rather than the rule for external integrations. You shouldn't be checking in code without at least some unit testing along for the ride, mocks or otherwise. It is indee…

It's simple enough take to extract the interface of the S3 calls you make into a definition that you can write a test stub that unit tests can pipe fake data into.
Post reply on HN