Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stop mocking your system

blog.bitgloss.ro

Post reply on HN
Print view View original

61–70 of 178 posts

Re: Stop mocking your system

#61

Post by bhawks »

Mocks make me sad. They are so often misused by people with the best of intentions. I have seen so many tests that literally only assert the mock expectations and say nothing about the code under test. I have watched upgrades and refactors take 5x longer because someone steps on a landmine of overmocking. It is so common to see people mocking dumb data objects, containers, and pure functions - creating Frankenstein widgets that break all contracts the original had except for the 1 code path the original developer used 6 months ago. I've listened to DRY/SRP fanatics defend mocks until they realized that their tests were littered with unreusable, low fidelity copies of their production system in the form of mocks.

Mocks are a power tool and you can use them to do things that they are not the right tool for the job.

Re: Stop mocking your system

#62

Post by JulianMorrison »

I feel this can be summarised as "integration tests > unit tests". And yes.

However, for tests you want to run locally and quickly, the question should be, can you get the speed advantage of unit tests with the largest possible amount of integration? Can you only mock things that are "outside the system" (such as DB, networked services, and file system)?

IMO the biggest risk of integration testing is non-cleanup. I feel that is one of the major positive use cases for mocking. A mock DB will not retain garbage from previous tests, or previous runs of the same test.

Re: Stop mocking your system

#63

Post by 0xbkt »

JulianMorrison wrote: post #62

I feel this can be summarised as "integration tests > unit tests". And yes. However, for tests you want to run locally and quickly, the question should be, can you get the speed advantage of unit tests with the largest possible amount of integration? Can you only mock things that are "outside the system" (such as DB, networked services, and file system)? IMO the biggest risk of integration testing is non-cleanup. I f…

Containers do a good job on this. One should check out the Testcontainers[1] project that makes a good use of containers in language level.

[1] https://github.com/testcontainers/testcontainers-go

Re: Stop mocking your system

#65

Post by sdeframond »

Recently, I thought about a process I call "Test Coverage Driven Testing". It is similar to TDD, but more adapted to when we write tests after the code (you know you do too, at least occasionally, don't lie).

It goes roughly like this:

- write one integration test for the "happy path".

- using some test coverage report, identify untested cases.

- write unit test for those.

I find I helps me find a good balance between time invested writing tests and benefits reaped from those tests.

Do you have a similar process?

Re: Stop mocking your system

#66

Post by fatbird »

If the system doesn't work but all the tests pass, then you're missing one of two things: code coverage, or articulated requirements. Either code that you're not testing is crucially responsible for the system outcome, in which case it should be tested; or the demands of a unit of code with full coverage and passing tests don't include something that a consuming unit needs it to do.

Neither of these conditions is an argument against unit testing or mocking. Both indicate a fairly basic gap in how you're testing that should be fixed directly. If integration tests would catch this gap, you're still missing the basic thing you need while incurring the cost and risks of integration tests.

Re: Stop mocking your system

#67

Post by JulianMorrison »

0xbkt wrote: post #63
JulianMorrison wrote: post #62

I feel this can be summarised as "integration tests > unit tests". And yes. However, for tests you want to run locally and quickly, the question should be, can you get the speed advantage of unit tests with the largest possible amount of integration? Can you only mock things that are "outside the system" (such as DB, networked services, and file system)? IMO the biggest risk of integration testing is non-cleanup. I f…

Containers do a good job on this. One should check out the Testcontainers[1] project that makes a good use of containers in language level. [1] https://github.com/testcontainers/testcontainers-go

They can do a good job, but you need to be very careful. "Oh, it's faster if you start the container once and then run all the tests in it." Yeah but now your tests are all peeing in each other's pool.

The upside of mocks is that the values returned are literally hard coded.

Re: Stop mocking your system

#68

Post by denton-scratch »

This resonates with me.

I'm not at all an expert in testing; but I instinctively take the attitude that every line of test code is code that itself has to be tested, even though it's never going into production. I don't entirely trust that instinct, but I'm suspicious of test rigs and mocks. But I don't know how to test a "module" without them. Que faire.

Re: Stop mocking your system

#69

Post by Chris_Newton »

EdSharkey wrote: post #40
pydry wrote: post #30

Earlier quoted context omitted.

>I think this is supposed to mean using dependency injection instead. I'm pretty sure he means "just use integration tests".

I think so too. Unfortunately, integration tests are too slow, so the practice doesn't scale if one is trying to TDD. Insinuating integration testing into every user story will lead to friction. Test run times will balloon, cycle times will get extended and resentment will grow for the test suite and the team's testing regime. If your test suites cannot complete quickly (seconds), then they cost more than they're wor…

Unfortunately, integration tests are too slow, so the practice doesn't scale if one is trying to TDD.

If integration tests get more useful outcomes than unit tests in some situation but TDD only works well with unit tests, maybe that means TDD isn’t the best process to use in that situation. Isn’t the essence of agile development to recognise what is working well and what is not, so you can make changes where they are needed?

If your test suites cannot complete quickly (seconds), then they cost more than they're worth.

I disagree with this. The goal of testing in software development is to improve results. Any testing methodology should be evaluated accordingly. Fast feedback is good, other things being equal. However, if going slower means getting more valuable feedback — identifying more defects, providing more actionable information about how to correct them, checking for a broader range of potential problems — then maybe going slower is the right choice.

Post reply on HN