Live data from Hacker News

Stop mocking your system

blog.bitgloss.ro

111–120 of 178 posts

Re: Stop mocking your system

#111

Another fanatic blog post about how something is always correct and another thing is always wrong. Blog posts like this will lure you into thinking that there is a single right approach. Don't fall for it. Do what makes sense. If it doesn't work try something else the next time. Becoming good is about growing your ability to make the right calls, not blindly following a methology.

> Another fanatic blog post about how something is always correct and another thing is always wrong.

The salesman techniques are just so obnoxious too:

"Yes, it is possible and highly desirable and don’t be in denial right now."

Re: Stop mocking your system

#112
I prefer to mock everything, with one exception: Refactoring old code that wasn't built with unit testing in mind.

For this, I'll go with the "social" unit tests up to the point where the code calls an external service or crosses a network boundary. I'll always mock these things because they slow down your tests massively. A 500ms external service or db call might not sound like much but multiply that over 5000 tests.... I'll still try to mock whatever I can that can be mocked with minimal effort.

Re: Stop mocking your system

#113
Nothing should ever be mocked. Period. If you don't agree you likely don't understand many things. Think about why mocks exist in the first place. Mocks exist for things that can't be tested. What you are doing is creating something completely new in place of that thing that can't be tested and testing that new thing instead. Utterly pointless. If it can't be tested, you can't test it period. It's like saying drugs can't be tested on humans, so you create a plastic dummy in the shape of a human to test the drug on instead. Come on man.

There are books, there are experts, there are people with years of experience who think they know what they're doing but the minute I see a mock in a code base which is probably 99% of what's out there I already know that the people who designed the system don't know what they're doing.

Nothing. Ever. Needs. To. Be. Mocked. Many of you are thinking you know better. You don't. Mocking is bad. Allow me to explain.

There are two parts of your code. Code that can be unit tested and code that can't be unit tested.

Code that can't be unit tested is simple. Any code that has to touch IO can't be unit tested. Period. Any code that doesn't touch IO can be unit tested. It's that simple.

Why do people mock? Because people write systems that are too tightly integrated with IO.

Imagine this function:

    function addTwo(x: socket) -> number{
         return socket.get_one_numer() + 2;
    }
Now you have a function that adds 2 to a number. But in order for it to be unit tested you have to Mock the socket. The socket is a parameter polluted with IO. If you have that parameter touch any part of your code then all of that code cannot be unit tested anymore and you have to mock that socket if you want to regain unit testing functionality.

What's the simple way to fix this? Easy Keep ALL IO segregated from the rest of your code. Keep IO functions and methods super small. Do not inject IO polluted objects into other parts of your code. It's trivial:

    function addTwo(x: number) -> number {
          return x + 2;
    } 

    function getNumber(void) -> number {
           return socket.get_one_number();
    }

There. No mocks. addTwo is a function that can be unit tested and getNumber is an IO function that can NEVER be unit tested. That's it. No need to mock it.

There are two types of IO functions. Input and Output. Inputs have void parameters. Outputs have void return values. These are the functions that can't be unit tested. If you keep these functions super small and tiny, guess what? Most of your code can be tested with unit tests and you're golden.

Instead what you'll see throughout your career is typically this garbage:

    class RandomObject(int param1, Socket paramSocket, IOService paramIOService, LogService logService){};
or some other overly complicated, over engineered structure that necessitates Dependency injection or some other garbage pattern that forces people to mock things to test.

Think about it. Every single method you put in that class cannot be unit tested. By using this stupid pattern you pollute that entire class file with IO and nothing can be unit tested unless you mock the socket and/or the IO service.

The problem is, this pattern even though it's so obviously detrimental is used practically everywhere because the complexity of the pattern makes it seem modular and "advanced" when really it's just bad.

Additionally I neglected to mention that it's not only IO. But overly complicated logic sometimes is mocked as well. To that I say it's the same problem as IO. If you find yourself mocking overly complicated logic to test some portion of your code it means that portion of your code is too tightly integrated with the rest of the universe. You need to loosen the coupling.

Re: Stop mocking your system

#114

Why stop there? Why not delete your test methods too and just test in production? Mocks are just test code, same as your test functions. And they’re necessary fur unit tests. If the thing you’re testing talks to another component without a mock, it’s now an integration test instead of a unit test. Unit tests test the API surface of a component. They’re useful for ensuring a component adhere to its documented API cont…

> Why not delete your test methods too and just test in production? This, but unironically. Depending on your use case, any local tests, whether mocked or using a swarm of local containers attempting to represent production may be a far stretch from production reality. Put everything behind feature flags and test your contracts, then release to production regularly and test against live data and live services.

You don't test in production. By definition, prod is what you care about and don't want to break.

This is what preprod is for: an environment that stores, receives and processes the same data as prod. It replicates prod as closely as possible and errors or unexpected differences are investigated.

Then there are rolling deployments in prod...

[and you don't need containers at all]

Re: Stop mocking your system

#115
post #102
post #73

Earlier quoted context omitted.

Don't forget a very important thing: unit test are hindering any refactoring. People will resist it because it means they have to rewrite their tests. And if you don't have enough e2e tests, you have nothing to check your refactoring efforts have not broken anything.

> Don't forget a very important thing: unit test are hindering any refactoring. People will resist it because it means they have to rewrite their tests. Well, unit tests verify a contract. If a developer wants to change code but is incapable of respecting a contract (i.e., preserving old invariants or adding new ones that make sense) then he should not be messing with that code at all, shouldn't he? In that sense alo…

> unit tests verify a contract

This is the most important thing. Rambo can delete all the unit tests he wants in his MR; our discussion will be about whether:

- the contracts have changed.

- the unit tests shouldn't have been there in the first place.

- he's violating contracts

Re: Stop mocking your system

#116
post #38
post #35

Earlier quoted context omitted.

Yeah, he's talking to you for sure.

Yep, and he did a very bad job at it (and so do you) if the goal was to change my mind. Do you maybe have arguments?

Mocks mean your code is too tightly coupled. You should be able to unit test your code by creating only fake data.

Things like dependency injection increase coupling to the point where you have to mock. Avoid dependency injection and other complexity within complexity features.

Re: Stop mocking your system

#117
post #51

True "unit" tests are: * faster to run * give you less confidence in the correctness of the system (per time spent writing them) * when they fail, give you more information about where the failure is The more integration-y/e2e-y a test is, the more it strays from this: slower to run, more confidence that the system is correct, less info about where the failures are. I think people have learned to undervalue the prope…

I think it's a failure to ever believe that unit tests provide any correctness proof. The only thing you achieve with unit tests is to prove that you are still bug for bug compatible when changing your code.

I think experience is needed to know when to unit test. Some code might not need any tests at all while other code might need quite a lot of tests.

My personal experience, for the code bases I work on, shows that for those code bases mocking is usually not beneficial, we find more problems with integration testing and fuzzing. That holds for those code bases, it might very well be different for other code bases, and here's where experience comes in again...

Re: Stop mocking your system

#118
post #48
post #24

The author seems to believe people either mock everything or don't mock anything. Obviously using mocks for all your tests is a very bad idea, but that's not how things are done generally. Unit tests allow you to validate a unit's behavior very quickly. If your unit test takes more than 1 second to run it is probably a bad unit test (some would argue 1/100 second max so your whole unit test suite can complete in a fe…

I've certainly seen people who mock almost everything to test units at the smallest scale possible because they think that's what they're supposed to. E.g., I once saw someone test a factory method like: def make_thing(a, b, c): return thing(a, b, c) with a unit test where they mocked `thing`, and ensured that calling `make_thing(a, b, c)` ended up calling `thing(a, b, c)`. They write just a shit ton of tests like th…

You should see the opposite of this. Where every module of code is unit testable with zero mocks and just a small subset of untestable IO functions packed in a neat corner.

Re: Stop mocking your system

#120
post #23

The mocks are the symptom. The problem is that your code doesn't restrict side effects in any way. And so you end up with integration tests for everything and setting up a single test requires recreating the universe from scratch and slightly tweaking it on every run. But that's what our program does, it talks to databases and file systems and HTTP servers! Sure, and the effect of doing those things is moving data ar…

>Sure, and the effect of doing those things is moving data around. What does your program do with the results of these side-effects? Does it parse it? Transform it in any way? Decide whether to run effect A next with the result or effect B? This is the code that, if extracted, can be tested in isolation of databases and HTTP servers. It's very frequent that this "decision making/calculation" code: * Doesn't do very m…

Be careful. Principled application of this advice can lead to separation of concerns. Programs that are broken down into independently verifiable modules can lead to an abundance of spare time and pursuit of new features in the absence of errors. Ensure your job security by making sure that it is difficult to test your code.

In all seriousness though it's smart to take a layered approach to testing. It is plain good engineering to be able to separate your program's concerns into independent, verifiable modules and to isolate side-effects from pure code. Definitely write a few integration tests to verify that the modules work together and that global properties like configuration have the desired behaviours. And by jolly live dangerously and test in production!

But don't do it dangerously. Make sure you have the right team and infrastructure to test in production safely without taking down production or frustrating customers. But test in production for reals. Integration tests are fine but nothing compares to prod. Not mocks, synthetic inputs, simulated environments, and not good intentions.

Post reply on HN