Write tests. Not too many. Mostly integration • Hacker News Board

Re: Write tests. Not too many. Mostly integration

#271

Post by hinkley » Fri, Oct 27, 2017, 4:47 PM UTC

sbergot wrote: post #46

hinkley wrote: post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

In some situations unit tests with lots of mocks will bring a negative value. Imagine a situation where you want to refactor a big piece of code with many dependencies but you don't want to change its public interface. If you mock everything, when you refactor, the test will break because the dependency structure will change, and the mocks are no longer relevant to the new implementation. You have to rewrite the test…

I agree with everything but your conclusion, but I have an aversion to mocks that isn’t shared by everyone.

If the code changed due to a big behavioral shift then your integration and E2E tests aren’t safe. It’s more than twice the work at the higher layers because people get trapped by the Sunk Cost Fallacy. They try and try to save the old tests before they finally rewrite.

That is the observation that convinced me to stick to unit tests. People aren’t emotionally attached to individual unit tests.

Re: Write tests. Not too many. Mostly integration

#272

Post by luord » Fri, Oct 27, 2017, 4:48 PM UTC

marcosdumay wrote: post #264

luord wrote: post #153

> You should very rarely have to change tests when you refactor code. I'm going to say this is poor, if not outright dangerous, advice. I'd argue the opposite: every time the code changes, it would be great if a test somewhere broke. Hell, if the tests originally were for a gigantic class with a ton of mocks but the code changed so there was lower coupling and the dependencies were injectable or separated in differen…

> every time the code changes, it would be great if a test somewhere broke So, it doesn't matter if your code is correct or not? > "the compiler catches some bugs right away!" as an advantage, which always makes me think "... and?" And that leaves more time to design things right and test the stuff that actually matters.

> So, it doesn't matter if your code is correct or not?

I'm going to need you to walk me so I can see how that part you quoted implies that the code being correct doesn't matter. I honestly don't see the correlation.

> And that leaves more time to design things right and test the stuff that actually matters.

I'm not gonna get into a religious flamewar; if you prefer static typing, all the power to you and I'm not interested in convincing you otherwise.

However, I fail to see how having to write "int", "str", etc before or after a variable name impacts the design process in any way.

As for testing:

- A test in Go:

    func TestAvg(t *testing.T) {
    	for _, tt := range []struct {
    		Nos    []int
    		Result int
    	}{
    		{Nos: []int{2, 4}, Result: 3},
    		{Nos: []int{1, 2, 5}, Result: 2},
    		{Nos: []int{1}, Result: 1},
    		{Nos: []int{}, Result: 0},
    		{Nos: []int{2, -2}, Result: 0},
    	} {
    		if avg := Average(tt.Nos...); avg != tt.Result {
    			t.Fatalf("expected average of %v to be %d, got %d\n", tt.Nos, tt.Result, avg)
    		}
    	}
    }

- The exact same test in python:

    def test_average():
        for param in [
        	{'nos': (2, 4), 'res': 3},
        	{'nos': (1, 2, 5), 'res': 2},
        	{'nos': (1,), 'res': 1},
        	{'nos': (), 'res': 0},
        	{'nos': (2, -2), 'res': 0},
        ]:
            assert average(*param['nos']) == param['res']

In a real situation, tests like this one need to be written and the fact that in Go we're specifying types doesn't change a thing, so I'm not convinced that not writing the types means that in a dynamic language I have to test stuff that doesn't matter. For that matter, I'm having a hard time imagining a scenario where the type wouldn't be tested anyway so back to my "... and?"

But, again, I'm also not trying to convince you otherwise. Different approaches for different folks.

Re: Write tests. Not too many. Mostly integration

#273

Post by hinkley » Fri, Oct 27, 2017, 4:49 PM UTC

sitkack wrote: post #178

sbergot wrote: post #46

Earlier quoted context omitted.

In some situations unit tests with lots of mocks will bring a negative value. Imagine a situation where you want to refactor a big piece of code with many dependencies but you don't want to change its public interface. If you mock everything, when you refactor, the test will break because the dependency structure will change, and the mocks are no longer relevant to the new implementation. You have to rewrite the test…

Testing internals forces future programmers of the codebase to maintain those invariants. All code is a liability.

Not in my experience. Convincing people to delete tests that only assert one invariant when the business changes its mind is easy. It’s the ones that have residual value after removing one invariant that trap people into spinning their wheels.

Re: Write tests. Not too many. Mostly integration

#274

Post by killjoywashere » Fri, Oct 27, 2017, 4:53 PM UTC

The title, by the way, is a play on Michael Pollan's famous essay "Unhappy Meals". The top line becomes the subtitle, the lesson, and eventually the title everyone googles for: "Eat food. Not too much. Mostly plants."

http://www.nytimes.com/2007/01/28/magazine/28nutritionism.t....

Re: Write tests. Not too many. Mostly integration

#275

Post by hinkley » Fri, Oct 27, 2017, 4:56 PM UTC

chewbacha wrote: post #53

growse wrote: post #42

Earlier quoted context omitted.

> If you test private methods: you're doing something wrong. Maybe a silly question, but why? If I refactor a class to pull a common piece of functionality into a private method, why would I not want a test for that? One of the principle benefits of tests I see is allowing me to change the implementation without worrying about the behaviour, and I'm not sure why that wouldn't apply to private methods?

One reason why is because you should be testing the public behavior of a function/class not the details. The reason for this is because the public interface is what other parts of the codebase will come to rely on. Refactoring generally shouldn’t change the public interface as it will break other pieces of code within your codebase, or other codebases if it’s a library, and other systems if it’s a network api. So, if…

Some people call this functional testing.

Re: Write tests. Not too many. Mostly integration

#276

Post by hinkley » Fri, Oct 27, 2017, 5:00 PM UTC

hvidgaard wrote: post #7

josteink wrote: post #4

Earlier quoted context omitted.

I work in a company where quite a few of the developers simply are incapable of writing anything but integration-tests. The reason? They don’t “believe” in unit-tests. They don’t think unit-testing “works in the real world”. They absolutely fail to accept that they need to write their code differently for automated testing to work well. How do you change such a mindset?

I doubt that you can. There was a study a while back, and I apologize in advance because I do not have a link, that showed projects written with unittests took significantly longer to reach the market, but with significantly less bugs. However, overall time spend on the code was less. So conclusion was that unittests are a commitment to a long term goal of minimizing developer time, and the tradeoff is that it takes…

That parallels my experiences. I got tired really early on with projects that ground to a halt because of brittleness and a lot of my focus is on building skill and confidence so that version 3 is no harder to ship than version 2 was. Every team I’ve left on good terms was more effective when I left than when I got there. The ones that fought me the whole way frustrate me and it shows.

Re: Write tests. Not too many. Mostly integration

#277

Post by hinkley » Fri, Oct 27, 2017, 5:03 PM UTC

fnord123 wrote: post #20

olavk wrote: post #18

Earlier quoted context omitted.

> Good lord. Why integration tests? Because they can find bugs and errors which unit tests cannot.

This is true. Areas where I have full unit test coverage tend not to have any bugs. What a waste of time to have written all these tests!

Except mine had plenty of bugs nobody ever saw because I spotted them while writing the tests...

Re: Write tests. Not too many. Mostly integration

#278

Post by floor__ » Fri, Oct 27, 2017, 5:04 PM UTC

I wouldn’t trust a person who says they kicked a pigeon.

Re: Write tests. Not too many. Mostly integration

#279

Post by hinkley » Fri, Oct 27, 2017, 5:06 PM UTC

pjmlp wrote: post #62

hinkley wrote: post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

Because there isn't proper way to write unit tests for GUIs for example. They can only test parts of its behavior, not everything, and are too brittle to any simple UI/UX change.

I can’t help but think this is because nobody writes testable GUI frameworks. You can’t build a castle on a swamp, unless you’re Monty Python.

Re: Write tests. Not too many. Mostly integration

#280

Post by hinkley » Fri, Oct 27, 2017, 5:11 PM UTC

crdoconnor wrote: post #37

hinkley wrote: post #3

Good lord. Why integration tests? I think the biggest thing you can do to write more integration tests is to just stop mocking so much stuff. Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. So they try to write E2E tests…

>Okay. The biggest problem I see with people trying to write unit tests is that they don’t want to change how they write code. They just want tests for it. It’s like watching an OO person try their hardest to write OO code in a functional language. I've seen what happens when a developer tries to abstract away a database in a database driven app so it can be "better unit tested". It's a goddamn mess. If your app reli…

In another thread I talk about splitting deciding from doing and I find that strikes a very easy balance for database heavy code. Unit tests for the logic and just concede the data transport to higher level tests. Preferably with a local database full of test fixtures.