Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Integration tests are a symptom of poor design

facebook.com

Post reply on HN
Print view View original

51–60 of 108 posts

Re: Integration tests are a symptom of poor design

#51

Post by mcphage »

WorldMaker wrote: post #17
pavement wrote: post #8

Networked resources have listeners. Listeners have ears. Ears have wax. A unit test verifies that your Q-tip is made of cotton, and that the cotton is soft and small enough to fit inside an ear to a depth that will fetch wax. Another unit test might confirm that a swabbed substance is actually ear wax, by reporting the qualities of a verified sample of earwax, and maybe also a sample of candle wax as a true negative.…

That's an interesting choice of analogy given that boxes of Q-Tips warn you to never try cleaning your ears with a Q-Tip and so do most professionals: http://time.com/4290668/q-tip-ear-wax-removal/ Current professional wisdom is that you shouldn't even bother cleaning your ears because wax buildup is healthier and the body's own wax removal process is good enough.

> the body's own wax removal process is good enough

It really isn't, though. If I go more than a few days without cleaning my ears they get really crusty and it gets harder to hear.

Re: Integration tests are a symptom of poor design

#52

Post by Blazespinnaker »

Chaebixi wrote: post #33
aeorgnoieang wrote: post #14

Earlier quoted context omitted.

I don't think you're really disagreeing with Kent. And I haven't found that integration (system) tests have a higher ROI – because they're usually much more expensive to write and maintain. And they can be more brittle than your actual code. If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? Maybe! But the more 'real' the test i…

> And they can be more brittle than your actual code. The brittlest tests I've ever seen have been mock-heavy unit tests. Some of them amount to little more than file-change detectors. Those tests can fail massively even when everything still actually works. They cry wolf so often the team ends up trained to unthinkingly change them so they "go green." > But I think I get what he's getting at in terms of – ideally –…

My experience with unit tests is that they force a developer to think through their logic. So the tests themselves,once passing don’t help that much (though a little), but the act of writing them did have high ROI. The coverage metrics also help management understand what is baked.

Re: Integration tests are a symptom of poor design

#53

Post by didibus »

GIVEN a function that returns one or zero, which is fully unit tested.

GIVEN a function that divides one number by another, which is fully unit tested.

WHEN I integrate both functions together.

THEN I should get back a number.

This is why you need integration tests. Interaction of fully tested units does not guarantee that they can be integrated together in arbitrary ways without the possibility of bugs being introduced from the integration.

The above example integ test would reveal a DivideByZero error. This is a super simplistic integration of two simple functions, now imagine a massive enterprise system and how many bugs could be due to the integration of its parts alone, even if individually each part is one hundred percent correct and bug free.

Re: Integration tests are a symptom of poor design

#54

Post by iainmerrick »

didibus wrote: post #53

GIVEN a function that returns one or zero, which is fully unit tested. GIVEN a function that divides one number by another, which is fully unit tested. WHEN I integrate both functions together. THEN I should get back a number. This is why you need integration tests. Interaction of fully tested units does not guarantee that they can be integrated together in arbitrary ways without the possibility of bugs being introdu…

That's what I was about to say too! But you said it better. :)

Unit tests are for standalone modules. Integration tests are for apps. It's definitely good to factor stuff out into modules where you can, but ultimately you'll still want to check that everything works when it's all hooked up together.

I guess the alternate approach would be writing "mocks" for the submodules, so the higher-level app can be treated as a module itself, and unit tested rather than integration tested. I've found that to be difficult and error-prone compared to integration testing (or normal unit testing for that matter) -- as you say, unexpected interactions between modules can easily fall through the cracks.

Re: Integration tests are a symptom of poor design

#55

Post by WalterSear »

mcphage wrote: post #49
WalterSear wrote: post #38

Earlier quoted context omitted.

> If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? That's a terrible idea. You've now coupled your tests to someone else's code. Test the endpoints separately. Wrap the endpoints to allow you convenient dependency injection. Test the wrapper. This wrapper is ideally a very thin layer of boilerplate, that you won't ever touch v…

> You've now coupled your tests to someone else's code. Your code is coupled to someone else's code. Which yeah, can lead to a lot of problems, but code that doesn't talk to someone else's code is usually not very useful.

> You've now coupled your tests to someone else's code.

Re: Integration tests are a symptom of poor design

#56

Post by cjalmeida »

WalterSear wrote: post #48
crdoconnor wrote: post #47

Earlier quoted context omitted.

>That's a terrible idea. You've now coupled your tests to someone else's code. It's a terrible idea not to. If your code isn't coupled to someone else's code your code isn't doing anything useful. Moreover, you should test against the code your code is coupled to as realistically as possible, which ideally means testing against the real thing.

>It's a terrible idea not to. If your code isn't coupled to someone else's code your code isn't doing anything useful. Paradoxically, you want your tests separate from the external inputs, because complete control over the inputs is required to systematically and programmatically hit every edge case with every pass of your test suite. You also need the modularity in your tests and code, so you can fix and replace 3rd…

I usually have this concept of integration tests covering the interaction of modules you are in control. 3rd party services are tested using something I call "edge tests".

This leads to few mocks, yet I know the expected behavior of the external services.

Re: Integration tests are a symptom of poor design

#57

Post by jcrben »

WalterSear wrote: post #55
mcphage wrote: post #49

Earlier quoted context omitted.

> You've now coupled your tests to someone else's code. Your code is coupled to someone else's code. Which yeah, can lead to a lot of problems, but code that doesn't talk to someone else's code is usually not very useful.

> You've now coupled your tests to someone else's code.

And so your tests actually demonstrate working code!

Re: Integration tests are a symptom of poor design

#58

Post by corpMaverick »

Kent is not arguing against integration tests.

> I take this as a challenge. I’m happy to write integration tests. I insist on it. But a part of me registers the need for integration tests as a failure or limitation of my design skills. I’ll put that frontier in the back of my mind and a month or a year or a decade later bing I’ll figure out how to raise the level of abstraction, put more of the system in that happy state where I have complete confidence in it, and I’ll have new tools for thinking. Yippee!

He is trying to figure out which design abstractions are yet to be invented. It is a challenge not a criticism.

Re: Integration tests are a symptom of poor design

#59

Post by dboreham »

aeorgnoieang wrote: post #2

A link (via Internet Archive) in case, like me, you can't access Facebook wherever you are: https://web.archive.org/web/20171206200104/https://www.faceb...

>like me, you can't access Facebook wherever you are

Wherever it is, it sounds wonderful.

Re: Integration tests are a symptom of poor design

#60

Post by matt_sales »

didibus wrote: post #53

GIVEN a function that returns one or zero, which is fully unit tested. GIVEN a function that divides one number by another, which is fully unit tested. WHEN I integrate both functions together. THEN I should get back a number. This is why you need integration tests. Interaction of fully tested units does not guarantee that they can be integrated together in arbitrary ways without the possibility of bugs being introdu…

you're missing a unit test case for your division function -- the case of 0. That test will reveal the function's domain is non-zero numbers. And you know this because you got the fast feedback from your unit test. No integration test using 0 necessary unless you want to test the failure scenario.

BTW, nobody is saying not to use integration tests. But unit tests are faster, more stable, and easier to maintain and execute.

Post reply on HN