Live data from Hacker News

Integration tests are a symptom of poor design

facebook.com

61–70 of 108 posts

Re: Integration tests are a symptom of poor design

#61

Earlier quoted context omitted.

>That's a terrible idea. You've now coupled your tests to someone else's code. It's a terrible idea not to. If your code isn't coupled to someone else's code your code isn't doing anything useful. Moreover, you should test against the code your code is coupled to as realistically as possible, which ideally means testing against the real thing.

>It's a terrible idea not to. If your code isn't coupled to someone else's code your code isn't doing anything useful. Paradoxically, you want your tests separate from the external inputs, because complete control over the inputs is required to systematically and programmatically hit every edge case with every pass of your test suite. You also need the modularity in your tests and code, so you can fix and replace 3rd…

You're missing a fundamental difference in philosophy. Why should we only care about bugs in our code? They aren't the only bugs our users care about.

A browser bug (for example) may not be our fault, but it is our problem. Therefore, we need to run our tests in real browsers to make sure it actually works, and keep running those tests on new browser releases to find any new bugs. Similarly, mobile apps need to be tested with a variety of different phones.

If you upgrade a dependency and it changes behavior, you're still responsible for fixing or working around it, hopefully before users find out. Lots of interesting bugs happen in the gaps between modules, especially when they're maintained by different teams.

A good reason to test code in isolation is that it makes diagnostics easier. However, if you only do that you're blind to a lot of other bugs.

Re: Integration tests are a symptom of poor design

#62

Disclaimer: I mean no offense. I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers. I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while. When they started their journey the wo…

No offense taken, but let me add this:

I too initially felt a bit weird about the insinuation that "Integration tests are a symptom of poor design", but it's worth pointing out that that's not really the title of his FB note: it's '"Unit" Tests?'. * In fact, going back and reading it a 2nd time, I can't really find anywhere where he's saying you're "doing it wrong" if you're writing integration tests - in fact he says he writes them himself. I think he's just relating a realization he had, that a test should perhaps only be considered an "integration test" if there are systems it depends on that we're not sure are bulletproof yet, i.e. we wonder how well they "integrate". For example, even our "Unit" tests rely on relatively complex systems, e.g. the interpreter of the scripting language you're using. This fits with my (limited) understanding of software testing.

It did seem like he was kind of presuming that pure functional implementations (e.g. of the Transactions objects in his example) would be so much better and more airtight that they probably wouldn't even need testing, but I'm not sure he was really implying that; perhaps he was just using that example to convey that once you really feel like you know the behavior of a subsystem, you take it for granted, and the tests right above it no longer have to "worry" about "integrating" it.

* To be fair, I guess it is KentBeck himself that posted this with that name, so that is a bit "considered harmful"-ish. Not sure what else you'd call it though. Anyways, I enjoyed his reflection, and didn't find it that preachy myself.

Re: Integration tests are a symptom of poor design

#63
post #53

GIVEN a function that returns one or zero, which is fully unit tested. GIVEN a function that divides one number by another, which is fully unit tested. WHEN I integrate both functions together. THEN I should get back a number. This is why you need integration tests. Interaction of fully tested units does not guarantee that they can be integrated together in arbitrary ways without the possibility of bugs being introdu…

Of course, but the point of the original article is that if the given functions work properly in isolation, it won't feel like writing an integration test. It's just an ordinary test.

Re: Integration tests are a symptom of poor design

#64
I liked this bit of perspective from the comments:

When people start arguing about "unit" vs "integration" tests, I think about Smalltalk, the home of the original xUnit testing framework: In Smalltalk, everything is an object. The constant integer value zero is an instance of a class. And your application may define new methods on that class. So in Smalltalk, you literally can't do anything without using some number of classes, many of which may have methods defined by your application. Also, in assembly language, the only abstractions handed to you are bytes, words, and various CPU status flags. The "units" you're working with are extremely low level.

So I conclude that the dividing line between "unit" and "integration" is largely arbitrary, and subject to reinterpretation, depending on the tools and libraries we happen to be using, and possibly how familiar and comfortable we are with them.

Re: Integration tests are a symptom of poor design

#65

Disclaimer: I mean no offense. I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers. I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while. When they started their journey the wo…

No offense taken, but let me add this: I too initially felt a bit weird about the insinuation that "Integration tests are a symptom of poor design", but it's worth pointing out that that's not really the title of his FB note: it's '"Unit" Tests?'. * In fact, going back and reading it a 2nd time, I can't really find anywhere where he's saying you're "doing it wrong" if you're writing integration tests - in fact he say…

I feel like some of these comments are just reading the title and immediately posting their disagreement. I'm with you, he says he writes integration tests himself. There's nothing controversial about this post, it's actually pretty interesting.

Re: Integration tests are a symptom of poor design

#66

Disclaimer: I mean no offense. I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers. I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while. When they started their journey the wo…

No offense taken, but let me add this: I too initially felt a bit weird about the insinuation that "Integration tests are a symptom of poor design", but it's worth pointing out that that's not really the title of his FB note: it's '"Unit" Tests?'. * In fact, going back and reading it a 2nd time, I can't really find anywhere where he's saying you're "doing it wrong" if you're writing integration tests - in fact he say…

I think that caveat that you cite is operative. Although I take it in a slightly different direction than the author.

The point at which integration tests become important is not really a question of bullet-proof-ness. It's a question of complexity. When the underlying system has sufficiently many moving parts/degrees of freedom, at some point, you need to see what it does when your system interacts with it in a certain way.

Generally this is not necessary for integers. In most domains, you're not going to come upon any boundary conditions in the way they are transformed or compared. Transactions are also a pretty simple concept with little in the way of moving parts. If your system doesn't rely on their behavior near boundary conditions, maybe they are transparent enough that you don't have to test your system's interaction with the transaction concept. But there are a lot of underlying systems that are not like that.

Re: Integration tests are a symptom of poor design

#67

Kent is not arguing against integration tests. > I take this as a challenge. I’m happy to write integration tests. I insist on it. But a part of me registers the need for integration tests as a failure or limitation of my design skills. I’ll put that frontier in the back of my mind and a month or a year or a decade later bing I’ll figure out how to raise the level of abstraction, put more of the system in that happy…

It would be a lot easier to evaluate his criticism, sorry, challenge, if he had actually provided an example of a design abstraction that did what he claimed, i.e. turned an integration test into a unit test.

Re: Integration tests are a symptom of poor design

#68

Earlier quoted context omitted.

No offense taken, but let me add this: I too initially felt a bit weird about the insinuation that "Integration tests are a symptom of poor design", but it's worth pointing out that that's not really the title of his FB note: it's '"Unit" Tests?'. * In fact, going back and reading it a 2nd time, I can't really find anywhere where he's saying you're "doing it wrong" if you're writing integration tests - in fact he say…

I think that caveat that you cite is operative. Although I take it in a slightly different direction than the author. The point at which integration tests become important is not really a question of bullet-proof-ness. It's a question of complexity. When the underlying system has sufficiently many moving parts/degrees of freedom, at some point, you need to see what it does when your system interacts with it in a cert…

I think that's right, that other than bullet-proof-ness, the boundary conditions and interfaces between nontrivial subsystems matter a lot and need to be integration-tested. Glad you pointed that out.

Re: Integration tests are a symptom of poor design

#69
I can feel him searching for the difference, and think he is so close but I have a slightly different take on it. He says over time you become confident enough in something that you don’t bother mocking it. Others have said they don’t bother mocking primitives. I would say you don’t bother mocking values. Objects with mutation, methods that cause side effects, these things need to be mocked, pure values never do. The system clock has been around a long time, and I trust it, but it has mutation, so I will mock it. If I make a new point class that is just an immutable wrapper for two immutable integers I won’t mock it. Pure functions never need to be mocked. Immutable values never need to be mocked. These don’t need time to become trusted, they are created trustworthy.

Re: Integration tests are a symptom of poor design

#70

Disclaimer: I mean no offense. I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers. I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while. When they started their journey the wo…

Stroustrup invented C++ and he's actively developing and designing it.

He also almost exclusively talks and writes about C++, he's not telling you anything about integration tests.

You just seem ignorant by including his name on that list.

Post reply on HN