Is there really such a hard boundary between integration tests, unit tests and other types of tests? If you do something that crunches only numbers and returns numbers you can probably live purely with simple unit tests. But once you get more complex systems you slowly creep into integration tests territory. I feel like it's more of a continuum.
Integration tests are a symptom of poor design
41–50 of 108 posts
Re: Integration tests are a symptom of poor design
#42Disclaimer: I mean no offense. I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers. I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while. When they started their journey the wo…
A long time ago we had Scott Meyers for a week long workshop. He definitely knew his stuff and showed a lot of real world experience. But with a lot of "gurus" I have my doubts. A lot of advice from Scrum gurus falls apart when it meets reality.
I guess the ultimate advice should be given by people like Linus Torvalds who run large projects.
Re: Integration tests are a symptom of poor design
#43Disclaimer: I mean no offense. I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers. I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while. When they started their journey the wo…
I am on the same page with you. In my personal experience, biggest problems occur between interaction of the modules and not so much within particular module. Without integration tests there is just no way to catch these things no matter how good of an architect you are.
Re: Integration tests are a symptom of poor design
#44Disclaimer: I mean no offense. I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers. I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while. When they started their journey the wo…
A long time ago we had Scott Meyers for a week long workshop. He definitely knew his stuff and showed a lot of real world experience. But with a lot of "gurus" I have my doubts. A lot of advice from Scrum gurus falls apart when it meets reality. I guess the ultimate advice should be given by people like Linus Torvalds who run large projects.
I will never question that he knows language and its obscurities, but real world in his case is even more different.
Scott Meyers is a C++ guy. I will bet that nowadays most of C++ programmers are in game development, HFT or other computationally expensive performance oriented areas, and there is this data-oriented design movement which goes against almost everything he ever said. There are entire constructs of language he is proficient in that are never used in real world.
Re: Integration tests are a symptom of poor design
#45Earlier quoted context omitted.
A long time ago we had Scott Meyers for a week long workshop. He definitely knew his stuff and showed a lot of real world experience. But with a lot of "gurus" I have my doubts. A lot of advice from Scrum gurus falls apart when it meets reality. I guess the ultimate advice should be given by people like Linus Torvalds who run large projects.
I will never question that he knows language and its obscurities, but real world in his case is even more different. Scott Meyers is a C++ guy. I will bet that nowadays most of C++ programmers are in game development, HFT or other computationally expensive performance oriented areas, and there is this data-oriented design movement which goes against almost everything he ever said. There are entire constructs of langu…
Back then I was at a C++ shop. He definitely knew how to deal with practical problems and wasn't just a language geek. This was more than 10 years ago so a lot of ideas from then are not fashionable anymore.
Re: Integration tests are a symptom of poor design
#46I can't agree with this. In his example (Transactions), he assumes that there is perfect knowledge of the underlying data and perfect knowledge on the assumptions made by the teams in the corner cases of each module, which is not the case in practice. Very very often, (one of the most common source of errors in programming large systems), two parts of a program are doing the "right" thing, within their assumption, bu…
I don't think you're really disagreeing with Kent. And I haven't found that integration (system) tests have a higher ROI – because they're usually much more expensive to write and maintain. And they can be more brittle than your actual code. If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? Maybe! But the more 'real' the test i…
"And I haven't found that integration (system) tests have a higher ROI – because they're usually much more expensive to write and maintain."
I think the worst code bases to modify are the ones with heavy use of mocking and dependency injection. You still get no feedback about the behavior when run against the real system and you have to maintain a potentially complex mock.
Re: Integration tests are a symptom of poor design
#47Earlier quoted context omitted.
I don't think you're really disagreeing with Kent. And I haven't found that integration (system) tests have a higher ROI – because they're usually much more expensive to write and maintain. And they can be more brittle than your actual code. If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? Maybe! But the more 'real' the test i…
> If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? That's a terrible idea. You've now coupled your tests to someone else's code. Test the endpoints separately. Wrap the endpoints to allow you convenient dependency injection. Test the wrapper. This wrapper is ideally a very thin layer of boilerplate, that you won't ever touch v…
>That's a terrible idea. You've now coupled your tests to someone else's code.
It's a terrible idea not to. If your code isn't coupled to someone else's code your code isn't doing anything useful.
Moreover, you should test against the code your code is coupled to as realistically as possible, which ideally means testing against the real thing.
Re: Integration tests are a symptom of poor design
#48Earlier quoted context omitted.
> If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? That's a terrible idea. You've now coupled your tests to someone else's code. Test the endpoints separately. Wrap the endpoints to allow you convenient dependency injection. Test the wrapper. This wrapper is ideally a very thin layer of boilerplate, that you won't ever touch v…
>That's a terrible idea. You've now coupled your tests to someone else's code. It's a terrible idea not to. If your code isn't coupled to someone else's code your code isn't doing anything useful. Moreover, you should test against the code your code is coupled to as realistically as possible, which ideally means testing against the real thing.
>It's a terrible idea not to. If your code isn't coupled to someone else's code your code isn't doing anything useful.
Paradoxically, you want your tests separate from the external inputs, because complete control over the inputs is required to systematically and programmatically hit every edge case with every pass of your test suite.
You also need the modularity in your tests and code, so you can fix and replace 3rd party endpoints without being afraid of polluting your code's inputs, or having to rewrite your integration suite. This is where the data contracts come in: they make sure that each module of the system and its partners are speaking the same language, even after they have been replaced or updated to handle new 3rd party behaviours.
Re: Integration tests are a symptom of poor design
#49Earlier quoted context omitted.
I don't think you're really disagreeing with Kent. And I haven't found that integration (system) tests have a higher ROI – because they're usually much more expensive to write and maintain. And they can be more brittle than your actual code. If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? Maybe! But the more 'real' the test i…
> If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? That's a terrible idea. You've now coupled your tests to someone else's code. Test the endpoints separately. Wrap the endpoints to allow you convenient dependency injection. Test the wrapper. This wrapper is ideally a very thin layer of boilerplate, that you won't ever touch v…
> You've now coupled your tests to someone else's code.
Your code is coupled to someone else's code. Which yeah, can lead to a lot of problems, but code that doesn't talk to someone else's code is usually not very useful.
Re: Integration tests are a symptom of poor design
#50Earlier quoted context omitted.
I don't think you're really disagreeing with Kent. And I haven't found that integration (system) tests have a higher ROI – because they're usually much more expensive to write and maintain. And they can be more brittle than your actual code. If your code depends on an external service, should your integration test actually make a (test) request to the real live service endpoints? Maybe! But the more 'real' the test i…
> But the more 'real' the test is, the less of your own test it covers relative to all of the other external dependencies. But the more 'real' the test is, the more of your own assumptions get about how your dependencies work get tested. Assumptions are where most bugs are born and hide. PS: Sorry for the multiple replies. HN is being glitchy and my previous comment kept getting truncated.
> But the more 'real' the test is, the more of your own assumptions get about how your dependencies work get tested. Assumptions are where most bugs are born and hide.
Very well put—I like this.