Testing like the TSA
37signals.com
Testing like the TSA
1–10 of 105 posts
Re: Testing like the TSA
#2A thought that folks reading this post might have an opinion on:
"Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..."
Agree or disagree?
Re: Testing like the TSA
#3A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?
That sounds good, with the caveat that most applications should have their business logic roughly organized into libraries. After those libraries get their unit tests, integration testing the applicationlibrary barrier makes a lot of sense.
Re: Testing like the TSA
#4STOP. BREATHE. THINK.
Here's the Stack Overflow post that I think triggered this post:
http://stackoverflow.com/questions/153234/how-deep-are-your-...
Yes, he said "fuck". Let's get past that, because it's a super boring discussion to have on HN.
Also: pointing out specific 37signals or Rails bugs as evidence that this view is flawed? Also a super super boring way to argue. Argue with the idea, not the proponent of the idea.
Ok, feel better? Now comment on this story.
Re: Testing like the TSA
#5Brilliant article. Testing for testings sake is wrong. Testing for 100% coverage sake is wrong. Write just enough tests at the level where it catches most of your regressions. Drill down into unit tests for complex logic, because you can test that more extensively and much faster than an integration test. Then leave a case or two for an integration test to make sure things are hooked up right.
Don't be afraid to unit test little complex things here and there. Are you writing a function to parse a string in a certain way? Pick that function, elevate its visibility if need be, write a simple unit test to make sure you didn't make a stupid off-by-one mistake. Does the rest of the class otherwise not loan itself to unit testing? That's OK, move on.
We've learned that each line of code is a liability, even if it's a configuration file, which is why we have come to appreciate things like DRY, convention over configuration, less verbose languages, less verbose APIs. Likewise, each line of test code is a liability, so each line better justify itself.
Re: Testing like the TSA
#6It is not fun to pull out the power tools on a huge piece of software with 50 thousand unit tests and see that 600 of them fail because you changed an important part if the code.
It means the unit tests will have to be deleted, and or updated. If each takes 10 Minutes that is 6000 Minutes.
On the other hand, seeing one unit test fail in another module you didn't expect to fail can save you a week of work, so it's all in how brittle and organized the tests are.
Re: Testing like the TSA
#7I don't get the "don't aim for 100%" point. What are you aiming for then? 50%? What happens when you reach 50% and remove some code - do you remove enough tests to match the ratio?
We may just phrase the same idea in 2 different ways, but I'd go with "don't force yourself to do 100% coverage if it's not that relevant" / "don't add a test for a simple getter if you have more important things to do". If you can do 100% and have time for it - you should definitely do that. For example in case someone rewrites your getter, but it's not that simple anymore.
Re: Testing like the TSA
#8A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?
Strongly agree, but also I think that the interesting thing is why this might be so and what it tells us about application architecture.
The basic thing about tests is that once you have them passing, they represent statements about constraints on the program. In other words, they express your opinion of things that should not change. Unit tests are a bet that certain aspects of the implementation will not change. Integration tests are a bet that certain aspects of the externally visible behaviour will not change.
Libraries tend to be smaller and with well-defined responsibility. Applications tend to be bigger and have many responsibilities. In general, I think it’s true that the requirements for libraries change less often than the requirements for applications. I think this leads us to expect that applications may need to be rewired “under the hood” and have their implementations changed as responsibilities are added, removed, or changed.
This, I believe, leads us to want to unit test applications less, because a unit test expresses implementation semantics, and we expect application implementations to change. No what about integration tests? Well, if we’re unit testing less in the application, we need to make up for it by integration testing more, otherwise where do we get our confidence?
Now if we throw the words “library” and “application” away, this suggests to me that those parts of the code that are small and tight and with a single, clear responsibility should be unit tested, while those parts that involve a lot of what the AOP people call ‘scattering and tangling,’ should be integration tested.
Thoughts?
Re: Testing like the TSA
#9A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?
It does sound right, where is this quote from?
Re: Testing like the TSA
#10I completely agree with David's assertion that there is too little focus paid to how to test properly or what over-testing looks like.
Here's a question to the HN community... For a library you write yourself to, say, access a Web service, how do you go about testing it? (For example, if you want to write tests against an Akismet gem)
I tend to write both unit tests and integration tests for it. My unit tests mock out the HTTP calls made by the library and only test that the library is able to handle both good and bad inputs.
My integration tests allow the library to speak directly to the web service in question, to test that the correct connection is being made and the service is providing the correct data back to the library.
Is this overkill?