Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing like the TSA

37signals.com

Post reply on HN
Print view View original

21–30 of 105 posts

Re: Testing like the TSA

#21

Post by tel »

raganwald wrote: post #8
jashkenas wrote: post #2

A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?

Strongly agree, but also I think that the interesting thing is why this might be so and what it tells us about application architecture. The basic thing about tests is that once you have them passing, they represent statements about constraints on the program. In other words, they express your opinion of things that should not change. Unit tests are a bet that certain aspects of the implementation will not change. In…

Unit testing didn't fully make sense to me until I played around with quickcheck (and eventually theorem proving in Coq). Unit tests vanish nicely to theorems and (empirical) proofs if your code expresses a succinct API. This is one end of the testing continuum.

I use this sort of stuff extensively when doing mathematical computing and statistics because there's usually a clear mathematical boundary. Once you're inside it, it's relatively easy to write down global properties (theorems) of your code's API.

The moment you cross that boundary your testing apparatuses have to get more complex and your tested properties less well-defined. Unit tests are hazier than quickcheck properties and integration tests hazier still.

This continuum seems to be precisely the same as the code reuse continuum. Highly abstracted, testable code with a shapely API is a highly reusable library whether you like it or not. Maybe it's being called by other code, maybe it's being called by your UI, maybe it's being called by the user themselves.

Re: Testing like the TSA

#22

Post by michaelfeathers »

jashkenas wrote: post #2

A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?

All I can say is that it's an idea. It's a shame there's no rationale.

I've seen teams that aggressively put all non-glue code in libraries, but I don't think that is what is meant here.

Re: Testing like the TSA

#24

Post by zmoazeni »

@dhh was talking about this on twitter before he made this post. And I think @dchelimsky has a very fair point that @dhh ignores: testing is all about effort to eliminate some amount of risk.

Test x but don't test y can't be universal. Risks are (often radically) different in every app/team.

(from https://twitter.com/#!/dchelimsky/status/190096365794230274 )

@dhh and 37Signals have a different perspective on acceptable risk for their own product (which they maintain every day) vs developers writing software for someone else/handing it off to someone else. 1 dev to 1 project vs 1 dev to x projects. As a consultant, my acceptable risk level is very different than an entrepreneur trying to push out a MVP and I think testing will reflect that.

Re: Testing like the TSA

#25

Post by thmzlt »

jashkenas wrote: post #2

A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?

100% agreed. Rails people have started to complain about difficult/slow tests because Rails (a framework, not a library) makes your life hard. From my own experience, you don't need all the Rails magic to get started on a project, and once you do, it just happens to be a completely different magic that you need.

Hence I try to divide my projects into lots of application agnostic code (the libraries that need to be unit tested), and little application specific code (the glue code that needs to be integration tested).

Re: Testing like the TSA

#26

Post by adrianhoward »

viraptor wrote: post #7

I don't get the "don't aim for 100%" point. What are you aiming for then? 50%? What happens when you reach 50% and remove some code - do you remove enough tests to match the ratio? We may just phrase the same idea in 2 different ways, but I'd go with "don't force yourself to do 100% coverage if it's not that relevant" / "don't add a test for a simple getter if you have more important things to do". If you can do 100%…

I don't get the "don't aim for 100%" point

For me the point of that one is that test coverage isn't the goal - good code is.

I've seen folk disappearing down a rabbit hole focusing on getting that last 2% of branch coverage using some baroque mock object monkey patched into the system. Their focus was on test coverage. What people who focus on test coverage get is an evil complex test suite that's very brittle in the face of change.

Other, smarter, folk go "damn - I can't test that easily - this code sucks", factor out unrelated functionality into appropriate classes, add some code that makes some duplication between branches obvious, factor out the duplication and end up with something that's better code, with simpler tests - and better test coverage too. Their focus is on the code - not the test coverage.

[Edited for... erm... English]

Re: Testing like the TSA

#27

Post by stcredzero »

astral303 wrote: post #5

Brilliant article. Testing for testings sake is wrong. Testing for 100% coverage sake is wrong. Write just enough tests at the level where it catches most of your regressions. Drill down into unit tests for complex logic, because you can test that more extensively and much faster than an integration test. Then leave a case or two for an integration test to make sure things are hooked up right. Don't be afraid to unit…

Write just enough tests at the level where it catches most of your regressions...make sure you didn't make a stupid off-by-one mistake.

One thing that I've seen in inexperienced coders (including myself in the past) is that they tend to think of every bug as a fluke one-off mistake in an otherwise mostly flawless and awesome record. New coders tend to want to just fix a bug, then pretend it didn't happen.

This is exactly the wrong attitude to take. As a discipline, we programmers should be studying our mistakes and taking steps to prevent them in the future. As a craftsperson striving to improve, each of us should be studying our own mistakes and taking steps to prevent them in the future.

Re: Testing like the TSA

#28

Post by regularfry »

viraptor wrote: post #7

I don't get the "don't aim for 100%" point. What are you aiming for then? 50%? What happens when you reach 50% and remove some code - do you remove enough tests to match the ratio? We may just phrase the same idea in 2 different ways, but I'd go with "don't force yourself to do 100% coverage if it's not that relevant" / "don't add a test for a simple getter if you have more important things to do". If you can do 100%…

Aiming for any specific ratio is missing the point. I think the only rule worth following once you're testing at all is never make it worse unless you understand why you are doing so.
Post reply on HN