Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing like the TSA

37signals.com

Post reply on HN
Print view View original

41–50 of 105 posts

Re: Testing like the TSA

#41

Post by adrianhoward »

skrebbel wrote: post #32

Nice read. I'm no Rails dev, so I'm curious about this one point from DHH: > 6. Don't use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests (and send me a bottle of fairy dust if you're there!) I mostly do C#, and teams I've recently been on have found SpecFlow tests to be an excellent time saver in communicating requirements and acceptance test criteria with customers. Has Cucumber not b…

Using BDD for having non-programmers write tests sounds far-fetched to me indeed. It's excellent to have them able to read and understand the tests, though. Any opinions? Is BDD as dead horse, or is DHH a little narrow minded here?

I thought what you thought - that this is DHH's working context speaking. For me Cuke-ish tests are about alignment and communication - not about having non-developers write tests.

(of course Cucumber isn't the only route for doing that)

Re: Testing like the TSA

#42

Post by damoncali »

skrebbel wrote: post #32

Nice read. I'm no Rails dev, so I'm curious about this one point from DHH: > 6. Don't use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests (and send me a bottle of fairy dust if you're there!) I mostly do C#, and teams I've recently been on have found SpecFlow tests to be an excellent time saver in communicating requirements and acceptance test criteria with customers. Has Cucumber not b…

My own observation is that some ruby/rails developers get so enthralled with testing that it becomes an obsession that eclipses the product that they are building. The result (strong opinion coming...) is libraries like rspec and cucumber. They're complex and burdensome, and tend to attempt to mimic english, but often do so poorly and are totally unsuitable for non-coders. You spend a lot of time learning "the way" to do things and wind up with code that is unintelligible to less experienced developers.

I use test::unit and minitest and it gets the job done without having to keep up with the latest trends. It's simple and can be written in a way that correlates well to actual English requirements. It takes all of an hour to digest minitest from zero.

That said, Cucumber and rspec are very popular, so I may be the weird one.

Re: Testing like the TSA

#43

Post by damoncali »

Don’t use Cucumber...

Thank God someone with a bullhorn finally said this. I was beginning to think I was alone in my hatred of Cucumber. (And my love of Test:Unit/Minitest.)

Re: Testing like the TSA

#44

Post by ejames »

This is why I left Microsoft. Automated testing was a separate discipline - meaning there were "dev devs" and "test devs". Automated tests were written based on what the "test devs" had time for, not on the need or usefulness of such tests for the actual code. I was hired as a "test dev" - I had no industry experience at the time and figured I would give it an unprejudiced try to see if I liked it.

I quickly realized that my job was futile - many of the "good" tests had already been written, while in other places, "bad" tests were entrenched and the "test dev" had the job of manning the scanner to check for nail clippers, or upgrading the scanner to find as many nail clippers as possible.

Here's a useful rule on the subject that I picked up from an Artificial Intelligence course back in the day: The value of a piece of information is proportional to the chance that you will act on it times the benefit of acting on it. We all realize there is no benefit in testing if you ignore failures rather than acting to fix the bugs, but in much the same way that doing nothing when tests fail has no benefit, doing nothing when tests pass also has no benefit - so tests which always pass are just as useless as failing tests you ignore, as are tests which only turn up corner-case bugs that you would have been comfortable with shipping.

If you're doing the right amount of testing, there should be a good chance, whenever you kick off a test run, that your actions for the next hour will change depending on the results of the run. If you typically don't change your actions based on the information from the tests, then the effort spent to write tests gathering that information was wasted.

Re: Testing like the TSA

#45

Post by orblivion »

My first unit testing experience (Django), it wasn't 1:2 or 1:3, it was more like 2:1. As far as time spent, I don't even know if I could separate the two, I spent so much time in tests and production at the same time. I actually want to scrap that code and restart it, it's not worth changing all those damn tests any time something else changes. I tend to go to extremes, I'm actually thinking of switching to Yesod next to see if I can mitigate a lot of the reasons for testing.

Re: Testing like the TSA

#46

Post by romaniv »

samd wrote: post #35

Testing isn't just about preventing bugs, it also encourages better design, and good design makes extending and enhancing your application much easier and faster. ...in general, of course, there's always exceptions, don't be dogmatic.

I think this aspect of unit testing is largely overrated. True, it does stop people from writing something ridiculous, like a 1000-line method that handles a dozen different things. But it also encourages developers to go overboard in the other direction: creating endless, needless layers of abstraction, chopping logic into bits that are to small to represent anything in reality and doing dependency injection when it's not needed. It's important to keep in mind that ease of unit testing doesn't automatically ensure ease of use of your library in real code.

In short, it's better to think of unit testing as bug-prevention mechanism. Thinking of it as an ultimate design mechanism doesn't end up well.

Re: Testing like the TSA

#47

Post by SoftwareMaven »

ejames wrote: post #44

This is why I left Microsoft. Automated testing was a separate discipline - meaning there were "dev devs" and "test devs". Automated tests were written based on what the "test devs" had time for, not on the need or usefulness of such tests for the actual code. I was hired as a "test dev" - I had no industry experience at the time and figured I would give it an unprejudiced try to see if I liked it. I quickly realized…

I disagree that there is no benefit in passing tests that don't change your behavior. Those tests are markers to prevent you from unknowingly doing something that should have changed your behavior. That is where the nuance enters: is this a marker I want to lay down or not? Some markers should be there; others absolutely should not and just introduce noise.

Re: Testing like the TSA

#48

Post by MaysonL »

raganwald wrote: post #8
jashkenas wrote: post #2

A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?

Strongly agree, but also I think that the interesting thing is why this might be so and what it tells us about application architecture. The basic thing about tests is that once you have them passing, they represent statements about constraints on the program. In other words, they express your opinion of things that should not change. Unit tests are a bet that certain aspects of the implementation will not change. In…

On this, I tend to think that many (most?) libraries, especially where they don't have extreme performance requirements, should have run-time testing. I.e., pre- and post-condition checks, or contracts.

Re: Testing like the TSA

#49

Post by sudonim »

If you're looking to manage code smell, check out Code Climate - http://codeclimate.com. We use it as an awesome gut check for your code coverage as you write more code. It helps you see at a macro level: "Are you writing better or worse code over time?". Pretty cool stuff.

Re: Testing like the TSA

#50

Post by [deleted] »

raganwald wrote: post #8
jashkenas wrote: post #2

A thought that folks reading this post might have an opinion on: "Libraries should be mostly unit tested. Applications should be mostly (and lightly) integration tested. Naturally, some parts of a complex app will behave like a library..." Agree or disagree?

Strongly agree, but also I think that the interesting thing is why this might be so and what it tells us about application architecture. The basic thing about tests is that once you have them passing, they represent statements about constraints on the program. In other words, they express your opinion of things that should not change. Unit tests are a bet that certain aspects of the implementation will not change. In…

[deleted]
Post reply on HN