Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Poll: Do you test your code?

news.ycombinator.com

Post reply on HN
Print view View original

231–240 of 351 posts

Re: Poll: Do you test your code?

#231

Post by Chris_Newton »

lparry wrote: post #215
Chris_Newton wrote: post #96

Earlier quoted context omitted.

There are many ways to improve code quality. Using an automated test suite is only one of them, and while it's one that is widely useful, it is of very limited value in some circumstances and I think for some developers it instils a false sense of security. Not having an automated test suite that covers a particular part of your code does not imply that the code is "untested" or of no value. It just means some other…

Not having automated test covering a piece of code does not imply that it's untested at the time it's written, but it sure as hell implies that it's not getting tested when seemingly unrelated feature X gets refactored and unknowingly breaks it. Tests are only marginally important at the time you're writing the code they test. The real value comes months later when something else causes the test to fail, and now you…

Sorry, but I simply can't agree with most of that. I do agree that automated tests are more valuable during maintenance than during initial development, though I think they help then too. It's the other details of your comments I'm disputing below.

Firstly, even if automated testing isn't appropriate for a particular part of the code, there should still be other forms of quality checking going on that would pick up a broken feature before the code is accepted, and certainly before the product ships. If this doesn't happen, you're relying on a limited set of automated tests as a substitute for things like proper code reviews and pre-release QA, in which case IMNSHO you're already doomed to ship junk on bad days.

Secondly, if you can break one piece of code by changing a completely unrelated bit of functionality elsewhere, you have other fundamental problems: your code isn't clearly organised with an effective modular design, and your developers demonstrably don't understand how the code works or the implications of the changes they are going to make before they dive in and start editing (or even afterwards). Again, you're already doomed: no amount of unit testing is going to save you from bugs creeping in under such circumstances.

Finally, unit tests are not a clear specification of anything, ever, other than the behaviour of a specific test.

Basically, if you consider automated unit testing a substitute for any of

(a) maintaining a clean design

(b) doing an impact analysis before making changes to existing code

(c) writing and updating proper documentation, including clear specifications, or

(d) proper peer review and QA processes

then I think you're suffering from precisely the false sense of security I mentioned earlier. In many contexts, unit tests can be great for sounding alarm bells early and giving some basic confidence, but even in the most ideal circumstances they can never replace those other parts of the development process.

Re: Poll: Do you test your code?

#232

Post by joe_the_user »

I do automated testing as much as I can, the main thing standing in my way is the problem of testing GUIs. GUI testing frameworks are inevitably painfully slow, don't test the appear of a GUI and don't test things like responsiveness and the varieties of behaviors of user message loop. I'd like to have the ability to test even more.

That said, I think TDD is trendy-consultant-crap. Writing a test before your write the code only works for simplistic code that doesn't need much testing and probably won't produce the right test for your code once you have written the code.

Also, for code I've just written, a ad-hoc manual test using the GUI is often much faster than writing a full test and I likely wouldn't ever need to run those tests again. The test suit takes quite a while to complete and if I could add every manual test I've ever run, it would take absolutely forever.

Something like "Zen Test", which runs the relevant tests in the background on code being changed sounds good but I don't think there's anything like it for c++. I'm a bit doubtful it could work on complex code in any language. A lot of R-and-R magic sounds like its creators never went code involving one model method, one controller method and one view method.

Re: Poll: Do you test your code?

#233

Post by bytemask »

I just want to say, it's always a nice feeling when I get all-green output from rspec and jenkins. The problem is that tests, like your code, are subject to the laws of entropy that comes from bit-rot.

So, I test things that matter and don't change too often - core business logic.

100% test coverage is just a goal, a bar to aim for.

And I'm totally with Zed Shaw when it comes to TDD - not worth it when you're still trying to get a full understanding of your problem domain.

Re: Poll: Do you test your code?

#234

Post by pbiggar »

hndl wrote: post #45

Another interesting question: how often do your tests run? Most folks probably run unit tests with continuous integration but what about functional and performance driven tests?

Continuous integration should run all your functional and performance tests if possible. Each "unit" (could be a commit or a push or a merge depending on your philosophy) can cause errors, and being able to pinpoint the unit in which the fail happened is immensely valuable.

If you have something really long running (eg you make a database and have a two week test), then you may be able to minimize your test (possibly automatically) and use git/hg bisecting to find it.

Fuzz testing (finding holes in your code) can be run separately, and again, you can find the root cause through minimization and bisecting.

Re: Poll: Do you test your code?

#236

Post by eric-hu »

sunir wrote: post #124
callmeed wrote: post #113

I answered "a few critical things" ... but, for the most part, testing is tedious, frustrating, and a time-sink for me. I recently paid someone $100+ an hour for some remote TDD coaching. It's helping a bit but hasn't really change my attitude towards testing (yet). What bugs me: - Testing frameworks and "best practices" change way faster than language frameworks and I simply can't keep up. What rspec version do I us…

People write and play with test frameworks because they are procrastinating from writing actual tests. Think about it. Just use Test::Unit and move on with your life. Write some tests. That's what counts.

I'm working on a single page app that's about 30% Rails and 70% Coffeescript/Backbone.js. Test::Unit is practically useless for us since users never hit any plain HTML pages besides the login page.

Imagine the current HTML5 Pandora having bugs with one particular song in Chrome only. How do you test for that using Test::Unit?

Re: Poll: Do you test your code?

#237

Post by davemo »

gruseom wrote: post #175
davemo wrote: post #58

Earlier quoted context omitted.

I went through an experience where 2 years ago I thought "I hate unit testing, don't know how to do it and don't see the value". 2 years later I think "I enjoy unit testing, know how to do it well, and see the value in unit testing _most_ of the time". I believe this transformation is entirely to do with the fact that I paired with a brilliant developer every day for 6 months who really helped to answer all my questi…

I paired with a brilliant developer every day for 6 months who really helped to answer all my questions Sounds great. What else did you learn?

Lots! How to test-drive (as opposed to test-after), how to mock out integration points, what mocks/testdoubles/spies are and how they differ. I also learned that Enterprise Java is a particular level of hell. In all, it was a good experience though :)

Re: Poll: Do you test your code?

#238

Post by Retric »

fragsworth wrote: post #210
harel wrote: post #195

Earlier quoted context omitted.

But then you'd still do the manual test after you complete your code. Nobody (I hope) codes blind hoping it would work or caught later by a test suite. Test suits don't reveal everything. Only what you tested for.

The process goes more like this: 1. Write some code 2. Test it 3. Debug your code 4. Test it 5. Debug your code 6. Test it ... 39. Debug your code 40. Test it; now it finally seems to work If all your "test it" steps are being done manually, you're being very inefficient. A good unit test can actually make development go by faster with the added bonus of defending your code from changes down the road that might screw…

That's mostly a beginners problem, I know several people where it's mostly:

  1. Write a lot of code
  2. Test it
  3. It works!
  4. Test it
  5. It works!
  6. Test it
  7. It works!
  ...
  15. Test it
  16. It works!
  17. Are you done?
TDD in no way makes 17 any clear, because every test they thought of before writing the code works more or less the first time. And that's the core problem with testing, for a solid developer what fails is has nothing to do with the code it's always a question of edge cases they did not think. (Wait, some sales people are their own managers and outside consultants at the same time? well just bob) You can force these people to write tests, but it really does just slow them down.

Re: Poll: Do you test your code?

#239

Post by smsm42 »

callmeed wrote: post #113

I answered "a few critical things" ... but, for the most part, testing is tedious, frustrating, and a time-sink for me. I recently paid someone $100+ an hour for some remote TDD coaching. It's helping a bit but hasn't really change my attitude towards testing (yet). What bugs me: - Testing frameworks and "best practices" change way faster than language frameworks and I simply can't keep up. What rspec version do I us…

My experience shows that tests is not very useful in protecting from "hard" mistaken (like unusual combination of inputs, missing condition branch coverage, etc) because even with 100% code coverage you don't actually cover 100% of input/state combinations. And things you didn't think in development are usually things you didn't think in tests too. Tests are, however, always been amazingly helpful for me in:

1. Protecting me from stupid mistakes like using wrong variable in parameters, etc. (yes, it is embarrassing to have something like this, but I better be embarrassed by test and fix it before anybody seen it than be embarrassed by somebody else hitting it when using my code).

2. Ensuring refactoring and adding new things didn't break anything.

3. After "hard" bug has been found, ensuring it never reoccurs again.

As for dealing with authentication, etc. - that's what unit tests are for, testing stuff that is under these layers directly. And I don't see it matters what you are using for tests - almost any framework would do fine, it's having tests that matters, not how you run them.

I think you can unit-test javascript too, though I never had to deal with it myself so I don't know how.

Re: Poll: Do you test your code?

#240

Post by X-Istence »

We run tests at Absio (the place I work). Everything is supposed to have full unit test coverage, but with ship-it mode that has slipped a little lately.

When you commit to a personal clone of mainline and push it up to the server Jenkins picks it up, builds it, and runs the tests, and if there are any failures notifies you over Jabber and or email to let you know it is broken and for you to go look at it.

We also integrate Jenkins with JIRA, so as soon as Jenkins builds something, pass/fail if there is a JIRA ID in the commit message a comment is automatically added there as well, which if people are watching the bugs they will get notified about.

This effectively allows people to see how they are coming along in terms of their progress and lets them see when stuff is broken almost instantly. Automated builds are nice because we can distribute the builds across a variety of different environments at the same time to see that if maybe something worked on Mac OS X that it doesn't build on Linux, well that needs to be fixed.

It definitely has made me code more defensively, nobody wants to have your Jenkins build show up as red on the status board, and nobody wants the extra scrutiny on code review when asking to merge something back into mainline. So far it has worked fairly well with most developers doing testing.

Post reply on HN