Testing like the TSA • Hacker News Board

Re: Testing like the TSA

#101

Post by nirvana » Thu, Apr 12, 2012, 2:00 PM UTC

One line in this article may have given me an epiphany: >" If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it."

My primary objection to TDD is that it doesn't seem to work, because when I've tried it, the tests caught no bugs at all. I believe that tests can be important for APIs to prevent regressions when you have to work with external components, but its frustrating to put time into tests and then never have the tests fail.

Its not that I'm a perfect programmer, its the kind of bugs I make. The kind of bugs I make are caught by the compiler. This may be because, over the years of my career (many of which occurred long before the idea of "test first" was widely heard) I've trained myself in a style of programming where I can trust the compiler to catch my mistakes (most of which are typos, frankly.)

I don't know if others can do this, but for me, it came about by doing things like:

Old way: if (variable == 1) then whatever New way: if (1 == variable) then whatever

Every time I mistype that as "(1 = variable) the compiler catches it because I can't redefine 1.

Re: Testing like the TSA

#102

Post by Porter_423 » Thu, Apr 12, 2012, 2:17 PM UTC

I like your point about 20% test-first and 80% test-after. I always felt dirty/wrong/stupid/naive when arguing for this type of approach. Sometimes I just need to explore a few approaches before I settle on one, and if I’m writing proper tests before each exploration, then I lose my creative momentum very quickly.

Re: Testing like the TSA

#103

Post by Splines » Thu, Apr 12, 2012, 3:13 PM UTC

ejames wrote: post #87

Splines wrote: post #72

Earlier quoted context omitted.

Hey Evan ;) > We all realize there is no benefit in testing if you ignore failures rather than acting to fix the bugs, but in much the same way that doing nothing when tests fail has no benefit, doing nothing when tests pass also has no benefit - so tests which always pass are just as useless as failing tests you ignore, as are tests which only turn up corner-case bugs that you would have been comfortable with shippi…

Hey Jonathan ;) This is the danger of having a 'test org' separate from the 'dev org'. When writing tests is tied to writing production code, your progress in one is tied to progress in the other. If it's easy to write tests for a particular feature, then the developer stops writing tests and writes the feature once they're done with the easy tests. It's much easier to understand your coverage when you're actually wo…

> This is the danger of having a 'test org' separate from the 'dev org'.
I completely agree with you, in light of how MSFT categorizes their ICs these days. There used to be three test/dev disciplines when I started (in 2001): "Software Test Engineers", "Software Development Engineers in Test", and "Software Development Engineers" (you likely know this already, but it might be useful history for others to know). Around 8-9 years ago the STE discipline disappeared; the company hired for SDETs (like yourself) from that point on.

The big loss here was that now all STEs were expected to be SDETs - writing code on a regular basis to test the application. There were (notice the past tense) many STEs who knew our application very* very well, and knew how to break it, hard. STEs provided quality feedback at a higher level than what a unit test provides - does the application feel right? Are customers getting their solutions solved? If you have your eyeballs stuck inside a unit test it's difficult to step 30 feet back and ask yourself if you're even doing the right thing.

Nowadays I feel like the pendulum is slowly swinging back the other way, and there is less drive to have test write automation (and maybe it is because where we are in the product cycle). I understand that high-level STEs "back in the day" were probably concerned about potential career progression at the time, which might be why they eliminated STEs (I have no idea of the real reasons - they've been lost to Outlook's mailbox size limits), but all-in-all I think MSFT is poorer because of it.

Re: Testing like the TSA

#104

Post by cageface » Thu, Apr 12, 2012, 4:05 PM UTC

damoncali wrote: post #43

Don’t use Cucumber... Thank God someone with a bullhorn finally said this. I was beginning to think I was alone in my hatred of Cucumber. (And my love of Test:Unit/Minitest.)

I really wish the Rails community would get over its crush on both RSpec and Cucumber. The whole point of Rails in the first place was to cast off all the ceremony and drudgery of web development in Java yet we've replaced it with these unnecessarily cryptic and complex tools just for the sake of some syntactic cuteness.

I'm so tired of reading Rails job postings hammering down RSpec and usually also Cucumber experience as a prerequisite.

Re: Testing like the TSA

#105

Post by gfodor » Thu, Apr 12, 2012, 5:37 PM UTC

aneth wrote: post #68

gfodor wrote: post #52

Hey, while we're slaughtering sacred cows, lets kill mocking, endotesting, expectation based testing, and the whole nine yards. It's a horrible practice that causes you to write too many tests, too many assertions, and results in tests that stay green even if you delete entire files from your codebase .

How would you test, say, payment processing, in any sane way, without mocking? How do you test failure handling code when the failure can not be reproduced in a known way by the current code base? I agree these tools are often overused, however they do have their roles.

Yeah I'm not referring to mocking external services, which is both smart and useful. I'm referring to the practice of endotesting whereby you mock your own objects and interfaces as a means to test interactions between them.