Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unit testing in Coders at Work

gigamonkeys.com

Post reply on HN
Print view View original

81–88 of 88 posts

Re: Unit testing in Coders at Work

#81

Post by toadpipe »

eru wrote: post #77
earl wrote: post #41

Earlier quoted context omitted.

Dude, a sudoku solver is really directly solved via brute force search. If a coder can't come up with the brute force algorithm for a 9x9 sudoku board, there's something wrong with that person.

Brute force? You can write a solver in 15 simple lines of Zimpl code (pun intended), that uses state-of-the-art constraint propagation and all that. http://www.zib.de/Publications/Reports/ZR-05-51.pdf

The point is that some people are too blinded by their ideology to even come up with a crude solution, let alone an elegant one.

Re: Unit testing in Coders at Work

#82

Post by silentbicycle »

maryrosecook wrote: post #79
geebee wrote: post #78

Earlier quoted context omitted.

"I have no idea how one writes tests when doing exploratory coding." You're probably already writing tests, especially when you're doing exploratory coding. How much code do you really write before you check to see if something works? You're almost certainly doing something . Maybe it's just a small main program where you validate that some methods are giving you the output you expected, or maybe it's a simple web fo…

You're right: I write a little, do a test run of the code to see what comes back, fix it if it doesn't work, write a bit more and so forth. And I see how saving those little tests could be useful.

There are testing packages for a couple languages (Python and Lua, for sure) that let you just copy in the text directly from the interpreter and use them as regression tests.

Re: Unit testing in Coders at Work

#83

Post by dkarl »

briancooley wrote: post #67
ZeroGravitas wrote: post #54

Earlier quoted context omitted.

I was amused by his comment on Knuth: "So Knuth too disagrees with the notion that unit testing always makes you go faster. Maybe he too is living in the stone age." This follows him describing writing a program, in pencil, in 1977.

I think the list of programmers throughout history who could reliably write something as complex as TeX using nothing but paper and pencil is incredibly short.

It's a lot easier than you think. When I was a kid I wrote longer programs in Basic using paper and pencil, because it was a lot easier for me than typing. I would type in the program after I was pretty sure it was right. The errors were almost always local. It seemed so easy to check the high-level structure of something when I could spread it out on the ground in front of me. Later, in college, I always printed out drafts of my papers and marked them up completely before I started editing.

Re: Unit testing in Coders at Work

#84

Post by toadpipe »

So now that I've voted up every plinkplonk comment, my comment is that most people obviously haven't read the book. The post is pretty good, but the book is better. You should read it.

If you do read the book, you will see that the discussion begins with Norvig saying that he thinks one of the most important things is being able to keep everything in your head at once. Extra tools come in when the problem gets too big to do that, but here's the key: he saw right from the very beginning that the Sudoku problem could be solved with two tools from the AI toolbox. In other words, he saw the entire solution immediately, and it was never too large to fit easily in his head.

Seibel's analysis fine as far as it goes, but he misses the really important thing here, which is that this is not just an example of someone recognizing a problem that they already knew how to solve. It's a case of someone with the mental tools that allow them to dramatically reduce the (apparent) complexity and size of a very large class of problems that happens to include Sudoku. Seibel takes a bottom up look at the data structures Norvig used, but this can be misleading because they weren't designed bottom up, they were designed all at once. You simply cannot do that unless you have the necessary training in abstract thinking (read: mathematics/formal logic/language development/ai techniques/etc).

No amount of code centric techniques or tools will ever make up for not having these tools. There will always be (relatively trivial) problems that you will never be able solve without them, because you will not be able to fit everything in your head and your ability to reason about the problem will be crippled by that. Debates about TDD are not even wrong, because they are at the wrong level of abstraction. Spending any significant amount of time discussing it is premature optimization.

Re: Unit testing in Coders at Work

#85

Post by jamesbritt »

jrockway wrote: post #18
jamesbritt wrote: post #14

Earlier quoted context omitted.

OK, but why ? Is it not useful to know that various operations and navigations through the GUI still work as expected? And if it's a matter of a poor benefit/effort ratio, why are GUIs so hard to test? Tools like Swinger are pretty easy to get rolling with for testing Swing UIs. Where do these things break down? Are these things worth fixing?

In my experience, I have sunk a lot of time making sure that foo div has bar css class when the quux link is clicked... but it has never saved me much time. I have to click through the site regularly (content changes, "does this work in IE", etc.) anyway, and errors are usually noticeable immediately. Other problems I've noticed are that they are either so specific that they fail for no reason (it was supposed to be…

"Anyway, I would be interested in hearing your UI testing success stories."

For Web apps, I use Selenium. I use the Firefox plug-in to record scenarios,with some hand-tweaking to replace any odd xpath stuff with more robust references to IDs. I then periodically run suites of UI tests to see that things still behavior as they should.

Yes, there are times when page content changes and breaks a test, but it's trivial to see where that is happening, so I've not had a problem keeping them up-to-date.

And having automated integration tests is way faster and more reliable than manually clicking through a site. I've caught numerous bugs this way, mostly in pages that do not typically get much use in real-life (but tend to be the first thing a clients tries when showing off code. Go figure.)

So, with Web+Selenium there's not much overhead to creating and maintaining a set of tests. It's a big win to be able to kick off a full suite and automatically run through a site far faster and more reliably than I could by hand.

It's not so good for desktop apps. I've been looking at Swinger, which marries Cucumber with Jemmy, for integration testing of Swing apps. As far as I know there are no good tools for recording user actions, so tests need to be constructed by hand. This makes it harder to assemble tests that capture assorted complex interactions.

But, at some point, someone has to actually test the app itself, so while it's time consuming to assemble automated UI tests, it may pay for itself over time since it reduces the effort in manually walking through the app.

So far neither of these would replace unit and functional testing, and neither catch all bugs as experienced by the end user, but they do reduce the number of problems that make it into a release. For Web apps, the effort spent is well worth it. For desktop apps, I've yet to reach that balance. However, things keep improving.

Re: Unit testing in Coders at Work

#86

Post by plinkplonk »

10ren wrote: post #47

One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age. This kind of attitude really pisses me off. It's so confident in its condescension and name-calling, I have a little wonder that they might be right. In fact, it's just dogmatic abuse, unencumbered by an objective factual appraisal of the issue at hand. Probably, I should just remember th…

" "One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age."

This kind of attitude really pisses me off. "

Then you'll really love "Uncle" Bob's latest blog entry (http://blog.objectmentor.com/articles/2009/10/06/echoes-from...) where he re iterates his inane "Stone Age Programmer" name calling. The comments on the blog post are more interesting (well ok, funnier) than the content of the post.

Re: Unit testing in Coders at Work

#87

Post by icefox »

dagw wrote: post #48
keyist wrote: post #4

If you enjoyed seeing the contrast in approaches between Norvig and Jeffries, you may also find the following interesting -- calculating bowling scores in 1. OCaml, no tests: http://alaska-kamtchatka.blogspot.com/2009/07/disfunctional-... 2. Clojure with unit tests: http://blog.objectmentor.com/articles/2009/07/19/uncle-bob-j...

One thing I've never really seen discussed is proving unit test coverage. Taking the clojure example above and its unit tests, nowhere does it make any attempt to show that the few test cases chosen provide a complete coverage of all possible bowling scores. Without that what have you really shown? Sure I can look at it and my gut feeling is, yea that should be enough, but I got that from looking at the OCaml code as…

I can't "prove" my code, but I can prove that my test executes every function, every line and takes every branch and every return. I run my test using valgrind (callgrind with a few options), then I wrote a valgrind parser and combined that with a c++ parser and output the results. Some fun tidbits:

Every time I discovered that a bit of code can never be executed.

The amount of functions that I never even call, even when I am using my test generator (built on top of the same c++ parser)

I enjoy coding. If I spent 1 hour coding a test that finds nearly all of the bugs in a class that means later I wont spent a week deciphering reports and fixing bugs that are annoying users. I get to hack more and my users get more stable code.

If you want to try out my code, http://arora-browser.org/

The c++ parser I used http://github.com/icefox/rpp/tree

My evil little test generator http://benjamin-meyer.blogspot.com/2007/11/auto-test-stub-ge...

My valgrind tools http://benjamin-meyer.blogspot.com/2007/12/valgrind-callgrin...

Re: Unit testing in Coders at Work

#88

Post by tetha »

10ren wrote: post #47

One day I hope Joel eventually realizes this. Programmers who say they don’t have time to write tests are living in the stone age. This kind of attitude really pisses me off. It's so confident in its condescension and name-calling, I have a little wonder that they might be right. In fact, it's just dogmatic abuse, unencumbered by an objective factual appraisal of the issue at hand. Probably, I should just remember th…

What I hate most about this article is that they stress the "always". It is impossible that unittesting is faster and better in every possible software for every possible hardware in every possible situation in this universe and time, just plain impossible.

Certainly, it will make you faster in a lot of cases (and a lot of regular buiseness-programming is among these cases) but there are certainly cases where it is wrong. Think of a code emitter in a compiler, or handcrafted assembler for some micro controller, or especially a problem where you just know the solution, and it is simple. In these cases, it just is faster to just hack down the solution.

Post reply on HN