Live data from Hacker News

Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

st.cs.uni-saarland.de

21–30 of 42 posts

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#21
post #9

We once had a bug that would cause a failure in our test suite, but very rarely. It was so rare that it would just "go away" and not show up for weeks. So we thought it was odd (maybe bad test data, etc) but moved on. When we decided to look into it when we had some downtime, we found out that it was a date bug that only reared its head on the 31st day of the month, hence only happening every two months. I'm not sure…

Thanks for the anecdote, because it gives me a new idea. We should be monitoring the rate of test failures -- per test -- in a timeseries and looking for periodicities. Just graphing them and seeing regular spikes will help, but having real numerics looking for approximate periodicities will catch this sort of thing (after a few months!).

Unless you are writing date specific code or have seen this bug arise, I probably wouldn't bother. Time bugs will likely creep in regardless, for example how would you find a bug that happens one month a year? How would you find a bug once every four years? Sometimes human knowledge/experience can't be tested out of a system. All test failures probably warrant investigation of some type.

But, as a general rule, I would try to brute force or fuzz your "inputs", a.k.a data out of your control. Random numbers, dates, user input, network packets, etc. Users and environments often act in unexpected ways and you want to make sure your bug finding isn't reduced to accidentally be working on the right day! :)

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#23
The GDB people have done it again. The new release 4.17 of the GNU debugger [6] brings several new features, languages, and platforms, but for some reason, it no longer integrates properly with my graphical front-end DDD [10]: the arguments specified within DDD are not passed to the debugged program. Something has changed within GDB such that it no longer works for me. Something? Between the 4.16 and 4.17 releases, no less than 178,000 lines have changed. How can I isolate the change that caused the failure and make GDB work again?

I guess this was before Git. I love git bisect

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#24
post #2

Reminds me of one of the nice features of Git: git bisect. Find the breaking commit in the smallest number of steps. Some random blog post about it: http://webchick.net/node/99

This is awesome. Didn't know about it, thanks! My approach has been to step exponentially. I go back 1 commit, 2 commits, 4 commits... etc. I'll be using bisect from now on though.

You may be implementing an intuitive weighted bisect :)

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#25

I took several courses with professor Zeller and with post-docs at his chair. We applied the described concept of "delta debugging" to finding programming errors in python programs. Suprisingly it was possible to implement this in the course although one had to fiddle so much with Python's internals. Zeller's chair homepage has a lot of additional infos. Here's a page on how the idea evolved after the initial paper i…

He actually has a course on debugging on Udacity.

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#26
post #9

We once had a bug that would cause a failure in our test suite, but very rarely. It was so rare that it would just "go away" and not show up for weeks. So we thought it was odd (maybe bad test data, etc) but moved on. When we decided to look into it when we had some downtime, we found out that it was a date bug that only reared its head on the 31st day of the month, hence only happening every two months. I'm not sure…

Thats where property based tests come in very useful

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#27
post #2

Reminds me of one of the nice features of Git: git bisect. Find the breaking commit in the smallest number of steps. Some random blog post about it: http://webchick.net/node/99

I think what blew my mind most was `git bisect run`. Write a test case, go get coffee, have git spit out the offending commit automatically. So cool.

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#28
post #2

Reminds me of one of the nice features of Git: git bisect. Find the breaking commit in the smallest number of steps. Some random blog post about it: http://webchick.net/node/99

This is awesome. Didn't know about it, thanks! My approach has been to step exponentially. I go back 1 commit, 2 commits, 4 commits... etc. I'll be using bisect from now on though.

Heh, I do a binary search. Find last known good build, go half way between, etc

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#29
post #28

Earlier quoted context omitted.

This is awesome. Didn't know about it, thanks! My approach has been to step exponentially. I go back 1 commit, 2 commits, 4 commits... etc. I'll be using bisect from now on though.

Heh, I do a binary search. Find last known good build, go half way between, etc

That's what git bisect does

Re: Yesterday, my program worked. Today, it does not. Why? (1999) [pdf]

#30
post #28

Earlier quoted context omitted.

This is awesome. Didn't know about it, thanks! My approach has been to step exponentially. I go back 1 commit, 2 commits, 4 commits... etc. I'll be using bisect from now on though.

Heh, I do a binary search. Find last known good build, go half way between, etc

This is what git-bisect does :)
Post reply on HN