Using tests as a debugging tool for logic errors
1–10 of 16 posts
Re: Using tests as a debugging tool for logic errors
#2This article seems like a very long-winded and complicated way to say that we should write tests. Am I missing something here? Wouldn't most developers write tests when creating algorithms, let alone something relating to finance as tax calculations? Yes, you should reproduce a defect by writing a failing tests first.
Where I hoped/thought this piece would go was to expand on the idea of error-prone[1] and apply it to the runtime.
Re: Using tests as a debugging tool for logic errors
#3This article seems like a very long-winded and complicated way to say that we should write tests. Am I missing something here? Wouldn't most developers write tests when creating algorithms, let alone something relating to finance as tax calculations? Yes, you should reproduce a defect by writing a failing tests first. Where I hoped/thought this piece would go was to expand on the idea of error-prone[1] and apply it t…
I thought it was interesting - not revolutionary but updated my thinking a bit.
Writing a failing test that reproduces a bug is something I learned pretty early on.
But I never consciously thought about and approached the test as a way to debug. I thought about it more of a TDD way - first write tests, then go off and debug/code until the test is green. Also practically, let's fill the gap in coverage and make sure this thing never happens again, especially if I had to deal with it on the weekend.
What was interesting to me about this was actively approaching the test as a way of debugging, designing it to give you useful information and using the test in conjunction with debugger
Re: Using tests as a debugging tool for logic errors
#4This reminds me of a talk that Leslie Lamport (author of LaTeX & prominent figure in the field of distributed computing) gave recently [1]. I remember him arguing that the difficult part in writing code is not to determine what code to write to compute something, but to determine what this something is in the first place. "Logic errors" are really about valid algorithms that end up computing the wrong thing - they're gonna compile, they're gonna run, but they won't do what you want them to do.
One example he gives is computing the maximum element in a sequence of numbers. This is something trivial to implement but you need to decide what to do with the obvious edge case: empty sequences. One solution is to return some kind of error or exception, but another is to extend what we mean by the largest element in a sequence the way mathematicians typically do. Indeed, the maximum function can be extended for empty sequences by letting max([]) := -infinity, the same way empty sums are often defined as 0, and empty products as 1. The alleged benefit of following the second approach is that it should lead to simpler code/algorithms, but it also requires more upfront thinking.
Re: Using tests as a debugging tool for logic errors
#5This article seems like a very long-winded and complicated way to say that we should write tests. Am I missing something here? Wouldn't most developers write tests when creating algorithms, let alone something relating to finance as tax calculations? Yes, you should reproduce a defect by writing a failing tests first. Where I hoped/thought this piece would go was to expand on the idea of error-prone[1] and apply it t…
I thought it was interesting - not revolutionary but updated my thinking a bit. Writing a failing test that reproduces a bug is something I learned pretty early on. But I never consciously thought about and approached the test as a way to debug. I thought about it more of a TDD way - first write tests, then go off and debug/code until the test is green. Also practically, let's fill the gap in coverage and make sure t…
I'm happy for you that you learned something and sad for me because you made me feel old and stupid.
I tend to forget that people don't know stuff I learned decades ago and consider them as general knowledge.
Before TDD became what it was, we used to create specific files for specific bug cases, or even get the files from the users themselves.
Re: Using tests as a debugging tool for logic errors
#6This article seems like a very long-winded and complicated way to say that we should write tests. Am I missing something here? Wouldn't most developers write tests when creating algorithms, let alone something relating to finance as tax calculations? Yes, you should reproduce a defect by writing a failing tests first. Where I hoped/thought this piece would go was to expand on the idea of error-prone[1] and apply it t…
I thought it was interesting - not revolutionary but updated my thinking a bit. Writing a failing test that reproduces a bug is something I learned pretty early on. But I never consciously thought about and approached the test as a way to debug. I thought about it more of a TDD way - first write tests, then go off and debug/code until the test is green. Also practically, let's fill the gap in coverage and make sure t…
> What was interesting to me about this was actively approaching the test as a way of debugging, designing it to give you useful information and using the test in conjunction with debugger
I'm curious, if you're using TDD weren't you already doing this? A test that doesn't give you useful information is not a useful test.
Re: Using tests as a debugging tool for logic errors
#7Earlier quoted context omitted.
I thought it was interesting - not revolutionary but updated my thinking a bit. Writing a failing test that reproduces a bug is something I learned pretty early on. But I never consciously thought about and approached the test as a way to debug. I thought about it more of a TDD way - first write tests, then go off and debug/code until the test is green. Also practically, let's fill the gap in coverage and make sure t…
> What was interesting to me about this was actively approaching the test as a way of debugging, designing it to give you useful information and using the test in conjunction with debugger I'm curious, if you're using TDD weren't you already doing this? A test that doesn't give you useful information is not a useful test.
I think the distinction is that if you write a test that reproduces the bug, that's a binary signal and doesn't by itself tell you anything about why the bug is happening.
In contrast, if you write tests that rule out particular causes of a bug you're incrementally narrowing down the potential causes of the bug. So each test gives you information that helps you solve the bug, without directly stepping through the code.
Unfortunately, I don't think the post is a great primer on the subject.
Re: Using tests as a debugging tool for logic errors
#8Earlier quoted context omitted.
> What was interesting to me about this was actively approaching the test as a way of debugging, designing it to give you useful information and using the test in conjunction with debugger I'm curious, if you're using TDD weren't you already doing this? A test that doesn't give you useful information is not a useful test.
I think the distinction is that if you write a test that reproduces the bug, that's a binary signal and doesn't by itself tell you anything about why the bug is happening. In contrast, if you write tests that rule out particular causes of a bug you're incrementally narrowing down the potential causes of the bug. So each test gives you information that helps you solve the bug, without directly stepping through the cod…
> Unfortunately, I don't think the post is a great primer on the subject.
It isn't, nor is it intended to be. It's an advert:
>> While mastering unit tests as debugging tools takes practice, AI-powered solutions like Qodo can significantly accelerate this journey. Qodo’s contextual understanding of your Java codebase helps it automatically generate tests that target potential logic vulnerabilities.
Re: Using tests as a debugging tool for logic errors
#9This article seems like a very long-winded and complicated way to say that we should write tests. Am I missing something here? Wouldn't most developers write tests when creating algorithms, let alone something relating to finance as tax calculations? Yes, you should reproduce a defect by writing a failing tests first. Where I hoped/thought this piece would go was to expand on the idea of error-prone[1] and apply it t…
I thought it was interesting - not revolutionary but updated my thinking a bit. Writing a failing test that reproduces a bug is something I learned pretty early on. But I never consciously thought about and approached the test as a way to debug. I thought about it more of a TDD way - first write tests, then go off and debug/code until the test is green. Also practically, let's fill the gap in coverage and make sure t…
> then go off and debug/code until
Yes, this is a missed opportunity! Well said. I try to write tests in place of print statements or debuggers, using assertions like xray glasses. Fun times!
Re: Using tests as a debugging tool for logic errors
#10Closely related are in-code assertions. I remember when I used to liberally use asserts inside a code (and you could disable them for production) to check pre-conditions, post-conditions, or any invariants. Nowadays, I don't think the pattern is recommended anymore, at least in certain popular languages.