Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TDD Doesn't Work

blog.cleancoder.com

Post reply on HN
Print view View original

121–130 of 133 posts

Re: TDD Doesn't Work

#121

Post by lisivka »

westurner wrote: post #114
lisivka wrote: post #74

Earlier quoted context omitted.

Robert Martin is author of Agile manifesto. https://www.quora.com/Why-does-Kent-Beck-refer-to-the-redisc... The original description of TDD was in an ancient book about programming. It said you take the input tape, manually type in the output tape you expect, then program until the actual output tape matches the expected output.

+1. TDD could be considered as a derivation of the Scientific Method (Hypothesis Testing). https://en.wikipedia.org/wiki/Scientific_method https://en.wikipedia.org/wiki/Hypothesis Test first isolates out a null hypothesis (that the test already passed); but not that it passes/fails because of some other chance variation (e.g. hash randomization and unordered maps). https://en.wikipedia.org/wiki/Null_hypothesis ... ht…

+1 Right on the spot.

TDD requires you to draw your target first, then hit or miss it with the code, like in science: hypotheses -> confirmation/declining via experiments -> working theory.

But in practice, lot of coders are hitting a point instead, then they draw target around that point, like in fake science: we throw coin 100 times, distribution is 60/40, our hypothesis: random coin flip has 60 to 40 ratio, our hypothesis confirmed by experiment, huge savings, hooray!

Re: TDD Doesn't Work

#122

Post by bitL »

lisivka wrote: post #120
bitL wrote: post #111

Earlier quoted context omitted.

My message is more like your confidence after writing hundreds of thousands tests might be just 50%. From my own experience, every single bad case that can happen in a complex system will happen at some point at some customer, wrecking their system and costing them potentially millions, in serious trading bugs even leading to a bankruptcy. Your testing suite won't catch these initially but reactively when you add tha…

In such cases, I use "torture" test cases: lengthy, random test cases, which are trying to abuse and overload system with no data, incorrect data, random data, huge data, high latencies, duplicated messages, missed messages or random aborts, random speaks, etc. They allows me to discover situations not covered by test cases. I also try to use underpowered hardware for such testing. Of course, I cannot imagine all pos…

That's a good approach as well! In addition, nowadays with VMs/containers you can even simulate nodes going randomly up and down, which is a bit of a challenge if you do it in a real testing cluster.

Re: TDD Doesn't Work

#123

Post by ambrop7 »

MaulingMonkey wrote: post #87
ambrop7 wrote: post #38

I must be one of the very few people who can write working and mostly bugless code and without writing any kind of test. Writing tests feels like the most wasteful and possibly harmful thing to me (like by people forcing dependency injection etc. where otherwise unneeded). I don't really know what to think of the situation? Is this how it has always been? Do most software engineers really have no idea what they're do…

> I must be one of the very few people who can write working and mostly bugless code and without writing any kind of test. Or you may be using a different definition of "mostly bugless" than the rest of us. I do gamedev. The ability to patch post-release is not a given, even today, for all platforms. Crashes, corruption, progress blockers, etc. are all VERY BAD in this environment. I see bellow you're writing network…

For my hobby projects I admittedly don't do much testing - simply because no one is paying me to. Only in rare cases do I intentionally write a test when something seems complex. And I do run address sanitizer etc. where that can reasonably be done (e.g. not on microcontroller code). Anyway my point is, my code has considerably less bugs than what you'd get from reasonably proficient programmers even if they DID write tests.

For example, I'm currently writing a TCP/IP stack for embedded systems [1]. While it's not quite complete yet (misses some essential code like fragmentation and congestion control), I'm very confident that it has (and will have when complete) much less bugs than related portions of lwIP; see for yourself all the bugs I've found in lwIP [2].

Again feel free to find bugs in my code. I very much appreciate people pointing out bugs, as it helps me make even fewer bugs :)

> We live in a world of exploitable 1-byte buffer overflows.

Indeed. But buffer overflows are so easy to avoid, just don't write over the end of the buffer. I doubt I've done a buffer overflow in years. The bugs that I do make, are much more complex.

[1] https://github.com/ambrop72/aprinter/tree/ipstack/aprinter/i...

[2] https://savannah.nongnu.org/bugs/index.php?go_report=Apply&g...

Re: TDD Doesn't Work

#124

Post by ambrop7 »

jacques_chester wrote: post #81
ambrop7 wrote: post #42

Earlier quoted context omitted.

That's a good question. There are different ways: - Play around with the program intelignetly and observe no bugs (you wouldn't believe how many bugs I've found by that method, that have been missed by very formal verification processes). - Don't call it done until you haven't proven to yourself that the code has no bugs. This is an informal process of self-code-review but which involves quite rigorous thinking about…

> Play around with the program intelignetly and observe no bugs What if the computer encoded your knowledge? > Don't call it done until you haven't proven to yourself that the code has no bugs. What happens when the software changes? Do you repeat every single desk-checking exercise to ensure nothing has broken? Do you even remember every click, every experimental input? Can you prove that you do? > This is an inform…

> What if the computer encoded your knowledge?

> Do you even remember every click, every experimental input?

> Can you prove that you do?

I agree some tests are a good idea depending on the project. Doesn't mean I have to like writing them!

> I assume that the code is smart and that my mental simulation of the code, which my brain helpfully and invisibly patches on the fly, is correct.

I try not to assume things until I've constructed associated proofs in my mind (and sometimes written them into comments). In fact keeping in mind what you've established (proven) and what not is a very important thing. Most of the bugs I've done are because I've simply forgotten to think about / prove something.

It's a completely different way of programming!

> Then I make those explicit statements executable. And then I run them frequently.

But I prefer to write down these explicit statements in the code itself, often as assertions. I can then prove them right on the spot!

Re: TDD Doesn't Work

#125

Post by ambrop7 »

lisivka wrote: post #85
z92 wrote: post #71

Earlier quoted context omitted.

If it works, then it works. That's the point. If I don't use it because it doesn't pass my peer review process, whatever that might be, then my competitor will use it and come up with a faster/cheaper solution than my one.

Peer review is not for filtering software. Peer review is process which used by (professional) developers to help each other to catch bugs and share experience and best practices. Basically, I said him that I found bugs and/or bad practices in his code.

Peer review is optimized for the average developers and developers in a specific team. And, when an extremely good programmer writes the code, peer review tends to:

- Mostly raise issues about nonfunctional aspects of the code. Often these are about the author knowingly violating coding guidelines, which they found stupid (at least in specific cases). Guidelines may conflict with the techniques that the author uses to achieve bugless code.

- Improve the skill of the reviewer much more than the author, as the reviewer observes high quality code and novel techniques.

> I found bugs

Please do let me know about bugs!

Re: TDD Doesn't Work

#126

Post by lisivka »

ambrop7 wrote: post #125
lisivka wrote: post #85

Earlier quoted context omitted.

Peer review is not for filtering software. Peer review is process which used by (professional) developers to help each other to catch bugs and share experience and best practices. Basically, I said him that I found bugs and/or bad practices in his code.

Peer review is optimized for the average developers and developers in a specific team. And, when an extremely good programmer writes the code, peer review tends to: - Mostly raise issues about nonfunctional aspects of the code. Often these are about the author knowingly violating coding guidelines, which they found stupid (at least in specific cases). Guidelines may conflict with the techniques that the author uses t…

Peer review is just request for short review. "Shortly look at this new piece of code, before I will commit it into our repository, and say your opinion." That's all.

If you will ask me to review piece of your code, written like that, I will immediately ask to write test cases, because team will not be able to improve code base without test coverage, and to add meaningful documentation, so we will spend our time on work instead of solving mental puzzles. In team, your freedom ends where freedom of other members begin.

If you are thinking that you are extremely good developer, then finish your projects, with your team, in fraction of time, comparing to other teams, otherwise you are extremely good programmer, but not an extremely good developer.

> Please do let me know about bugs!

There is no obvious bugs in C code. There are few tiny problems in bash scripts (e.g. error messages are not printed to STDERR and error messages are not helpful). There are lot of problems with style and documentation, which will affect team velocity and maintenability. Peer review is not a bug review. I just need to ensure myself that new code will not create problems for us when committed, and the easier way to do that in less than 10 minutes is to look at test cases, so write test cases and documentation first. When they will be OK, I may say that your code is OK. ;-)

Re: TDD Doesn't Work

#127

Post by cjwoodall »

GrinningFool wrote: post #16
keithnz wrote: post #5

Earlier quoted context omitted.

so what's your approach to ensuring the software you deploy is correct?

How does TDD prove its correctness? TDD suffers from the same limitations as the code - it generally only covers what you could think of. It's a powerful tool, but I think any belief that sufficient test coverage (in most common cases) actively proves correctness is misguided. In the general case, even full test coverage proves only that you've tested for the conditions you expect - but does nothing to verify the cor…

Testing validates that the program is correct according to some base of assumptions. You run into this a whole lot in embedded systems, where mocking hardware is difficult. You can't feasibly unit test against real hardware (most of the time), so instead you unit test against your assumption of what the hardware does and verify you respond according to requirements.

Has the benefit of proving correctness of your assumptions, which makes it easier to debug once you insert it in system and things inevitably are not 100% right. It gives you a way to reason about what your code does, what might be different, and then allows you to revise your assumptions and get your new solution in place and tested without the often long wait times to do manual testing on deeply embedded hardware.

Sometimes traditional TDD is the answer, sometimes simulations are the answer, and sometimes you need to just get out of your chair and test it out. It is a tool!

Re: TDD Doesn't Work

#128

Post by ambrop7 »

lisivka wrote: post #126
ambrop7 wrote: post #125

Earlier quoted context omitted.

Peer review is optimized for the average developers and developers in a specific team. And, when an extremely good programmer writes the code, peer review tends to: - Mostly raise issues about nonfunctional aspects of the code. Often these are about the author knowingly violating coding guidelines, which they found stupid (at least in specific cases). Guidelines may conflict with the techniques that the author uses t…

Peer review is just request for short review. "Shortly look at this new piece of code, before I will commit it into our repository, and say your opinion." That's all. If you will ask me to review piece of your code, written like that, I will immediately ask to write test cases, because team will not be able to improve code base without test coverage, and to add meaningful documentation, so we will spend our time on w…

As has already been mentioned, there are industries where the code simply must have no bugs and in those code reviews will be different. Most critical components may be written by a very small team perhaps even a single person, as multiple people working on the same thing will generally result in more bugs.

Again I didn't say tests are a bad idea. In industry projects they are usually a very good idea. My point was that people should be able to write working bug-less code without using tests. When a test catches a real bug, it is to be understood as a failure of the developer(s).

Re: TDD Doesn't Work

#129

Post by ambrop7 »

lisivka wrote: post #126
ambrop7 wrote: post #125

Earlier quoted context omitted.

Peer review is optimized for the average developers and developers in a specific team. And, when an extremely good programmer writes the code, peer review tends to: - Mostly raise issues about nonfunctional aspects of the code. Often these are about the author knowingly violating coding guidelines, which they found stupid (at least in specific cases). Guidelines may conflict with the techniques that the author uses t…

Peer review is just request for short review. "Shortly look at this new piece of code, before I will commit it into our repository, and say your opinion." That's all. If you will ask me to review piece of your code, written like that, I will immediately ask to write test cases, because team will not be able to improve code base without test coverage, and to add meaningful documentation, so we will spend our time on w…

> If you are thinking that you are extremely good developer, then finish your projects, with your team, in fraction of time, comparing to other teams, otherwise you are extremely good programmer, but not an extremely good developer.

Actually, I do! My work code does have more comments and tests (for my hobby projects, I do less of that and instead write other code, getting more done faster).

> There is no obvious bugs in C code. Indeed you still haven't pointed out a single bug, so you have not yet invalidated my assertion that I can write (mostly) bugless code :)

Re: TDD Doesn't Work

#130

Post by lisivka »

ambrop7 wrote: post #129
lisivka wrote: post #126

Earlier quoted context omitted.

Peer review is just request for short review. "Shortly look at this new piece of code, before I will commit it into our repository, and say your opinion." That's all. If you will ask me to review piece of your code, written like that, I will immediately ask to write test cases, because team will not be able to improve code base without test coverage, and to add meaningful documentation, so we will spend our time on w…

> If you are thinking that you are extremely good developer, then finish your projects, with your team, in fraction of time, comparing to other teams, otherwise you are extremely good programmer, but not an extremely good developer. Actually, I do! My work code does have more comments and tests (for my hobby projects, I do less of that and instead write other code, getting more done faster). > There is no obvious bug…

I pointed to one bug: error messages are not redirected to stderr in shell scripts. It's small bug, because it will affect development only, but I found it. I need to find just one bug to prove my point, isn't?
Post reply on HN