Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software testing, and why I'm unhappy about it

nhaehnle.blogspot.com

Post reply on HN
Print view View original

21–30 of 76 posts

Re: Software testing, and why I'm unhappy about it

#21

Post by quadrifoliate »

13of40 wrote: post #17

I have a lot of bitter things to say about automated testing, having spent 14 years of my life trying to knead it into a legitimate profession, but here's the most significant: You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. Because if you just throw in s…

At an old job, one manager would put in his employees' annual reports stuff like "Developer X wrote N automated tests, fixed M bugs, and filed P new bugs this quarter..."

The obvious result of Goodhart's Law ensued, leading to test cases like you mention.

Lesson to leaders: Please stop your bad managers from pulling stupid crap like this. It wastes a lot more time in the longer run.

Re: Software testing, and why I'm unhappy about it

#22

Post by [deleted] »

invalidname wrote: post #19
ramesh31 wrote: post #18

Earlier quoted context omitted.

>You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. This is why I found Gherkin/Cucumber (and BDD in general) to be a total revelation when I first encountered it. No one should be writing tests any other way IMO. https://cucumber.io/docs/gherkin/reference/

This sounds like a good theory but the practice of it is really hard. Pretty quickly you end up with tests that "say" one thing but have nuanced different behavior in the underlying implementation. Then try to debug a "document"... I like the idea. But having tried it at scale, it becomes a mess. Code I can understand. I can read English comments. I can't debug English.

[deleted]

Re: Software testing, and why I'm unhappy about it

#23

Post by randomdata »

ramesh31 wrote: post #18
13of40 wrote: post #17

I have a lot of bitter things to say about automated testing, having spent 14 years of my life trying to knead it into a legitimate profession, but here's the most significant: You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. Because if you just throw in s…

>You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. This is why I found Gherkin/Cucumber (and BDD in general) to be a total revelation when I first encountered it. No one should be writing tests any other way IMO. https://cucumber.io/docs/gherkin/reference/

Gherkin/Cucumber reintroduce the very problem TDD/BDD was intended to solve: Documentation falling out of sync with the implementation.

The revelation of TDD, which was later rebranded as BDD to deal with the confusion that arose with other types of testing, was that if your documentation was also executable the machine could be used to prove that the documentation is true. The Gherkin/Cucumber themselves are not executable and require you to re-document the function in another language with no facilities to ensure that the two are consistent with each other.

If you are attentive enough to ensure that the documentation and the implementation are aligned, you may as well write it in plain English. It will give you all of the same benefits without the annoying syntax.

Re: Software testing, and why I'm unhappy about it

#24

Post by dgb23 »

13of40 wrote: post #17

I have a lot of bitter things to say about automated testing, having spent 14 years of my life trying to knead it into a legitimate profession, but here's the most significant: You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. Because if you just throw in s…

What about data driven tests where you lay out several variants, including edge cases, for function arguments? Seems pretty clear to me.

Re: Software testing, and why I'm unhappy about it

#25

Post by brabel »

invalidname wrote: post #19
ramesh31 wrote: post #18

Earlier quoted context omitted.

>You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. This is why I found Gherkin/Cucumber (and BDD in general) to be a total revelation when I first encountered it. No one should be writing tests any other way IMO. https://cucumber.io/docs/gherkin/reference/

This sounds like a good theory but the practice of it is really hard. Pretty quickly you end up with tests that "say" one thing but have nuanced different behavior in the underlying implementation. Then try to debug a "document"... I like the idea. But having tried it at scale, it becomes a mess. Code I can understand. I can read English comments. I can't debug English.

We have it at scale, and no, it doesn't become a mess.

We use Spock, which make "comments" a very expected thing, which helps us not let tests without comments pass a code review.

Just use a tool that helps you and stop writing stupid tests whose impl code looks worse than the code being tested.

Re: Software testing, and why I'm unhappy about it

#26

Post by vincnetas »

randomdata wrote: post #20
13of40 wrote: post #17

I have a lot of bitter things to say about automated testing, having spent 14 years of my life trying to knead it into a legitimate profession, but here's the most significant: You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. Because if you just throw in s…

Which is funny as the purpose of testing is to explain to other other developers what the code under test assumes and what should be expected of it under various conditions. It is documentation. If you have to document your documentation, you might be missing something fundamental in how you are writing your first order documentation. Not to mention that in doing so you defeat the reason for writing your documentatio…

So i understand correctly, that your position is "code is the documentation"?

Over time im inclined to value human written documentation. Especially when things involve integrations of multiple systems. I had real cases, where two parties point at code and say their code is correct. And in isolation code looks correct. But when time comes to integrate these systems. It breaks. And then if you have human readable document where intentions and expectations are specified it's much easier to come to common (working) solution.

Not all languages have capability to express complex intentions so code as documentation does not work most of the time.

Re: Software testing, and why I'm unhappy about it

#27

Post by charcircuit »

randomdata wrote: post #20
13of40 wrote: post #17

I have a lot of bitter things to say about automated testing, having spent 14 years of my life trying to knead it into a legitimate profession, but here's the most significant: You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. Because if you just throw in s…

Which is funny as the purpose of testing is to explain to other other developers what the code under test assumes and what should be expected of it under various conditions. It is documentation. If you have to document your documentation, you might be missing something fundamental in how you are writing your first order documentation. Not to mention that in doing so you defeat the reason for writing your documentatio…

>the purpose of testing is to explain to other other developers what the code under test assumes and what should be expected of it under various conditions

No, the point of automated testing is to verify that what is under test behaves correctly and to be able to scale this verification cheaper than having humans do it. Documenting what it verifies and under what conditions is just a side effect.

Re: Software testing, and why I'm unhappy about it

#28

Post by pjerem »

vincnetas wrote: post #26
randomdata wrote: post #20

Earlier quoted context omitted.

Which is funny as the purpose of testing is to explain to other other developers what the code under test assumes and what should be expected of it under various conditions. It is documentation. If you have to document your documentation, you might be missing something fundamental in how you are writing your first order documentation. Not to mention that in doing so you defeat the reason for writing your documentatio…

So i understand correctly, that your position is "code is the documentation"? Over time im inclined to value human written documentation. Especially when things involve integrations of multiple systems. I had real cases, where two parties point at code and say their code is correct. And in isolation code looks correct. But when time comes to integrate these systems. It breaks. And then if you have human readable docu…

Same. I’m sick of people escaping writing documentation by saying that "code is the doc" and in the meantime, writing unreadable code abstracted over dozens of code files.

They almost convinced me somewhere in my career. But the hard truth I learnt is that most people are saying this because they aren’t capable of verbalizing what they are programming.

If your "code is doc", it should be extremely easy to add a little sentence above your method to explain what it does. And no, doc doesn’t stale. If your documentation isn’t up to what your function does, it’s probably because you should have written a brand new function instead of changing a function’s behavior.

Re: Software testing, and why I'm unhappy about it

#29

Post by jmcomets »

randomdata wrote: post #20
13of40 wrote: post #17

I have a lot of bitter things to say about automated testing, having spent 14 years of my life trying to knead it into a legitimate profession, but here's the most significant: You test case is more useless than a turd in the middle of the dining room table unless you put a comment in front of it that explains what it assumes, what it attempts, and what you expect to happen as a result. Because if you just throw in s…

Which is funny as the purpose of testing is to explain to other other developers what the code under test assumes and what should be expected of it under various conditions. It is documentation. If you have to document your documentation, you might be missing something fundamental in how you are writing your first order documentation. Not to mention that in doing so you defeat the reason for writing your documentatio…

Disregarding the "code is doc" position, it's still common to have an overview or index for documentation, which points readers in the right direction instead of dumping pages of detailed docs on them.

Now, you could also have a well organized test suite that goes from most obvious to most detailed, split into sections for each use-case, but this sounds a lot more tedious than "write a one-line comment describing the unit test".

Re: Software testing, and why I'm unhappy about it

#30

Post by tjrDL6MjB2Zwwa »

pjerem wrote: post #28
vincnetas wrote: post #26

Earlier quoted context omitted.

So i understand correctly, that your position is "code is the documentation"? Over time im inclined to value human written documentation. Especially when things involve integrations of multiple systems. I had real cases, where two parties point at code and say their code is correct. And in isolation code looks correct. But when time comes to integrate these systems. It breaks. And then if you have human readable docu…

Same. I’m sick of people escaping writing documentation by saying that "code is the doc" and in the meantime, writing unreadable code abstracted over dozens of code files. They almost convinced me somewhere in my career. But the hard truth I learnt is that most people are saying this because they aren’t capable of verbalizing what they are programming. If your "code is doc", it should be extremely easy to add a littl…

[dead]
Post reply on HN