Pynguin – Generate Python unit tests automatically
1–10 of 80 posts
Re: Pynguin – Generate Python unit tests automatically
#2What kind of "reassurance" do a tool like pynguin provide?
In my head tests should be written from requirements, before - or at least alongside - code.
Re: Pynguin – Generate Python unit tests automatically
#3preprint available for those curious about how this works: https://arxiv.org/abs/2007.14049
Re: Pynguin – Generate Python unit tests automatically
#4Sometimes automated testing is the only way to go.
Consider a case when you have a code that takes about 100 input parameters and makes a decision on whether something should or should not happen. The logic that determines the decision is about 1,000 LOC that doesn't divide easily into smaller chunks. The code consists of a lot of different conditions and calculations based on the input parameters and some constants.
Nobody knows for sure or can explain how every single line works or why is it there, but it seems that everything works correctly based on the decision.
Writing tests manually might be feasible, but is very impractical considering that you don't know whether the code is correct or not in the first place. The code is being modified on regular basis: new inputs are added and logic is changed. Any such change would break the manual test. In my experience, nobody will spend an hour studying why the test broke and do the "right" fix. It will most likely be the easiest fix that makes the test pass.
Re: Pynguin – Generate Python unit tests automatically
#5What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.
One reason to use a tool like this is to supplement hand-written unit tests when doing a refactor. Ultimately, a lot of unit tests' jobs are just to catch unexpected side-effects of a code change.
> tests should be written from requirements
TDD might be too big of a debate to fit in an HN comment thread :)
Re: Pynguin – Generate Python unit tests automatically
#6What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.
If you do TDD, yes.
But there are a lots of projects out there that don't, and write tests afterward.
I do.
It's usually much, much faster and easier for me to write the tests after I figure out how to make what I want to work. I sometimes need to change the API a little to be more testable, but it's still faster.
In fact, if I write a test first, I have to write it with an abstract idea on how my code will work and need to understand the problem perfectly, which I rarely do. My understanding of the problem and solution grows as I write the code. It helps me ask the right questions, it reveals issues I didn't think about, I shows part of the API I didn't know by heart and above all, it cleans my initial idea of the workflow for this part of the code.
So if I write the test first, it will take a lot of time to think it through, and I will realize later I got it wrong and have to rewrite it anyway. You could argue I need to better define my problem or get better specs, but my experience is that they are always wrong as long as you haven't written any code: "no plan survive the contact with the enemy".
Not to mention you may not have written the untested code in the first place.
So I'd love a tool that can output test boiler plate for me.
Re: Pynguin – Generate Python unit tests automatically
#7What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.
I can't talk about this specific tool because I never used it and they are a bit short on the details. But if they ensure code coverage then you know at least that every line of code has been run at least once. Tools may also inspect every "if" and "while" and look for typical boundary conditions (when applicable) such as underflow, overflow, off-by-one, None, etc.
Those two checks alone can take you far, specially when compared to the alternative "I tried it once and it worked".
Re: Pynguin – Generate Python unit tests automatically
#8> mature tools exist—for statically typed languages, such as Java
Is that really a thing Java developers use? I am one, I remember doing a PoC with Jtest (now Parasoft) in 2008, and it was utterly useless, never come across any such since, I'd be genuinely interested to learn more (I'll go duckduckgo now).
Re: Pynguin – Generate Python unit tests automatically
#9The README lacks of an example of input and output.
Re: Pynguin – Generate Python unit tests automatically
#10Sometimes automated testing is the only way to go. Consider a case when you have a code that takes about 100 input parameters and makes a decision on whether something should or should not happen. The logic that determines the decision is about 1,000 LOC that doesn't divide easily into smaller chunks. The code consists of a lot of different conditions and calculations based on the input parameters and some constants.…
I'm intrigued by this example. What's an example of a domain or problem set that produces a function with 100 input parameters and a boolean result that can't be broken down cleanly?