Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pynguin – Generate Python unit tests automatically

github.com

Post reply on HN
Print view View original

21–30 of 80 posts

Re: Pynguin – Generate Python unit tests automatically

#21

Post by 411111111111111 »

BiteCode_dev wrote: post #6
_ZeD_ wrote: post #2

What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.

If you do TDD, yes. But there are a lots of projects out there that don't, and write tests afterward. I do. It's usually much, much faster and easier for me to write the tests after I figure out how to make what I want to work. I sometimes need to change the API a little to be more testable, but it's still faster. In fact, if I write a test first, I have to write it with an abstract idea on how my code will work and…

Slightly off-topic but I thought it might be interesting: you can easily modify tests to adapt to your changing architecture as well.

Personally I use TDD to have an easy entry point to run through my code, so the first instance of a test for an api might only check status code

It makes the process very iterative, as you can continuously run through your code just by saving

Re: Pynguin – Generate Python unit tests automatically

#22

Post by jeffreygoesto »

bobthecowboy wrote: post #12
BiteCode_dev wrote: post #9

The README lacks of an example of input and output.

It does, but their documentation's Quickstart has an example: https://pynguin.readthedocs.io/en/latest/user/quickstart.htm...

This looks like they combined the exploration strategy of property based testing with unit testing. This is not even testing, it tries to find some examples that result in different code paths. There is no concept of "expected result, given a certain input" and it does not try to find extreme cases or equivalence classes. Using it will result in a heap of buggy code that has a useless "but tested" sticker.

Re: Pynguin – Generate Python unit tests automatically

#23

Post by moksly »

_ZeD_ wrote: post #2

What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.

Well yes, but that kind of testing has also proven to be a waste of resources(1).

So an automated tool may be the only path forward as far as testing goes. Whether this one adds value or not is hard to say as their documentation is extremely sparse (probably why they made the tool in the first place).

1) http://www.knosof.co.uk/ESEUR/

Re: Pynguin – Generate Python unit tests automatically

#24

Post by 4140tm »

zerof1l wrote: post #14
cdogl wrote: post #10

Earlier quoted context omitted.

I'm intrigued by this example. What's an example of a domain or problem set that produces a function with 100 input parameters and a boolean result that can't be broken down cleanly?

The decision is actually 5 different variables. Types are: boolean, a decimal percent and integer being a monetary value. The domain is fintech: making a decision on how much money to give and some other aspects about the loan.

> The domain is fintech

What you describe is the opposite of what I expect from (modern) fintech.

* no tests

* god method with no proper documentation/requirements

* no one actually understanding the function deciding how much money to send! Isn't this one of the core functionalities of fintech products?

Re: Pynguin – Generate Python unit tests automatically

#25

Post by BiteCode_dev »

411111111111111 wrote: post #21
BiteCode_dev wrote: post #6

Earlier quoted context omitted.

If you do TDD, yes. But there are a lots of projects out there that don't, and write tests afterward. I do. It's usually much, much faster and easier for me to write the tests after I figure out how to make what I want to work. I sometimes need to change the API a little to be more testable, but it's still faster. In fact, if I write a test first, I have to write it with an abstract idea on how my code will work and…

Slightly off-topic but I thought it might be interesting: you can easily modify tests to adapt to your changing architecture as well. Personally I use TDD to have an easy entry point to run through my code, so the first instance of a test for an api might only check status code It makes the process very iterative, as you can continuously run through your code just by saving

Let's say you test an api, since yoy are taling about status code. You have to test all kind of cases with wrong inputs, various user states and so on.

You can of course write the test to simulate all that, but's way harder than just using your user UI which lets you discover it and put a breakpoint in the endpoint to inspect. I may not remember where to get the header data in this framework, or maybe the client send me a weird queryset param and I must figure out why, or I my validation is failing in some case and I must check it out. I can't simulate something I don't know about or don't understand yet.

Plus using the UI will make you realize you forgot to limit input length or what happen if the user doesn't have any phone number or if it's tuesday.

The minimal use case you will write your code against will likely be only good in theory in your mind unless you do something very basic. You will probably throw it away in the end in exhange for IRL code learned by feed back, and you'll have written all that for nothing.

This is were TDD proponents say ypu shoud write the minimal case and see all that stuff incrementally, but again, it's way easier and faster to just fiddle with the UI and write the code I need then test each case that imagine each case incrementally, and find the proper incatation to make the whole thing works.

I'm exagerating for the sake of the argument, but that's the result I get every time I work on a project where the team requires TDD. Incidently, those are the projects with the worst UI as well, because dev don't use it enough.

Re: Pynguin – Generate Python unit tests automatically

#26

Post by 4140tm »

zerof1l wrote: post #4

Sometimes automated testing is the only way to go. Consider a case when you have a code that takes about 100 input parameters and makes a decision on whether something should or should not happen. The logic that determines the decision is about 1,000 LOC that doesn't divide easily into smaller chunks. The code consists of a lot of different conditions and calculations based on the input parameters and some constants.…

If no one knows whether the code is correct a generated test could easily do more harm than good.

If no one is able/willing to refactor the code into understandable and testable bits, there's not much left to guarantee the correctness of it.

Re: Pynguin – Generate Python unit tests automatically

#27

Post by pjc50 »

411111111111111 wrote: post #21
BiteCode_dev wrote: post #6

Earlier quoted context omitted.

If you do TDD, yes. But there are a lots of projects out there that don't, and write tests afterward. I do. It's usually much, much faster and easier for me to write the tests after I figure out how to make what I want to work. I sometimes need to change the API a little to be more testable, but it's still faster. In fact, if I write a test first, I have to write it with an abstract idea on how my code will work and…

Slightly off-topic but I thought it might be interesting: you can easily modify tests to adapt to your changing architecture as well. Personally I use TDD to have an easy entry point to run through my code, so the first instance of a test for an api might only check status code It makes the process very iterative, as you can continuously run through your code just by saving

> you can easily modify tests to adapt to your changing architecture as well.

Can you? Surely it depends; most systems end up with more LOC in the tests than the tested code, and the tests multiply the cost of refactoring.

I've had success with the "happy path first" kind of TDD, where you write a use case and implement to that and only then come back and fill in other tests.

Re: Pynguin – Generate Python unit tests automatically

#28

Post by cjfd »

zerof1l wrote: post #14
cdogl wrote: post #10

Earlier quoted context omitted.

I'm intrigued by this example. What's an example of a domain or problem set that produces a function with 100 input parameters and a boolean result that can't be broken down cleanly?

The decision is actually 5 different variables. Types are: boolean, a decimal percent and integer being a monetary value. The domain is fintech: making a decision on how much money to give and some other aspects about the loan.

The whole thing does not sound very competent. Like noone could possibly guarantee that the result is not off by a factor of 2 or 3 or perhaps even 10 or more in some cases. What would happen in that case? It also sounds like a domain where mistakes could be very expensive or lead to lawsuits or similar such things.

Re: Pynguin – Generate Python unit tests automatically

#29

Post by totorovirus »

zerof1l wrote: post #14
cdogl wrote: post #10

Earlier quoted context omitted.

I'm intrigued by this example. What's an example of a domain or problem set that produces a function with 100 input parameters and a boolean result that can't be broken down cleanly?

The decision is actually 5 different variables. Types are: boolean, a decimal percent and integer being a monetary value. The domain is fintech: making a decision on how much money to give and some other aspects about the loan.

Does this mean there are at least 2^5 = 32 paths that a procedure can take? Seriously, is it even documentable?

Re: Pynguin – Generate Python unit tests automatically

#30

Post by dmos62 »

zerof1l wrote: post #4

Sometimes automated testing is the only way to go. Consider a case when you have a code that takes about 100 input parameters and makes a decision on whether something should or should not happen. The logic that determines the decision is about 1,000 LOC that doesn't divide easily into smaller chunks. The code consists of a lot of different conditions and calculations based on the input parameters and some constants.…

What you're describing is testing without understanding the code. I've not yet encountered inunderstandable code, and can't conceive what it might look like, though I've encountered code that I'd rather not try to understand.
Post reply on HN