Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pynguin – Generate Python unit tests automatically

github.com

Post reply on HN
Print view View original

61–70 of 80 posts

Re: Pynguin – Generate Python unit tests automatically

#61

Post by mristin »

mristin wrote: post #57
gjulianm wrote: post #40

Earlier quoted context omitted.

Coding with contracts has always been interesting to me but I haven't had the option/time to try it seriously in a project. I assume you've had experience with it, how much productivity/code maintanability you gain compared to not using it (or using type annotations)?

In my anecdotal experience, it takes very little time for juniors to pick up adding contracts to their code. You need to grasp implication, equivalence, exclusive or, and get used to declarative code, but then it's easy. (I often compare it to SQL.) I find contracts personally super useful as I can express a lot of relationships in the code trivially and have them automatically verified. For example, when this input…

P.S. I think the important bit is not to be religious about contracts and tests. Sometimes it's easy to write the contracts and have the function automatically tested, sometimes unit tests are clearer.

I tend to limit myself to simple contracts and do table tests for the complex cases to reap the benefits of the both approaches.

Re: Pynguin – Generate Python unit tests automatically

#62

Post by iib »

zelphirkalt wrote: post #44

Once I found these names funny, replacing any letter after a "P" in a word with a "y" to make it a PYthon library name. It has become a bit trite over time though. What other funny "strategies" are there in Python or other programming language ecosystems?

This is a small excerpt from a draft blog post I am currently writing:

- Python forces the `py` affix in words: scrapy, scapy, rpyc

- Python does wordplay on standard library modules' names: pickle => dill

- Emacs and Julia append the file extension to the end of the name: restclient.el, HTTP.jl, JSON.jl

- Emacs does wordplay on the package the extension is based on: git => magit

These are the ones I was most familiar with. If the replies to your comment provide more insights, I may be able to extend that part of the blog post :)

Re: Pynguin – Generate Python unit tests automatically

#63

Post by ChrisRackauckas »

iib wrote: post #62
zelphirkalt wrote: post #44

Once I found these names funny, replacing any letter after a "P" in a word with a "y" to make it a PYthon library name. It has become a bit trite over time though. What other funny "strategies" are there in Python or other programming language ecosystems?

This is a small excerpt from a draft blog post I am currently writing: - Python forces the `py` affix in words: scrapy, scapy, rpyc - Python does wordplay on standard library modules' names: pickle => dill - Emacs and Julia append the file extension to the end of the name: restclient.el, HTTP.jl, JSON.jl - Emacs does wordplay on the package the extension is based on: git => magit These are the ones I was most familia…

https://github.com/JuliaSIMD/Polyester.jl is a cheap threading model in Julia.

Re: Pynguin – Generate Python unit tests automatically

#64

Post by stjohnswarts »

kortilla wrote: post #56
ptx wrote: post #46

Earlier quoted context omitted.

This warning quoted from the README doesn't seem to suggest anything unexpected or unusual. Obviously running the tests would execute the module under test. (Although I suppose if someone wanted to generate tests without ever executing them the warning would be relevant.)

This warning is very important. It’s absolutely not expected that generating tests means executing module code, side effects and all.

Was anyone thinking it would "break" their computer just generating the tests? I would have though that only such a thing could happen later particularly if your python code was doing a bunch of file related stuff.

Re: Pynguin – Generate Python unit tests automatically

#65

Post by stjohnswarts »

4140tm wrote: post #24
zerof1l wrote: post #14

Earlier quoted context omitted.

The decision is actually 5 different variables. Types are: boolean, a decimal percent and integer being a monetary value. The domain is fintech: making a decision on how much money to give and some other aspects about the loan.

> The domain is fintech What you describe is the opposite of what I expect from (modern) fintech. * no tests * god method with no proper documentation/requirements * no one actually understanding the function deciding how much money to send! Isn't this one of the core functionalities of fintech products?

Just plain scary especially with money if they can't at the very least test the extremes and some in the middle values. Good gravy! What is the name of this company so I can avoid it...

Re: Pynguin – Generate Python unit tests automatically

#66

Post by HelloNurse »

mristin wrote: post #61
mristin wrote: post #57

Earlier quoted context omitted.

In my anecdotal experience, it takes very little time for juniors to pick up adding contracts to their code. You need to grasp implication, equivalence, exclusive or, and get used to declarative code, but then it's easy. (I often compare it to SQL.) I find contracts personally super useful as I can express a lot of relationships in the code trivially and have them automatically verified. For example, when this input…

P.S. I think the important bit is not to be religious about contracts and tests. Sometimes it's easy to write the contracts and have the function automatically tested, sometimes unit tests are clearer. I tend to limit myself to simple contracts and do table tests for the complex cases to reap the benefits of the both approaches.

If a software module maybe has tests and maybe has contracts, I "reap the benefit" of not being able to rely on either.

Do you have in mind some technique to turn contracts into tests or tests into contracts automatically, in order to close the gaps in quality control?

Re: Pynguin – Generate Python unit tests automatically

#67

Post by the_af »

moksly wrote: post #23
_ZeD_ wrote: post #2

What kind of "reassurance" do a tool like pynguin provide? In my head tests should be written from requirements, before - or at least alongside - code.

Well yes, but that kind of testing has also proven to be a waste of resources(1). So an automated tool may be the only path forward as far as testing goes. Whether this one adds value or not is hard to say as their documentation is extremely sparse (probably why they made the tool in the first place). 1) http://www.knosof.co.uk/ESEUR/

Which kind of testing has been proven to be a waste of resources? TDD?

I'm no fan of TDD -- I think there's a lot of cargo culting about it -- but I wasn't aware anything had been "proven" about it. I've seen papers claiming it helps, papers claiming it sort of helps, and some people claiming it mostly doesn't help (or that what actually helps are some practices that go along with it).

I thought there was no industry consensus that it doesn't help.

Re: Pynguin – Generate Python unit tests automatically

#68

Post by mristin »

HelloNurse wrote: post #66
mristin wrote: post #61

Earlier quoted context omitted.

P.S. I think the important bit is not to be religious about contracts and tests. Sometimes it's easy to write the contracts and have the function automatically tested, sometimes unit tests are clearer. I tend to limit myself to simple contracts and do table tests for the complex cases to reap the benefits of the both approaches.

If a software module maybe has tests and maybe has contracts, I "reap the benefit" of not being able to rely on either. Do you have in mind some technique to turn contracts into tests or tests into contracts automatically, in order to close the gaps in quality control?

Sorry, I did not express myself clearly. For certain functions you can express all the properties in contracts and have them automatically tested.

For other functions, you write some obvious contracts (so that those are also tested in integration tests or end-to-end tests), but since writing all the contracts would be too tedious or unmaintainable, you test these functions additionally using, say, table-driven tests where you specifically test with data points for which you could not write the preconditions or you check the post-conditions manually given the input data. For example, sometimes is easier to generate the input, manually inspect the results and write the expected results in the test table.

> [...] turn contracts into tests [...]

icontract-hypothesis allows you to ghostwrite Hypothesis strategies which you can further refine yourself.

Re: Pynguin – Generate Python unit tests automatically

#69

Post by one2three4 »

>> Pynguin can cause serious harm to your computer, for example, wipe your entire hard disk! We recommend running Pynguin in an isolated environment, for example, a Docker container, to minimise the risk of damaging your system.

That sounds ominous.

Why do they execute the generated tests though? I mean I'd expect this to go only as far as generating the tests. Also that means that at least someone needs to review the tests.

Re: Pynguin – Generate Python unit tests automatically

#70

Post by wpietri »

mmmkkaaayy wrote: post #48

There wasn't much info on the GitHub repo about how it actually works. Here's the paper from the authors: https://arxiv.org/abs/2007.14049 From what I gather from the paper, they frame the problem of test generation as a search problem. An evolutionary algorithm randomly mutates a randomly generated test suite. The evolutionary algorithm optimizes for greater branch coverage. Excerpt from the abstract: "Our experimen…

I have trouble even understanding the point of this. To me "automatic test generation" seems as bad an idea as "automatic code generation".

The point of tests for me is to express intent for what the software should do. The code then expresses the details of how we do it.

I could see a use for evolutionary algorithms to probe code for hidden bugs. But even there it seems limited. I definitely see a use for things like Hypothesis, which makes test expression more powerful: https://hypothesis.readthedocs.io/en/latest/

It makes sense to me that this comes from an academic perspective, and not from people who actually make software for a living.

Post reply on HN