Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Python Tests That Write Themselves

timothycrosley.github.io

Post reply on HN
Print view View original

31–39 of 39 posts

Re: Show HN: Python Tests That Write Themselves

#31

Post by Izkata »

ryall wrote: post #24
mrfusion wrote: post #21

This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.

I remember finding something like this a few years ago for testing rest apis. It was very cool but for the life of me I can't recall the name :(

Possibly vcr [0], except that's for mocking out APIs in unit tests. There's a whole bunch of ports to other languages midway down the page.

[0] https://github.com/vcr/vcr

Re: Show HN: Python Tests That Write Themselves

#32

Post by Izkata »

duckerude wrote: post #17
timothycrosley wrote: post #9

Earlier quoted context omitted.

While I agree, that `_param` generally means private, it is also a common way to allow parameters to be passed into a function where all other parameters are passed directly along via args and *kwargs to avoid naming collisions. An example of this is NamedTuple: https://docs.python.org/3/library/collections.html#namedtupl... .

I believe Raymond Hettinger now considers that a mistake, and wishes he had gone with a trailing underscore (param_) instead. A trailing underscore is just as unlikely to lead to a collision, but less confusing.

Personally, I like how django does it, where one of the kwargs accepts a dict, and that dict is what's passed along as kwargs to the next level.

Re: Show HN: Python Tests That Write Themselves

#34

Post by anonytrary »

mrfusion wrote: post #21

This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.

This sounds like snapshot testing. It is the responsibility of the person to browse the produced snapshots to ensure they are correct.

Re: Show HN: Python Tests That Write Themselves

#35

Post by timothycrosley »

duckerude wrote: post #17
timothycrosley wrote: post #9

Earlier quoted context omitted.

While I agree, that `_param` generally means private, it is also a common way to allow parameters to be passed into a function where all other parameters are passed directly along via args and *kwargs to avoid naming collisions. An example of this is NamedTuple: https://docs.python.org/3/library/collections.html#namedtupl... .

I believe Raymond Hettinger now considers that a mistake, and wishes he had gone with a trailing underscore (param_) instead. A trailing underscore is just as unlikely to lead to a collision, but less confusing.

Updated in latest release to use trailing underscore: https://timothycrosley.github.io/hypothesis-auto/CHANGELOG/ figured it would be the least painful the earlier I did it :)

Re: Show HN: Python Tests That Write Themselves

#36

Post by duckerude »

timothycrosley wrote: post #35
duckerude wrote: post #17

Earlier quoted context omitted.

I believe Raymond Hettinger now considers that a mistake, and wishes he had gone with a trailing underscore (param_) instead. A trailing underscore is just as unlikely to lead to a collision, but less confusing.

Updated in latest release to use trailing underscore: https://timothycrosley.github.io/hypothesis-auto/CHANGELOG/ figured it would be the least painful the earlier I did it :)

Great!

It takes a bit of manual juggling, but if you take kwargs anyway you could inspect them to check for the old-style parameters, and use them if they're present, perhaps with a warning. That way you don't have to break compatibility.

Re: Show HN: Python Tests That Write Themselves

#38

Post by ACow_Adonis »

noobiemcfoob wrote: post #23
mrfusion wrote: post #21

This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.

I really like this idea. So, broken out, would the development process look like: 1) Write code 2) Open interpreter (ipython) 3) Import code and call it manually 4) This magic library will have observed and created a testcase module from your execution If the library can use the observed and recorded testcase to extrapolate far beyond your manual call, this could be a really interesting way of testing python code.

This is actually the idea behind a little library I developed for my own use in common lisp called testy.

Because the common lisp environment already defines special variables that contain the last form evaluated and the return value of the last form evaluated, when you evaluate code at the REPL and it succeeds, you call the (testy:test) function, which captures them and wraps them up as a test for the package you're working on, adding it to the database of tests for the package.

Not only was it useful for development (no need to specify your tests up front, you write your code until it works, then that state "is" the test), but it was also helpful for fixing bugs that you hadn't thought of up front (you fix the bug and then that state is captured as a new test as well).

Re: Show HN: Python Tests That Write Themselves

#39

Post by somada141 »

While I had really liked the idea of hypothesis in Python I found that the edge-cases it was uncovering were the ones that were obviously gonna break but at the same time cases I didn't care to guard against, e.g., using 3-mile long integers, or cases that wouldn't work with the underlying libraries eg NumPy. Thus, I found myself spending more time adding constraints on the generated inputs than fledging out my test-suite. So my adventures with hypothesis were short-lived.

I don't mean to detract from this library, I think its a great combination of strong-typing and property-based testing but has anyone had any experience employing property-based testing on complex functions outside of the whole add/subtract/multiple stuff? What kinda thing have you used it on?

Post reply on HN