21–30 of 39 posts
It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs.
You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
If the library can use the observed and recorded testcase to extrapolate far beyond your manual call, this could be a really interesting way of testing python code.
This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
A lower barrier way of trying this out could be to run MonkeyType in combination with OP's hypothesis-auto (which uses type annotations to generate the tests).
This is neatly packaged, but it's not immediately clear what advantages it has over Hypothesis' native offerings for accomplishing this? ( https://hypothesis.readthedocs.io/en/latest/details.html#inf... and https://hypothesis.readthedocs.io/en/latest/data.html#hypoth... )
OP's extension is inferring arguments to the fn under test, and then generating a test like 'assert isinstancee(fn_under_test(a, *kw), expected_ret_type)` to go with them.
You can do more types of test with `infer` (i.e. not just 'does return' and 'returns correct type' but also 'returns correct value'), but this is an easy way to cover a lot of basic return checking ground.
This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
I know, coverage is Yet Another Metric, but if you don't game it, it can help you track down branches you haven't written tests for.
So my hesitation is I can see people running this, gettting 100% code coverage and thinking, "hooray, it's fully tested!"
This actually gave me another idea. What do you all think, I’d be up for trying to build it. It would watch your program during execution and record each function calls input and output. And then create tests for each function using those inputs and outputs. You could always go over the created tests manually and fix them or review them but if nothing else it could be a good start.
Instagram created 'MonkeyType' to do something similar for type annotations. A lower barrier way of trying this out could be to run MonkeyType in combination with OP's hypothesis-auto (which uses type annotations to generate the tests).
[1] https://github.com/google/pytype
[2] https://github.com/dropbox/pyannotate