Show HN: Python Tests That Write Themselves
timothycrosley.github.io
Show HN: Python Tests That Write Themselves
1–10 of 39 posts
Re: Show HN: Python Tests That Write Themselves
#2Nice concept but does it work with real-world application.
I failed to understand how will it work with methods like `authenticate_user(user)` or `load_permissions_from_db(user, db)`.
Re: Show HN: Python Tests That Write Themselves
#3Nice concept but does it work with real-world application. I failed to understand how will it work with methods like `authenticate_user(user)` or `load_permissions_from_db(user, db)`.
Hi @kburman,
For cases like that it would be best combined with a mock:
@auto_pytest()
def test_add(test_case, mocker):
mocker.patch('db.call')
test_case()
However, I would also note, that just because you have some methods that have potentially dangerous side effects, in most large code-bases, not all functions do. Which is why it operates at a function by function basis. You can use this for new pure functions, while continuing to write other tests in the same fashion by hand.
Re: Show HN: Python Tests That Write Themselves
#4Thoughts behind project creation live here: https://timothycrosley.com/project-5-hypothesis-auto
Re: Show HN: Python Tests That Write Themselves
#5Does this do anything other than assert that functions return the right basic type given extremely basic inputs?
Is naive type testing a thing that people actually bother doing vs testing that the functions actually do what they're supposed to?
If a function takes two ints and returns an int that is the integer division of the two, is the evaluation of exceptions and returned types not already implicit in the evaluation of the actual results?
If you know that divide(a, b) should return c for sufficient candidates a, b, and c, then you _know_ that divide returns the right type without explicitly checking. And knowing that the divide function happens to return ints when given ints doesn't actually tell you that it's doing anything even close to the right behavior. So this both doesn't reduce the number of tests you need to write and is also obsoleted by actually writing the tests that you need.
Re: Show HN: Python Tests That Write Themselves
#6This is neatly packaged, but it's not immediately clear what advantages it has over Hypothesis' native offerings for accomplishing this? ( https://hypothesis.readthedocs.io/en/latest/details.html#inf... and https://hypothesis.readthedocs.io/en/latest/data.html#hypoth... )
Re: Show HN: Python Tests That Write Themselves
#7I like this idea. One piece of feedback, a parameter with a leading underscore feels very odd. In python I interpret leading underscores to indicate the programmer thinks of this as an internal / pseudo-private property. Exposing it through the api makes it "public" which means (to me) that it shouldn't have a leading underscore.
This is especially true if the usecase is common enough to put in the top level examples
Re: Show HN: Python Tests That Write Themselves
#8Does this do anything other than assert that functions return the right basic type given extremely basic inputs? Is naive type testing a thing that people actually bother doing vs testing that the functions actually do what they're supposed to? If a function takes two ints and returns an int that is the integer division of the two, is the evaluation of exceptions and returned types not already implicit in the evaluat…
By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). To get the most value out of it, you can further specify what you expect from the result using the _auto_verify=Callable() parameter, or by looking at the individual test cases.
Simple example:
from my_library import add
from hypothesis_auto import auto_pytest
@auto_pytest()
def test_add(test_case):
add_result = test_case()
if test_case.params.kwargs['number_1'] > 0 and test_case.params.kwargs['number_1']:
assert add_result > test_case.params.kwargs['number_1']
assert add_result > test_case.params.kwargs['number_1']Re: Show HN: Python Tests That Write Themselves
#9I like this idea. One piece of feedback, a parameter with a leading underscore feels very odd. In python I interpret leading underscores to indicate the programmer thinks of this as an internal / pseudo-private property. Exposing it through the api makes it "public" which means (to me) that it shouldn't have a leading underscore. This is especially true if the usecase is common enough to put in the top level examples
While I agree, that `_param` generally means private, it is also a common way to allow parameters to be passed into a function where all other parameters are passed directly along via args and *kwargs to avoid naming collisions. An example of this is NamedTuple: https://docs.python.org/3/library/collections.html#namedtupl....
Re: Show HN: Python Tests That Write Themselves
#10This is neatly packaged, but it's not immediately clear what advantages it has over Hypothesis' native offerings for accomplishing this? ( https://hypothesis.readthedocs.io/en/latest/details.html#inf... and https://hypothesis.readthedocs.io/en/latest/data.html#hypoth... )
It's an extension for hypothesis, and its value is 100% just convenience and accessibility. Really, it's meant to be a sort of gateway to doing full property-based testing, in the least barrier way possible. I wrote a bit more about why I created it here: https://timothycrosley.com/project-5-hypothesis-auto