Live data from Hacker News

Show HN: Python Tests That Write Themselves

timothycrosley.github.io

11–20 of 39 posts

Re: Show HN: Python Tests That Write Themselves

#11

Thoughts behind project creation live here: https://timothycrosley.com/project-5-hypothesis-auto

Thanks for the write up - explains a lot (including what hypothesis is).

I think there's a typo in the last code box? It looks like you repeated "from hypothesis_auto import"? (that, or I understand even less python that I thought!)

Re: Show HN: Python Tests That Write Themselves

#12
post #5

Does this do anything other than assert that functions return the right basic type given extremely basic inputs? Is naive type testing a thing that people actually bother doing vs testing that the functions actually do what they're supposed to? If a function takes two ints and returns an int that is the integer division of the two, is the evaluation of exceptions and returned types not already implicit in the evaluat…

By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). To get the most value out of it, you can further specify what you expect from the result using the _auto_verify=Callable() parameter, or by looking at the individual test cases. Simple example: from my_library import add from hypothesis_auto import auto_pytest @auto_pytes…

> By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself).

So it's a fuzzer?

Re: Show HN: Python Tests That Write Themselves

#13
post #12

Earlier quoted context omitted.

By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). To get the most value out of it, you can further specify what you expect from the result using the _auto_verify=Callable() parameter, or by looking at the individual test cases. Simple example: from my_library import add from hypothesis_auto import auto_pytest @auto_pytes…

> By default, the biggest value from this is catching unexpected runtime exceptions with certain values (the edge cases you don't think of yourself). So it's a fuzzer?

Yes! A smart one though, that limits itself to the type constraints you've specified to avoid redoing the work of your type checker.

Re: Show HN: Python Tests That Write Themselves

#14
post #5

Does this do anything other than assert that functions return the right basic type given extremely basic inputs? Is naive type testing a thing that people actually bother doing vs testing that the functions actually do what they're supposed to? If a function takes two ints and returns an int that is the integer division of the two, is the evaluation of exceptions and returned types not already implicit in the evaluat…

It's also useful to check the function doesn't crash.

I use hypothesis to test a DB abstraction, and it often caught bugs in edge cases I didn't consider: timestamps/timezones too large, strings containing null bytes, etc.

These would be caught by the tests generated by hypothesis-auto.

Re: Show HN: Python Tests That Write Themselves

#15

Thoughts behind project creation live here: https://timothycrosley.com/project-5-hypothesis-auto

Thanks for the write up - explains a lot (including what hypothesis is). I think there's a typo in the last code box? It looks like you repeated "from hypothesis_auto import"? (that, or I understand even less python that I thought!)

You are correct! Thanks for catching this! I'll fix the example later today

Re: Show HN: Python Tests That Write Themselves

#16
I like the idea, but I think I would be more comfortable with it generating test files that I could keep beside my other tests.

Also, `auto_pytest_magic` doesn't seem to exist.

    ImportError: cannot import name 'auto_pytest_magic' from 'hypothesis_auto' (/home/.../.local/lib/python3.7/site-packages/hypothesis_auto/__init__.py)

Re: Show HN: Python Tests That Write Themselves

#17

I like this idea. One piece of feedback, a parameter with a leading underscore feels very odd. In python I interpret leading underscores to indicate the programmer thinks of this as an internal / pseudo-private property. Exposing it through the api makes it "public" which means (to me) that it shouldn't have a leading underscore. This is especially true if the usecase is common enough to put in the top level examples

While I agree, that `_param` generally means private, it is also a common way to allow parameters to be passed into a function where all other parameters are passed directly along via args and *kwargs to avoid naming collisions. An example of this is NamedTuple: https://docs.python.org/3/library/collections.html#namedtupl... .

I believe Raymond Hettinger now considers that a mistake, and wishes he had gone with a trailing underscore (param_) instead. A trailing underscore is just as unlikely to lead to a collision, but less confusing.

Re: Show HN: Python Tests That Write Themselves

#18
post #16

I like the idea, but I think I would be more comfortable with it generating test files that I could keep beside my other tests. Also, `auto_pytest_magic` doesn't seem to exist. ImportError: cannot import name 'auto_pytest_magic' from 'hypothesis_auto' (/home/.../.local/lib/python3.7/site-packages/hypothesis_auto/__init__.py)

> I think I would be more comfortable with it generating test files that I could keep beside my other tests.

This is a very interesting idea I might explore more!

`auto_pytest_magic` is only made available if you have pytest installed. You can enforce this by doing:

  pip3 install -U hypothesis-auto[pytest]

Re: Show HN: Python Tests That Write Themselves

#19

Earlier quoted context omitted.

While I agree, that `_param` generally means private, it is also a common way to allow parameters to be passed into a function where all other parameters are passed directly along via args and *kwargs to avoid naming collisions. An example of this is NamedTuple: https://docs.python.org/3/library/collections.html#namedtupl... .

I believe Raymond Hettinger now considers that a mistake, and wishes he had gone with a trailing underscore (param_) instead. A trailing underscore is just as unlikely to lead to a collision, but less confusing.

That's a great call! I'll update this to use trailing underscore in a later version

Re: Show HN: Python Tests That Write Themselves

#20
post #16

I like the idea, but I think I would be more comfortable with it generating test files that I could keep beside my other tests. Also, `auto_pytest_magic` doesn't seem to exist. ImportError: cannot import name 'auto_pytest_magic' from 'hypothesis_auto' (/home/.../.local/lib/python3.7/site-packages/hypothesis_auto/__init__.py)

> I think I would be more comfortable with it generating test files that I could keep beside my other tests. This is a very interesting idea I might explore more! `auto_pytest_magic` is only made available if you have pytest installed. You can enforce this by doing: pip3 install -U hypothesis-auto[pytest]

Oh man, for some reason I thought pytest was a synonym for Python's unittest module.
Post reply on HN