Might be more useful pointed [here][1], perhaps? [1]: http://hypothesis.readthedocs.org/en/latest/
Well, it is a link to the "official" announcement, and the announcement directly links there, but I can see you point too.
Hypothesis 1.0: A property-based testing library for Python
21–30 of 31 posts
Re: Hypothesis 1.0: A property-based testing library for Python
#22This is a very awesome library. I am attempting to integrate it into the test suites of my various projects this past week and the author has been more than helpful, even when I filed bug reports that turns out to be issues in my own code.
Slightly OT ramble ahead :)
> even when I filed bug reports that turns out to be issues in my own code.
Every time I've used or built a property based testing system I've had problems when starting because there's something broken in it. This has then turned out to be an actual bug in the system being tested.
Building a quickcheck style tester in AS3 drove me nuts at one point before finding out that the conversions between strings and floating point numbers has a bunch of weird issues. There are some obvious ones, but then also things like certain numbers convert differently if you add a trailing zero (so 0.7362856270 is turned into a different number to 0.736285627 and 0.73628562700). So my little "decode(encode(x)) == x" test example broke!
My favourite thing that it found was in a menuing library we were building (and why I built the tester). I set it up to test a library by making library calls that a developer might make and test various properties of the overall menu (all elements reachable, for some if you move left then right you're back on the same item, etc). One property was that if there was at least one item in the list and the list had focus, then one item in that list had focus. This was found to be broken by starting with only one item, removing it and adding a new one (reduced test cases are incredibly useful).
That's a fairly boring bug, but the interesting thing (to me) was that when I fixed it my unit tests broke. I had an explicit test to ensure that this behaviour was happening, and I'd also written it down in my spec for what should happen.
The testing tool forced me to consider higher level things of what should be true and drove out an inconsistency in my library. It's a 'bug' that would have bitten me many times, but rarely enough that it would probably have regularly ended up going live.
Re: Hypothesis 1.0: A property-based testing library for Python
#23This is a very awesome library. I am attempting to integrate it into the test suites of my various projects this past week and the author has been more than helpful, even when I filed bug reports that turns out to be issues in my own code.
Slightly OT ramble ahead :) > even when I filed bug reports that turns out to be issues in my own code. Every time I've used or built a property based testing system I've had problems when starting because there's something broken in it. This has then turned out to be an actual bug in the system being tested. Building a quickcheck style tester in AS3 drove me nuts at one point before finding out that the conversions…
> Building a quickcheck style tester in AS3 drove me nuts at one point before finding out that the conversions between strings and floating point numbers has a bunch of weird issues. There are some obvious ones, but then also things like certain numbers convert differently if you add a trailing zero (so 0.7362856270 is turned into a different number to 0.736285627 and 0.73628562700). So my little "decode(encode(x)) == x" test example broke!
I have in fact had to deal with exactly this problem in Hypothesis internals. The example saving code didn't work correctly with floats in the first edition, because it was serializing them as JSON, which loses some information encoded in the actual float. In the end I serialize floats (which are actually doubles) by converting them to a 64-bit integer with the same bitwise representation first.
The bug aryx is talking about though actually illustrates an amusing feature of Hypothesis, which is that the code in it is just weird enough that it tends to do unexpected things that trigger bugs which have nothing to do with the properties being tested. :-) In this case it was putting unicode objects onto sys.path, which is 100% allowed but causes problems for certain code running on python 2.7 on windows that previously appeared to work.
Re: Hypothesis 1.0: A property-based testing library for Python
#24Earlier quoted context omitted.
You're right, of course. I typed that on my cellphone, and were a little quick looking up alternatives for @double_application_turns_this_into_an_identity_function -- that made more sense, and was a little more succinct. Maybe @left_inverse would be more appropriate? It's been a while since I had to classify functions.
If you're curious, the math-y term for an operation that is its own inverse is involution . I don't know that I've ever heard the word in a programming context, but there it is! :)
An involution played a rather important role in this programming context:
http://blog.moertel.com/posts/2013-12-14-great-old-timey-gam...
Re: Hypothesis 1.0: A property-based testing library for Python
#25I read about QuickChck over 10 years ago in Uni, I was just reintroduced to property based testing this week in HelsinkiJS meetup and now this Python implementation hit Hacker News.
Question: most of my backend code is rather dull CRUD and business logic code. Are there good tutorials about randomly generating objects with a few of fields and relationships between them and testing business logic using hypothesis?
Re: Hypothesis 1.0: A property-based testing library for Python
#26Earlier quoted context omitted.
Slightly OT ramble ahead :) > even when I filed bug reports that turns out to be issues in my own code. Every time I've used or built a property based testing system I've had problems when starting because there's something broken in it. This has then turned out to be an actual bug in the system being tested. Building a quickcheck style tester in AS3 drove me nuts at one point before finding out that the conversions…
> Building a quickcheck style tester in AS3 drove me nuts at one point before finding out that the conversions between strings and floating point numbers has a bunch of weird issues. There are some obvious ones, but then also things like certain numbers convert differently if you add a trailing zero (so 0.7362856270 is turned into a different number to 0.736285627 and 0.73628562700). So my little "decode(encode(x)) =…
> I have in fact had to deal with exactly this problem in Hypothesis internals.
Hah, wonderful. It's a great example of how this type of testing can really dig out odd bugs. Once you've tried property based testing, I think you never really trust things quite the same :)
> The bug aryx is talking about though actually illustrates an amusing feature of Hypothesis, which is that the code in it is just weird enough that it tends to do unexpected things that trigger bugs which have nothing to do with the properties being tested
Nice :)
Thanks for releasing this library, it's great to see more work being done in this area. There are usually a few random testing libraries floating about but adding things like minimisation (and I'm still reading through the templating & other new stuff) really makes it stand out.
The API testing example in particular actually comes at a perfect time for me, so it'll definitely be getting some use.
Re: Hypothesis 1.0: A property-based testing library for Python
#27I read about QuickChck over 10 years ago in Uni, I was just reintroduced to property based testing this week in HelsinkiJS meetup and now this Python implementation hit Hacker News. Question: most of my backend code is rather dull CRUD and business logic code. Are there good tutorials about randomly generating objects with a few of fields and relationships between them and testing business logic using hypothesis?
Have you seen http://hypothesis.readthedocs.org/en/latest/examples.html#fu... ? It doesn't test very much - mostly just that the API doesn't produce a 500 error - but it's a decent example of how you can generate structured data with some constraints. http://hypothesis.readthedocs.org/en/latest/examples.html#co... is also a decent example where the data is more uniform but requires a bunch of massaging to satisfy the constraints.
Re: Hypothesis 1.0: A property-based testing library for Python
#28This is awesome! And it comes right as I've been getting my feet wet with Haskell (and property-based testing). It was really easy to get up and running with a simple test: https://github.com/ecordell/pymacaroons/blob/property-tests/... and I'm excited to use it in more places. Though I'll have to configure tox/travis not to run hypothesis tests on Python 2.6.
Cool! I've added a comment to your commit as I think you've made a mistake in the strategy setup, but it looks good as a test.
Sorry about the lack of 2.6 support. I looked into it and I could probably do it but I just don't care enough about 2.6 to put in the work and make everything uglier. :-)
Re: Hypothesis 1.0: A property-based testing library for Python
#29Earlier quoted context omitted.
> Building a quickcheck style tester in AS3 drove me nuts at one point before finding out that the conversions between strings and floating point numbers has a bunch of weird issues. There are some obvious ones, but then also things like certain numbers convert differently if you add a trailing zero (so 0.7362856270 is turned into a different number to 0.736285627 and 0.73628562700). So my little "decode(encode(x)) =…
> I have in fact had to deal with exactly this problem in Hypothesis internals. Hah, wonderful. It's a great example of how this type of testing can really dig out odd bugs. Once you've tried property based testing, I think you never really trust things quite the same :) > The bug aryx is talking about though actually illustrates an amusing feature of Hypothesis, which is that the code in it is just weird enough that…
> Once you've tried property based testing, I think you never really trust things quite the same :)
Oh god, tell me about it. So far I've hit 4 pypy bugs, 3 cpython bugs, two pytz bugs (one caused by a cpython bug), and I've learned far more about the edge cases of the language than I ever wanted to know.
Lets just not talk about the number of bugs I've found in Hypothesis itself in the course of testing bits of the system I was sure worked perfectly.
Re: Hypothesis 1.0: A property-based testing library for Python
#30Earlier quoted context omitted.
You're right, of course. I typed that on my cellphone, and were a little quick looking up alternatives for @double_application_turns_this_into_an_identity_function -- that made more sense, and was a little more succinct. Maybe @left_inverse would be more appropriate? It's been a while since I had to classify functions.
If you're curious, the math-y term for an operation that is its own inverse is involution . I don't know that I've ever heard the word in a programming context, but there it is! :)
Thank you! A quick look through my bookshelf reveals that out of four books on discrete mathematics, only one contain involution in the index: and there it is mentioned once in a supplementary exercise... I guess that explains why I couldn't seem to recall a term for "self-inverse" [ed: No, that's wrong to, lol. Lets just stick with involution].