Hypothesis: Property-Based Testing for Python • Hacker News Board

Re: Hypothesis: Property-Based Testing for Python

#161

Post by tybug » Fri, Nov 07, 2025, 8:48 PM UTC

dragonwriter wrote: post #155

tybug wrote: post #151

Earlier quoted context omitted.

(Hypothesis maintainer here) If you have recommendations for a better example on the front page, I'd love to hear them! (I mean this entirely genuinely and non-sarcastically; I agree sorting can give misleading ideas, but it is also concise and well understood by every reader).

The more I think about it the more I think calling it a bad example may be unfair. It can be extremely misleading for someone unfamiliar with the concept coming at it with a particular viewpoint, but I’m less sure, with more time to think, that an example that is better for that wouldn’t be worse in other ways. I like sorting as an example, and I like that using the built-in is concise, and reimplementing the behavio…

Appreciate the thoughts <3. I do think there might be stronger examples we could choose. Possibly json encode/decode..

Re: Hypothesis: Property-Based Testing for Python

#162

Post by dmitry_dygalo » Sat, Nov 08, 2025, 9:52 PM UTC

agumonkey wrote: post #101

groundhogstate wrote: post #62

Earlier quoted context omitted.

You might find schemathesis useful for API testing (which IIRC is built on Hypothesis). Certainly helped me find a stack of unhandled edge cases. YMMV, but once I first set up Schemathesis on one of my MVPs, it took several hours to iron out the kinks. But thereafter, if built into CI, I've found it guards against regression quite effectively.

pretty nice tool, helped us poke into our app in ways we didn't foresee also pretty happy to see the reports of thousands or api calls that matched the expected responses, that increases the confidence you have in your system

Hi! Author of Schemathesis here, really glad to hear it helped you uncover those edge cases. If any of those bugs were in public or open-source APIs, I’d love to feature them in our new “trophy case”: https://github.com/schemathesis/schemathesis/issues/new?temp...

Re: Hypothesis: Property-Based Testing for Python

#163

Post by ibizaman » Tue, Nov 11, 2025, 1:32 PM UTC

imiric wrote: post #129

dragonwriter wrote: post #120

Earlier quoted context omitted.

> One drawback I see is that property-based tests inevitably need to be much more complex than example-based ones. I don’t think that’s true, I just think the complexity is more explicit (in code) rather than implicit (in the process of coming up with examples). Example-based testing usually involves defining conditions and properties to be tested, then involves constructing sets of examples to test them and which at…

> Property based testing involves defining the conditiosn and properties, writing code that generates the conditions and for each property writing a bit of code that can refute it by passing if and only if it is true of the subject under test for a particular set of inputs. You're downplaying the amount of code required to properly setup a property-based test. In the linked article, the author implemented a state mac…

You’re right it’s always a trade off. One unexpected but very welcomed side effect of having those stateful property tests is we could use them to design high fidelity stubs. I wrote a follow-up blog post about it https://blog.tiserbox.com/posts/2024-07-08-make-good-stubs-w...

Re: Hypothesis: Property-Based Testing for Python

#164

Post by ibizaman » Tue, Nov 11, 2025, 1:34 PM UTC

Jtsummers wrote: post #121

imiric wrote: post #61

Earlier quoted context omitted.

Thanks for sharing! Your article illustrates well the benefits of this approach. One drawback I see is that property-based tests inevitably need to be much more complex than example-based ones. This means that bugs are much more likely, they're more difficult to maintain, etc. You do mention that it's a lot of code, but I wonder if the complexity is worth it in the long run. I suppose that since testing these scenari…

My experience is that PBT tests are mostly hard in devising the generators, not in the testing itself. Since it came up in another thread (yes, it's trivial), a function `add` is no easier or harder to test with examples than with PBT, here are some of the tests as both PBT-style and example-based style: @given(st.integers()) def test_left_identity_pbt(a): assert add(a, 0) == a def test_left_identity(): assert add(10…

For sure, the hardest part is to create meaningful generators for the problem at hand which can test interesting cases in a finite amount of time. That’s where the combinatory explosion takes place in my experience.

I wanted to highlight one unexpected but very welcomed side effect of having those stateful property tests is we could use them to design high fidelity stubs. I wrote a follow-up blog post about it https://blog.tiserbox.com/posts/2024-07-08-make-good-stubs-w...