Live data from Hacker News

Hypothesis: Property-Based Testing for Python

hypothesis.readthedocs.io

51–60 of 164 posts

Re: Hypothesis: Property-Based Testing for Python

#51

Property based testing is fantastic. Why is it not more popular? My theory is that only code written in functional languages has complex properties you can actually test. In imperative programs, you might have a few utils that are appropriate for property testing - things like to_title_case(str) - but the bulk of program logic can only be tested imperatively with extensive mocking.

But wouldn't that apply just as much to example based testing?

Re: Hypothesis: Property-Based Testing for Python

#53

Property based testing is fantastic. Why is it not more popular? My theory is that only code written in functional languages has complex properties you can actually test. In imperative programs, you might have a few utils that are appropriate for property testing - things like to_title_case(str) - but the bulk of program logic can only be tested imperatively with extensive mocking.

I actually used property testing very successfully to test a DB driver and a migration to another DB driver in Go. I wrote up about it here https://blog.tiserbox.com/posts/2024-02-27-stateful-property...

Re: Hypothesis: Property-Based Testing for Python

#54

Property based testing is fantastic. Why is it not more popular? My theory is that only code written in functional languages has complex properties you can actually test. In imperative programs, you might have a few utils that are appropriate for property testing - things like to_title_case(str) - but the bulk of program logic can only be tested imperatively with extensive mocking.

>> Why is it not more popular?

Property, fuzzy, snapshot testing. Great tools that make software more correct and reliable.

The challenge for most developers is that they need to change how they design code and think about testing.

I’ve always said the hardest part of programming isn’t learning, it’s unlearning what you already know…

Re: Hypothesis: Property-Based Testing for Python

#56
I love the idea of hypothesis! Haven't found a lot of use cases for it yet, I think the quick start example helps explain why.

Essentially, you're testing that "my_sort" returns the same as python's standard "sort". Of course, this means you need a second function that acts the same as the function you wrote. In real life, if you had that you probably wouldn't have written the function "my_sort" at all.

Obviously it's just a tiny example, but in general I often find writing a test that will cover every hypothetical case isn't realistic.

Anyone here use this testing in the wild? Where's it most useful? Do you have the issue I described? Is there an easy way to overcome it?

Re: Hypothesis: Property-Based Testing for Python

#57

I love the idea of hypothesis! Haven't found a lot of use cases for it yet, I think the quick start example helps explain why. Essentially, you're testing that "my_sort" returns the same as python's standard "sort". Of course, this means you need a second function that acts the same as the function you wrote. In real life, if you had that you probably wouldn't have written the function "my_sort" at all. Obviously it'…

On more complex logic you can often reason about many invariants. I don't think using sorting and then comparing to the existing sort is a good example. Precisely for the reason you mention.

A sort itself is a good example though. You can do much better then just compare to an existing sort implementation. Because you can easily check if an array is sorted with a linear scan. If your sort is stable you can do an additional check that for all pairs of adjacent elements that compare as equal the order of them is the same as in the input array.

Re: Hypothesis: Property-Based Testing for Python

#58

I love the idea of hypothesis! Haven't found a lot of use cases for it yet, I think the quick start example helps explain why. Essentially, you're testing that "my_sort" returns the same as python's standard "sort". Of course, this means you need a second function that acts the same as the function you wrote. In real life, if you had that you probably wouldn't have written the function "my_sort" at all. Obviously it'…

Indeed. For a sorting algorithm it would be more insightful to show it test the actual property of something that is sorted: every consecutive element is larger than the previous one (or the other way around). You don’t need a sorting function to test the “sorted” property.

Re: Hypothesis: Property-Based Testing for Python

#59
I'm using sqlglot to parse hundreds of old mysql back up files to find diffs of schemas. The joys of legacy code. I've found hypothesis to be super helpful for tightening up my parser. I've identified properties (invariants) and built some strategies. I can now generate many more permutations of DDL than I'd thought of before. And I have absolutely confidence in what I'm running.

I started off TDD covered the basics. Learned what I needed to learn about the files I'm dealing with, edge cases, sqlglot and then I moved onto Hypothesis for extra confidence.

I'm curious to see if it'll help with commands for APIs. I nothing else it'll help me appreciate how liberal my API is when perhaps I don't want it to be?

Re: Hypothesis: Property-Based Testing for Python

#60

I love the idea of hypothesis! Haven't found a lot of use cases for it yet, I think the quick start example helps explain why. Essentially, you're testing that "my_sort" returns the same as python's standard "sort". Of course, this means you need a second function that acts the same as the function you wrote. In real life, if you had that you probably wouldn't have written the function "my_sort" at all. Obviously it'…

> Anyone here use this testing in the wild? Where's it most useful? Do you have the issue I described? Is there an easy way to overcome it?

One example would be when you have a naive implementation of some algorithm and you want to introduce a second one, optimized but with much more complex implementation. Then this naive one will act as a model for comparisons.

Another case that comes to mind is when you have rather simple properties to test (like: does it finish without a crash, within a given time?, does not cross some boundaries on the output?), and want to easily run over a sensible set of varying inputs.

Post reply on HN