21–30 of 37 posts
Interesting! I wonder how this could be applied to a Rails project (since that's what ruby is most used for). For example how would it know to test a wide selection of possible values for different variables if they're not explicitly set up?
Can anyone recommend an introduction to PBT for someone who knows nothing about it? Edited to add: D'oh! There's an explanation in the linked repo.
Very cool. Wonder how much time this adds to a test suite as you incorporate it more. Might need to have some different test targets to keep things fast.
In my opinion, PBT should be used in combination with example-based testing. Besides, since example-based testing cases account for the majority of tests, I think it's rare that PBT's execution time is dominant.
As for the combination and usages, refs: https://medium.com/criteo-engineering/introduction-to-proper...
Excited to see libraries using Ractors as somebody who has also fallen in love with them ♡
Very specific question: when tests fail due to a single example, you suggest creating a specific assertion with the given seed. I imagine the seed is used to generate data and depending on the order of your generators, it produces different results. For example, in: PBT.assert(seed = ..) do PBT.property(PBT.integer, PBT.string) ... end Would changing the order of parameters to `property` change the actual test case?
(not OP but I would be surprised if the answer wasn't) yes, because you're changing the order in which the random draws are interpreted. But this isn't a problem in practice because you generally aren't changing the generator in the middle of debugging a failure.
Correct. The test inputs are determined by a seed and generators (including the order of generators).
Earlier quoted context omitted.
(not OP but I would be surprised if the answer wasn't) yes, because you're changing the order in which the random draws are interpreted. But this isn't a problem in practice because you generally aren't changing the generator in the middle of debugging a failure.
So, this means refactoring becomes potentially difficult. While the gem is still a great accomplishment and very useful, I'd have to engineer my way around this issue before using it with things like a Rails Model which could have changing shape. @OP: I wonder if the README (and possibly runner) should suggest writing a test-case that doesn't rely on PBT when the user wants to preserve a case for future testing. The…
- Create a test case that doesn't depend on PBT as you suggest.
- Fix the production code being tested since its failure is an unexpected bug.
- Fix the PBT itself. This means that the programmer has had wrong assumption for the test target.
I think it's difficult for the tool to know which choice is the best on a failure. But if there's any good idea, I'd like to incorporate it. :)
[1] https://github.com/DRMacIver/minithesis
[2] https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.E...
It would be cool if we could eventually make use of RBS or inline Sorbet or something better than both to get the types for property testing for free.
Earlier quoted context omitted.
Ruby just simply isn’t the glue language winner because of the heavy emphasis on rails. From a systems perspective, I had to switch to python because it has pyroute2, which supports rtnl, devlink, ethtool and more. I would have thought ruby had a full-fledged netlink library right now considering the stability of chef and puppet. But all I could find was this from 8 years ago: https://github.com/BytemarkHosting/netli…
> I wish it was all ruby Why do you not wish it was all Python?