Live data from Hacker News

Bombadil: Property-based testing for web UIs

github.com

61–70 of 112 posts

Re: Bombadil: Property-based testing for web UIs

#61

My kingdom for a way to stop this godforsaken industry from stripping Tolkien's fiction for parts.

We can go strip Shakespeare instead.

(For those who don't get it, "My kingdom for ___" is from Richard III: "my horse, my horse, my kingdom for a horse")

Re: Bombadil: Property-based testing for web UIs

#62

My kingdom for a way to stop this godforsaken industry from stripping Tolkien's fiction for parts.

All the companies you’re thinking of are likely paying the Tolkien estate a very fat fee. This repo will likely have to rename.

Yeah. Unfortunately the people running the estate today appear to share ~none of the late Professor's values.

Re: Bombadil: Property-based testing for web UIs

#63

My kingdom for a way to stop this godforsaken industry from stripping Tolkien's fiction for parts.

Let's start naming things after Iain Banks ships.

Elon Musk has already had a head start on that idea for over 10 years, unfortunately:

https://scifi.stackexchange.com/q/81164

Re: Bombadil: Property-based testing for web UIs

#64

Earlier quoted context omitted.

Thanks, glad you like it! Do you mean the temporal logic aspect of it?

Yeah, the temporal logic aspect of it is exactly what I was referring to :)

Cool. I think that is a very neat way of expressing properties of UIs (and stateful systems more generally) that works out nicely in testing. There are some gotchas related to the finiteness of testing, but it's manageable.

Re: Bombadil: Property-based testing for web UIs

#66
post #12
post #4

I'm a huge fan of property based testing, I've built some runners before, and I think it can be great for UI things too so very happy to see this coming around more. Something I couldn't see was how those examples actually work, there are no actions specified. Do they watch a user, default to randomly hitting the keyboard, neither and you need to specify some actions to take? What about rerunning things? Is there shr…

How effective is property based testing in practice? I would assume it has no trouble uncovering things like missing null checks or an inverted condition because you can cover edge cases like null, -1, 0, 1, 2^n - 1 with relatively few test cases and exhaustively test booleans. But beyond that, if I have a handful of integers, dates, or strings, then the state space is just enormous and it seems all but impossible to…

"Exhaustively covering the search space" or "hitting specific edge cases" is the wrong way to think about property tests, in my experience. I find them most valuable as insanity checks, i.e. they can verify that basic invariants hold under conditions even I wouldn't think of testing manually. I'd check for empty strings, short strings, long strings, strings without spaces, strings with spaces, strings with weird characters, etc. But I might not think of testing with a string that's only spaces. The generator will.

Re: Bombadil: Property-based testing for web UIs

#67
post #12
post #4

I'm a huge fan of property based testing, I've built some runners before, and I think it can be great for UI things too so very happy to see this coming around more. Something I couldn't see was how those examples actually work, there are no actions specified. Do they watch a user, default to randomly hitting the keyboard, neither and you need to specify some actions to take? What about rerunning things? Is there shr…

How effective is property based testing in practice? I would assume it has no trouble uncovering things like missing null checks or an inverted condition because you can cover edge cases like null, -1, 0, 1, 2^n - 1 with relatively few test cases and exhaustively test booleans. But beyond that, if I have a handful of integers, dates, or strings, then the state space is just enormous and it seems all but impossible to…

One of the founders of Antithesis gave a talk about this problem last week; diversity in test cases is definitely an issue they're trying to tackle. The example he gave was Spanner tests not filling its cache due to jittering near zero under random inputs. Not doing that appears to be a company goal.

https://github.com/papers-we-love/san-francisco/blob/master/...

Re: Bombadil: Property-based testing for web UIs

#68
post #12

Earlier quoted context omitted.

How effective is property based testing in practice? I would assume it has no trouble uncovering things like missing null checks or an inverted condition because you can cover edge cases like null, -1, 0, 1, 2^n - 1 with relatively few test cases and exhaustively test booleans. But beyond that, if I have a handful of integers, dates, or strings, then the state space is just enormous and it seems all but impossible to…

One of the founders of Antithesis gave a talk about this problem last week; diversity in test cases is definitely an issue they're trying to tackle. The example he gave was Spanner tests not filling its cache due to jittering near zero under random inputs. Not doing that appears to be a company goal. https://github.com/papers-we-love/san-francisco/blob/master/...

Glad you enjoyed the talk! Making Bombadil able to take advantage of the intelligence in the Antithesis platform is definitely a goal, but we wanted to get a great open source tool into peoples’ hands ASAP first.

Re: Bombadil: Property-based testing for web UIs

#70

Earlier quoted context omitted.

Let's start naming things after Iain Banks ships.

Elon Musk has already had a head start on that idea for over 10 years, unfortunately: https://scifi.stackexchange.com/q/81164

Weird - Iain m Banks would’ve found Musk’s politics abhorrent
Post reply on HN