Structural Equality for Better Tests
blog.ploeh.dk
Structural Equality for Better Tests
1–9 of 9 posts
Re: Structural Equality for Better Tests
#2I write all my tests by first extracting aspects of the compared items to values that are then compared structurally. Nothing else makes sense.
Re: Structural Equality for Better Tests
#3Probably worth mentioning that we sometimes need a normalisation step. The most common example is comparing two lists for equality, when their order doesn't actually matter; this can produce false-positives if some later refactor changes the order of the elements. In this case we can normalise the values by sorting them before comparing.
One example I keep hitting is a widely used JSON library for Haskell, whose test suite checks a pair of values for equality, when they just-so-happen to be in the opposite order on 32bit x86. Since I do lots of dependency pinning, this same bug keeps cropping up in different contexts :(
Re: Structural Equality for Better Tests
#4very easy is to serialize both objects and compare binary data.
Re: Structural Equality for Better Tests
#5very easy is to serialize both objects and compare binary data.
It seems clear that if the serialization matches, the objects are equal. It is not at all clear that if the serialization mismatches, that the objects are necessarily not equal, just that they are possibly not equal.
Re: Structural Equality for Better Tests
#6Probably worth mentioning that we sometimes need a normalisation step. The most common example is comparing two lists for equality, when their order doesn't actually matter; this can produce false-positives if some later refactor changes the order of the elements. In this case we can normalise the values by sorting them before comparing. One example I keep hitting is a widely used JSON library for Haskell, whose test…
Subsetting, mapping and normalization.
I think about it like running "an SQL query" on the thing I'm comparing and the resulting value structure (made up of maps/lists/numbers/bool/string) is the result set of the query. Then I compare queries.
Re: Structural Equality for Better Tests
#7Probably worth mentioning that we sometimes need a normalisation step. The most common example is comparing two lists for equality, when their order doesn't actually matter; this can produce false-positives if some later refactor changes the order of the elements. In this case we can normalise the values by sorting them before comparing. One example I keep hitting is a widely used JSON library for Haskell, whose test…
I wrote a hacky helper for this sort of thing the other day. It takes a vector of values and a vector of functions. For each value it scans the functions until one returns true, and then pops off that function. If either list isn't empty it prints the unmatched items and panics.
Re: Structural Equality for Better Tests
#8very easy is to serialize both objects and compare binary data.
It seems clear that if the serialization matches, the objects are equal. It is not at all clear that if the serialization mismatches, that the objects are necessarily not equal, just that they are possibly not equal.
Not so fun fact. ROS1 does this to compute hashes for their message format, but they do it wrong, so that every ROS1 system has potentially catastrophic deserialisations going on.
It's a wontfix because they'd have to recompile their packages to fix it. Yeah, talk about quality...
Re: Structural Equality for Better Tests
#9Probably worth mentioning that we sometimes need a normalisation step. The most common example is comparing two lists for equality, when their order doesn't actually matter; this can produce false-positives if some later refactor changes the order of the elements. In this case we can normalise the values by sorting them before comparing. One example I keep hitting is a widely used JSON library for Haskell, whose test…
I think that using a better datastructure is often better than normalizing in tests: if order doesn’t matter, use a bag or set instead of arrays. I’ve generally found that this sort of thinking reduces boilerplate across the board.