Hyrum's Law • Hacker News Board

Re: Hyrum's Law

#11

Post by dgacmu » Sat, Feb 17, 2024, 1:24 AM UTC

cpeterso wrote: post #3

One approach to guard against Hyrum’s Law is GREASE (aka “Generate Random Extensions And Sustain Extensibility” used in the TLS 1.3 protocol) i.e. behavior randomization to avoid inadvertent dependencies on unspecified behavior: https://textslashplain.com/2020/05/18/a-bit-of-grease-keeps-... What are other approaches?

The Go hash table approach is similar to this: it's randomizedish but not strongly random enough to depend on it being actually random...

https://dev.to/wallyqs/gos-map-iteration-order-is-not-that-r...

(Which I find a hilarious second order example of Hyrum's law - if you add true randomness to prevent people from depending on the iteration order, they might use it as a way to randomly access items in the map!)

Re: Hyrum's Law

#12

Post by wrsh07 » Sat, Feb 17, 2024, 1:24 AM UTC

cpeterso wrote: post #2

What are some fun or unusual examples of Hyrum’s Law that you’ve run into? Were you the user, inadvertently depending on some implementation detail? Or were users depending on your implementation’s details?

At Google (long time off and on home of Hyrum), they migrated to swisstable from other hash maps. Unordered map iterates through the elements in insertion order. They explicitly couldn't support that without adding overhead to their new table so they randomized iteration order of the map

The migrations to the new table were fun because you would migrate some code and then find a test that assumed the order things were inserted into some seemingly distant protocol buffer array were in insertion order of the map

Usually the ordering was just a defect of the test, but it required actually digging in to be sure

Re: Hyrum's Law

#13

Post by david_allison » Sat, Feb 17, 2024, 1:36 AM UTC

cpeterso wrote: post #2

What are some fun or unusual examples of Hyrum’s Law that you’ve run into? Were you the user, inadvertently depending on some implementation detail? Or were users depending on your implementation’s details?

I implemented a development-only feature which span up a thread and permanently held a lock on the database, effectively making the app useless until restarted.

A couple months later, I had a user thank me for it. I still don't know why

Re: Hyrum's Law

#14

Post by tanx16 » Sat, Feb 17, 2024, 2:22 AM UTC

cpeterso wrote: post #3

One approach to guard against Hyrum’s Law is GREASE (aka “Generate Random Extensions And Sustain Extensibility” used in the TLS 1.3 protocol) i.e. behavior randomization to avoid inadvertent dependencies on unspecified behavior: https://textslashplain.com/2020/05/18/a-bit-of-grease-keeps-... What are other approaches?

Our team went with a similar approach when refactoring Protobuf debug APIs (https://bughunters.google.com/blog/6405366705946624/fixing-d...). People were relying on debug output and trying to parse it, so in the new implementation we threw up big warning flags and made the output unstable so that you couldn't make the mistake.

The key lesson to take away is: if you want something to be an implementation detail, make sure to have multiple differing implementations :)

Re: Hyrum's Law

#15

Post by tialaramex » Sat, Feb 17, 2024, 2:24 AM UTC

wrsh07 wrote: post #12

cpeterso wrote: post #2

What are some fun or unusual examples of Hyrum’s Law that you’ve run into? Were you the user, inadvertently depending on some implementation detail? Or were users depending on your implementation’s details?

At Google (long time off and on home of Hyrum), they migrated to swisstable from other hash maps. Unordered map iterates through the elements in insertion order. They explicitly couldn't support that without adding overhead to their new table so they randomized iteration order of the map The migrations to the new table were fun because you would migrate some code and then find a test that assumed the order things wer…

IIRC There's a (CppCon?) talk about this rollout where Hyrum is in the audience so that he can heckle each time they describe a change which "obviously" can't break anybody because of course Hyrum's Law did cause that to break at Google when they did it.

Maybe somebody who remembers better can link it and/or correct my memory of exactly what's going on.

Re: Hyrum's Law

#16

Post by tialaramex » Sat, Feb 17, 2024, 2:31 AM UTC

dgacmu wrote: post #11

cpeterso wrote: post #3

One approach to guard against Hyrum’s Law is GREASE (aka “Generate Random Extensions And Sustain Extensibility” used in the TLS 1.3 protocol) i.e. behavior randomization to avoid inadvertent dependencies on unspecified behavior: https://textslashplain.com/2020/05/18/a-bit-of-grease-keeps-... What are other approaches?

The Go hash table approach is similar to this: it's randomizedish but not strongly random enough to depend on it being actually random... https://dev.to/wallyqs/gos-map-iteration-order-is-not-that-r... (Which I find a hilarious second order example of Hyrum's law - if you add true randomness to prevent people from depending on the iteration order, they might use it as a way to randomly access items in the map!)

Matt Kulukundis (sp?) mentions this problem is his talk about Google's rather fancier Swiss tables.

At that point their debug builds would do a "coin toss" with 50.3% chance to be heads during hash table insertion and he expected some idiots would use that to generate random bits then be annoyed if it breaks. I believe in production it's entirely seeded from ASLR, so you're actually leaking address layout info if you try to use these bits as entropy instead of just to make the hash table defeat your bad unit tests.

Re: Hyrum's Law

#17

Post by andrewingram » Sat, Feb 17, 2024, 2:34 AM UTC

One could argue that React’s strict mode behaviour in dev is a guard against this —- as it prevented you from building things that will break with (the then) upcoming functionality.

Re: Hyrum's Law

#18

Post by sebastianconcpt » Sat, Feb 17, 2024, 2:35 AM UTC

One thing I found surprising is how disregarding many companies are about creating and nurturing know-how transfer culture among teammates. They seem to value compartmentalisation so much that they forget they are not de-risking themselves from Bus Factor.

Re: Hyrum's Law

#19

Post by dang » Sat, Feb 17, 2024, 2:58 AM UTC

Hyrum's Law - https://news.ycombinator.com/item?id=33283849 - Oct 2022 (53 comments)

Hyrum's Law - https://news.ycombinator.com/item?id=29848295 - Jan 2022 (36 comments)

Hyrum's Law - https://news.ycombinator.com/item?id=27386818 - June 2021 (5 comments)

Hyrum's Law: An Observation on Software Engineering - https://news.ycombinator.com/item?id=21515225 - Nov 2019 (6 comments)

Hyrum's Law - https://news.ycombinator.com/item?id=19249199 - Feb 2019 (1 comment)

Re: Hyrum's Law

#20

Post by nonethewiser » Sat, Feb 17, 2024, 2:59 AM UTC

My attempt to generalize it because i dont think it its only true of apis.

With enough consumers, all observable system behaviors will be relied upon, regardless of the promises made by the system provider.