Live data from Hacker News

SeaHash: Explained

ticki.github.io

21–30 of 57 posts

Re: SeaHash: Explained

#21
post #2

> SeaHash has mathematically provable statistical guarantees I'm all for proofs but would love to see an empirical head-to-head against metroHash. Is it as good or better output distribution?

MetroHash's main transformation actually loses entropy, so that's, well, pretty bad. It still passes Smhasher, though.

Re: SeaHash: Explained

#22

Could someone explain what the point is? Is there a use-case for this for "general hashing" and such, where sha256 is _genuinely_ insufficient?

A good example is for a bloom filter: using sha256 is much too slow for good filter performance, you want something like siphash or some other non-cryptographic hash.

Here's a good story of the performance benefits of switching from cryptographic to non-crypto hashes: https://github.com/bitly/dablooms/pull/19

(But I don't recommend you use murmur anymore: https://emboss.github.io/blog/2012/12/14/breaking-murmur-has... (although tbh I could be wrong on this one, not an expert))

(Shameless plug for my bloom filter tutorial https://llimllib.github.io/bloomfilter-tutorial/ )

Re: SeaHash: Explained

#23

I've been using blake2(b) for file hashing in some content addressable db stuff. What might be a scenario where i would choose Seahash over Blake2? My main concern was speed and assurance that i would not see collisions. Beyond that, i am clearly naive on the subject.

There's a huge difference between cryptographic and non-cryptographic. Note that blake2 has various length, whereas SeaHash is fixed to 64-bit (although I suppose it's not to hard to make a version with bigger length), and thus naturally collisions will happen.

If you need fingerprints, don't use SeaHash, but if you are looking to insert into e.g. a hash table, you shouldn't use BLAKE2. It's awfully slow for that.

Re: SeaHash: Explained

#24

Earlier quoted context omitted.

Hi, I recently used a hash function like this as part of a React-style delta calculation engine but for Swift / iOS. I needed a hash function that was fast but collision-resistant, and I did not need a cryptographic hash, as all data is trusted (and a hash collision would not actually matter that much). I chose SpookyHash V2 based on the advice of some peers and http://aras-p.info/blog/2016/08/09/More-Hash-Function-T…

Even when I've not needed a cryptographic hash, I've still used one, because why not? I've never not needed one so bad as to resort to some barely studied, homemade hashing algorithm. > A hash collision wouldn't matter that much. Interesting. What was the use for the hash function then?

In hash tables, you never use cryptographic hash functions. Why? Because they're slower.

Take SHA3, which is around 50x slower than SeaHash. That is really really bad for hash tables.

When hash collisions happen in hash tables, they're resolved through collision-resolution strategy, such a linear proping.

Fingerprints are one very narrow usecase for hash functions, and there are tousands of other uses.

Re: SeaHash: Explained

#25

Earlier quoted context omitted.

Hi, I recently used a hash function like this as part of a React-style delta calculation engine but for Swift / iOS. I needed a hash function that was fast but collision-resistant, and I did not need a cryptographic hash, as all data is trusted (and a hash collision would not actually matter that much). I chose SpookyHash V2 based on the advice of some peers and http://aras-p.info/blog/2016/08/09/More-Hash-Function-T…

Even when I've not needed a cryptographic hash, I've still used one, because why not? I've never not needed one so bad as to resort to some barely studied, homemade hashing algorithm. > A hash collision wouldn't matter that much. Interesting. What was the use for the hash function then?

> Even when I've not needed a cryptographic hash, I've still used one, because why not?

To avoid wasting cpu cycles preserving a property you don't need. Seahash should be 50x faster than sha3

Re: SeaHash: Explained

#27

Earlier quoted context omitted.

Hi, I recently used a hash function like this as part of a React-style delta calculation engine but for Swift / iOS. I needed a hash function that was fast but collision-resistant, and I did not need a cryptographic hash, as all data is trusted (and a hash collision would not actually matter that much). I chose SpookyHash V2 based on the advice of some peers and http://aras-p.info/blog/2016/08/09/More-Hash-Function-T…

Even when I've not needed a cryptographic hash, I've still used one, because why not? I've never not needed one so bad as to resort to some barely studied, homemade hashing algorithm. > A hash collision wouldn't matter that much. Interesting. What was the use for the hash function then?

Caching often relies on hash functions. If you run the Cloudflare cache, you'll start caring, considering hashing is usually >50% of the CPU workload and an optimized non-cryptographic hash function can be 20x faster.

Re: SeaHash: Explained

#28
post #2

> SeaHash has mathematically provable statistical guarantees I'm all for proofs but would love to see an empirical head-to-head against metroHash. Is it as good or better output distribution?

Should be easy to plug it into smhasher.

According to the author, it passes.

Re: SeaHash: Explained

#29

Earlier quoted context omitted.

Hi, I recently used a hash function like this as part of a React-style delta calculation engine but for Swift / iOS. I needed a hash function that was fast but collision-resistant, and I did not need a cryptographic hash, as all data is trusted (and a hash collision would not actually matter that much). I chose SpookyHash V2 based on the advice of some peers and http://aras-p.info/blog/2016/08/09/More-Hash-Function-T…

Even when I've not needed a cryptographic hash, I've still used one, because why not? I've never not needed one so bad as to resort to some barely studied, homemade hashing algorithm. > A hash collision wouldn't matter that much. Interesting. What was the use for the hash function then?

A trade off between collision potential vs. speed. Sometimes you need speed more than you need cryptographic levels of collision avoidance.

For example, finding unique files on the file system. After looking at size, first and last bytes, it would be better to filter quickly on an imperfect hash (with, say, a 1 in 1^56 chance of collision) than slowly on a perfect hash (with a 1 in 1^256 chance).

Re: SeaHash: Explained

#30

Earlier quoted context omitted.

Even when I've not needed a cryptographic hash, I've still used one, because why not? I've never not needed one so bad as to resort to some barely studied, homemade hashing algorithm. > A hash collision wouldn't matter that much. Interesting. What was the use for the hash function then?

A trade off between collision potential vs. speed. Sometimes you need speed more than you need cryptographic levels of collision avoidance. For example, finding unique files on the file system. After looking at size, first and last bytes, it would be better to filter quickly on an imperfect hash (with, say, a 1 in 1^56 chance of collision) than slowly on a perfect hash (with a 1 in 1^256 chance).

I hope you mean 2^56 and 2^256 :)
Post reply on HN