11–20 of 44 posts
Looks like two interleaved 128-bit hashes, don't see any cross-half mixing. Code style needs a bit of cleanup, the sindex stuff obscures the algorithm a bit.
-Austin, SMHasher/Murmur author
Not bad, I like that the author stuck with plain C instead of sse/aes. Looks like two interleaved 128-bit hashes, don't see any cross-half mixing. Code style needs a bit of cleanup, the sindex stuff obscures the algorithm a bit. -Austin, SMHasher/Murmur author
Should I use this as an everyday hash function (for non-security purposes)? I am always interested when these get attention, but I don't know enough about the implications to switch over from just using SHA.
If you're shipping a black-box component that needs to use hashing internally, and the hash outputs don't leak out of the black-box, consider switching to a faster non-cryptographic hash to gain performance. Consider the implications of switching -- dependencies, trust, performance profile, hash output size, documentation, customer expectation -- and you will have to discard or otherwise invalidate the meaning behind of your past hash outputs.
If you're occasionally applying a hash function and obtain a digest that gets put into long-lived files or records, e.g. you're checksumming your own files for sanity and then verify them later against these records, then you may value availability and stability more than you value performance. If so, don't switch.
If you are fine with the current performance profile, the cost and complexity of switching (or any nontrivial change) may outweigh the benefits of leaving everything as-is.
What is the memory bandwidth on that instance? I ask because I'm not seeing that listed [1] but it would be a useful point of comparison. Maybe run Doctor Bandwidth's STREAM benchmark [2].
DiscoHash is included in SMHASHER [3] but its benchmark results aren't.
[1] https://cloud.google.com/compute/docs/machine-types
This is a hash designed for non-cryptographic use, like in hash tables or bloom filters. You can tell by their small output size, which greatly reduces the cost of a brute-force collision search. It's also in the linked readme. Hash function families with a similar target usecase include: cityhash, falkhash, farmhash, FNV, meowhash, metrohash, murmur, t1ha, wyhash, xxh. The SMHasher suite tests hash functions for spe…
The Readme specifically says "you can modify it to yield 128-bits or more if you want a cryptographically secure hash."
Which is a problematic statement, because it is not designed for cryptography even if you extended the output to 256 bit. It's not the output length that makes it cryptographicaly secure. (Rather it's the difference between "you won't find collisions by accident" and "you won't find collisions even if you try really hard using very sophisticated math", but there are more requirements.)
This is similar to the issues with a different hash by the same Github user posted two weeks ago:
https://news.ycombinator.com/item?id=23103521
Making fast non-cryptographic hash functions is a fun challenge and I appreciate the projects, but please, please do not make any claims about cryptographic properties!
Not bad, I like that the author stuck with plain C instead of sse/aes. Looks like two interleaved 128-bit hashes, don't see any cross-half mixing. Code style needs a bit of cleanup, the sindex stuff obscures the algorithm a bit. -Austin, SMHasher/Murmur author
Was it the inconsistent use of curly braces that made you comment on the code style?
Tested at ~ 5GB/s @ 3Gz, (Google Cloud Platform, N1 CPU) What is the memory bandwidth on that instance? I ask because I'm not seeing that listed [1] but it would be a useful point of comparison. Maybe run Doctor Bandwidth's STREAM benchmark [2]. DiscoHash is included in SMHASHER [3] but its benchmark results aren't. [1] https://cloud.google.com/compute/docs/machine-types [2] https://www.cs.virginia.edu/stream/ [3] ht…
They are included, but understandably you missed them because it's also called BEBB4185, stated in README. Find BEBB4185 line in SMHASHER Readme.
Good question on the memory bandwidth. From memory it was a multicore system, so I think that has a higher memory bandwidth than a single core. Thanks for the STREAM thing!
Earlier quoted context omitted.
Was it the inconsistent use of curly braces that made you comment on the code style?
Predeclared aliased pointers, STATEM/HSTATEM constants, len/Len, that kind of stuff.
BTW, not sure exactly what you mean by cross-half mixing, but I'm assuming it means mixing both halves of the 128 bit state with each other. There is that, on 3 lines 91[0], 100 and 108.
The 128 bit mix, stirs two adjacent blocks of 64 bits. Those 3 lines can mix blocks 1 & 2.
Subsequent mixes on either half then propagate that cross mix. So it does use the full 256 bit state.
Is that what you meant?
[0]: https://github.com/cris691/discohash/blob/master/src/discoha...
This is a hash designed for non-cryptographic use, like in hash tables or bloom filters. You can tell by their small output size, which greatly reduces the cost of a brute-force collision search. It's also in the linked readme. Hash function families with a similar target usecase include: cityhash, falkhash, farmhash, FNV, meowhash, metrohash, murmur, t1ha, wyhash, xxh. The SMHasher suite tests hash functions for spe…
I think SipHash is the better choice for non-cryptographic use cases (e.g. hash tables) https://131002.net/siphash/
This is a hash designed for non-cryptographic use, like in hash tables or bloom filters. You can tell by their small output size, which greatly reduces the cost of a brute-force collision search. It's also in the linked readme. Hash function families with a similar target usecase include: cityhash, falkhash, farmhash, FNV, meowhash, metrohash, murmur, t1ha, wyhash, xxh. The SMHasher suite tests hash functions for spe…
All of those mentioned have Hash flooding issues due to invertability or lack of diffusion. I think SipHash is the better choice for non-cryptographic use cases (e.g. hash tables) https://131002.net/siphash/
FWIW, the SMHasher test suite takes the view [1] that defense against hash flooding attacks is a concern for the hash table's collision resolution method, which is a fair point. Nonetheless, SipHash was subsequently adopted by several programming languages' standard libraries for use in hash tables. SipHash is also notable for its clear and concise specification, including security claims, preliminary cryptanalysis, and a discussion on hash flooding [2].
[1] https://github.com/rurban/smhasher#security [2] https://eprint.iacr.org/2012/351.pdf