this..seems extremely cool (?) does someone have a reference or pointer for further study about how this is 'perfect' and not just 'really good'
A perfect hash function has no collisions. Typically, people are interested in perfect hashes constructed on types that themselves are not represented in the number of output bits but with an input cardinality that is less than or equal to the output's cardinality. What the author is describing here is more commonly called a mixing function. There are many fast mixing functions. https://gist.github.com/badboy/6267743
Fast Perfect Hashing
11–20 of 43 posts
Re: Fast Perfect Hashing
#12> A perfect hash function is one that is collision-free. By implication, the hash must be at least as many bytes as the key I don't understand this lemma. An indexed array is a degenerate example in which the "hash" (pointer) is presumably a lot smaller than the key (index). BTW for those not familiar with a perfect hash function they are great for things like symbol tables that are immutable at runtime. You can make…
Re: Fast Perfect Hashing
#13> A perfect hash function is one that is collision-free. By implication, the hash must be at least as many bytes as the key I don't understand this lemma. An indexed array is a degenerate example in which the "hash" (pointer) is presumably a lot smaller than the key (index). BTW for those not familiar with a perfect hash function they are great for things like symbol tables that are immutable at runtime. You can make…
It's smaller than the key, but the size of the domain is the number of items you've put in the array.
Re: Fast Perfect Hashing
#14The OP seems to be confusing terminology. What they construct is a permutation. Everything they say is relevant to permutations, not perfect hashing. In particular: > A perfect hash function is one that is collision-free. By implication, the hash must be at least as many bytes as the key This is just false. Perfect hashing is often if not always performed on subsets of the full domain, e.g. the keywords in a programm…
Re: Fast Perfect Hashing
#15Re: Fast Perfect Hashing
#16The OP seems to be confusing terminology. What they construct is a permutation. Everything they say is relevant to permutations, not perfect hashing. In particular: > A perfect hash function is one that is collision-free. By implication, the hash must be at least as many bytes as the key This is just false. Perfect hashing is often if not always performed on subsets of the full domain, e.g. the keywords in a programm…
It's really nice work though! Invertible mixers are usually building blocks for more general hash function, for example MurmurHash3 uses this 64-bit mixer
https://github.com/aappleby/smhasher/blob/master/src/MurmurH...
Using this trick could produce a faster murmurhash-type function.
Re: Fast Perfect Hashing
#17this..seems extremely cool (?) does someone have a reference or pointer for further study about how this is 'perfect' and not just 'really good'
The first sentence of the article explains what a perfect hash function is : "A perfect hash function is one that is collision-free.". Essentially, a perfect hash function is one where no two domain elements map to the same range element. If you want to explore it further, you can search for injective functions and one-to-one mapping. An example of a function that produces collisions is the ABS function. ABS( 2 ) = 2…
Re: Fast Perfect Hashing
#18> A perfect hash function is one that is collision-free. By implication, the hash must be at least as many bytes as the key and the function is theoretically reversible, though not always tractably so.
This is true because of the pigeonhole principle, but most people won't assume that the key size is equivalent to the domain size. Usually when using perfect hashing, you are hashing (much) fewer elements than the total range a key can represent.
A perfect hash function that uniquely assigns hash values to the eight items you need to store, but gives you back integers anywhere in the 32 bit range isn't super helpful. Or, at least, it's not obvious to me why it would be. They do state the algorithm can be scaled down to 8 or 16 bits, so I guess there's that.
> Because the hash is no smaller than the key, the primary use case is randomizing small values like integral types.
This just doesn't make sense to me. The key can be much larger than the hash, even with perfect hashing, as long as the number of keys in your data set is smaller than the maximum hash value.
The author says "primary use case" but what they really mean is that this isn't what most people would consider a perfect hash at all. It's just an integer permuter. In other words, here as an even faster implementation of what the author would describe as a perfect hash over small integer keys:
uint32_t hash(uint32_t key) { return key; }
This also guarantees there are no collisions so is a "perfect hash". It's just not a very useful one.The author assumes the reader knows the value of randomizing your integer keys. I'm not sure what that value is in cases where you know you don't have collisions anyway.
Re: Fast Perfect Hashing
#19> A perfect hash function is one that is collision-free. By implication, the hash must be at least as many bytes as the key I don't understand this lemma. An indexed array is a degenerate example in which the "hash" (pointer) is presumably a lot smaller than the key (index). BTW for those not familiar with a perfect hash function they are great for things like symbol tables that are immutable at runtime. You can make…
In cases where you can’t associate that metadata directly using a pointer (by using structures as keys embedding the pointer in the key) or a direct index into a metadata array, a perfect hash generated at build time is about as good as you’re going to get.
The other thing is that often this is easy to bolt on after the fact. As an example, in network devices it is common for a syslog mnemonic to be associated with an instance of a rate limiter. A good solution is to compile your log messages using an errmsg definition and compiler tool (which also allows you to build documentation) but this requires a lot of up-front planning and build orchestration and makes the code awkward in sone circumstances. An alternative is that you allow people to declare their mnemonic (and, for example, rate limit) in-place and then allow the logger library to use that to look up other properties that are joined from other sources (if there are any) and glean the structures that need to be created.
Since extracting the full set on mnemonics is necessary anyway (for tripwires, for documentation, ...) you can now build the hash function after-the-fact. This definitely simplified the build considerations (and avoids unnecessary rebuilds when, for example, only documentation changes).
The after-the-factness-but-with-low-overhead part is what’s nice about it. Yes, it’s better to structure things up front to make the system optimal but it’s nice to be able to get close.
Re: Fast Perfect Hashing
#20Earlier quoted context omitted.
It's smaller than the key, but the size of the domain is the number of items you've put in the array.
Actually, I might have misunderstood you. In your example, the index in the array is treated like the hash of the item you've put in there, why are we suddenly considering pointers?