Live data from Hacker News

Advanced techniques to implement fast hash tables

attractivechaos.wordpress.com

11–20 of 22 posts

Re: Advanced techniques to implement fast hash tables

#11

Earlier quoted context omitted.

Of course. Notable libraries implementing quadratic probing all use addition instead of multiplication. The example in the blog post is just for demonstration purpose.

>addition instead of multiplication What is the advantage of this?

Addition being cheaper than multiplication on many CPUs?

Re: Advanced techniques to implement fast hash tables

#13

The author of this post wrote klib/khash, and uses it in his very popular cpu-intensive bioinformatics programs such as bwa (short read alignment against the human ref genome). I've been learning rust recently. As a learning exercise, I compared the robin-hood hashing of std::collections::HashMap in rust to his klib/khash he mentions in this article , and then tried various hash functions to try and match his perform…

How does that compare to the dict implementation of Python 3.6 ? It's supposed to be damned fast but I don't have the skills to make such comparisons.

Re: Advanced techniques to implement fast hash tables

#15

The author of this post wrote klib/khash, and uses it in his very popular cpu-intensive bioinformatics programs such as bwa (short read alignment against the human ref genome). I've been learning rust recently. As a learning exercise, I compared the robin-hood hashing of std::collections::HashMap in rust to his klib/khash he mentions in this article , and then tried various hash functions to try and match his perform…

How does that compare to the dict implementation of Python 3.6 ? It's supposed to be damned fast but I don't have the skills to make such comparisons.

The split keys and values arrays in Python 3.6 dictionaries make it more compact than dictionaries from earlier versions of Python, which makes it more cache friendly. But Python dictionaries (and sets) use a variant of double hashing for collision resolution, which is less cache friendly than Robin Hood hashing's linear probing. It's certainly possible to combine both techniques to come up with a hybrid dictionary with the best aspects of both split keys and Robin Hood hashing.

Re: Advanced techniques to implement fast hash tables

#16
post #6

I love this stuff but how many of us really get to (have the privilege) to doing this kind of work?

Your choice: either you want to earn money to buy toys or feed a family, or you can have poetic fun with high level maths and algorithmics.

Re: Advanced techniques to implement fast hash tables

#20

The author of this post wrote klib/khash, and uses it in his very popular cpu-intensive bioinformatics programs such as bwa (short read alignment against the human ref genome). I've been learning rust recently. As a learning exercise, I compared the robin-hood hashing of std::collections::HashMap in rust to his klib/khash he mentions in this article , and then tried various hash functions to try and match his perform…

How does that compare to the dict implementation of Python 3.6 ? It's supposed to be damned fast but I don't have the skills to make such comparisons.

Python, ruby, php hash tables are roughly 2x slower. perl5's being the worst maybe 4x slower.

Damn fast is only relative. It's damn fast compared to the previous implementations. Wikipedia even dares to say that the worst hash tables of all, perl5, is one of the very best. Bias all over.

SIMD optimized hash tables, such as the Swiss tables tricks have the potential to be much faster than khash. And khash is not cache oblivious at all, with its double hashing scheme.

Post reply on HN