Live data from Hacker News

Hashed sorting is typically faster than hash tables

reiner.org

41–50 of 67 posts

Re: Hashed sorting is typically faster than hash tables

#41
̶F̶o̶r̶ ̶t̶h̶e̶ ̶s̶p̶a̶r̶s̶e̶-̶m̶a̶t̶r̶i̶x̶-̶m̶u̶l̶t̶i̶p̶l̶i̶c̶a̶t̶i̶o̶n̶ ̶e̶x̶a̶m̶p̶l̶e̶ ̶g̶i̶v̶e̶n̶,̶ ̶w̶o̶u̶l̶d̶n̶'̶t̶ ̶i̶t̶ ̶b̶e̶ ̶b̶e̶t̶t̶e̶r̶ ̶t̶o̶ ̶p̶r̶e̶-̶s̶o̶r̶t̶ ̶e̶a̶c̶h̶ ̶v̶e̶c̶t̶o̶r̶ ̶a̶n̶d̶ ̶d̶o̶ ̶n̶^̶2̶ ̶w̶a̶l̶k̶s̶ ̶o̶n̶ ̶p̶a̶i̶r̶s̶ ̶o̶f̶ ̶p̶r̶e̶s̶o̶r̶t̶e̶d̶ ̶v̶e̶c̶t̶o̶r̶s̶,̶ ̶r̶a̶t̶h̶e̶r̶ ̶t̶h̶a̶n̶ ̶(̶a̶s̶ ̶I̶ ̶t̶h̶i̶n̶k̶ ̶w̶a̶s̶ ̶i̶m̶p̶l̶i̶e̶d̶)̶ ̶d̶o̶i̶n̶g̶ ̶n̶^̶2̶ ̶h̶a̶s̶h̶e̶d̶ ̶s̶o̶r̶t̶i̶n̶g̶ ̶o̶p̶e̶r̶a̶t̶i̶o̶n̶s̶?̶ ̶(̶F̶o̶r̶ ̶t̶h̶a̶t̶ ̶m̶a̶t̶t̶e̶r̶,̶ ̶I̶ ̶w̶o̶n̶d̶e̶r̶ ̶w̶h̶y̶ ̶s̶p̶a̶r̶s̶e̶ ̶m̶a̶t̶r̶i̶c̶e̶s̶ ̶w̶o̶u̶l̶d̶n̶'̶t̶ ̶a̶l̶r̶e̶a̶d̶y̶ ̶b̶e̶ ̶r̶e̶p̶r̶e̶s̶e̶n̶t̶e̶d̶ ̶i̶n̶ ̶s̶o̶r̶t̶e̶d̶-̶a̶d̶j̶a̶c̶e̶n̶c̶y̶-̶l̶i̶s̶t̶ ̶f̶o̶r̶m̶ ̶i̶n̶ ̶t̶h̶e̶ ̶f̶i̶r̶s̶t̶ ̶p̶l̶a̶c̶e̶)̶ ̶

EDIT: ah no I'm being dense, you'd aggregate the union of all the set-columns indices across rows and the union of the set-row indices across the columns, keeping track of the source locations, and do the hashed sorting on those union vectors to find all the collision points. You could still get a small win I think by sorting the row-aggregation and column-aggregation separately though?

Re: Hashed sorting is typically faster than hash tables

#42
Interesting article, I particularly like the reference to real-world workloads.

Do I understand correctly, that the data being tested is fully random? If so I'd be curious to see how the results change if a Zipfian distribution is used. I don't have hard data for sorting specifically, but I suspect the majority of real-world data being sorted - that isn't low cardinality or already nearly or fully sorted - to follow a Zipfian distribution rather than true randomness. The Rust std lib sort_unstable efficiently filters out common values using the pdqsort repeat pivot flip partition algorithm.

Re: Hashed sorting is typically faster than hash tables

#43

> Hash tables win the interview O(n) vs O(n log n), If the original table has n entries, the hash must have at least log(n) bits to get most of the time a different hash. I don't know the current state of the art, but I think the recommendation was to have at most a 95% filled array, so the are not too many collision. So both methods are O(n log n), one under the hood and the other explicitly.

[deleted]

Re: Hashed sorting is typically faster than hash tables

#44
post #38

Earlier quoted context omitted.

Do-What-I-Mean isn't possible. What Rust does give you is Do-What-I-Say which mostly leaves the problem of saying what you mean, a significant task but one that is hopefully what software engineering was training you to be most effective at. One important trick is ruling out cases where what you said is nonsense. That can't be what you meant so by ruling it out we're helping you fall into the pit of success and Rust…

What is "Rice" referring to?

Rice's Theorem: https://en.wikipedia.org/wiki/Rice%27s_theorem

Or the "why we can't have nice things" theorem. Colloquially, anything interesting about a Turing-complete program is unprovable. You may have proved specific instance of this if you went through a formal computer science program and reduced problems like "this program never uses more than X cells on the tape" to the halting problem.

Re: Hashed sorting is typically faster than hash tables

#45
Very interesting and cool article, if you love low-level optimisation (like myself!)

Interestingly, recently I've been thinking that basically the Big-O notation is essentially a scam, in particular the log(N) part.

For small values of N, log(N) is essentially a constant, O(N).

For large values, even so-called linear algorithms (e.g. linear search) are actually O(N log(N)), as the storage requirements for a single element grow with log(N) (i.e. to store distinct N=2^32 elements, you need N log(N) = 2^32 * 32 bits, but to store N=2^64 elements, you need 2^64 * 64 bits).

Cache locality consideration make this effect even more pronounced.

Re: Hashed sorting is typically faster than hash tables

#46
post #45

Very interesting and cool article, if you love low-level optimisation (like myself!) Interestingly, recently I've been thinking that basically the Big-O notation is essentially a scam, in particular the log(N) part. For small values of N , log(N) is essentially a constant, O(N) . For large values, even so-called linear algorithms (e.g. linear search) are actually O(N log(N)) , as the storage requirements for a single…

> For small values of N, log(N) is essentially a constant, How can I unread this?

Re: Hashed sorting is typically faster than hash tables

#47

> Hash tables win the interview O(n) vs O(n log n), If the original table has n entries, the hash must have at least log(n) bits to get most of the time a different hash. I don't know the current state of the art, but I think the recommendation was to have at most a 95% filled array, so the are not too many collision. So both methods are O(n log n), one under the hood and the other explicitly.

N bits doesn’t mean n operations.

Re: Hashed sorting is typically faster than hash tables

#48
post #45

Very interesting and cool article, if you love low-level optimisation (like myself!) Interestingly, recently I've been thinking that basically the Big-O notation is essentially a scam, in particular the log(N) part. For small values of N , log(N) is essentially a constant, O(N) . For large values, even so-called linear algorithms (e.g. linear search) are actually O(N log(N)) , as the storage requirements for a single…

> i.e. to store distinct N=2^32 elements, you need N log(N) = 2^32 * 32 bits, but to store N=2^64 elements, you need 2^64 * 64 bits

N is the number of bits, not the number of elements, so no.

It is helpful to use N=# of elements, since the elements are often fixed/limited size. If elements aren't a fixed size, it's necessary to drop down to # of bits.

Re: Hashed sorting is typically faster than hash tables

#49
post #45

Very interesting and cool article, if you love low-level optimisation (like myself!) Interestingly, recently I've been thinking that basically the Big-O notation is essentially a scam, in particular the log(N) part. For small values of N , log(N) is essentially a constant, O(N) . For large values, even so-called linear algorithms (e.g. linear search) are actually O(N log(N)) , as the storage requirements for a single…

> For small values of N, log(N) is essentially a constant, How can I unread this?

I've given the following exercise to developers in a few workplaces:

What's the complexity of computing the nth fibonacci number? Make a graph of computation time with n=1..300 that visualizes your answer.

There are those that very quickly reply linear but admit they can't get a graph to corroborate, and there are those that very quickly say linear and even produce the graph! (though not correct fibonacci numbers...)

Re: Hashed sorting is typically faster than hash tables

#50
post #5

Interesting article. It’s actually very strange that the dataset needs to be “big” for the O(n log n) algorithm to beat the O(n). Usually you’d expect the big O analysis to be “wrong” for small datasets. I expect that in this case, like in all cases, as the datasets become gallactically large, the O(n) algorithm will start winning again.

The key here is in the cache lines.

Sorting (really scanning through an array) is very efficient in cache lines, while hash tables are very inefficient.

In the example, every memory access in hashing gets 8 byte of useful data, while every memory access in sorting gets 64 bytes of useful data.

So you have to be 8x more efficient to win out.

For this problem, the radix sort is log_1024(n) passes, which for a key size of 64 bits can’t ever get above 7 passes.

If you increase the key size then the efficiency win of sorting drops (128 bit keys means you have to run less than 4 passes to win).

Essentially the size of the key compared to the cache line size is a (hidden) part of the big O.

Post reply on HN