Live data from Hacker News

Word-Aligned Bloom Filters

lemire.me

81–82 of 82 posts

Re: Word-Aligned Bloom Filters

#81
post #30

Earlier quoted context omitted.

yep, great fan of his work here. thanks for sharing that podcast. that kind of optimization requires you to know your machine architecture quite well. SIMD optimizations aren't new. but it's always amazing to see these performance increases on a single machine! our CPUs and GPUs are quite amazing. we have decided, as a field, that we can get enough virtual CPUs, GPUs, or RAM on-demand. and that we shouldn't concern o…

I was recently disappointed and frustrated to learn that GCC will be enabling vectorization at -O2 soon. The realization that you have to specify both -O3 AND an architecture to get it to use AVX basically invalidated a bunch of benchmarking and testing I had done. What's the point of building with x86-64-v3 if all your code is built at -O2 without vectorization enabled? Doh!

It's a tricky situation. CPUs reduce their clock speed when you use vector instructions so you can't enable everything by default. You have to use LTO+PGO or the compiler won't know if your code needs to be vectorised at the expense of slowing down everything else. That's one reason why O3 is sometimes slower than O2 or Os

Re: Word-Aligned Bloom Filters

#82

I'm really suspicious about whether this would really work out in most workloads. On modern architectures, it's the random memory reference that kills performance. Predictive pre-fetching in the pipeline has a impossible time figuring out what to get ready in the caches with random lookups; it's not like a linear sweep. This style of bloom-filter needs to be quite large, and it pulls only one cache-line in per key lo…

If you're still a part of this site can i ask a few questions about my RSU's and how to handle them ?
Post reply on HN