Haydex: From Zero to 178.6B rows a second in 30 days
1–10 of 23 posts
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#2178.6e9rows/s/30days = 66150rows/s^2
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#3I usually just call it 178 billion
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#4178 billion? That's nothing. I did trillions just this morning. I went to the grocery store and picked an item off the shelf, effectively filtering out the trillions of other products that I could have picked but didn't.
They did not process 178 billion rows per second. They did a search that found something in a large data set by eliminating the parts of the data set that could not have contained the item. Same way I did by picking one grocery store and going straight to the shelf.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#5I usually just call it 178 billion
But only if your billion is 10^9, not 10^12.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#6This kind of reads like an action or war novel
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#7That looks like a variation on a Bloom filter to me.
https://en.wikipedia.org/wiki/Bloom_filter
In the 80's or so when I thought I was being really clever I came up with another variation on this and I recall being quite annoyed when someone on HN pointed out (many years later) that this was a staple of computing science for longer than that I had been busy with computers. So much for having original thoughts...
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#8That looks like a variation on a Bloom filter to me. https://en.wikipedia.org/wiki/Bloom_filter In the 80's or so when I thought I was being really clever I came up with another variation on this and I recall being quite annoyed when someone on HN pointed out (many years later) that this was a staple of computing science for longer than that I had been busy with computers. So much for having original thoughts...
Author here, indeed a variation of bloom filters: https://x.com/lemire/status/1971279371131646063
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#9That looks like a variation on a Bloom filter to me. https://en.wikipedia.org/wiki/Bloom_filter In the 80's or so when I thought I was being really clever I came up with another variation on this and I recall being quite annoyed when someone on HN pointed out (many years later) that this was a staple of computing science for longer than that I had been busy with computers. So much for having original thoughts...
It does go to show that a huge number of inventions we consider foundational are really just from a normal person being in the right place at the right time. When a field is emerging there is a lot of low hanging fruit you can get your name stamped upon.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#10Same EventDB as https://github.com/ahri/eventdb or proprietary?