Same EventDB as https://github.com/ahri/eventdb or proprietary?
Proprietary.
Haydex: From Zero to 178.6B rows a second in 30 days
11–20 of 23 posts
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#12This kind of reads like an action or war novel
As edited by ChatGPT…
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#13178 billion? That's nothing. I did trillions just this morning. I went to the grocery store and picked an item off the shelf, effectively filtering out the trillions of other products that I could have picked but didn't. They did not process 178 billion rows per second. They did a search that found something in a large data set by eliminating the parts of the data set that could not have contained the item. Same way…
Hm, if I understand their product correctly they are building a DB and their filtering actually returns correct results.
So, the analogy doesn't really hold true unless you actually have these trillions of alternate products stored in your brain and manage to cite the matching subset on demand.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#14Re: Haydex: From Zero to 178.6B rows a second in 30 days
#15That looks like a variation on a Bloom filter to me. https://en.wikipedia.org/wiki/Bloom_filter In the 80's or so when I thought I was being really clever I came up with another variation on this and I recall being quite annoyed when someone on HN pointed out (many years later) that this was a staple of computing science for longer than that I had been busy with computers. So much for having original thoughts...
Author here, indeed a variation of bloom filters: https://x.com/lemire/status/1971279371131646063
Ok. I have blocked X at the router level here since Elon went certifiable so I can't read that link but I will happily take your word for it.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#16178 billion? That's nothing. I did trillions just this morning. I went to the grocery store and picked an item off the shelf, effectively filtering out the trillions of other products that I could have picked but didn't. They did not process 178 billion rows per second. They did a search that found something in a large data set by eliminating the parts of the data set that could not have contained the item. Same way…
[deleted]
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#17Earlier quoted context omitted.
As edited by ChatGPT…
Yeah, it's very clearly LLM-edited, but it's fun to read. The LLM did a good job. It's not just a tech blog post - it's a thriller. ;)
Write an adventure where we implemented Bloofi Multidimensional Bloom Filters from this 2015 article https://arxiv.org/abs/1501.01941. At the end mention the second author, don’t mention the algorithm nor that it is based on the 1970 Bloom-filter algorithm https://en.wikipedia.org/wiki/Bloom_filter. Make me the main character that did all the hard work and caused our customer to win. The adventure should be some 5000 words long and use each of these words 20-30 times: data, axiom, filter, query, hashcolumn, haydex.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#18Earlier quoted context omitted.
Author here, indeed a variation of bloom filters: https://x.com/lemire/status/1971279371131646063
Ok. I have blocked X at the router level here since Elon went certifiable so I can't read that link but I will happily take your word for it.
It's funny how this comment chain is about how names stick to ideas in somewhat arbitrary ways, and you are using "Elon" to explain a personal policy for information grooming.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#19178 billion? That's nothing. I did trillions just this morning. I went to the grocery store and picked an item off the shelf, effectively filtering out the trillions of other products that I could have picked but didn't. They did not process 178 billion rows per second. They did a search that found something in a large data set by eliminating the parts of the data set that could not have contained the item. Same way…
Hm, if I understand their product correctly they are building a DB and their filtering actually returns correct results. So, the analogy doesn't really hold true unless you actually have these trillions of alternate products stored in your brain and manage to cite the matching subset on demand.
Seemingly, their way of thinking goes roughly like this:
If I have 10 billion rows in an SQL database, with a UNIQUE index, and do SELECT * FROM t1 WHERE pk=, then I have “processed” 10 billion rows.
If I do 10k of these queries per second, I have processed 100T rows per second.
Re: Haydex: From Zero to 178.6B rows a second in 30 days
#20Earlier quoted context omitted.
Ok. I have blocked X at the router level here since Elon went certifiable so I can't read that link but I will happily take your word for it.
It's funny how this comment chain is about how names stick to ideas in somewhat arbitrary ways, and you are using "Elon" to explain a personal policy for information grooming.
I think 'don't give your data to assholes' is a pretty good policy, regardless of whether it is personal or business.