Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No, QuestDB is not Faster than ClickHouse

telegra.ph

Post reply on HN
Print view View original

41–50 of 73 posts

Re: No, QuestDB is not Faster than ClickHouse

#41

Post by Dzugaru »

bluestreak wrote: post #29
Dzugaru wrote: post #25

Earlier quoted context omitted.

"while QuestDB utilizes its full indexing strategy to read just a tiny fraction of the actual data" Can you please elaborate on this?

Full disclosure: I am CTO of QuestDB and I took part in JIT implementation. The quote above is not mine, it was written by Clickhouse staff. "utilizes its full indexing strategy" statement is false and is news to me.

So you do a full scan and it's ~50 CPU cycles per row (48 CPUs at 4 GHZ), correct? This is possible I guess? And in this case Clickhouse is wrong.

Re: No, QuestDB is not Faster than ClickHouse

#42

Post by olluk »

Dzugaru wrote: post #5
thegeomaster wrote: post #2

Sounds like they didn't re-do the QuestDB benchmark with same change to the indexes, and so their claim is that Clickhouse is 27x faster with a specific index than QuestDB without that index. Which is not a fair comparison. Also, the tone of the post sounds really arrogant. They try to hide it a bit, I feel, but it just seeps through.

Well, I don't know how QuestDB works, and I couldn't find anything in the original benchmark, but probably they already have some sort of (geo)index in place? It's really strange to search geo-data by scanning the whole surface of the Earth. The point that Clickhouse outperforms this by just sorting on one axis (and even not using any fancy 2D indices) is reasonable.

No, there are no indexes in QuestDB in the article. None. Zero. That's bold mistake in the ClickHouse article. Should be named Yes, QuestDb is Faster.

Re: No, QuestDB is not Faster than ClickHouse

#43

Post by Dzugaru »

olluk wrote: post #42
Dzugaru wrote: post #5

Earlier quoted context omitted.

Well, I don't know how QuestDB works, and I couldn't find anything in the original benchmark, but probably they already have some sort of (geo)index in place? It's really strange to search geo-data by scanning the whole surface of the Earth. The point that Clickhouse outperforms this by just sorting on one axis (and even not using any fancy 2D indices) is reasonable.

No, there are no indexes in QuestDB in the article. None. Zero. That's bold mistake in the ClickHouse article. Should be named Yes, QuestDb is Faster.

Yeah, I've read more carefully and it seems they're doing full scan.

Re: No, QuestDB is not Faster than ClickHouse

#44

Post by [deleted] »

olluk wrote: post #42
Dzugaru wrote: post #5

Earlier quoted context omitted.

Well, I don't know how QuestDB works, and I couldn't find anything in the original benchmark, but probably they already have some sort of (geo)index in place? It's really strange to search geo-data by scanning the whole surface of the Earth. The point that Clickhouse outperforms this by just sorting on one axis (and even not using any fancy 2D indices) is reasonable.

No, there are no indexes in QuestDB in the article. None. Zero. That's bold mistake in the ClickHouse article. Should be named Yes, QuestDb is Faster.

[deleted]

Re: No, QuestDB is not Faster than ClickHouse

#45

Post by [deleted] »

olluk wrote: post #42
Dzugaru wrote: post #5

Earlier quoted context omitted.

Well, I don't know how QuestDB works, and I couldn't find anything in the original benchmark, but probably they already have some sort of (geo)index in place? It's really strange to search geo-data by scanning the whole surface of the Earth. The point that Clickhouse outperforms this by just sorting on one axis (and even not using any fancy 2D indices) is reasonable.

No, there are no indexes in QuestDB in the article. None. Zero. That's bold mistake in the ClickHouse article. Should be named Yes, QuestDb is Faster.

[deleted]

Re: No, QuestDB is not Faster than ClickHouse

#46

Post by qoega »

PeterZaitsev wrote: post #35
bluestreak wrote: post #11

Our article in question can be found here: https://questdb.io/blog/2022/05/26/query-benchmark-questdb-v... The intent of the article was to showcase JIT-optimised WHERE clause and we did not use any indexes on QuestDB.

If your intent it to showcase the new optimization in the product it is best to compare it to your own old version

Agree. And for a blog post it can even have a story like: "We compared with ClickHouse and we were 10x slower, than we looked at this case and made it 100x faster. Thank you, benchmark and ClickHouse developers that showed us use case where we could do better."

For me benchmarking is usual - "Why this query takes so long? We need to improve it. Sometimes 1000x times."

Re: No, QuestDB is not Faster than ClickHouse

#47

Post by xenator »

olluk wrote: post #39
PeterZaitsev wrote: post #36

Earlier quoted context omitted.

I wonder what "every effort to be fair" means ? The first thing you could have done is reach out to ClickHouse Community to ask for optimization suggestions

What if the purpose of the article is to compare queries without indexes?

So, maybe do not create specific scenarios for corner cases and then generalize outcome? And write articles about common scenarios that is important for people who will use technology on daily basis.

Re: No, QuestDB is not Faster than ClickHouse

#48

Post by olluk »

xenator wrote: post #47
olluk wrote: post #39

Earlier quoted context omitted.

What if the purpose of the article is to compare queries without indexes?

So, maybe do not create specific scenarios for corner cases and then generalize outcome? And write articles about common scenarios that is important for people who will use technology on daily basis.

My personal view is that having fast queries without indexes is quite general outcome.

Re: No, QuestDB is not Faster than ClickHouse

#49

Post by olluk »

thegeomaster wrote: post #2

Sounds like they didn't re-do the QuestDB benchmark with same change to the indexes, and so their claim is that Clickhouse is 27x faster with a specific index than QuestDB without that index. Which is not a fair comparison. Also, the tone of the post sounds really arrogant. They try to hide it a bit, I feel, but it just seeps through.

There were 2 queries in the QuestDB benchmark over the same table. ClickHouse didn't even try to match both of them choosing one as a victim. I guess that's what happens when you optimise the data storage for one query.

Re: No, QuestDB is not Faster than ClickHouse

#50

Post by jsnell »

olluk wrote: post #39
PeterZaitsev wrote: post #36

Earlier quoted context omitted.

I wonder what "every effort to be fair" means ? The first thing you could have done is reach out to ClickHouse Community to ask for optimization suggestions

What if the purpose of the article is to compare queries without indexes?

Doesn't matter, since that clearly wasn't the purpose of the article. After all, they were totally happy to add an index for another competing DB as long as they happened to win that comparison. Then they crow about how they beat having an index.

Pretty sleazy.

Post reply on HN