ClickHouse as an alternative to Elasticsearch for log storage and analysis

Post by hodgesrm » Tue, Mar 02, 2021, 6:35 PM UTC

jetter wrote: post #35

tgtweak wrote: post #32

I think it's an unfair comparison, notably because: 1) Clickhouse is rigid-schema + append-only - you can't simply dump semi-structured data (csv/json/documents) into it and worry about schema (index definition) + querying later. The only clickhouse integration I've seen up close had a lot of "json" blobs in it as a workaround, which cannot be queried with the same ease as in ES. 2) Clickhouse scalability is not as s…

I address your concern from #1 in "2. Flexible schema - but strict when you need it" section - take a look at https://www.youtube.com/watch?v=pZkKsfr8n3M&feature=emb_titl... Regarding #2: Clickhouse scalability is not simple, but I think Elasticsearch scalability is not that simple, too, they just have it out of the box, while in Clickhouse you have to use Zookeeper for it. I agree that for 200 nodes ES may be a bett…

Post by AurimasJLT » Tue, Mar 02, 2021, 6:40 PM UTC

dominotw wrote: post #15

How does clickhouse compare to druid, pinot, rockset (commercial), memsql (commercial). I know clickhouse is easier to deploy. But from user's perspective is clickhouse superior to the others?

Post by advaita » Tue, Mar 02, 2021, 6:42 PM UTC

caust1c wrote: post #53

dominotw wrote: post #15

How does clickhouse compare to druid, pinot, rockset (commercial), memsql (commercial). I know clickhouse is easier to deploy. But from user's perspective is clickhouse superior to the others?

When Cloudflare was considering clickhouse, we did estimates on just the hardware cost and it was well over 10x what clickhouse was based on druids given numbers on events processed per compute unit.

Post by hodgesrm » Tue, Mar 02, 2021, 6:45 PM UTC

jetter wrote: post #64

hodgesrm wrote: post #56

Earlier quoted context omitted.

I'm the author of at least one of the ClickHouse video presentations referenced in the article as well as here on HN. ElasticSearch is a great product, but three of your points undersell ClickHouse capabilities considerably. 1.) ClickHouse JSON blobs are queryable and can be turned into columns as needed. The Uber engineering team posted a great write-up on their new log management platform, which uses these capabili…

Thank you for what you guys do. Altinity blog and videos are an outstanding source of practical in-depth knowledge on the subject, so much needed for Clickhouse recognition.

Post by dominotw » Tue, Mar 02, 2021, 6:55 PM UTC

jfim wrote: post #57

phillc73 wrote: post #47

> SQL is a perfect language for analytics. Slightly off topic, but I strongly agree with this statement and wonder why the languages used for a lot of data science work (R, Python) don't have such a strong focus on SQL. It might just be my brain, but SQL makes so much logical sense as a query language and, with small variances, is used to directly query so many databases. In R, why learn the data.tables (OK, speed) o…

SQL doesn't compose all that well. For example, imagine that you have a complex query that handles a report. If someone says "hey we need the same report but with another filter on X," your options are to copy paste the SQL query with the change, create a view that can optionally have the filter (assuming the field that you'd want to filter on actually is still visible at the view level), or parse the SQL query into…