Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Observability with ClickHouse

cmtops.dev

Post reply on HN
Print view View original

1–10 of 26 posts

Re: Building Observability with ClickHouse

#2

Post by zokier »

I see lot of hype around ClickHouse these days. Few years ago I remember TimescaleDB making the rounds, arguably being predecessor for this sort of "observability on SQL" thinking. The article has short paragraph mentioning Timescale, but unfortunately it doesn't really go into comparing it to ClickHouse. How does HN see the situation these days, is ClickHouse simply overtaking Timescale on all axis? That sounds bit of a shame; I have used Timescale a bit and enjoyed it, but just on such small scale that it's operational aspects did not really come up.

Re: Building Observability with ClickHouse

#3

Post by k_bx »

Another project I want to give shout out to is Databend. It's built around the idea of storing your data at S3-compatible storage as Parquet files, and querying as SQL or other protocol.

Like many popular Data Lake solutions, but it's open-source and written in Rust, which means quite easy to extend for many who know it already.

Re: Building Observability with ClickHouse

#5

Post by valyala »

zokier wrote: post #2

I see lot of hype around ClickHouse these days. Few years ago I remember TimescaleDB making the rounds, arguably being predecessor for this sort of "observability on SQL" thinking. The article has short paragraph mentioning Timescale, but unfortunately it doesn't really go into comparing it to ClickHouse. How does HN see the situation these days, is ClickHouse simply overtaking Timescale on all axis? That sounds bit…

ClockHouse outperforms TimescaleDB in every aspect on large volumes of data. https://benchmark.clickhouse.com/

If you have small volumes of data (let's say less than a terabyte of data), then TimescaleDB is OK to use if you are OK with not so fast query performance.

Re: Building Observability with ClickHouse

#6

Post by ekabod »

zokier wrote: post #2

I see lot of hype around ClickHouse these days. Few years ago I remember TimescaleDB making the rounds, arguably being predecessor for this sort of "observability on SQL" thinking. The article has short paragraph mentioning Timescale, but unfortunately it doesn't really go into comparing it to ClickHouse. How does HN see the situation these days, is ClickHouse simply overtaking Timescale on all axis? That sounds bit…

Clickhouse has been popular for many years. Even before Timescale.

Re: Building Observability with ClickHouse

#7

Post by valyala »

dakiol wrote: post #4

Such a PITA. Unless you have a dedicated team to handle observability, you are in for pain, no matter the tech stack you use.

That's not truth. There are solutions for logging, which are very easy to setup and operate. For example, VictoriaLogs [1] (I'm its' author). It is designed from the grounds up to be easy to configure and use. It contains a single self-contained executable without external dependencies, which runs optimally on any hardware starting from Raspberry Pi and ending with a monster machine containing hundreds of CPU cores and terabytes of RAM. It accepts logs over all the popular data ingestion protocols [2]. It provides very easy to use query language for typical querying tasks over logs - LogsQL [3].

[1] https://docs.victoriametrics.com/victorialogs/

[2] https://docs.victoriametrics.com/victorialogs/data-ingestion...

[3] https://docs.victoriametrics.com/victorialogs/logsql/

Re: Building Observability with ClickHouse

#8

Post by valyala »

k_bx wrote: post #3

Another project I want to give shout out to is Databend. It's built around the idea of storing your data at S3-compatible storage as Parquet files, and querying as SQL or other protocol. Like many popular Data Lake solutions, but it's open-source and written in Rust, which means quite easy to extend for many who know it already.

Databend performance looks good! https://benchmark.clickhouse.com/#eyJzeXN0ZW0iOnsiQWxsb3lEQi...

It looks like it has slightly worser on-disk data compression than ClickHouse, and slightly worser performance for some query types when the queried data isn't cached by the operating system page cache, according to the link above (e.g. when you query terabytes of data, which doesn't fit RAM).

Are there additional features other than S3 storage, which can convince ClickHouse user switching to Databend?

Re: Building Observability with ClickHouse

#10

Post by shin_lao »

zokier wrote: post #2

I see lot of hype around ClickHouse these days. Few years ago I remember TimescaleDB making the rounds, arguably being predecessor for this sort of "observability on SQL" thinking. The article has short paragraph mentioning Timescale, but unfortunately it doesn't really go into comparing it to ClickHouse. How does HN see the situation these days, is ClickHouse simply overtaking Timescale on all axis? That sounds bit…

Timescale "doesn't scale" - in a nutshell.

Clikchouse performance is better because it's truly column oriented and it has powerful partitioning tools.

However, Clickhouse has quirks and isn't great if you need low latency data updates or if your data is mutable.

Post reply on HN