Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

blog.timescale.com

Post reply on HN
Print view View original

31–40 of 184 posts

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#31

Post by mfreed »

yumraj wrote: post #21
carlotasoto wrote: post #6

Earlier quoted context omitted.

(Timescale Team member here) We've been working really hard on our launches / releases this month! We called it "Always Be Launching" - we've been aiming for releasing multiple things per week during October :)

That sounds great. However, as a DB where users may store critical data, should you really be "Always be launching"? That sounds a little like FB's "move fast and break things". There's a reason why some of the mission critical open source technologies move slowly.

We actually are only having one database software release this month (TimescaleDB v2.5), which is aligned with our normal database release cadence.

Timescale (the company) also provides a managed cloud offering, as well as Promscale (an observability product built on top of TimescaleDB).

So #AlwaysBeLaunching is a company-wide effort across different product & engineering teams, as well as folks in Developer Advocacy and others (e.g., who worked on this comparison benchmarks).

What might be also interesting is our introduction of Experimental Schema features in TimescaleDB - explicitly so that we can "Move fast, but don't break things" (which is also key to getting good community feedback):

https://blog.timescale.com/blog/move-fast-but-dont-break-thi...

(Timescale co-founder)

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#32

Post by ryanbooz »

rkwasny wrote: post #27
ryanbooz wrote: post #20

Earlier quoted context omitted.

(Post author) I'm not sure why you think that's creative engineering. What you're pointing to is the depth of available configuration that the contributors to TSBS have exposed for each database. It's totally open source and anyone is welcome to add more configuration and options! I believe (although not totally sure) that Altinity and ClickHouse folks added their code a few years ago - at least it wasn't anyone on t…

Ah so the tests you have used are not the ones in https://github.com/timescale/tsbs ?

All the same tests. You simply pointed to a shell script that's configurable to run tests for each database. We provided details in the blog post of exactly what settings we used for each database (cardinality, batch size, time range, TimescaleDB chunk size, etc.) so you can use those script to configure and run the tests too.

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#33

Post by encoderer »

claytonjy wrote: post #15
andrejserafim wrote: post #4

Our anecdata: we store telemetry per thing. After loading a month worth of data - timescaldb as hosted by their cloud ran a difference aggregation in seconds. Clickhouse routinely did it in 20 millis. Simple avg, etc were better, but always clickhouse was an order of magnitude faster than timescale. We didn't invest a whole bunch into optimization other than trying some indexing strategies in timescaledb. So for our…

Was this for your primary source-of-truth, or more of a downstream data warehouse, or something else? I'm struggling to imagine a case where these are the two things being considered; Timescale is the obvious choice for a primary database, Clickhouse the obvious choice for a warehouse. I wouldn't let my user-facing app write to Clickhouse, and while I could potentially get away with a read-only Timescale replica for…

> I wouldn't let my user-facing app write to Clickhouse

I’ve been thinking of doing exactly that. What are your concerns?

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#36

Post by avthar »

yumraj wrote: post #21
carlotasoto wrote: post #6

Earlier quoted context omitted.

(Timescale Team member here) We've been working really hard on our launches / releases this month! We called it "Always Be Launching" - we've been aiming for releasing multiple things per week during October :)

That sounds great. However, as a DB where users may store critical data, should you really be "Always be launching"? That sounds a little like FB's "move fast and break things". There's a reason why some of the mission critical open source technologies move slowly.

Timescale team member here. We take our responsibility to build a rock-solid platform very seriously. We have multiple "levels" of product within Timescale. At our core, we have the open-source database, TimescaleDB. This product releases on a more deliberate and careful cadence, always making sure that we are optimizing for reliability, security, and performance. This has been our approach since our initial launch [0], where we embraced the mantra "boring is awesome", recognizing that for our users stability and reliability is of paramount importance.

Within the core database, we offer features that are carefully marked as "experimental", which we discuss at length in this blog post [1].

Beyond TimescaleDB, we also offer other products that are more SaaS-y in nature. While they're all based on the rock-solid foundation of TimescaleDB, we are also able to ship new features more quickly because they are UI components that make using the database even easier.

Finally, some of our "launches" are more textual in nature, such as this benchmark, which we have spent months researching and compiling.

[0]: https://blog.timescale.com/blog/when-boring-is-awesome-build...

[1]: https://blog.timescale.com/blog/move-fast-but-dont-break-thi...

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#38

Post by ryanbooz »

rkwasny wrote: post #26
rkwasny wrote: post #17

There is some creative engineering going here :) have a look: https://github.com/timescale/tsbs/blob/master/scripts/load/l... vs https://github.com/timescale/tsbs/blob/master/scripts/load/l...

Also this queries are different? order by "time" vs order by "created_at" https://github.com/timescale/tsbs/blob/a045665d9c94426bbc405... https://github.com/timescale/tsbs/blob/a045665d9c94426bbc405...

We were using tags, so that "else" block isn't the one being used for ClickHouse. Regardless, the table that is created (by the community and verified by former CH engineers) orders by created_at, not time and so that query should be the "fastest" the distinct possible.

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#39

Post by ryanbooz »

sam0x17 wrote: post #18

Can someone give me a real-world example of a scenario where they actually need a time series database, like an example query with the business use case / justification? Just super curious.

(Post author)

This is a great post to give you some talking points:

https://blog.timescale.com/blog/what-the-heck-is-time-series...

I also love this recent one we did with some non-standard time-series data that the NFL provided! Really fun working on that data set.

https://blog.timescale.com/blog/hacking-nfl-data-with-postgr...

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#40

Post by ryanbooz »

GordonS wrote: post #29
andrejserafim wrote: post #4

Our anecdata: we store telemetry per thing. After loading a month worth of data - timescaldb as hosted by their cloud ran a difference aggregation in seconds. Clickhouse routinely did it in 20 millis. Simple avg, etc were better, but always clickhouse was an order of magnitude faster than timescale. We didn't invest a whole bunch into optimization other than trying some indexing strategies in timescaledb. So for our…

How many data points were those aggregations being computed over? How much memory does your Postgres server have, and are you using SSD storage (with associated postgres config tweaks)?

(Post author)

Howdy! We provided all of those details in the post and you're welcome to join us next week when we live-stream our setup and test!

https://blog.timescale.com/blog/what-is-clickhouse-how-does-...

Post reply on HN