Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data • Hacker News Board

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#181

Post by swyx » Mon, Oct 25, 2021, 6:02 AM UTC

mfreed wrote: post #31

yumraj wrote: post #21

Earlier quoted context omitted.

That sounds great. However, as a DB where users may store critical data, should you really be "Always be launching"? That sounds a little like FB's "move fast and break things". There's a reason why some of the mission critical open source technologies move slowly.

We actually are only having one database software release this month (TimescaleDB v2.5), which is aligned with our normal database release cadence. Timescale (the company) also provides a managed cloud offering, as well as Promscale (an observability product built on top of TimescaleDB). So #AlwaysBeLaunching is a company-wide effort across different product & engineering teams, as well as folks in Developer Advocacy…

how long does #AlwaysBeLaunching last? seems like a good initiative but also might cause some burnout

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#182

Post by ryanbooz » Mon, Oct 25, 2021, 11:47 AM UTC

cachemiss wrote: post #176

Having used both TSDB and ClickHouse in anger I have some thoughts on this: They are both fantastic engines, I really like that both have made very specific tradeoffs and can be very clear in what they are good and bad at. Having worked on database engines, I can appreciate the complexity that they are solving. My most recent use is with ClickHouse, which is great and I think a complete game-changer for the company.…

Thanks for your excellent contribution to this discussion. As the post author I wholly agree with your approach: if a solution hits the sweet spot for you in the context of your requirements that's the one you choose. Thank you for considering TimescaleDB alongside ClickHouse in what was obviously a well thought through assessment of these two excellent technologies.

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#183

Post by ryanbooz » Mon, Oct 25, 2021, 3:05 PM UTC

ryanbooz wrote: post #182

cachemiss wrote: post #176

Having used both TSDB and ClickHouse in anger I have some thoughts on this: They are both fantastic engines, I really like that both have made very specific tradeoffs and can be very clear in what they are good and bad at. Having worked on database engines, I can appreciate the complexity that they are solving. My most recent use is with ClickHouse, which is great and I think a complete game-changer for the company.…

Thanks for your excellent contribution to this discussion. As the post author I wholly agree with your approach: if a solution hits the sweet spot for you in the context of your requirements that's the one you choose. Thank you for considering TimescaleDB alongside ClickHouse in what was obviously a well thought through assessment of these two excellent technologies.

Heh - somehow missed that I had already responded to this one, my apologies. (and no immediate way to edit after the fact).

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#184

Post by sarlalian » Thu, Nov 04, 2021, 7:58 AM UTC

sam0x17 wrote: post #18

Can someone give me a real-world example of a scenario where they actually need a time series database, like an example query with the business use case / justification? Just super curious.

I use TimescaleDB to store data from logs (syslog, http logs, render logs), timeseries host metrics (CPU, Disk, Memory, Network), system inventory data, FDW to LDAP, network flows, etc. I do analysis across multiple dimensions for a variety of business uses.

1. Render job logs, combined with host metrics to determine if any systems are being under utilized. 2. Intranet network usage for monitoring purposes. 3. General monitoring of log events, system metrics, etc with alerting based on specific query criteria. 4. Temperature / Humidity monitoring and trend analysis.

Basically if there are things that happen somewhat often (http logs, system logs, host metrics, power usage, temperature change, etc) and you want to analyze that data, and one of the factors in your analysis is time, then time series databases are for you.

In the end each TSDB has different trade offs, and there are a number of considerations to make, and how you weigh them will change a lot depending on scale. If you are writing 1-20GB/Day vs 1-20TB/Day, you will have different considerations. If you have relatively simple queries vs complex queries with complex joins on other data you will have different considerations. In the end you need to have some idea what your data is, how much of it there is, what the growth looks like and what your write/query usage patterns are going to look like. Insert / Query performance isn't everything, unless for your business it is.