Live data from Hacker News

Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

blog.timescale.com

171–180 of 184 posts

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#171
post #144
post #136

I’m surprised Timescale hasn’t given a comparison with SingleStoreDB. I’ve found SingleStore column scans at parity with ClickHouse in speed. At same time SingleStore uses a hybrid skip-list, columnstore data structure in their universal storage (which is default table format). So you have high throughput transactions, as well as insanely fast aggregate scans. Usually in column stores, they are great at append, not s…

Beyond being closed source, SingleStoreDB's License explicitly prohibits benchmarking: https://www.singlestore.com/assets/contracts/singlestore-fre... 2. Restrictions. You acknowledge that the Software, and its structure, organization, and source code, constitute SingleStore’s and its suppliers’ valuable trade secrets, and the Software is subject to the following restrictions. Specifically, Customer shall not, and sh…

Wow. Not expected to see that in Modern Database Product.

What is the most funny though I remember them in MemSQL days showing benchmarks vs Oracle (which as I understand also has the same restriction)

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#172
post #164

Earlier quoted context omitted.

The compressed column segment is stored in a single row in TOAST. More info: https://blog.timescale.com/blog/building-columnar-compressio...

Does timescale do it’s own compression alg too? I see in pg 14 toast column compression can be lz4 instead of ootb pglz which has a few probs appr, I see mentions on the mailing list of significant possible optimizations. When dealing with EBS style storage where read latencies can be multi millis compression is always going to be a win, but is an easy optimization either way I’d think.

Timescale implements its own compression algorithms. It includes several ones, and automatically applies the choice of algorithm based on the data types of columns.

- Gorilla compression for floats

- Delta-of-delta + Simple-8b with run-length encoding compression for timestamps and other integer-like types

- Whole-row dictionary compression for columns with a few repeating values (+ LZ compression on top)

- LZ-based array compression for all other types

This means within even the same table, different columns will be compressed using different algorithms based on their type (or inferred entropy).

More information for those interests:

- General TimescaleDB compression post: https://blog.timescale.com/blog/building-columnar-compressio...

- Deep dive on compression algorithms it employs: https://blog.timescale.com/blog/time-series-compression-algo...

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#173
Last time I looked at TimescaleDB a few months ago, it appeared to me that you had to ingest data into what they call an access node, which made it a non-starter for our use case, but matched perfectly with ClickHouse, which is a symmetric deployment, where every node can be an ingest node.

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#175

Earlier quoted context omitted.

Few comments: - The CH table schema generated by TSBS isn't optimized for the queries. First of all, it doesn't uses CODEC ( https://altinity.com/blog/2019/7/new-encodings-to-improve-cl... ) and many other optimizations CH have. > We tried multiple batch sizes and found that in most cases there was little difference in overall insert efficiency This is wrong in CH world where batch size matters a lot. I would recomme…

This is what tend to make all vendor benchmarks "benchmarketing" - while many of us fully intend to give a fair shot to other technologies we tend to know best practices for our own software better than "competition"

It also puzzled me when I started benchmarking and helping/reviewing PRs on the tsbs project.

I even wrote an idea about promoting the "time series racing contest" a few months ago: https://gist.github.com/jonatas/a84b734645d288051fb861d9522f...

Orchestrate the race will require a lot of collaboration from all players involved :)

Another small step would be encourage DB companies to bring the most optimized defaults to tsbs or extra scripts and templates to tune the DB server.

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#176
Having used both TSDB and ClickHouse in anger I have some thoughts on this:

They are both fantastic engines, I really like that both have made very specific tradeoffs and can be very clear in what they are good and bad at. Having worked on database engines, I can appreciate the complexity that they are solving.

My most recent use is with ClickHouse, which is great and I think a complete game-changer for the company. However there's a lot of issues (that are being worked on, the core team is great, though there are a few personalities that are a bit frosty to deal with). All of these comments come with love for the system.

1. Joins really need some work, both in the kinds of algorithms (pk aware, merge joins that don't do a full sort etc.), and in query optimizer work to make them better. We have analysts that use our system, and telling them to constantly write subqueries for simple joins is a total PITA. Not having PK aware joins is a massive blocker for higher utilization at our company, which really loves CH otherwise.

2. Some personalities will tell you that not having a query optimizer is a feature, and from an operational standpoint, it is nice to know that a query plan won't change, or try and force the optimizer to do the right thing. However, given #1, making joins performant (we have one huge table with trillions of rows, and a few smaller ones with billions) is really rough.

3. The operations story really needs some work, especially the distribution model. The model of local tables with a distributed table over it is difficult to work with personally. It would be nice to just be able to plug servers in without alot of work, like Scylla, and not have two tables that you have to keep schemas consistent with. THere's also just some odd behavior, like if you insert async into a distributed table, and only have a few shards, it'll only use a thread per shard to move that data over. It would be nice if there wasn't as much to think about.

4. Following #3, there's just too many knobs, maybe if they had a tuning tool or something that would help, but configuring thread pools is difficult to get right. I suspect CH could use a dedicated scheduler like Scylla's, that could dispatch the work, instead of relying on the OS.

5. The storage system relies a lot on the underlying FS and settings on when to fsync etc. I suspect if they had a more dedicated storage engine (controlled by the scheduler above), things could be more reliable. I still don't fully trust data being safe with CH.

6. Deduplication - This is a hard problem, but one that is really difficult to solve in CH. We solve it by having our inserters coordinate so that they always produce identical blocks, using replacing merge trees to catch stragglers (maybe), but it isn't perfect. A suggestion if possible is to try and put the same keys into the same parts, so they'll always get merged out by the replacing merge tree (I understand this is difficult).

The CH team is great, and these will be fixed in time, but these were the problems we ran into with CH.

TSDB was really solid, but we never used it at a scale where it would tip over. Our use case is really aligned with Yandex's so a lot of the functionality they have built is useful to us in a way that TSDB's isn't. (Also, being able to page data to S3 is amazing).

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#177

Having used both TSDB and ClickHouse in anger I have some thoughts on this: They are both fantastic engines, I really like that both have made very specific tradeoffs and can be very clear in what they are good and bad at. Having worked on database engines, I can appreciate the complexity that they are solving. My most recent use is with ClickHouse, which is great and I think a complete game-changer for the company.…

(post author)

Thanks for the great, thoughtful feedback. We (Timescale) couldn't agree more that there is a lot to love about ClickHouse, especially where it truly excels.

Information like this is helpful for others currently in the "choose the right tool" part of the job and to the developers of the product. I can't imagine how different all of our offerings will look in a few more years! :-)

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#178

Last time I looked at TimescaleDB a few months ago, it appeared to me that you had to ingest data into what they call an access node, which made it a non-starter for our use case, but matched perfectly with ClickHouse, which is a symmetric deployment, where every node can be an ingest node.

You are correct. The current multi-node deployment (if you need it for your workload) does have one "managing" access node. All ANs and DNs can be replicated and configured with tooling such as patroni for HA.

We have a few users that have larger multi-node clusters setup this way (one at 40+ nodes so far) and happy with the offering overall. Obviously YMMV depending on requirements/IT support.

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#179

Earlier quoted context omitted.

You are right about transactional differences between ClickHouse and PostgreSQL but you are comparing apples and oranges. ClickHouse prioritizes speed, efficiency, and scale over consistency. These are reasonable choices, especially in the largely append-only use cases which dominate analytics. 1. I've seen relatively few messed up source tables and mat views over thousands of support cases. When they happen they can…

I could go on, but I think these points illustrate that ClickHouse has a different set of design choices for different problems. I would never use it for eCommerce, but it's great for analytics. I agree with this. You are poking at a straw man. My reply was in response to this comment by the OP: > No, you're not there yet: doing timeseries with timescale is way riskier than with clickhouse, which is both a bit older…

Hi Ajay! Thanks for the thoughtful response and email. I would love a direct meeting and will contact you shortly.

I don't mean to gloss over ClickHouse imperfections. There are lots of them. For my money the biggest is that it still takes way too much expertise in ClickHouse for ordinary developers to use it effectively. Part of that is SQL compatibility, part of it is lack of tools of which simple backup is certainly one. To the extent that ClickHouse is risky, the risk is finding (and retaining) staff who can use it properly. Our business at Altinity exists in large part because of this risk, so I know it's real.

The big aha! experience for me has been that the things like lack of ACID transactions or weak backup mechanisms are not necessarily the biggest issues for most ClickHouse users. I came to ClickHouse from a long background in RDBMS and transactional replication. Things that would be game ending in that environment are not in analytic systems.

What's more interesting (mind-expanding even) is that techniques like deduplication of inserted blocks and async multi-master replication turn out to be just as important as ACID & backups to achieve reliable systems. Furthermore, services like Kafka that allow you to have DC-level logs are an essential part of building analytic applications that are reliable and performant at scale. We're learning about these mechanisms in the same way that IBM and others developed ACID transaction ideas in the 1970s--by solving problems in real systems. It's really fun to be part of it.

My comment didn't convey this clearly, for which I heartily apologize. I certainly don't intend to portray ClickHouse as perfect and still less to bash Timescale. I don't know enough about the latter to make any criticism worth reading.

p.s., Non-transactional insert (specifically non-atomicity across blocks and tables) is an undisputed problem. It's being fixed in https://github.com/ClickHouse/ClickHouse/issues/22086. Altinity and others are working on backups. Backup comes up in my job just about every day.

Re: Comparing ClickHouse to PostgreSQL and TimescaleDB for time-series data

#180
post #59

More war stories: found Timescale easier to setup (maybe just because more familiar), but raw query perf is not something you just magically get for free. Timescale requires a lot of investment in planning. In one project we had simple time range scan queries against a less-than-RAM-sized table taking tens of seconds to complete. ClickHouse has a bit more ops overhead, but requires very little in the way of pre-plann…

I found clickhouse very easy to setup '__') haven't tried timescaledb tho.
Post reply on HN