161–170 of 184 posts
Earlier quoted context omitted.
> I wouldn't let my user-facing app write to Clickhouse I’ve been thinking of doing exactly that. What are your concerns?
https://blog.cloudflare.com/http-analytics-for-6m-requests-p... has some good thoughts. The main thing you'll likely need is some sort of a buffer layer so you can do bulk inserts. Do not write a high-volume of single-row inserts into Clickhouse.
Earlier quoted context omitted.
(Remember that clickhouse is not reliable. It doesn’t pretend to be. Clickhouse is great for lots of common query workloads, but if losing your data would be a big deal then it makes a lot of sense to have your data in a reliable and backed up place (eg timescale or just s3 files or whatever) too. Of course lots of times people chuck stuff into clickhouse and it’s fine if they lose a bit sometimes. YMMV.)
I have not found this to be the case. Like any system you need to take precautions (replicas and sharding) to ensure no data loss, but I didn't find that to be challenging. In what way have you found ClickHouse particularly risky in this way?
Clickhouse (and other systems with the same basic architecture, like elastic search and, shudder, mongodb) work very well on happy path. They are not advertising themselves as ACID.
1) Clickhouse allows to trivially setup a batch buffer on server side. It will always be faster on insert with low disk usage.
2) Suspicious axis on query benchmark. It seems it represents diff in query latencies. So if one executes query in 1.00 and another in 1.34 on graph it will be whopping 34.
Earlier quoted context omitted.
With TimescaleDB compression, 1000 rows of uncompressed data are compressed into column segments, moved to external TOAST pages, and then pointers to these column segments are stored in the table's "row" (along with other statistics, including some common aggregates). So while the query processor might still be "row-by-row", each "row" it processes actually corresponds to a column segment for which parallelization/ve…
yeah very interesting, i was wondering how timescale pushed postgres more towards columnar without rewriting a bunch of postgres itself. My understanding of TOAST is that it itself is just a bunch of rows in a toast table that split the compressed "row" or in this case "1000 rows of 1 column" across as many rows as required to store the data whilst remaining within the postgres page size limits (normally 8kb). With t…
More info: https://blog.timescale.com/blog/building-columnar-compressio...
Earlier quoted context omitted.
yeah very interesting, i was wondering how timescale pushed postgres more towards columnar without rewriting a bunch of postgres itself. My understanding of TOAST is that it itself is just a bunch of rows in a toast table that split the compressed "row" or in this case "1000 rows of 1 column" across as many rows as required to store the data whilst remaining within the postgres page size limits (normally 8kb). With t…
The compressed column segment is stored in a single row in TOAST. More info: https://blog.timescale.com/blog/building-columnar-compressio...
Earlier quoted context omitted.
yeah very interesting, i was wondering how timescale pushed postgres more towards columnar without rewriting a bunch of postgres itself. My understanding of TOAST is that it itself is just a bunch of rows in a toast table that split the compressed "row" or in this case "1000 rows of 1 column" across as many rows as required to store the data whilst remaining within the postgres page size limits (normally 8kb). With t…
The compressed column segment is stored in a single row in TOAST. More info: https://blog.timescale.com/blog/building-columnar-compressio...
Earlier quoted context omitted.
I have not found this to be the case. Like any system you need to take precautions (replicas and sharding) to ensure no data loss, but I didn't find that to be challenging. In what way have you found ClickHouse particularly risky in this way?
It’s basic computer science. Clickhouse doesn’t fsync etc. Clickhouse (and other systems with the same basic architecture, like elastic search and, shudder, mongodb) work very well on happy path. They are not advertising themselves as ACID.
Can someone give me a real-world example of a scenario where they actually need a time series database, like an example query with the business use case / justification? Just super curious.
We're using TimescaleDB to store log events - so each row has a timestamp, some properties, and a log message. And a lot of that is actually in a JSONB column. Not the archetypal time series use case, but TimescaleDB is still really useful. TSDB's compression means we can store a huge volume of data in a fraction of the space of a standard Postgres table. You can achieve even better compression ratios and performance…
Earlier quoted context omitted.
I have not found this to be the case. Like any system you need to take precautions (replicas and sharding) to ensure no data loss, but I didn't find that to be challenging. In what way have you found ClickHouse particularly risky in this way?
It’s basic computer science. Clickhouse doesn’t fsync etc. Clickhouse (and other systems with the same basic architecture, like elastic search and, shudder, mongodb) work very well on happy path. They are not advertising themselves as ACID.