21–30 of 56 posts
What about TimescaleDB vs a lot of other OLAP databases like ClickHouse, Druid, Pinot, Databend, etc.? Or hybrid databases like StarRocks or TiDB?
https://www.timescale.com/blog/what-is-clickhouse-how-does-i...
Whenever I see these posts from TimescaleDB, I always want to ask them how it compares in performance to alternative extensions that implement the same features, rather than just comparing TimescaleDB to vanilla PostgreSQL. For example, they mention their automated data retention and how it's achieved with one SQL command, and how DELETEing records is a very costly operation, and how "even if you were using Postgres…
Though I didn't write this post, I'd imagine at least part of it is that it's already nearly 4000 words and a 15 minute read and we just didn't want to add another set of things to it, to be perfectly honest.
`pg_partman` is cool! I haven't used it in a while, and because it uses declarative partitioning, it has some locking issues that we address with our partitioning scheme, but implying that it is OSS and we're not in terms of things like data retention features is a bit misleading as well. The `drop_chunks` command used for data retention is in the Apache 2 licensed portion of Timescale.
What about TimescaleDB vs a lot of other OLAP databases like ClickHouse, Druid, Pinot, Databend, etc.? Or hybrid databases like StarRocks or TiDB?
https://benchmark.clickhouse.com/
Timescale apparently lags pretty far behind modern columnstore engines.
Looks like these are quite different in features https://docs.timescale.com/timescaledb/latest/timescaledb-ed...
Is Timescale suited for keeping the history of the data and then query against any point in time?
Whenever I see these posts from TimescaleDB, I always want to ask them how it compares in performance to alternative extensions that implement the same features, rather than just comparing TimescaleDB to vanilla PostgreSQL. For example, they mention their automated data retention and how it's achieved with one SQL command, and how DELETEing records is a very costly operation, and how "even if you were using Postgres…
Disclosure, I work at Timescale. Though I didn't write this post, I'd imagine at least part of it is that it's already nearly 4000 words and a 15 minute read and we just didn't want to add another set of things to it, to be perfectly honest. `pg_partman` is cool! I haven't used it in a while, and because it uses declarative partitioning, it has some locking issues that we address with our partitioning scheme, but imp…
> For example, one recent query planner improvement excludes data more efficiently for relative now()-based queries (e.g., WHERE time >= now()-’1 week’::interval). This could be done by just calculating the start date in code too. > When hypertables are compressed the amount of data that queries need to read is reduced, leading to dramatic increases in performance of 1000x or more. At my workplace we recently experim…
We are kindred spirits I think! I did this too [0] a while back at a previous company and it actually served as part of the inspiration for our compression work! It's fun, but a bit difficult to query at times. Our compressed columns do also get TOASTed and stored out of line.
I'm not sure that it's going to be much more efficient than the Timescale format once it's compressed, we have some pretty good compression algos, but I might be missing something about your case, we generally can achieve close to 10x compression, but right now you can't write directly compressed data, so you would save on the write side I suppose.
It is true that you need to put the uncompressed version into memory at some point, but we do try to limit that and in many cases you end up IO limited moreso than memory limited. We're also thinking about doing some work to push processing down towards the compressed data, but that's still in the "glint in our eye" stage, but I think it has a lot of promise.
(As a side note, TOAST is still the best acronym around ;) ).
Earlier quoted context omitted.
Disclosure, I work at Timescale. Though I didn't write this post, I'd imagine at least part of it is that it's already nearly 4000 words and a 15 minute read and we just didn't want to add another set of things to it, to be perfectly honest. `pg_partman` is cool! I haven't used it in a while, and because it uses declarative partitioning, it has some locking issues that we address with our partitioning scheme, but imp…
But almost all of your posts and benchmarks are based on the closed source version of Timescale. Everywhere I have seen it is always recommend to use the closed source version to get decent performance out of it.
Just to clarify: Nothing on Timescale is closed-source. It is all source available, all on Github. Some of it is Apache2 licensed, some of it is Timescale Licensed. And it is all free.
Whenever I see these posts from TimescaleDB, I always want to ask them how it compares in performance to alternative extensions that implement the same features, rather than just comparing TimescaleDB to vanilla PostgreSQL. For example, they mention their automated data retention and how it's achieved with one SQL command, and how DELETEing records is a very costly operation, and how "even if you were using Postgres…
(Timescale co-founder) That's a fair question. We find that most developers storing time-series data on Postgres are doing so without pg_partman. So we first wanted to provide a benchmark that would be useful to most developers. This benchmark was also the result of months of dedicated work. So the team did spend a lot of time on this. Unfortunately, they ran out of time to cover pg_partman. But that comparison is on…
So the relevant question isn't what a typical PostgreSQL user is doing, but whether someone wanting to optimize their storage should look at a PostgreSQL extension, or an upstart database like TimescaleDB.