Building a distributed time-series database on PostgreSQL
blog.timescale.com
Building a distributed time-series database on PostgreSQL
1–10 of 98 posts
Re: Building a distributed time-series database on PostgreSQL
#2Are you afraid the Postgres tuple-at-a-time iterator architecture is going to be limiting for your long term performance optimizations?
Presumably the custom operators you’ve implemented in your query plan can push projection and predicates down to a more efficient architecture without so many indirect function calls / branches / etc, but once you get up past that, aren’t you back in iterator land?
Re: Building a distributed time-series database on PostgreSQL
#3What are some use cases for a time-series database?
Re: Building a distributed time-series database on PostgreSQL
#4What are some use cases for a time-series database?
The big use cases are devops/monitoring data and IoT data. But it also applies to pretty much any use case where you want to answer questions about data in relation to time.
Re: Building a distributed time-series database on PostgreSQL
#5What are some use cases for a time-series database?
They're commonly used for collecting metrics, such as performance or traffic data.
Re: Building a distributed time-series database on PostgreSQL
#6What are some use cases for a time-series database?
Pretty much anything that is naturally generating time series data, and when your queries are going to be about things-over-time in relationship to each other.
Think automotive monitoring (external or internal), algorithmic trading, retail monitoring, aviation, etc.
Re: Building a distributed time-series database on PostgreSQL
#7Are you afraid the Postgres tuple-at-a-time iterator architecture is going to be limiting for your long term performance optimizations? Presumably the custom operators you’ve implemented in your query plan can push projection and predicates down to a more efficient architecture without so many indirect function calls / branches / etc, but once you get up past that, aren’t you back in iterator land?
We actually haven't been running against any limits here. One thing to keep in mind is that postgres remote-fetch operations aren't tuple-at-a-time, so this shouldn't be a bottleneck for our multi-node operations.
Re: Building a distributed time-series database on PostgreSQL
#8Are you afraid the Postgres tuple-at-a-time iterator architecture is going to be limiting for your long term performance optimizations? Presumably the custom operators you’ve implemented in your query plan can push projection and predicates down to a more efficient architecture without so many indirect function calls / branches / etc, but once you get up past that, aren’t you back in iterator land?
We actually haven't been running against any limits here. One thing to keep in mind is that postgres remote-fetch operations aren't tuple-at-a-time, so this shouldn't be a bottleneck for our multi-node operations.
Have you done any analysis of your per-core scan rates for simple aggregations like sum/count + group by with a reasonably large cardinality key? Or has anyone published a benchmark you trust on queries of that variety?
An example would be TPC-H Q1, which is a little weak on the group by cardinality, but is good for testing raw aggregation performance.
Re: Building a distributed time-series database on PostgreSQL
#9Timescale has improved greatly since first released and is pretty solid on a single-node.
Wish they would tone down the hype in the blog posts though, a shard/chunk/partition are all the same. How you define the splits is completely arbitrary and every database uses its own algorithm, including multiple levels.
Re: Building a distributed time-series database on PostgreSQL
#10What are some use cases for a time-series database?
A good article for some background: https://dzone.com/articles/what-the-heck-is-time-series-data...