OpenTSDB – A Distributed, Scalable Time Series Database
1–10 of 27 posts
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#2It is alot of work to keep up. I would use KairosDB instead. https://kairosdb.github.io/.
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#3It is alot of work to keep up. I would use KairosDB instead. https://kairosdb.github.io/ .
What sort of work are you referring to when you say it's a lot to keep up?
We've been using OpenTSDB here in an extremely high traffic setup for about a year now, with absolutely no issues at all. It took a few hours at most to setup and figure out scaling.
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#4We've been running OpenTSDB at a very large scale (~100k writes/s) for the past year and a half and have been pretty happy with it. It's not as reliable on the read side as something like ganglia and the query options aren't as powerful as InfluxDB, but it can scale better than any other option I know of.
The ability to look back into the past and get full-resolution metrics is a huge deal. With Ganglia or Graphite (which aggressively downsample old data) there's a lot of squinting to try to make out patterns past a week or so.
That said, we also use HBase as our main data store, so we have a lot of experience with operating it. I'm not sure I could recommend OpenTSDB to an organization that lacks that expertise.
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#5We've been running OpenTSDB at a very large scale (~100k writes/s) for the past year and a half and have been pretty happy with it. It's not as reliable on the read side as something like ganglia and the query options aren't as powerful as InfluxDB, but it can scale better than any other option I know of.
The ability to look back into the past and get full-resolution metrics is a huge deal. With Ganglia or Graphite (which aggressively downsample old data) there's a lot of squinting to try to make out patterns past a week or so.
That said, we also use HBase as our main data store, so we have a lot of experience with operating it. I'm not sure I could recommend OpenTSDB to an organization that lacks that expertise.
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#6We've been running OpenTSDB at a very large scale (~100k writes/s) for the past year and a half and have been pretty happy with it. It's not as reliable on the read side as something like ganglia and the query options aren't as powerful as InfluxDB, but it can scale better than any other option I know of. The ability to look back into the past and get full-resolution metrics is a huge deal. With Ganglia or Graphite (…
If you were to create an application that relies heavily on a time series database, what would you recommend?
I'm writing an application of this sorts, and my requirements won't be too bad (about 1000-4000 metrics on a per second basis). I will also rely heavily on predefined queries (rollups to 10s, 1m, 10m, 1h, etc.).
I'm hesitant to use OpenTSDB because the community "buy-in" seems to be going down.
EDIT: I can't lose data to downsampling. I will need to store this for both troubleshooting and regulatory purposes.
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#7We've been running OpenTSDB at a very large scale (~100k writes/s) for the past year and a half and have been pretty happy with it. It's not as reliable on the read side as something like ganglia and the query options aren't as powerful as InfluxDB, but it can scale better than any other option I know of. The ability to look back into the past and get full-resolution metrics is a huge deal. With Ganglia or Graphite (…
Can you talk a little bit about how many nodes it takes to maintain that write rate? We have our own system that's currently handling around 1% of that on a single node, with decent headroom but I've been looking for ways to distribute it. Certainly word on the street is that OpenTSDB performance is not good, at least relative to the resources you throw at it; however since it's based on HBase you can keep scaling even if it's expensive.
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#8We've been running OpenTSDB at a very large scale (~100k writes/s) for the past year and a half and have been pretty happy with it. It's not as reliable on the read side as something like ganglia and the query options aren't as powerful as InfluxDB, but it can scale better than any other option I know of. The ability to look back into the past and get full-resolution metrics is a huge deal. With Ganglia or Graphite (…
If you were to create an application that relies heavily on a time series database, what would you recommend? I'm writing an application of this sorts, and my requirements won't be too bad (about 1000-4000 metrics on a per second basis). I will also rely heavily on predefined queries (rollups to 10s, 1m, 10m, 1h, etc.). I'm hesitant to use OpenTSDB because the community "buy-in" seems to be going down. EDIT: I can't…
If it's just a few streams at high frequency, my take would be to bucket it (eg every second or something) and stick it on your favorite database. You could use postgres arrays, the json columntype, or frankly just a blob in those rows containing all the data for a second.
You can then write some simple postgres queries to create the views your talking about and won't have to mess with any of the "look what I wrote last week" "databases".
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#9We've been running OpenTSDB at a very large scale (~100k writes/s) for the past year and a half and have been pretty happy with it. It's not as reliable on the read side as something like ganglia and the query options aren't as powerful as InfluxDB, but it can scale better than any other option I know of. The ability to look back into the past and get full-resolution metrics is a huge deal. With Ganglia or Graphite (…
If you were to create an application that relies heavily on a time series database, what would you recommend? I'm writing an application of this sorts, and my requirements won't be too bad (about 1000-4000 metrics on a per second basis). I will also rely heavily on predefined queries (rollups to 10s, 1m, 10m, 1h, etc.). I'm hesitant to use OpenTSDB because the community "buy-in" seems to be going down. EDIT: I can't…
I wouldn't worry too much about what the community thinks. TSDB fits the bill and works great. That being said, I've also heard wonderful things about druid.io (http://druid.io/), however I've yet to personally try it. We're going to start testing and throwing data at druid early next year though.
Re: OpenTSDB – A Distributed, Scalable Time Series Database
#10So, we need a new database engine instead of mere using tables with partitioning in PostgreSQL with a timestamp as a primary key?
And it is Java - waste two times more resources than the data you have ™