Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Time series database Graphite seems to be falling into disfavor

vividcortex.com

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Time series database Graphite seems to be falling into disfavor

#61

Post by js2 »

jrv wrote: post #57

If you prefer running an open-source solution yourself, Prometheus ( http://prometheus.io/ ) addresses exactly those points and works especially well in a dynamic cloud / microservices / container scheduler world. It has a dimensional data model, a powerful query language to go with it, and covers aspects from instrumentation to storing data, all the way to alerting and dashboarding. The latest version of Grafana has…

I chose InfluxDB about a year ago because it was the only option that allowed me to put each individual event into a time series and then do the grouping/counting after the fact. The prometheus docs used to say this it wasn't suited to that use case. Is that still the case?

edit: yup, the prometheus docs indicate InfluxDB is better suited to this use case.

http://prometheus.io/docs/introduction/comparison/

Aside, I really appreciate this comparison page and the prometheus docs in general are well done.

Re: Time series database Graphite seems to be falling into disfavor

#62

Post by e12e »

chaotic-good wrote: post #45
gtrubetskoy wrote: post #4

I've been in search of a good time-series database solution for some time now, and have pretty much given up on it and am in the process of rolling my own: https://github.com/grisha/timeriver My issues with the present state of TS isn't the volume. I was looking for using TS outside of the DevOps world. Everyday things like your heart rate over time, price of gas at the nearest station, number of people in line at yo…

Can't agree more. Time-series data analysis should be considered the most important, not storage. Storage is a solved problem. Let's imagine that we're getting ECG data from many patients. What most time-series databases is able to do with this data? They can draw a nice graph. OK. They can resample this data. Well.. this is mostly meaningless for ECG. Can graphite/opentsdb/whatsoever find specific patterns indicatin…

> Can graphite/opentsdb/whatsoever find specific patterns indicating a disease (for example T-wave inversion)? No, but this is OK, because they're not designed for this. We should build new tools to solve new problems, not the old ones (like monitoring in DevOps).

You know, that sounds like exactly the kind of thing you'd want to do in a DevOps world (leverage your data and new ideas to predict problems).

So, you're not just right that we need to look at timeseries because there are many interesting new problems that we might solve with them, but also because there are still many problems we want to solve in DevOps.

Re: Time series database Graphite seems to be falling into disfavor

#63

Post by KyleBrandt »

With http://bosun.org (Stack Overflows alerting system: Expressions, Notification Templates, Historical testing etc) we have hedged our bets by adding in different query functions to the expression language. Currently it can query:

- OpenTSDB

- InfluxDB

- Graphite

- Elastic (Expects to be populated by logstash)

OpenTSDB was the original time series database so that has some extra UI features compared to others for graphing. My hope is that InfluxDB will mature to the point in availability that we can use it and ditch the hbase dependency. But Currently OpenTSDB is the best option for us.

Re: Time series database Graphite seems to be falling into disfavor

#64

Post by jrv »

js2 wrote: post #61
jrv wrote: post #57

If you prefer running an open-source solution yourself, Prometheus ( http://prometheus.io/ ) addresses exactly those points and works especially well in a dynamic cloud / microservices / container scheduler world. It has a dimensional data model, a powerful query language to go with it, and covers aspects from instrumentation to storing data, all the way to alerting and dashboarding. The latest version of Grafana has…

I chose InfluxDB about a year ago because it was the only option that allowed me to put each individual event into a time series and then do the grouping/counting after the fact. The prometheus docs used to say this it wasn't suited to that use case. Is that still the case? edit: yup, the prometheus docs indicate InfluxDB is better suited to this use case. http://prometheus.io/docs/introduction/comparison/ Aside, I r…

Yes, Prometheus is fundamentally a store for numeric time series (with a set of dimensions attached), not a store for individual events or log entries.

Re: Time series database Graphite seems to be falling into disfavor

#65

Post by cwyers »

mintplant wrote: post #37
dang wrote: post #22

Earlier quoted context omitted.

Ok, we changed the title to use representative language from the article. Btw, this article was heavily flagged. It's not really legit to flag a story just because people don't like the title. Plenty of good stories have problematic titles. Depriving others of a chance to read the content, especially when there's a good discussion going on in the thread, is a bad use of flagging power.

I didn't flag it, but perhaps others did because the author is a Graphite competitor, yet this isn't disclosed until the end.

It's not like this is a Medium post or something, it's on the company blog.

Re: Time series database Graphite seems to be falling into disfavor

#66

Post by nemo44x »

svckr wrote: post #41
js2 wrote: post #6

Earlier quoted context omitted.

I'm responsible for a solution at $dayjob where I ended up feeding the data into multiple sources. I basically have incoming "events" and they get fed into Sentry (which in turn stores them in MySQL and Redis), InfluxDB (to use with Graphana), and soon Elastic Search (to use with Kibana). Each of these just provides a different way to visualize the data.

I find it confusing that these solutions seem to conflate time series storage/querying with visualization. Is there a reason for this? I recall that Kibana is a plugin for Elastic Search but don't know about the others. Isn't it possible to connect a hypothetical standalone visualization frontend to any TSDB?

Kibana 3 and earlier was a plugin for Elasticsearch. However, Kibana 4 is a stand alone application. Also, Kibana 4 uses Elasticsearch aggregations instead of Lucene Facets.

Re: Time series database Graphite seems to be falling into disfavor

#67

Post by gh02t »

svckr wrote: post #41
js2 wrote: post #6

Earlier quoted context omitted.

I'm responsible for a solution at $dayjob where I ended up feeding the data into multiple sources. I basically have incoming "events" and they get fed into Sentry (which in turn stores them in MySQL and Redis), InfluxDB (to use with Graphana), and soon Elastic Search (to use with Kibana). Each of these just provides a different way to visualize the data.

I find it confusing that these solutions seem to conflate time series storage/querying with visualization. Is there a reason for this? I recall that Kibana is a plugin for Elastic Search but don't know about the others. Isn't it possible to connect a hypothetical standalone visualization frontend to any TSDB?

Grafana is sort of that. It's a visualization tool that can use a bunch of different TSDBs as the backend. I use it with InfluxDB to track and plot sensor data from my home IoT setup. Basically all sensors (primarily temp because it controls my thermostat) are connected to an isolated WLAN and broadcast do their readings via MQTT. A process on the server collects them and stores them in Influx, which Grafana can use to make nice plots. It's quite easy to use actually.

The TSDBs I tested when I was building this setup (Graphite, OpenTSDB and InfluxDB) all had some sort of basic plotting facilities, but they were extremely minimal and all pointed towards Grafana if you wanted something more substantial.

Re: Time series database Graphite seems to be falling into disfavor

#68

Post by iheartmemcache »

e12e wrote: post #62
chaotic-good wrote: post #45

Earlier quoted context omitted.

Can't agree more. Time-series data analysis should be considered the most important, not storage. Storage is a solved problem. Let's imagine that we're getting ECG data from many patients. What most time-series databases is able to do with this data? They can draw a nice graph. OK. They can resample this data. Well.. this is mostly meaningless for ECG. Can graphite/opentsdb/whatsoever find specific patterns indicatin…

> Can graphite/opentsdb/whatsoever find specific patterns indicating a disease (for example T-wave inversion)? No, but this is OK, because they're not designed for this. We should build new tools to solve new problems, not the old ones (like monitoring in DevOps). You know, that sounds like exactly the kind of thing you'd want to do in a DevOps world (leverage your data and new ideas to predict problems). So, you're…

I mean, that's sort of a "solved' solution. DevOps (well they were called greybeard Perl developers, but they effectively filled the role of devops 20 years ago) data munged with Perl and Unix tools to 'leverage' your data (i.e., everything from simple transforms, all the way to complicated aggregate analysis. Did you know you could topologically build a di-graph with an out of the box default UNIX tool 'tsort'? Yeah you can).

It's kind of now accessible to the masses (previously you'd have to pay SAP a few million for BusinessObjects then EMC another million and a half for enough storage to setup a DW), but open source tools have been available for at least a decade. Now it just has a fancy new marketable name "predictive analysis" that Oracle et al can charge an extra couple million for with their price-gouging RAC licenses.

Re: Time series database Graphite seems to be falling into disfavor

#69

Post by Mahn »

lumpypua wrote: post #12

What does a modern metrics stack look like? There are so many words... Graphfana, Kibana, StatsD, Graphite, etc. Which bits should I choose and attach together?

Grafana + InfluxDB are the simplest to get up and running. Literally two binaries that you download and run, that's it. Can't beat the simplicity of this set up and you can get a lot done with it, if you are just starting your metrics infrastructure I can recommend it.

Re: Time series database Graphite seems to be falling into disfavor

#70

Post by krakensden »

meowface wrote: post #23
Karunamon wrote: post #13

Earlier quoted context omitted.

I've got to advise against Kibana/Elasticsearch until the company gets more mature. I was 100% on board with them up until the redesign. Not so much for the design (oh look it's white instead of black, who cares), but the way they've handled it subsequently. The 3.x -> 4.x transition for Kibana left a product that was missing really basic features (like, the ability to set graph colors for one - a bug/feature request…

After many years of using Splunk at my job, switching to ELK for personal project use was quite a disappointment for me. Of course, ELK is free and Splunk is very expensive, but I was still surprised at the gap. This might be derailing the thread a bit, but is there any log management platform like ELK or Splunk that has an expressive and versatile query language like Splunk's? My biggest issue with ELK is that analy…

I know, right? I recently did a pilot project with a Splunk competitor called Jut, that was totally query language oriented. It was great, hugely refreshing after Kibana. They're planning on going open source, too, so maybe the language will even survive.
Post reply on HN