Show HN: Grafana Live Streaming Datasource Implementation
1–10 of 15 posts
Re: Show HN: Grafana Live Streaming Datasource Implementation
#2I saw this presentation at grafanacon and it was awesome. I really hope the team takes this and turns it into a first class feature.
Re: Show HN: Grafana Live Streaming Datasource Implementation
#3[deleted]
Re: Show HN: Grafana Live Streaming Datasource Implementation
#4Very cool, but real time visualization use cases are surprisingly hard to concoct. Almost any signal worth capturing visually at one second or sooner intervals would be better served through automation.
For example, all SCADA systems already have command and control alerting for instant degradation response.
Re: Show HN: Grafana Live Streaming Datasource Implementation
#5Very cool, but real time visualization use cases are surprisingly hard to concoct. Almost any signal worth capturing visually at one second or sooner intervals would be better served through automation. For example, all SCADA systems already have command and control alerting for instant degradation response.
And not helped by placating BI vendors conflating "real time" (with a precise technical definition) with something akin to "zero friction" self service analytics platforms (you can start to answer a question as soon as you can conceive it.)
Cf https://www.sisense.com/blog/dont-real-time-analytics-insigh...
Re: Show HN: Grafana Live Streaming Datasource Implementation
#6Very cool, but real time visualization use cases are surprisingly hard to concoct. Almost any signal worth capturing visually at one second or sooner intervals would be better served through automation. For example, all SCADA systems already have command and control alerting for instant degradation response.
Also think about having a dashboard refresh at 5s with 50k datapoints, literally transferring gigabytes of data to every browser looking at that dashboard. If you streamed the data (even at slow rates) it would be a massive traffic savings.
Re: Show HN: Grafana Live Streaming Datasource Implementation
#7Very cool, but real time visualization use cases are surprisingly hard to concoct. Almost any signal worth capturing visually at one second or sooner intervals would be better served through automation. For example, all SCADA systems already have command and control alerting for instant degradation response.
Agreed, watching dashboards is no substitute for alerting. However, one of the motivators for doing this is to reduce pressure on the metric backend. It's common to have Grafana reports refresh at 5s intervals, which if you have sub-second data, means that you request the same (potentially tens of thousands of) points over and over again. Some have worked around this by adding a caching layer (see https://github.com/Comcast/trickster), but streaming feels more elegant :)
Re: Show HN: Grafana Live Streaming Datasource Implementation
#8Very cool, but real time visualization use cases are surprisingly hard to concoct. Almost any signal worth capturing visually at one second or sooner intervals would be better served through automation. For example, all SCADA systems already have command and control alerting for instant degradation response.
Also think about having a dashboard refresh at 5s with 50k datapoints, literally transferring gigabytes of data to every browser looking at that dashboard. If you streamed the data (even at slow rates) it would be a massive traffic savings.
This was the thing I liked the best about it. Even if you wanted to not make it "real time" making it so I didn't have to transfer all my data every refresh would be benefit enough.
Re: Show HN: Grafana Live Streaming Datasource Implementation
#9Very cool, but real time visualization use cases are surprisingly hard to concoct. Almost any signal worth capturing visually at one second or sooner intervals would be better served through automation. For example, all SCADA systems already have command and control alerting for instant degradation response.
Agreed, watching dashboards is no substitute for alerting. However, one of the motivators for doing this is to reduce pressure on the metric backend. It's common to have Grafana reports refresh at 5s intervals, which if you have sub-second data, means that you request the same (potentially tens of thousands of) points over and over again. Some have worked around this by adding a caching layer (see https://github.com/…
Exactly this. Especially in an environment where you have tens or hundredths of people/screens with the dashboard opened.
Non streaming dashboards are just stupidly hammering the backends (and the network transferring the same data over and over). So this has nothing to do with how cool it looks (also dashboard users rarely complain that they refresh or scroll too fast).
Working on such a backend at the moment, I was missing this feature so much I would probably have started to work on this on my own at some point.
Re: Show HN: Grafana Live Streaming Datasource Implementation
#10Very cool, but real time visualization use cases are surprisingly hard to concoct. Almost any signal worth capturing visually at one second or sooner intervals would be better served through automation. For example, all SCADA systems already have command and control alerting for instant degradation response.
Also think about having a dashboard refresh at 5s with 50k datapoints, literally transferring gigabytes of data to every browser looking at that dashboard. If you streamed the data (even at slow rates) it would be a massive traffic savings.
Recently I was looking into Plot.ly Dash (https://plot.ly/products/dash/) for a Python-based real-time updating dashboard.
...until I noticed that it transfers the entire chart and all the datapoints each update. (example: https://dash-gallery.plotly.host/dash-live-model-training/)