Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Netdata: Open-source real-time monitoring platform

github.com

Post reply on HN
Print view View original

81–90 of 108 posts

Re: Netdata: Open-source real-time monitoring platform

#81

Post by ohthehugemanate »

Abishek_Muthian wrote: post #54
Havoc wrote: post #2

It’s very neat for individual servers Doesn’t work well for monitoring multiple servers though from what I can tell.

Neat project but I'm also confused with 'distributed'; It sounds like designed for monitoring multiple systems in a single dashbaord OOB with 'zero-config', But on further digging it seems like the distributed monitoring works 'only' with their cloud service[1]. Further clarity on this would be appreciated. [1] https://github.com/netdata/netdata/blob/master/docs/quicksta...

It's OOTB configured to use their free cloud service, but with 2 lines of config you can run your own central collection point instead. That's what I do for my home install.

BUT the UI for this is just a dropdown for each of your monitored servers. I've found I actually want to export data to a more robust system so I can view patterns across machines, too.

Re: Netdata: Open-source real-time monitoring platform

#82

Post by gregwebs »

mtmsr wrote: post #74

I have played around with netdata just yesterday on my home server. Great tool, but the defaults are overkill for my needs. After spending an hour trying to simplify (=disable most of the "collectors") using the documentation, I finally gave up. Settled on neofetch [1] instead: pure bash, wrote my own custom inputs including color coding for incident reporting in less time than it took me to strip down netdata. Highl…

Thanks for the link: neofetch seems a good tool when you just want to manually see what is going on. Netdata is also designed to alert, forward data to other locations, monitor at 1 second granularity, and to store historical data efficiently if you want to see what went on in the recent past.

Re: Netdata: Open-source real-time monitoring platform

#83

Post by sdesol »

Croftengea wrote: post #69
ktsaou wrote: post #62

Earlier quoted context omitted.

Thank you for this feedback. I am the founder of Netdata. Netdata is about making our lives easier. If you need to tweak Netdata, please open a github issue to let us know. It is a bug. Netdata should provide the best possible dashboards and alerts out of the box. If it does not for you, we missed something and we need your help to fix it, so please open a github issue to let us know of your use case. We want Netdata…

> our free-forever SaaS offering that complements the open-source agent How do you make or plan to make money?

I was analyzing the activity in the netdata project and what I found interesting was this project is less active than I would have thought. See the following for insights into the project:

https://public-001.gitsense.com/insights/github/repos?q=wind...

In the last 30 days, there were 2 frequent and 3 occasional contributors. I honestly thought frequent contributors would have been much higher, which leads me to believe the project is quite mature and they don't need a lot of people to work on netdata.

Based on Crunchbase, they've raised about 33 million so far and if the number of people required to maintain netbase is low (relatively speaking that is), I can see them not really needing to worry about making money and I'm guessing they are finding value in gathering data for ML.

Re: Netdata: Open-source real-time monitoring platform

#84

Post by Abishek_Muthian »

ohthehugemanate wrote: post #81
Abishek_Muthian wrote: post #54

Earlier quoted context omitted.

Neat project but I'm also confused with 'distributed'; It sounds like designed for monitoring multiple systems in a single dashbaord OOB with 'zero-config', But on further digging it seems like the distributed monitoring works 'only' with their cloud service[1]. Further clarity on this would be appreciated. [1] https://github.com/netdata/netdata/blob/master/docs/quicksta...

It's OOTB configured to use their free cloud service, but with 2 lines of config you can run your own central collection point instead. That's what I do for my home install. BUT the UI for this is just a dropdown for each of your monitored servers. I've found I actually want to export data to a more robust system so I can view patterns across machines, too.

Thanks, So monitoring multiple machines is possible in a central console although not in a single dashboard. Hopefully it will be available soon and the project seems useful as it is as I like the idea of getting the system information of all the Servers, SBCs in my network.

Re: Netdata: Open-source real-time monitoring platform

#85

Post by andrewm4894 »

sneak wrote: post #51
odyslam wrote: post #49

Earlier quoted context omitted.

That's a bit unfair. In the docs we are being very upfront that you can opt-out of anonymous telemetry: https://learn.netdata.cloud/docs/get we use the data we gather in order to make smarter product decisions. We want to invest resources where it matters, so we need to know how our users use the product. We are also very detailed on what we gather: https://learn.netdata.cloud/docs/agent/netdata-security Lastly, we j…

The telemetry isn't anonymous: it includes the client IP; the method you use to transmit the data cannot work anonymously. Additionally, what's actually unfair is that you proceed with this spying without the consent of the user. Being upfront about it is not obtaining consent: it's just informing the user you're about to violate their (lack of) consent. You must obtain consent from the user first, before transmittin…

Disclaimer - i work for Netdata Cloud.

We actually mask the ip address (https://github.com/netdata/dashboard/blob/master/src/domains...) so it's not even sent - we just send "127.0.0.1" as the IP into our self hosted PostHog. Likewise with any URL, referrer type event properties that could leak a hostname to us - we don't want that data at all so explicitly mask it before even capturing it in our telemetry system.

Previously, when using a fairly standard Google Analytics implementation, we could not really have this level of control all that easily.

So the hope is that with PostHog we can do better here while still enabling some really useful product telemetry to help us understand how to make the product better over time and try catch bugs and issues quicker too.

Oh and we have removed Google Tag Manager (GTM) from the agent dashboard too so that that's no longer around as a possibility for loading other third party tags too.

You can read more here: https://github.com/netdata/netdata/blob/master/docs/anonymou...

p.s. PostHog is really cool - check it out: https://posthog.com/docs#philosophy, https://github.com/PostHog/posthog

Re: Netdata: Open-source real-time monitoring platform

#86

Post by ktsaou »

Croftengea wrote: post #69
ktsaou wrote: post #62

Earlier quoted context omitted.

Thank you for this feedback. I am the founder of Netdata. Netdata is about making our lives easier. If you need to tweak Netdata, please open a github issue to let us know. It is a bug. Netdata should provide the best possible dashboards and alerts out of the box. If it does not for you, we missed something and we need your help to fix it, so please open a github issue to let us know of your use case. We want Netdata…

> our free-forever SaaS offering that complements the open-source agent How do you make or plan to make money?

The same way GitHub, Slack or Cloudflare provide massively free-forever SaaS offerings while making money.

We believe that the world will greatly benefit by a monitoring solution that is massively battle tested, highly opinionated, incorporating all the knowledge and experience of the community for monitoring infrastructure, systems and applications. A solution that is installed in seconds, even mid-crisis and is immediately effective in identifying performance and stability issues.

The tricky part is to find a way to support this and sustain it indefinitely. We believe we nailed it!

So, we plan to avoid selling monitoring features. Our free offering will never have a limit on the number of nodes monitored, the number users using it, the number of metrics collected, analyzed and presented, no limit on the granularity of data, the number of war-rooms, of dashboards, the number of alarms configured, the notifications sent, etc. All these will always be free.

And no, we are not collecting any data for ML or any other purpose. The opposite actually: we plan to release ML at the edge, so that each server will learn its own behavior.

We plan to eventually sell increased convenience features, enforcement of compliance to business policies and enterprise specific integrations, all of them on top of the free offering.

Re: Netdata: Open-source real-time monitoring platform

#87

Post by gregwebs »

tifadg1 wrote: post #24

Could someone enlighten me on the internals, how is netdata able to get realtime granularity, whereas prometheus defaults to 15s?

Prometheus is designed around metric centralization and running a scraper at some interval (every 15s). Netdata was originally focused on running on a single node and collecting at small intervals. Centralizing that data every second is a separate task, and you could avoid it with Netdata simply by viewing Netdata on the node in question. Netdata can also be configured to stream data to a central node.

The centralized pull architecture of Prometheus does not lend itself towards small interval updates or towards resiliency (you actually need to run 2 Prometheus and double scrape for that).

Re: Netdata: Open-source real-time monitoring platform

#89

Post by ktsaou »

sdesol wrote: post #83
Croftengea wrote: post #69

Earlier quoted context omitted.

> our free-forever SaaS offering that complements the open-source agent How do you make or plan to make money?

I was analyzing the activity in the netdata project and what I found interesting was this project is less active than I would have thought. See the following for insights into the project: https://public-001.gitsense.com/insights/github/repos?q=wind... In the last 30 days, there were 2 frequent and 3 occasional contributors. I honestly thought frequent contributors would have been much higher, which leads me to belie…

> they've raised about 33 million

yes, this is right

> if the number of people required to maintain netbase is low (relatively speaking that is)

The Netdata agent is a robust and mature product. We maintain it and we constantly improve it, but:

- most of our efforts go to Netdata.Cloud

- most of the action in the agent is in internal forks we have. For example, we are currently testing ML at the edge. This will eventually go into the agent, but is not there yet. Same with EBPF. We do a lot of work to streamline the process of providing the best EBPF experience out there.

> I can see them not really needing to worry about making money

We are going to make money on top of the free tier of Netdata.Cloud. We are currently building the free tier. In about a year from now we will start introducing new paid features to Netdata.Cloud. Whatever we will have released by then, will always be free.

> I'm guessing they are finding value in gathering data for ML

No, we are not gathering any data for any purpose. Our database is distributed. Your data are your data. We don't need them.

Re: Netdata: Open-source real-time monitoring platform

#90

Post by andrewm4894 »

sdesol wrote: post #83
Croftengea wrote: post #69

Earlier quoted context omitted.

> our free-forever SaaS offering that complements the open-source agent How do you make or plan to make money?

I was analyzing the activity in the netdata project and what I found interesting was this project is less active than I would have thought. See the following for insights into the project: https://public-001.gitsense.com/insights/github/repos?q=wind... In the last 30 days, there were 2 frequent and 3 occasional contributors. I honestly thought frequent contributors would have been much higher, which leads me to belie…

oh cool that's a nice tool.

p.s. i am the only person working on ML at Netdata and i can confirm we don't gather any data for ML purposes, which is actually my biggest challenge right now :) - convincing people the ML can be useful without having lots of nice labeled data from real netdata users to be able to quantify that with typical metrics like accuracy etc. I'm hoping to introduce mainly unsupervised ML features into the product that don't rely on lots of labeled data and have thumbs up/down type feedback and we can then use that to figure out if new ML based features are working or being useful for users. So any models that would be trained would be trained on the host and live on the host as opposed to in Netdata Cloud somewhere.

Post reply on HN