Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

121–130 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#121

Post by bpodgursky »

jakozaur wrote: post #115
carlosdp wrote: post #46

Gotta say though, having rolled grafana and prometheus and such on my own plenty of times before, if you are a startup and can afford Datadog, use Datadog.

DataDog per host pricing can be very expensive. Metrics are provided by many platforms. If you need logs too, you may look at Sumo Logic which got way cheaper metrics in typical use case. Disclaimer: I work at Sumo Logic.

Like any SaaS dev tool, when at scale, you negotiate and pay a fraction of the list price.

It's meaningless to look at the price of Datadog@5 hosts -- at 500 or 5000, you're paying a completely detached number from the website list price, likely a small fraction.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#122

Post by samstave »

avthar wrote: post #82
jimaek wrote: post #81

Earlier quoted context omitted.

While influx is pretty bad overall it's super simple to deploy and configure unlike timescale. It's the main reason we decided to use influx in our small team with simple enough timeseries needs

Curious what you found difficult to deploy/ configure? Is this in a self-managed context?

I found this to be a funny, subtle insult. :-)

"Why is it difficult, because you're self managed?"

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#123

Post by hnarn »

thethethethe wrote: post #119
hnarn wrote: post #104

Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…

Possibly stupid question: Can you use Nagios to stream metrics exported from your applications binary in real time? For example, can you use Nagios record each http request processed by your application webserver, tagged with http method, code, latency etc?

Nagios is not so much a "recorder" as it is a "state inspector", basically plugins run on a schedule and inspect that things are up to spec. The situation you describe may be better suited for something like the ELK stack which can hook into your HTTPD logs.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#124

Post by dijit »

rattray wrote: post #91
dijit wrote: post #68

Earlier quoted context omitted.

I just costed a datadog deployment (based on your comment) and it would cost me my yearly salary every month. No thanks. :/

Can you say something vague about your deployment scale?

2,500~ compute instances.

Looking at the pricing page the cost $15/instance so $37,500

Middle-of-the-road developer salary is like $35k in most of Europe, outside of the capitals.

Although it does say: "Volume discounts available (500+ hosts/mo). Contact us." at the bottom, so I guess 500 is a lot.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#125

Post by cevian »

rattray wrote: post #96

I've only ever used third party monitoring tools, but hope to set up a startup again soon and want to do OSS if I can. Can anyone comment on Prometheus vs Timescale? What are the tradeoffs? Or would I use Prometheus on top of Timescale?

(Timescale-Prometheus team lead here) I'd suggest using Timescale-Prometheus to connect Prometheus with TimescaleDB.

Repo: https://github.com/timescale/timescale-prometheus Design Doc: https://tsdb.co/prom-design-doc

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#126

Post by mhall119 »

cbzbc wrote: post #112
mhall119 wrote: post #108

Earlier quoted context omitted.

`snap install telegraf` is pretty easy though

Not everything uses or even can run ubuntu. I have a similar setup and have a number of openwrt devices that also feed things into grafana via collectd - so I already have that up and running.

What kind of packaging does openwrt use? We publish tarballs of Telegraf that should work, but if there's a different packaging format that would make it easier to install we can look into adding that.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#127

Post by user5994461 »

klohto wrote: post #97
cs-szazz wrote: post #92

Earlier quoted context omitted.

Wait, you're saying it costs 12x as much as your salary? That doesn't seem right... the company I'm at uses it pretty heavily and we're at about 1x of a FTE salary (and it's still super worth it)

World’s salaries aren’t only US based :) Datadog for a smaller Western Europe startup is going to cost more than their devs salaries.

Western salaries are not that low and true costs to the employer is typically double the perceived salary. That means we're talking tens of thousands of euros, so thousands of hosts (list price is certainly negotiable at this scale).

If they've got a thousand of hosts, the costs of the infrastructure itself must dwarf the salary of any developer by orders of magnitude, the salary of a developer is simply irrelevant when it comes to acquiring software/hardware.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#128

Post by tilolebo »

Papric0re wrote: post #74

[Offtopic (a bit)] Lots of you are talking about metric monitoring. But do you have recommendations when it comes to (basic) security Monitoring? I would usually go for the Elastic-Stack for that purpose, especially because Kibana offers lots of features for security monitoring. But I feel like these stacks are so big and bloated. I basically need something to monitor network traffic (Flows and off-Database retention…

Grafana together with Loki would be a good match for you

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#129

Post by hnarn »

brightball wrote: post #118
hnarn wrote: post #104

Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…

Ever looked at Zabbix?

I haven't, unfortunately, but it looks promising from just looking into it briefly. Open source monitoring is always an area that needs more competition.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#130

Post by mhall119 »

rattray wrote: post #103
rbetts wrote: post #23

Earlier quoted context omitted.

(I'm the VPE at Influxdata). I appreciate this sentiment. We've been focused on a building a SaaS version of Influxdata and are committed to a paired open source version of that. The open source version has been lagging as we work on the SaaS side. However, we are committed to shipping a GA version of the OSS 2.0 stack around the end of Q3 that offers an in-place data migration capability from 1.x OSS. We've spoken a…

Well, guess I know to never use Influx as a startup

Why is that? It takes minimal effort to get it up and running and you can either self-host or use the SaaS offering on any of the major clouds. There's even a free tier on the SaaS your startup can use that won't cost you a dime until your usage becomes significant.
Post reply on HN