Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grafana releases OnCall open source project

grafana.com

Post reply on HN
Print view View original

111–120 of 134 posts

Re: Grafana releases OnCall open source project

#111

Post by SEJeff »

CSMastermind wrote: post #24
juliennakache wrote: post #5

Looking forward to trying this out. I've always felt that PagerDuty was absurdly expensive for the feature set they were offering. It costs something at least $250 per user for organization larger than 5 person - even if you're not an engineer who is ever directly on call. At my previous company, IT had to regularly send surveys to employees to assess if they really needed to have a PagerDuty account. Alerts are a ke…

> I've always felt that PagerDuty was absurdly expensive for the feature set they were offering For anyone out there in the same spot, I'll say that I switched my last company to Atlassian's OpsGenie and it was a 10x cost savings for the same feature set.

We did the same thing. We were fairly early and heavy users of the pagerduty API. We'd been using it for over a year and had integrated it fairly deeply into our SRE stack. When the renewal came up (we had a 3 year contract), they wanted to massively up the price due to our API usage. IIRC, it was more than 5x more. Our CTO had us port all of the tooling to opsgenie and we've never looked back.

I'm generally not a fan of Atlassian anything, but Opsgenie was really good before Atlassian purchased them and they're still really good.

Re: Grafana releases OnCall open source project

#112

Post by airstrike »

mkl95 wrote: post #66
pphysch wrote: post #23

A bit disappointed by the architecture -- it's a Django stack with MySQL, Redis, RabbitMQ, and Celery -- for what is effectively AlertManager (a single golang binary) with a nicer web frontend + Grafana integration + etc. I'm curious why/if this architecture was chosen. I get that it started as a standalone product (Amixr), but in the current state it is hard to rationalize deploying this next to Grafana in my curren…

> Django stack with MySQL, Redis, RabbitMQ, and Celery MySQL is a weird if not slightly disturbing choice. Other than that it's a boring, battle-tested stack that is relatively easy to scale. I agree that Go is nicer, but I'm biased by several years of dealing with horrific Flask / Django projects.

> several years of dealing with horrific Flask / Django projects.

I think you misspelled "beautiful"

Re: Grafana releases OnCall open source project

#113

Post by redfoxx22 »

motakuk wrote: post #11

Hello HN! Matvey Kukuy, ex-CEO of Amixr and a head of the OnCall project here. We've been working hard for a few months to make this OSS release happen. I believe it should make incident response features (on-call rotations, escalations, multi-channel notifications) and best practices more accessible to the wider audience of SRE and DevOps engineers. Hope someone will be able to finally sleep well at night being sure…

No post body was provided.

Re: Grafana releases OnCall open source project

#114

Post by deathanatos »

pm90 wrote: post #110
juliennakache wrote: post #5

Looking forward to trying this out. I've always felt that PagerDuty was absurdly expensive for the feature set they were offering. It costs something at least $250 per user for organization larger than 5 person - even if you're not an engineer who is ever directly on call. At my previous company, IT had to regularly send surveys to employees to assess if they really needed to have a PagerDuty account. Alerts are a ke…

I knew Pagerduty was going down the toilet when their sales folks started aggressively pitching BS products nobody really needed before their IPO. They couldn’t even release a proper incident management tool. I really hope this project gets good enough to ditch PD. PD should literally lay off most of its staff and just maintain the existing product, cut costs and focus mostly on integrations. There is no way they hav…

I agree with the weird pushiness on other products that add specious business value, but I do think there is room for feature growth in the main product.

For example, the auto-merge functionality is somewhat useful, but it sometimes gets it wrong. Both merging and splitting alerts is extremely clunky. I'd also love to be able to have policies like "this alert is low-priority outside of business hours". The ability to re-open an accidentally closed alert. The ability to edit the alerting rules (the Global Ruleset) without needing to be a super-admin, as the Power That Be are reluctant to hand that out, also, the ability to just read the Global Ruleset with my peon privileges.

… the sort of spit & polish that doesn't happen, IMO, anymore, because everything is an MVP feature, before the agile scrum PM moves on to the next MVP feature. "Polish" just accumulates in the JIRA graveyard.

Re: Grafana releases OnCall open source project

#115

Post by Tao3300 »

knicholes wrote: post #25
motakuk wrote: post #11

Hello HN! Matvey Kukuy, ex-CEO of Amixr and a head of the OnCall project here. We've been working hard for a few months to make this OSS release happen. I believe it should make incident response features (on-call rotations, escalations, multi-channel notifications) and best practices more accessible to the wider audience of SRE and DevOps engineers. Hope someone will be able to finally sleep well at night being sure…

Being on-call has never made me sleep better at night!

I've said it before, I'll say it again. I don't care if it's the Joint Chiefs and the whole United Nations calling me up to tell me an asteroid will destroy the planet unless I sign on [0]. If it's after hours, it'll have to wait until tomorrow, possibly Monday.

[0] Spoiler alert: it's not.

Re: Grafana releases OnCall open source project

#116

Post by Tao3300 »

motakuk wrote: post #11

Hello HN! Matvey Kukuy, ex-CEO of Amixr and a head of the OnCall project here. We've been working hard for a few months to make this OSS release happen. I believe it should make incident response features (on-call rotations, escalations, multi-channel notifications) and best practices more accessible to the wider audience of SRE and DevOps engineers. Hope someone will be able to finally sleep well at night being sure…

> it should make incident response features (on-call rotations, escalations, multi-channel notifications) and best practices more accessible to the wider audience

Is that a net positive?

Re: Grafana releases OnCall open source project

#117

Post by pphysch »

heavyset_go wrote: post #102
pphysch wrote: post #23

A bit disappointed by the architecture -- it's a Django stack with MySQL, Redis, RabbitMQ, and Celery -- for what is effectively AlertManager (a single golang binary) with a nicer web frontend + Grafana integration + etc. I'm curious why/if this architecture was chosen. I get that it started as a standalone product (Amixr), but in the current state it is hard to rationalize deploying this next to Grafana in my curren…

That's a tried and true stack, and a very good one for maintaining sane levels of reliability, consistency, durability etc. Resource wise, at least with Celery, RabbitMQ and Django, they're also pretty lean. It even ships in containers along with Docker Compose files and Helm charts, which would suit the deployment use cases of 99% of users. I understand that you're not using containers, but I don't think that's a li…

> Docker Compose takes about 5 minutes and you don't have to think about it again

Except when you need to pin and repin versions to comply with a security policy, which may be why you aren't even running containers in the first place

Re: Grafana releases OnCall open source project

#118

Post by twright0 »

CSMastermind wrote: post #24
juliennakache wrote: post #5

Looking forward to trying this out. I've always felt that PagerDuty was absurdly expensive for the feature set they were offering. It costs something at least $250 per user for organization larger than 5 person - even if you're not an engineer who is ever directly on call. At my previous company, IT had to regularly send surveys to employees to assess if they really needed to have a PagerDuty account. Alerts are a ke…

> I've always felt that PagerDuty was absurdly expensive for the feature set they were offering For anyone out there in the same spot, I'll say that I switched my last company to Atlassian's OpsGenie and it was a 10x cost savings for the same feature set.

I made that leap too (moved from a company that used Pagerduty to a company that used Opsgenie) and it is the same feature set but not the same quality. Pagerduty in my experience is rock solid - I used it on various high-page-frequency rotations for something like seven straight years and literally never saw a dropped alert/notification once.

On the flip side in the 8ish months I used Opsgenie I saw a litany of issues, like the mobile app silently logging people out (and thus not delivering push notifications) and the app failing to send SMS notifications. We had pages get dropped by the primary because they never got a notification.

I want exactly one feature from my pager, delivering 100% of my pages. It's not a situation where I'm ok with 99% success rate, and that seemed to be the tradeoff and what you are paying for with Pagerduty.

Re: Grafana releases OnCall open source project

#119

Post by echelon »

juliennakache wrote: post #5

Looking forward to trying this out. I've always felt that PagerDuty was absurdly expensive for the feature set they were offering. It costs something at least $250 per user for organization larger than 5 person - even if you're not an engineer who is ever directly on call. At my previous company, IT had to regularly send surveys to employees to assess if they really needed to have a PagerDuty account. Alerts are a ke…

Time and time again.

"Business should focus on its core competency"

* Outsource in-house infra to cloud. This begets lock-in as every engineer is doing heaven knows what with Lambda. Still need that huge infra team to manage AWS.

* Outsource in-house metrics and visibility to SignalFx, Splunk, DataDog, NewRelic, etc. Still need a team to manage it. Costs get raised by more than double because we're beholden, so now we need to fund 20+ engineer quarters to migrate everything ASAP.

* Feature flagging system built in house works like a charm and needs one engineer for maintenance. Let's fund a team to migrate it all to LaunchDarkly. Year+ later and we still don't have proper support or rollout and their stuff doesn't work as expected.

Madness.

Expensive madness.

SaaS won't magically reduce your staffing needs. Open source solutions won't reduce your staffing needs either, but they'll make costs predictable. As these tools become more prevalent and standard, you can even hire experts for them.

Re: Grafana releases OnCall open source project

#120

Post by Too »

vhold wrote: post #33
pphysch wrote: post #23

A bit disappointed by the architecture -- it's a Django stack with MySQL, Redis, RabbitMQ, and Celery -- for what is effectively AlertManager (a single golang binary) with a nicer web frontend + Grafana integration + etc. I'm curious why/if this architecture was chosen. I get that it started as a standalone product (Amixr), but in the current state it is hard to rationalize deploying this next to Grafana in my curren…

AlertManager is one component of a more complicated infrastructure. https://prometheus.io/docs/introduction/overview/#architectu... https://kubernetes.io/docs/concepts/overview/components/

That picture shows the complete monitoring stack with lots of optional components (pushgw, service discovery, Grafana).

When comparing to OnCall you need OnCall AND still the rest of that Prometheus picture.

Compare with this picture where everything in the leftmost Alert Detection box is what you see in the Prometheus architecture. https://grafana.com/docs/oncall/latest/getting-started/

Post reply on HN