We could stop normalising "on-call" instead.
Show HN: I built an open-source tool to make on-call suck less
101–110 of 174 posts
Re: Show HN: I built an open-source tool to make on-call suck less
#102how can you prove it works and doesnt hallucinate?
do you have any actual users that have installed it and found it useful?
Re: Show HN: I built an open-source tool to make on-call suck less
#103> It reduces alert fatigue by classifying alerts as actionable or noisy and providing contextual information for handling alerts. grimace face I might be missing context here, but this kind of problem speaks more to a company’s inability to create useful observability, or worse, their lack of conviction around solving noisy alerts (which upon investigation might not even be “just” noise)! Your product is welcome and…
Agreed. This doesn't make the problem better, it's a bandaid solution that can make the problem worse by allowing you to ignore it for longer. Iterating on your alarms is super informative about the underlying product. It'll point to how you might improve your KPI measurements, or find bugs you didn't know were there.
Bandaids are a valuable and useful product used billions of times around the globe every year.
Re: Show HN: I built an open-source tool to make on-call suck less
#104is this only on the frontpage because this is an HN company?
Re: Show HN: I built an open-source tool to make on-call suck less
#105Is this for missile defense systems or something? What's possibly so important that you need to be woken up for it?
System goes down or degrades in some other way at night and important customers with a different timezone get angry, threatening to leave? (happened with us a few times)
But I would'nt use LLM for it due to hallucinations
Re: Show HN: I built an open-source tool to make on-call suck less
#106Shameless question tangential related to the topic. We are based in Europe and have the problem that some of us sometimes just forget we're on call or are afraid that we'll miss OpsGenie notifications. We're desparately looking for a hardware solution. I'd like something similar to the pagers of the past but at least here in Germany they don't really seem to exist anymore. Ideally I'd have a Bluetooth dongle that ale…
A candy bar cell phone, paid for by your employer and handed to whoever is on call. People who don't want it can just forward it to their phone.
Yup, this is something we've thought about as well.
We're a remote company so everyone would need to get their own but this option is definitely on the table.
Re: Show HN: I built an open-source tool to make on-call suck less
#107co-founder of merlinn here: https://merlinn.co | https://github.com/merlinn-co/merlinn
We're also building a tool in the same space with the option of choosing your own model (private llms) + we're open source with a multitude of integrations.
good to see more options in this space! especially OS. I think de-noising is a good feature given alert fatigue is one of the repeating complaints of on-callers.
Re: Show HN: I built an open-source tool to make on-call suck less
#108It feels to me that using LLM to classify alerts as noisy is just adding risk instead of fixing the root cause of the problem. If an alert is known to be noisy and have appeared on slack before (which is how the LLM would figure out it's a noisy alert), then just remove the alert? Otherwise, how will the LLM know it's noise? Either it will correctly annoy you or hallucinate a reason it figures that alert is just noise.
Re: Show HN: I built an open-source tool to make on-call suck less
#109> It reduces alert fatigue by classifying alerts as actionable or noisy and providing contextual information for handling alerts. grimace face I might be missing context here, but this kind of problem speaks more to a company’s inability to create useful observability, or worse, their lack of conviction around solving noisy alerts (which upon investigation might not even be “just” noise)! Your product is welcome and…
> enabling bad cultural practices I strongly disagree. There is nothing culturally bad in a system issuing an error if there is an error. Sometimes systems issue errors that are considered noise by supporters because they are not actionable, but forcing a system to not issue an error just because your support team cannot directly take action on it is an extremely odd leakage of team responsibilities and bound to have…
The context here is alerts triggering on-call.
If the error is not-actionable, why wake someone up in the middle of the night because of it?
I don't think anyone is rejecting the observability of these errors, but just that there's no point in having it alert/wake someone up unnecessarily.
Re: Show HN: I built an open-source tool to make on-call suck less
#110It feels to me that using LLM to classify alerts as noisy is just adding risk instead of fixing the root cause of the problem. If an alert is known to be noisy and have appeared on slack before (which is how the LLM would figure out it's a noisy alert), then just remove the alert? Otherwise, how will the LLM know it's noise? Either it will correctly annoy you or hallucinate a reason it figures that alert is just nois…
yeah, thats the goal of adding the context and the report - to hopefully bring awareness to the team that this alert should be removed.
My rationale for flagging the alert was to help prioritization for the on-call (lets say there are multiple alerts going off at the same time)