Earlier quoted context omitted.
What does AI add to your playooks ?
I'm guessing the being awake and fresh at 3am within a few seconds of the incident occuring part.
Leveraging AI for efficient incident response
51–58 of 58 posts
Re: Leveraging AI for efficient incident response
#52Re: Leveraging AI for efficient incident response
#53Im really interested in the implied restriction/focus on “code changes.” IME a very very large number of impacting incidents arent strictly tied to “a” code change, if any at all. It _feels_ like theres an implied solution to tying running version back to deployment rev, to deployment artifacts, and vcs. Boundary conditions and state changes in the distributed system were the biggest bug bear I ran in to at AWS. Then…
> IME a very very large number of impacting incidents arent strictly tied to “a” code change, if any at all Usually this implies there are bigger problems. If something keeps breaking without any change (config / code) then it was likely always broken and just ignored. So when companies do have most of the low hanging fruit resolved it's the changes that break things. I've seen places where everything is duck taped t…
Re: Leveraging AI for efficient incident response
#54Re: Leveraging AI for efficient incident response
#55Earlier quoted context omitted.
I can execute a playbook without AI at 3am in a few seconds using some orchestration tools. Without any AI.
Are you happy about waking up to do so?
This is standard SRe/ Ops practice. Monitoring system detects failures and automatically runs remediation.
You didn’t read the part where I said “using orchestration to tools”.
Re: Leveraging AI for efficient incident response
#56> The biggest lever to achieving 42% accuracy was fine-tuning a Llama 2 (7B) model 42% accuracy on a tiny, outdated model - surely it would improve significantly by fine-tuning Llama 3.1 405B!
Re: Leveraging AI for efficient incident response
#57Earlier quoted context omitted.
Are you happy about waking up to do so?
Been automatically executing playbooks (Ansible) since before you were born. I sleep fine. This is standard SRe/ Ops practice. Monitoring system detects failures and automatically runs remediation. You didn’t read the part where I said “using orchestration to tools”.
This made me look up how old Ansible was.
> Initial release: February 20, 2012; 12 years ago