Benchmarking OpenTelemetry: Can AI trace your failed login? • Hacker News Board

Re: Benchmarking OpenTelemetry: Can AI trace your failed login?

#81

Post by vardalab » Thu, Jan 29, 2026, 11:04 PM UTC

hakanderyal wrote: post #41

Anyone that have spent serious time with agents know that you cannot expect out-of-the-box success without good context management, despite what the hyping crowd would claim. Have AI document the services first into a concise document. Then give it proper instructions about what you expect, along with the documentation created. Opus would pass that. We are not there yet, the agents are not ready to replace the driver…

Yep, I've been doing a lot of Ansible and Terraform automation with agents, and success has been continually updating our learnings, so to speak, capturing them in skills. It really does help in the long run. And it's gotten much smoother. Opus 4.5 was specifically almost like a step change and combined with decent skills, it has been effective in my homelab.

Re: Benchmarking OpenTelemetry: Can AI trace your failed login?

#82

Post by fragmede » Thu, Jan 29, 2026, 11:05 PM UTC

dudeinhawaii wrote: post #70

ndriscoll wrote: post #65

Earlier quoted context omitted.

Are these the same people who say it doesn't work well? I've been experimenting with writing what I actually mean by that (with the help of an LLM, funny enough), and it seems to be giving me much better code than the typical AI soup. e.g. - functional core, imperative shell. prefer pure helpers. - avoid methods when a standalone function suffices - use typed errors. avoid stringly errors. - when writing functions, c…

To play devils advocate, why do we have to layout a simple task in PAINSTAKING DETAIL to an AI model which is "PHD LEVEL" and going to take our jobs in 6-12 months? Why am I still holding its hand like it has the intellect and experience of a new-hire intern that's coded one project in college? I would never expect to have to layout every detail about "how to write code" to someone I hired to code on my team, at the…

Don't worry about devil's advocate, if < 100 words feels like a gargantuan amount of documentation effort ("PAINSTAKING DETAIL"), well, there are certain stereotypes about developers (not) writing comments or documentation that come to mind. Whoever coined the term "prompt engineering" may have the last laugh (before the robots take over) after all.

Re: Benchmarking OpenTelemetry: Can AI trace your failed login?

#83

Post by stared » Thu, Jan 29, 2026, 11:36 PM UTC

dang wrote: post #78

Submitters: " Please use the original title, unless it is misleading or linkbait; don't editorialize. " - https://news.ycombinator.com/newsguidelines.html If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... (Submitt…

I am sorry for that.

As a context, I felt that the original title (the first time I posted get a few upvotes, but not more). At the same time, I shouldn’t have editorialized it, as it is a slippery slope from „just a bit better title”, through optimization, to a clickbait.

Thank you dang for keeping the spirit and quality of HN.

Re: Benchmarking OpenTelemetry: Can AI trace your failed login?

#84

Post by [deleted] » Fri, Jan 30, 2026, 3:30 AM UTC

stared wrote: post #83

dang wrote: post #78

Submitters: " Please use the original title, unless it is misleading or linkbait; don't editorialize. " - https://news.ycombinator.com/newsguidelines.html If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so... (Submitt…

I am sorry for that. As a context, I felt that the original title (the first time I posted get a few upvotes, but not more). At the same time, I shouldn’t have editorialized it, as it is a slippery slope from „just a bit better title”, through optimization, to a clickbait. Thank you dang for keeping the spirit and quality of HN.

[deleted]

Re: Benchmarking OpenTelemetry: Can AI trace your failed login?

#85

Post by rixed » Fri, Jan 30, 2026, 4:51 AM UTC

sathish316 wrote: post #72

the_duke wrote: post #22

This is very confusingly written. From the post I expected that the tasks were about analysing traces, but all the tasks in the repository are about adding instrumentation to code! Some of the instructions don't give any guidance how to do it, some specify which libraries to use. "Use standard OTEL patterns" ... that's about as useful as saying "go write some code". There are a lot of ways to do instrumentation.... I…

+1 I’m not sure if tasks like Add OTel instrumentation belongs more in a Coding bench than an SRE bench. I came here expecting to see things like, this is how Models perform on finding the root cause in 50 complicated microservice failure scenarios. For AI-SRE tasks like finding root cause of bugs and errors, I believe the key is to provide tools to the agent to query metrics, logs, traces and understand the problem.…

I'm surprised by how many people think that SRE's job is to debug.

SRE's job is to make the software reliable, for instance by adding telemetry, understanding and improving the failure modes, the behavior under load etc.

So a better SRE test would not be "read the logs and fix the bug", but rather "read the code and identify potential issues".