41–50 of 85 posts
Have AI document the services first into a concise document. Then give it proper instructions about what you expect, along with the documentation created.
Opus would pass that.
We are not there yet, the agents are not ready to replace the driver.
Anyone that have spent serious time with agents know that you cannot expect out-of-the-box success without good context management, despite what the hyping crowd would claim. Have AI document the services first into a concise document. Then give it proper instructions about what you expect, along with the documentation created. Opus would pass that. We are not there yet, the agents are not ready to replace the driver…
I would wager the main reason for this is the same reason it’s also hard to teach these skills to people: there’s not a lot of high quality training for distributed debugging of complex production issues. Competence comes from years of experience fighting fires. Very few people start their careers as SREs, it’s generally something they migrate into after enjoying it and showing aptitude for it. With that said, I woul…
When we ask it to generate an image, any image will do it. We couldn't care less. Try to sculpt it, try to rotate it 45 degrees and all hell breaks loose. The image would be rotated but the hair color could change as well. Pure vibes!
When you ask it to refactor your code, any pattern would do it. You could rearrange the code in infinite ways, rename variables in infinite ways without fundamentally breaking logic. You could make as many arbitrary bullshit abstraction and call it good, as people have done it for years with OOP. It does not matter at all, any result would do it in this cases.
When you want to hit an specific gRPC endpoint, you need an specific address and the method expects an specific contract to be honored. This either matches or it doesn't. When you wish the llms could implement a solution that captures specifics syscalls from specifics hosts and send traces to an specific platform, using an specific protocol, consolidating records on a specific bucket...you have one state that satisfy your needs and 100 requirement that needs to necessarily be fulfilled. It either meet all the requirements or it's no good.
It truly is different from Vibing and llms will never be able to do in this. Maybe agents will, depending on the harnesses, on the systems in place, but one model just generate words words words with no care about nothing else
Also LLM is a very advanced autocomplete algorithm. And autocomplete isn’t designed to write for you, you have to write first.
Is it clicking a different result from same search?
It’s possible that the requirements here are not clear, given that the instructions don’t detail how to handle such a situation and it’s not obvious to me as a human.
This is very confusingly written. From the post I expected that the tasks were about analysing traces, but all the tasks in the repository are about adding instrumentation to code! Some of the instructions don't give any guidance how to do it, some specify which libraries to use. "Use standard OTEL patterns" ... that's about as useful as saying "go write some code". There are a lot of ways to do instrumentation.... I…
>Some of the instructions don't give any guidance how to do it, some specify which libraries to use. In supporting a piece of cloud software with a lot of microservices I think this is a more generalized problem for humans. The app I work with demanded some logging requirements like the library to use. But that was it, different parts by different teams ended up with all kinds of different behaviors. As for the AI si…
Anyone that have spent serious time with agents know that you cannot expect out-of-the-box success without good context management, despite what the hyping crowd would claim. Have AI document the services first into a concise document. Then give it proper instructions about what you expect, along with the documentation created. Opus would pass that. We are not there yet, the agents are not ready to replace the driver…
Sounds like it'd be faster to just do it yourself.
That's just misleading phrasing on this post
I'm an SRE, AI does NOT struggle with 'simple SRE tasks' OTel instrumentation by no measure is a 'simple SRE task'
Earlier quoted context omitted.
isn't that what trace IDs are for?
Yeah I don't know their stack but I have a service that is a collection of microservices and Opus can debug them fine by aggregating the logs tied to the same faulty request ID. In general for those tasks though the question is more "How would a human do it". If it's impossible for a human because your tooling is so bad you can't even get the logs across services for a single ID, that seems like a pretty serious desi…
I've seen some places that pretty much say
"Good luck, we hope you can swim. Life preserver not provided"