11–20 of 35 posts
This was clearly written by a bot heavily trained on linkedin posts, or someone horrifically addicted to linkedin. Its nauseating to read.
This was clearly written by a bot heavily trained on linkedin posts, or someone horrifically addicted to linkedin. Its nauseating to read.
This was clearly written by a bot heavily trained on linkedin posts, or someone horrifically addicted to linkedin. Its nauseating to read.
Why does the gate say “blocked” if the stuff is clearly still flowing through it? Having no image is better than shitty ones.
We have been using few different SRE agents and they all fucking suck. The way they are promoted and run always makes them eager to “please” by inventing processes, services, and work-arounds that don’t exist or make no sense. Giving examples will always sound pity or “dumb”. Every time I have to explain to management where SRE agent failed they just hand wave it and assume it’s a small problem. And the problem is, I totally get it. When the SRE agent says “DNS propagation issues are common. I recommend flushing dns cache or trying again later” or “The edge proxy held a bad cache entry. Cache will eventually get purged and the issue should be solved eventually” sounds so reasonable and “smart”. The issue was in DNS or in the proxy configuration. How smart was the SRE agent to get there? They think it’s phenomenal and it may be. But I know that the “DNS issue” isn’t gonna resolve itself because we have a bug in how we update DNS. I know the edge proxy cache issue is always gonna cause a particular use case to fail because the way cache invalidation is implemented has a bug. Everyone loves deflection (including me) and “self correcting” systems. But it just means that a certain class of bugs will forever be “fine” and maybe that’s fine. I don’t know anymore.
Logging, tracing, observability, and control plane (flags, etc.) should be open. We built 100% in-house pieces for all of this at a major fintech a decade ago. Everything worked and single teams could manage these systems. Someone in leadership said we had to get rid of all "weirdware". Open solutions weren't robust, so we went commerical. SignalFX got acquired, immediately 10x'd our prices and put all hands on deck…
> These systems need to be open and owned by us. Managed is okay, but they shouldn't be proprietary offerings.
You could say this about all software in the world, but good luck with that... people who make money off of making things and selling things are going to keep doing so in non-open ways, because it's advantageous. And customers will keep buying them, because it's better than the alternative.
Edit: a couple of comments pointed out that the blog does mention paying Datadog. Leaving my comment as is below, because I still find the whole interaction weird. It makes me wonder if the story is fabricated. > we lost visibility into production systems that depend fundamentally on continuous observability signals to operate safely. The Datadog message implies that Deductive wasn't paying for any service from Datad…
It may very well be a canned message since such issues would typically be detected during the evaluation period.
SRE agents are the worst agents. I totally get why business and management will demand them and love them. After all, they are the n+1 of customer support chat bot that you get frustrated talking to before you find the magic way to get to a person. We have been using few different SRE agents and they all fucking suck. The way they are promoted and run always makes them eager to “please” by inventing processes, servic…
That said, I think you're right that you can't really replace an Operations staff, as there will always need to be a human making complex, multi-dimensional decisions around constantly changing scenarios, in order to keep a business operational.