121–130 of 268 posts
From what I understand customer support chatbots have had some pretty good outcomes from ai agents. Or does that not count?
Link does not work for me but as someone who does a lot of work with LLMs I am also betting against agents. Agents have captivated the minds of groups of people in each large engineering org. I have no idea what their goal is other then they work on “GenAI”. For over a year now they have been working on agents with the promise that the next framework that MSFT or Alphabet publishes will solve their woes. They don’t a…
I have no idea what agents are for, could be my own ignorance. That said, I have been using LLMs for a while now with great benefit. I did not notice anything missing, and I am not sure what agents bring to the table. Do you know?
I'll manage my whiney emotions over the term Agents, but you'll have to hold a gun to my head before I embrace "Agentic", which is a thoroughly stupid word. "Scripted workflow" is what it is, but I know there are some true "visionaries" out there ready to call it "Sentient workflow".
I also build agents/ai automation for a living. Coding agents or anything open-ended is just a stupid idea. It's best to have human validated checkpoints, small search spaces and very specific questions/prompts (does this email contain an invoice? YES/NO). Just because we'd love to have fully intelligent, automatic agents, doesn't mean the tech is here. I don't work on anything that generates content (text, images, c…
In general I would agree, however the resulting systems of such an approach tend to be "just" expensive workflow systems, which could be done with old tech as well... Where is the real need for anything LLM here?
> I've built 12+ production AI agent systems across development, DevOps, and data operations It's hard to make *one* good product (see startup failure rates). You couldn't make 12 (as seemingly a solo dev?) and you're surprised? we've been working on Definite[0] for 2 years with a small team and it only started getting really good in the past 6 months. 0 - data stack + AI agent: https://www.definite.app/
Something seems off about that...
Earlier quoted context omitted.
An agent is an LLM + a tool call loop - it is quite a step up in terms of value in my experience
Agents are more than that. Agents, besides tool use, also have memory, can plan work towards a goal, and can, through an iterative process (Reflect - Act), validate if they are on the right track.
Earlier quoted context omitted.
thats good to hear, theyre on their way there! on a personal note, I'm happy to hear that. I've been apprehensive and haven't tried it, purely due to my fear of the cost.
The standard way to use Claude Code is with a constant-cost subscription; one of their standard website accounts. It’s rate-limited but still generous. You can also use API tokens, yes, but that’s 5-10x more expensive. So I wouldn’t.
The size of the code base you are working in also matters. On an old, large code base, the cost does go up, though still not real high. On a new or relatively small code base, it is not unusual for my requests to cost a tenth of a cent. For what I am doing, paying with an API key is much cheaper than a subscription would be
Very nice article. The point about mathematical reliability is interesting. I generally agree with it, but humans aren't 100% reliable, or even 99% reliable, so how do we manage to create things like the Linux kernel or the Mars landers without AI? Clearly we have some sort of goal-based self-correction mechanism. I wonder if there's research into AI on that thread?
Humans can try things, learn, and iterate. LLMs still can't really do the second thing, you can feed back an error message into the prompt but the learning isn't being added to its weights so its knowledge doesn't compound with experience like it does for us.
I think there are still a few theoretical breakthroughs needed for LLMs to achieve AGI and one of them is "active learning" like this.
Earlier quoted context omitted.
Why this timeline? What’s missing today that would make it possible in 5-10 years?
Just empirical observations. It takes time to propagate technology down to general businesses and business methods up to technology developers. The "propagate down to business methods" is the slower path, as it requires business leaders to become familiar enough with technology to get ideas on how to leverage it. This is not a new observation -- Clark's note on overestimating short term and underestimating long term…
(End quote)
Isn't this just wrong? Isn't the author conflating accuracy of LLM output in each step to accuracy of final artifact which is a reproducible deterministic piece of code?
And they're completely missing that a person in the middle is going to intervene at some point to test it and at that point the output artifact's accuracy either goes to 100% or the person running the agent would backtrack.
Either am missing something or this does not seem well thought through.