Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current hype around autonomous agents, and what actually works in production

utkarshkanwat.com

Post reply on HN
Print view View original

131–140 of 268 posts

Re: The current hype around autonomous agents, and what actually works in production

#131

Post by airstrike »

an0malous wrote: post #128
actinium226 wrote: post #123

Very nice article. The point about mathematical reliability is interesting. I generally agree with it, but humans aren't 100% reliable, or even 99% reliable, so how do we manage to create things like the Linux kernel or the Mars landers without AI? Clearly we have some sort of goal-based self-correction mechanism. I wonder if there's research into AI on that thread?

> Clearly we have some sort of goal-based self-correction mechanism. Humans can try things, learn, and iterate. LLMs still can't really do the second thing, you can feed back an error message into the prompt but the learning isn't being added to its weights so its knowledge doesn't compound with experience like it does for us. I think there are still a few theoretical breakthroughs needed for LLMs to achieve AGI and…

100% and it seems like we need a whole new architecture to get there, because right now training a model takes so much time.

At the risk of making a terrible analogy, right now we're able to "give birth" to these machines after months of training, but once they're born, they can't really learn. Whereas animals learn something new every day, got to sleep, clean up their memories a bit, deleting some, solidifying others, and waking up with an improved understanding of the world.

Re: The current hype around autonomous agents, and what actually works in production

#133

Post by psadri »

an0malous wrote: post #128
actinium226 wrote: post #123

Very nice article. The point about mathematical reliability is interesting. I generally agree with it, but humans aren't 100% reliable, or even 99% reliable, so how do we manage to create things like the Linux kernel or the Mars landers without AI? Clearly we have some sort of goal-based self-correction mechanism. I wonder if there's research into AI on that thread?

> Clearly we have some sort of goal-based self-correction mechanism. Humans can try things, learn, and iterate. LLMs still can't really do the second thing, you can feed back an error message into the prompt but the learning isn't being added to its weights so its knowledge doesn't compound with experience like it does for us. I think there are still a few theoretical breakthroughs needed for LLMs to achieve AGI and…

You could instruct the LLM to formulate a “lesson” based on the error and add this to the tool instructions for future runs.

Re: The current hype around autonomous agents, and what actually works in production

#134

Post by hungryhobbit »

hannofcart wrote: post #130

> Let's do the math. If each step in an agent workflow has 95% reliability, which is optimistic for current LLMs,then: 5 steps = 77% success rate 10 steps = 59% success rate 20 steps = 36% success rate Production systems need 99.9%+ reliability. (End quote) Isn't this just wrong? Isn't the author conflating accuracy of LLM output in each step to accuracy of final artifact which is a reproducible deterministic piece o…

Did you even finish the article? The end is all about the trade-off of when "a person in the middle is going to intervene".

In fact, the point of the whole article isn't that AI doesn't work; to the contrary, it's that long chains of (20+) actions with no human intervention (which many agentic companies promise) don't work.

Re: The current hype around autonomous agents, and what actually works in production

#135

Post by coliveira »

hannofcart wrote: post #130

> Let's do the math. If each step in an agent workflow has 95% reliability, which is optimistic for current LLMs,then: 5 steps = 77% success rate 10 steps = 59% success rate 20 steps = 36% success rate Production systems need 99.9%+ reliability. (End quote) Isn't this just wrong? Isn't the author conflating accuracy of LLM output in each step to accuracy of final artifact which is a reproducible deterministic piece o…

He's not wrong. The numbers are too pessimistic, however when building software the numbers don't need to be as high for a complete disaster to happen. Even if just 1% of the code is bad, it is still very difficult to make this work.

And you mention testing, which certainly can be done. But when you have a large product and the code generator is unreliable (which LLMs always are), then you have to spend most of your time testing.

Re: The current hype around autonomous agents, and what actually works in production

#136

Post by alpha_squared »

One thing I'll add that isn't touched on here is about context windows. While not "infinite", humans have a very large context window for problems they're specialized in solving. Models can often overcome their context window limitations by having larger and more diverse training sets, but that still isn't really a solution to context windows.

Yes, I get the context window increases over time and that for many purposes it's already sufficient enough, but the current paradigm forces you to compress your personal context into a prompt to produce a meaningful result. In a language as malleable as English, this doesn't feel like engineering so much as it feels like incantations and guessing. We're losing so, so much by skipping determinism.

Re: The current hype around autonomous agents, and what actually works in production

#137

Post by coliveira »

exe34 wrote: post #95
infecto wrote: post #84

Earlier quoted context omitted.

That’s a bit reductive and misses the core issue. Of course companies want to reduce headcount or boost productivity, but many are pursuing these initiatives without a clear problem in mind. If the mandate were, say, “we’re building X to reduce customer support staff by 20%,” that would be a different story. Instead, it often feels like solution-first thinking without a clear target. Edit: not even going to reply to…

> “we’re building X to reduce customer support staff by 20%,” I've never understood the "do X to increase/decrease Y by Z%". I remember working at McDonalds and the managers worked themselves up into a frenzy to increase "sale of McSlurry by 10%". All it meant was that they nagged people more and sold less of something else. It's not like people's stomachs got 10% larger.

The sad part is that companies doing this will very soon figure out that the 20% less staff they "achieved" is only at a cost of 100% increase in development and fees to LLM vendor. Moreover, after a few years these fees will skyrocket because their businesses are now dependent on this technology and unlike people, LLMs are monopolized by just a few robber barons.

Re: The current hype around autonomous agents, and what actually works in production

#138

Post by swader999 »

jvanderbot wrote: post #111

My AI tool use has been a net positive experience at work. It can take over small tasks when I need a break, clean up or start momentum, and generally provide a good helping hand. But even if it could do my job, the costs pile up really quickly. Claude Code can burn $25/ 1-2 hrs, easily on a large codebase, and that's creeping along at a net positive rate assuming I can keep it on task and provide corrections. If you…

Subscription?

Re: The current hype around autonomous agents, and what actually works in production

#139

Post by georgeplusplus »

apwell23 wrote: post #112
georgeplusplus wrote: post #104

Earlier quoted context omitted.

Maybe you are in a job where it’s not a good use case but there are fields that are handling massive amounts of data or have a huge amount of time waiting for processing data before moving to the next step that I think handing it off to an AI agent to solve then a human puts the pieces together based on its own logic and experiences would work quite nice.

not quite sure what you are proposing here. what exactly is AI agent solving in this example? I keep hearing vague stuff exactly like your comment at work from management. Its so infuriating.

For instance cyber security toolsets like mde capture a lot of data. That data is made meaningless unless someone is looking through it, at my org there isn’t enough manpower to do that, so one solution is using an agent to help characterize that network log data into suspicious or what’s worthy of a human to follow up on.

Re: The current hype around autonomous agents, and what actually works in production

#140

Post by Vetch »

an0malous wrote: post #128
actinium226 wrote: post #123

Very nice article. The point about mathematical reliability is interesting. I generally agree with it, but humans aren't 100% reliable, or even 99% reliable, so how do we manage to create things like the Linux kernel or the Mars landers without AI? Clearly we have some sort of goal-based self-correction mechanism. I wonder if there's research into AI on that thread?

> Clearly we have some sort of goal-based self-correction mechanism. Humans can try things, learn, and iterate. LLMs still can't really do the second thing, you can feed back an error message into the prompt but the learning isn't being added to its weights so its knowledge doesn't compound with experience like it does for us. I think there are still a few theoretical breakthroughs needed for LLMs to achieve AGI and…

Compounding with learn and iterate, humans also build abstractions which significantly shorten the number of steps required. These are more expressive programming languages, compilers and toolchains. We also build engines, libraries, DSLs and invent appropriate data-structures to simplify the landscape or reuse existing work. Besides abstractions, we build tools like better type systems, error testing and borrow checkers to help eliminate certain classes of errors. Finally, after all is said and done, we still have QA teams and major bugs.
Post reply on HN