Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current hype around autonomous agents, and what actually works in production

utkarshkanwat.com

Post reply on HN
Print view View original

171–180 of 268 posts

Re: The current hype around autonomous agents, and what actually works in production

#171

Post by ankit219 »

These are all solvable problems. The issue is given the race to get to a certain ARR quickly, many startups end up not focusing on these. There is some truth to AI agents being not as useful as their promise, but the problems mentioned are engineering problems, and once we start seeing them with a different lens, they would start working. (This is not to say I believe orchestration or multi step agents are a way to go, I personally lean heavily towards RL. Just that the criticisms here assumes the state would remain the same even without AI advancement).

Eg: you need good verifiers (to understand whether a task is done successfully or not). Many tasks have easier verifications than doing the task. YOu have five parallel generations with 80% accuracy, the probablity of getting one right (and a verifier which can pick that) goes to 99.96%. With multi step too, the math changes in a similar manner. It just needs a different approach than how we have built software till date. He even hints at a paradigm with 3-5 discrete step workflow which works superbly well. We need to build more in that way.

Re: The current hype around autonomous agents, and what actually works in production

#172

Post by torginus »

jvanderbot wrote: post #111

My AI tool use has been a net positive experience at work. It can take over small tasks when I need a break, clean up or start momentum, and generally provide a good helping hand. But even if it could do my job, the costs pile up really quickly. Claude Code can burn $25/ 1-2 hrs, easily on a large codebase, and that's creeping along at a net positive rate assuming I can keep it on task and provide corrections. If you…

My somewhat cynical 2 cents say, it that these thinking LLMs, that constantly re-prompt themselves in a loop to fix their own mistakes, combined with the 'you don't need RAG, just dump the all code into the 1m token context windows' align well with the 'we charge per token' business model.

Re: The current hype around autonomous agents, and what actually works in production

#173

Post by throwaway423342 »

ankit219 wrote: post #171

These are all solvable problems. The issue is given the race to get to a certain ARR quickly, many startups end up not focusing on these. There is some truth to AI agents being not as useful as their promise, but the problems mentioned are engineering problems, and once we start seeing them with a different lens, they would start working. (This is not to say I believe orchestration or multi step agents are a way to g…

Is it reasonable to assume the five generations are independent?

Re: The current hype around autonomous agents, and what actually works in production

#174

Post by ankit219 »

throwaway423342 wrote: post #173
ankit219 wrote: post #171

These are all solvable problems. The issue is given the race to get to a certain ARR quickly, many startups end up not focusing on these. There is some truth to AI agents being not as useful as their promise, but the problems mentioned are engineering problems, and once we start seeing them with a different lens, they would start working. (This is not to say I believe orchestration or multi step agents are a way to g…

Is it reasonable to assume the five generations are independent?

They are not completely independent. It's a good assumption though. If a model encounters something out of distribution then all five of the generations will fail. If the model knows and went in a wrong direction (due to lack of reliability), within five generations, it can be corrected. You need evals, runtime verifiers as basic harness for AI systems.

Re: The current hype around autonomous agents, and what actually works in production

#175

Post by sfink »

infecto wrote: post #84
wooque wrote: post #82

Earlier quoted context omitted.

[flagged]

That’s a bit reductive and misses the core issue. Of course companies want to reduce headcount or boost productivity, but many are pursuing these initiatives without a clear problem in mind. If the mandate were, say, “we’re building X to reduce customer support staff by 20%,” that would be a different story. Instead, it often feels like solution-first thinking without a clear target. Edit: not even going to reply to…

I agree, and would like to hear examples of where this has not been the case. I'm sure they're out there. But pretty much everything has been "how can we use LLMs" and "it doesn't matter if it was a problem that we had that needed to be solved; we need to gain experience now because AI is The Future and we can't be left behind".

Occasionally it works and people stumble across a problem worth solving as they go about applying their solution to everything. But that's not planning or top-down direction. That's not identifying a target in advance.

Re: The current hype around autonomous agents, and what actually works in production

#176

Post by holler »

ghuntley wrote: post #63
johnisgood wrote: post #61

Earlier quoted context omitted.

What is the use case? What does it solve exactly, or what practical value does it give you? I am not sure what a tool call loop is.

> I am not sure what a tool call loop is. See https://ampcode.com/how-to-build-an-agent

that was a great read, thanks! - agentic noob

Re: The current hype around autonomous agents, and what actually works in production

#178

Post by stavros »

fleebee wrote: post #77
jrexilius wrote: post #50

Earlier quoted context omitted.

The thing that sucks about it is maybe his english is bad (not his native language) so he relies on LLM output for his posts. Im inclined to cut people slack for this. But the rub is that it is indistinguishable from spam/slop generated for marketing/ads/whatever. Or it's possible that he is one of those people that _realy_ adopted LLMs into _all_ their workflow, I guess, and he thinks the output is good enough as is…

The graphs don't line up. I'm inclined to believe they were hallucinated by an LLM and the author either didn't check them or didn't care. Judging by the other comments this is clearly low-effort AI slop. > LLMs have certainly damaged trust in general internet reading now, that's for sure. I hate that this is what we have to deal with now.

I don't know why you do. I found the article interesting, derived value from it. I don't care if it's an LLM or a human that gave me the value. I don't see why it should matter.

Re: The current hype around autonomous agents, and what actually works in production

#179

Post by stavros »

GeneralMayhem wrote: post #165
vntok wrote: post #48

Earlier quoted context omitted.

If "the bridge collapses and people die" because the team has a 1min26 "downtime" on a specific day, which is what you are arguing, then you have much bigger problems to solve than the performance of AI agents.

Uptime and reliability are not the same thing. Designing a bridge doesn't require that the engineer be working 99.9% of minutes in a day, but it does require that they be right in 99.9% of the decisions they make.

Another way to think about it is that, if the engineer isn't right in 99.9% of decision, the bridge will have 99.9% uptime.

Re: The current hype around autonomous agents, and what actually works in production

#180

Post by stavros »

Pasorrijer wrote: post #26
vntok wrote: post #9

> Production systems need 99.9%+ reliability This is not remotely true. Think of any business process around your company. 99.9% availability would mean only 1min26 per day allowed for instability/errors/downtime. Surely your human collaborators aren't hitting this SLA. A single coffee break immediately breaks this (per collaborator!). Business Process Automation via AI doesn't need to be perfect. It simply needs to…

I think you're crossing reliability and availability. Reliability means 99.9% of the time when I hand something off to someone else it's what they want. Availability means I'm at my desk and not at the coffee machine. Humans very much are 99.9% accurate, and my deliverable even comes with a list of things I'm not confident about

An interesting comment I read in another post here is that humans aren't even 99.9% accurate in breathing, as around 1 in 1000 breaths requires coughing or otherwise cleaning the airways.
Post reply on HN