Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current hype around autonomous agents, and what actually works in production

utkarshkanwat.com

Post reply on HN
Print view View original

51–60 of 268 posts

Re: The current hype around autonomous agents, and what actually works in production

#51

Post by constantcrying »

No, it is not "mathematically impossible". It is empirically implausible. There is no statement in mathematics that says that agents can not have a 99.999% reliability rate.

Also, if you look at any human process you will realize that none of them have a 100% reliability rate. Yet, even without that we can manufacture e.g. a plane, something which takes millions of steps, each without a 100% success rate.

I actually think the article makes some good points, but especially when you are making good points it is unnecessary to stretch credibility with exaggerating your arguments.

Re: The current hype around autonomous agents, and what actually works in production

#53

Post by vntok »

Pasorrijer wrote: post #26
vntok wrote: post #9

> Production systems need 99.9%+ reliability This is not remotely true. Think of any business process around your company. 99.9% availability would mean only 1min26 per day allowed for instability/errors/downtime. Surely your human collaborators aren't hitting this SLA. A single coffee break immediately breaks this (per collaborator!). Business Process Automation via AI doesn't need to be perfect. It simply needs to…

I think you're crossing reliability and availability. Reliability means 99.9% of the time when I hand something off to someone else it's what they want. Availability means I'm at my desk and not at the coffee machine. Humans very much are 99.9% accurate, and my deliverable even comes with a list of things I'm not confident about

> Humans very much are 99.9% accurate

This is an extraordinary claim, which would require extraordinary evidence to prove. Meanwhile, anyone who spends a few hours with colleagues in a predominantly typing/data entry/data manipulation service (accounting, invoicing, presales, etc.) KNOWS the rate of minor errors is humongous.

Re: The current hype around autonomous agents, and what actually works in production

#55

Post by delis-thumbs-7e »

paradite wrote: post #4
tomhow wrote: post #117

[stub for offtopicness]

This is obviously AI generated, if that matters. And I have an AI workflow that generates much better posts than this.

I wonder why a person from Bombay India might use AI to aid with an English language blog post…

Perhaps more interesting is whether their argument is valid and whether their math is correct.

Re: The current hype around autonomous agents, and what actually works in production

#56

Post by conartist6 »

DavidPiper wrote: post #46
pmg101 wrote: post #33

Earlier quoted context omitted.

Only one of these outcomes will be correct, so worth putting money on it if you think they're wrong a la The Big Short.

Not OP, but I've been thinking about this and concluded it's not quite so clear-cut. If I was going to go down this path, I think I would bet on competitors, rather than against incumbents. My thinking: In a financial system collapse (a la The Big Short), the assets under analysis are themselves the things of value. Whereas betting on AI to collapse a technology business is at least one step removed from actual valua…

If you want to bet on a competitor, let's talk cause I'm your guy. While everyone else was looking the other way, I stole home: https://github.com/bablr-lang

Re: The current hype around autonomous agents, and what actually works in production

#57

Post by johnisgood »

infecto wrote: post #37

Link does not work for me but as someone who does a lot of work with LLMs I am also betting against agents. Agents have captivated the minds of groups of people in each large engineering org. I have no idea what their goal is other then they work on “GenAI”. For over a year now they have been working on agents with the promise that the next framework that MSFT or Alphabet publishes will solve their woes. They don’t a…

I have no idea what agents are for, could be my own ignorance.

That said, I have been using LLMs for a while now with great benefit. I did not notice anything missing, and I am not sure what agents bring to the table. Do you know?

Re: The current hype around autonomous agents, and what actually works in production

#58

Post by mhog_hn »

johnisgood wrote: post #57
infecto wrote: post #37

Link does not work for me but as someone who does a lot of work with LLMs I am also betting against agents. Agents have captivated the minds of groups of people in each large engineering org. I have no idea what their goal is other then they work on “GenAI”. For over a year now they have been working on agents with the promise that the next framework that MSFT or Alphabet publishes will solve their woes. They don’t a…

I have no idea what agents are for, could be my own ignorance. That said, I have been using LLMs for a while now with great benefit. I did not notice anything missing, and I am not sure what agents bring to the table. Do you know?

An agent is an LLM + a tool call loop - it is quite a step up in terms of value in my experience

Re: The current hype around autonomous agents, and what actually works in production

#59

Post by macleginn »

constantcrying wrote: post #51

No, it is not "mathematically impossible". It is empirically implausible. There is no statement in mathematics that says that agents can not have a 99.999% reliability rate. Also, if you look at any human process you will realize that none of them have a 100% reliability rate. Yet, even without that we can manufacture e.g. a plane, something which takes millions of steps, each without a 100% success rate. I actually…

This is a good point, but it seems, empirically, that most parts of a standard passenger airplane have reliability approximating 100% in a predefined time window with proper inspection and maintenance, otherwise passenger transit would be impossible. When the system does start to degrade, e.g. because replacement parts and maintenance becomes unavailable or too costly (cf. the use of imported planes by Russian airlines after the sanctions hit), incidents quickly start piling up.

Re: The current hype around autonomous agents, and what actually works in production

#60

Post by ilaksh »

Retr0id wrote: post #3

> Each new interaction requires processing ALL previous context I was under the impression that some kind of caching mechanism existed to mitigate this

Yes, prompt caching helps a lot with the cost. It still adds up if you have some tool outputs with long text. I have found that breaking those out into subtasks makes the overall cost much more reasonable.
Post reply on HN