Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current hype around autonomous agents, and what actually works in production

utkarshkanwat.com

Post reply on HN
Print view View original

251–260 of 268 posts

Re: The current hype around autonomous agents, and what actually works in production

#251

Post by HumblyTossed »

globular-toast wrote: post #201
lxgr wrote: post #142

Earlier quoted context omitted.

Humans don't have this fixed split into "context" and "weights", at least not over non-trivial time spans. For better or worse, everything we see and do ends up modifying our "weights", which is something current LLMs just architecturally can't do since the weights are read-only.

This is why I actually argue that LLMs don't use natural language. Natural language isn't just what's spoken by speakers right now. It's a living thing. Every day in conversation with fellow humans your very own natural language model changes. You'll hear some things for the first time, you'll hear others less, you'll say things that get your point across effectively first time, and you'll say some things that requir…

> All I hear from LLM people is "you're just not using it right" or "it's all in the prompt" etc. That's not natural language. That's no different from programming any computer system.

This right here is the nail on the head. When you use (a) language to ask a computer to return you a response, there's a word for that and it's "programming". You're programming the computer to return data. This is just programming at a higher level, but we've always been increasing the level at which we program. This is just a continuation of that. These systems are not magical, nor will they ever be.

Re: The current hype around autonomous agents, and what actually works in production

#252

Post by lukaslalinsky »

I was one of the early adopters of GitHub Copilot and generally a proponent of AI assisted coding. I've recently tried "vibe coding" and oh my god, the experience couldn't be more different to what I was used to. It feels like a super expensive machine, making all kinds of mistakes and charging me for all of them. So many trial and error attempts. I ask it to do X, it conveniently does a lot of work around it, but in the end X does not work, so it just comments it out as a minor issue. It requires so much micro management, that I don't really see the purpose. Much easier and faster to just write the code myself and let it help me in that process. With agents, I feel like I'm the one helping it get a job done. I honestly can't imagine trusting this with any kind of production process.

Re: The current hype around autonomous agents, and what actually works in production

#253

Post by christianqchung »

medbrane wrote: post #247
actinium226 wrote: post #220

Earlier quoted context omitted.

Air Canada did this a bit ago, and their AI gave the customer a fake process for submitting claims for some sort of discount on airfare due to bereavement (the flight was for a funeral). The customer sued and Air Canada's defense was that he shouldn't have trusted the Air Canada AI chatbot. Air Canada lost.

That was in 2022, before LLMs, and they "lost" as in they had to pay back $482 USD.

Weird deflection. GPT 3, a 175 billion parameter model, came out in 2020, so it very much is not before LLMs. I guess you can say the story happened before ChatGPT by a few weeks. As for putting lost in quotation marks, I have no idea why you think the quantity of money is relevant to the outcome. No one was expecting them to file bankruptcy over this, only to follow the original agreement.

Re: The current hype around autonomous agents, and what actually works in production

#254

Post by Shypangz »

DavidPiper wrote: post #46
pmg101 wrote: post #33

Earlier quoted context omitted.

Only one of these outcomes will be correct, so worth putting money on it if you think they're wrong a la The Big Short.

Not OP, but I've been thinking about this and concluded it's not quite so clear-cut. If I was going to go down this path, I think I would bet on competitors, rather than against incumbents. My thinking: In a financial system collapse (a la The Big Short), the assets under analysis are themselves the things of value. Whereas betting on AI to collapse a technology business is at least one step removed from actual valua…

It’s tricky to predict how AI impacts business value directly. You might want to check out MailsAI for insights on how companies adjust their strategies in uncertain times. It helped me see the bigger picture without getting lost in assumptions.

Re: The current hype around autonomous agents, and what actually works in production

#255

Post by vrighter »

lmeyerov wrote: post #170

I used to believe the error rate fallacy, but: 1. Multi-turn agents can correct themselves with more steps, so the reductive error cascade thinking here is more wrong than right in my experience 2. The 99.9% production requirement is so contextual and misleading, when the real comparison is often something like "outage", "dead air", "active incident", "nobody on it", "prework before/around human work", "proactive tas…

How about those large swaths are done with LLMs, but instead of spending all that time reviewing that code (really reviewing it, not just a brief LGTM), which would make the time savings moot, you just decide to personally assume responsibility for that code being dead wrong sometimes and the consequences it causes (you cannot blame the AI. As far as anyone is concerned, you wrote the code and signed off on it). As in, legal liability. Would you take the deal?

Re: The current hype around autonomous agents, and what actually works in production

#256

Post by vrighter »

hannofcart wrote: post #130

> Let's do the math. If each step in an agent workflow has 95% reliability, which is optimistic for current LLMs,then: 5 steps = 77% success rate 10 steps = 59% success rate 20 steps = 36% success rate Production systems need 99.9%+ reliability. (End quote) Isn't this just wrong? Isn't the author conflating accuracy of LLM output in each step to accuracy of final artifact which is a reproducible deterministic piece o…

How is it that the final result is a reproducible deterministic piece of code, when the prompts become the "source code" itself, and the underlying model used is constantly changing (being updated), which is equivalent to your programming language changing its semantics every other day and refusing to tell you exactly what has changed (because they can't). Not to mention the nondeterminism that a lot of times is present due to nondeterministic order of evaluation when parallelizing?

Re: The current hype around autonomous agents, and what actually works in production

#257

Post by vrighter »

vntok wrote: post #9

> Production systems need 99.9%+ reliability This is not remotely true. Think of any business process around your company. 99.9% availability would mean only 1min26 per day allowed for instability/errors/downtime. Surely your human collaborators aren't hitting this SLA. A single coffee break immediately breaks this (per collaborator!). Business Process Automation via AI doesn't need to be perfect. It simply needs to…

This comment makes the assumption that the software is cloud based and all that matters is uptime.

I used to work on a backup application, it ran locally on our clients' machines. We had over 10000 clients. A 99.9% reliability would mean that there are 10 of our customers, at any one point, having a problem. It's not a question of uptime. It's a question of data integrity in this case. So 99.9% reliability could even leave us open to, potentially, 10 lawsuits. Also, about 10 support calls per day.

Now we only had about 10k customers at the time. Imagine if it were millions.

Re: The current hype around autonomous agents, and what actually works in production

#258

Post by dickersnoodle »

A4ET8a8uTh0_v2 wrote: post #73
infecto wrote: post #37

Link does not work for me but as someone who does a lot of work with LLMs I am also betting against agents. Agents have captivated the minds of groups of people in each large engineering org. I have no idea what their goal is other then they work on “GenAI”. For over a year now they have been working on agents with the promise that the next framework that MSFT or Alphabet publishes will solve their woes. They don’t a…

I think it is a mix of fomo and the 'upside' potential of being able to minimize ( ideally remove ) the expensive "human component". Note, I am merely trying to portray a specific world model. Preaching to the choir man. We just got custom AI tool ( which manages to have all my industry specific restrictions rendering it kinda pointless, low context making it annoying, and slower than normal, because it now has to go…

>I think it is a mix of fomo and the 'upside' potential of being able to minimize ( ideally remove ) the expensive "human component". Note, I am merely trying to portray a specific world model.

IOW, it's a case of C-suite "monkey see, monkey do" kicked off by management consultants with crap to sell for very high prices...

Re: The current hype around autonomous agents, and what actually works in production

#259

Post by sarchertech »

stavros wrote: post #238
sarchertech wrote: post #227

Earlier quoted context omitted.

If you’re going by bugs per lines of code, I’m far higher than 99% reliable.

When measuring the reliability of my vacuum, I don't go by component, so I don't go by line when I measure the reliability of my code.

Ok if we measure uptime of the software I’ve written it’s far higher than 99%.

Re: The current hype around autonomous agents, and what actually works in production

#260

Post by stavros »

sarchertech wrote: post #259
stavros wrote: post #238

Earlier quoted context omitted.

When measuring the reliability of my vacuum, I don't go by component, so I don't go by line when I measure the reliability of my code.

Ok if we measure uptime of the software I’ve written it’s far higher than 99%.

Again, we're not measuring uptime. We're measuring how many of them had at least one bug.
Post reply on HN