Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current hype around autonomous agents, and what actually works in production

utkarshkanwat.com

Post reply on HN
Print view View original

241–250 of 268 posts

Re: The current hype around autonomous agents, and what actually works in production

#241

Post by raxxorraxor »

throwehshdhdy wrote: post #204
Arn_Thor wrote: post #200

I spoke with an Amazon AI production engineer who’s talking with prospective clients about implementing AI in our business. When a colleague asked about using generative AI in customer facing chats the engineer said he knows of zero companies who don’t have a human in the loop. All the automatic replies are non-generative “old” tech. Gen AI is just not reliable enough for anyone to stake their reputation on it.

Plenty of tech companies have started using gen AI for live chat support. Off the top of head I know off sonder.com and wealthsimple.com. If the LLM can’t answer a query it usually forwards the chat to a human support agent.

That is for selling support. A drone for a consumer drone. Nothing more than a little more sophisticated advertising banner.

This is not being part of a defined workflow that requires structured output.

Re: The current hype around autonomous agents, and what actually works in production

#242

Post by RamblingCTO »

la_fayette wrote: post #98
RamblingCTO wrote: post #78

I also build agents/ai automation for a living. Coding agents or anything open-ended is just a stupid idea. It's best to have human validated checkpoints, small search spaces and very specific questions/prompts (does this email contain an invoice? YES/NO). Just because we'd love to have fully intelligent, automatic agents, doesn't mean the tech is here. I don't work on anything that generates content (text, images, c…

In general I would agree, however the resulting systems of such an approach tend to be "just" expensive workflow systems, which could be done with old tech as well... Where is the real need for anything LLM here?

you are 100% right. LLMs are perfect for anything that required heuristics. "is that project a good fit for client A given the following specifications ... rate it from 1-10". stuff like that. I use it as a solution for an undefined search space/problem essentially.

Re: The current hype around autonomous agents, and what actually works in production

#243

Post by RamblingCTO »

lxgr wrote: post #143
RamblingCTO wrote: post #78

I also build agents/ai automation for a living. Coding agents or anything open-ended is just a stupid idea. It's best to have human validated checkpoints, small search spaces and very specific questions/prompts (does this email contain an invoice? YES/NO). Just because we'd love to have fully intelligent, automatic agents, doesn't mean the tech is here. I don't work on anything that generates content (text, images, c…

Human validation is certainly the most reliable way of introducing checkpoints, but there's others: Running unit tests, doing ad-hoc validations of the entire system etc.

that goes without saying. but I'd argue: HITL is more of a workflow pattern, the rest are engineering patterns

Re: The current hype around autonomous agents, and what actually works in production

#244

Post by RamblingCTO »

dlisboa wrote: post #160
anon191928 wrote: post #150

Earlier quoted context omitted.

it would take months with old tech to create a bot that can check multiple websites for specific data or information? so LLM reduces the time a lot? am I wrong?

Months? Scraping wasn’t a hard problem then. Classifying information is a different and more complex thing, which is what these models are very good at. Then again we had other means of classification before LLMs without having to go through chat bots.

I think classical ML should still be compared when you use an LLM as a classifier. some problems are so well defined you can just drop an SVM on it and be done with it. the biggest benefit of LLMs I think is you can get results that are ok quite fast. no need to split or clean data, compare metrics etc. etc.

Re: The current hype around autonomous agents, and what actually works in production

#245

Post by RamblingCTO »

murukesh_s wrote: post #157
RamblingCTO wrote: post #78

I also build agents/ai automation for a living. Coding agents or anything open-ended is just a stupid idea. It's best to have human validated checkpoints, small search spaces and very specific questions/prompts (does this email contain an invoice? YES/NO). Just because we'd love to have fully intelligent, automatic agents, doesn't mean the tech is here. I don't work on anything that generates content (text, images, c…

I am also building an agent framework and also used chat coding (not vibe coding) to generate work - I was easily able to save 50% of my time just by asking GPT. But it generates mistakes like say 1 in 10 times and I do not see it getting fixed unless we drastically change the LLM architecture. In future I am sure we will have much more robust systems if the current hype cycle doesn't ruin its trust with devs. But th…

I just left my CTO job for personal reasons. we tried coding agents, agentic coding, LLM-driven coding whatever. the code any of these generate is subpar (a junior would get the PR rejected for what it produces) and you just waste so much time prompting and not thinking. people don't know the code anymore, don't check the code and it's all just gotten very sloppy. so not hiring coders because of AI is a dangerous thing to do and I'd advise heavily against it. your risk just got way higher because of hype. maintainability is out of the window, people don't know the code and there are so many hidden deviations to the spec that it's just not worth it.

the truth is that we stop thinking when we code like that.

Re: The current hype around autonomous agents, and what actually works in production

#246

Post by Arn_Thor »

throwehshdhdy wrote: post #204
Arn_Thor wrote: post #200

I spoke with an Amazon AI production engineer who’s talking with prospective clients about implementing AI in our business. When a colleague asked about using generative AI in customer facing chats the engineer said he knows of zero companies who don’t have a human in the loop. All the automatic replies are non-generative “old” tech. Gen AI is just not reliable enough for anyone to stake their reputation on it.

Plenty of tech companies have started using gen AI for live chat support. Off the top of head I know off sonder.com and wealthsimple.com. If the LLM can’t answer a query it usually forwards the chat to a human support agent.

Perhaps. But it’s telling that someone whose job is selling those kinds of services wasn’t aware of any personally.

Re: The current hype around autonomous agents, and what actually works in production

#247

Post by medbrane »

actinium226 wrote: post #220
throwehshdhdy wrote: post #204

Earlier quoted context omitted.

Plenty of tech companies have started using gen AI for live chat support. Off the top of head I know off sonder.com and wealthsimple.com. If the LLM can’t answer a query it usually forwards the chat to a human support agent.

Air Canada did this a bit ago, and their AI gave the customer a fake process for submitting claims for some sort of discount on airfare due to bereavement (the flight was for a funeral). The customer sued and Air Canada's defense was that he shouldn't have trusted the Air Canada AI chatbot. Air Canada lost.

That was in 2022, before LLMs, and they "lost" as in they had to pay back $482 USD.

Re: The current hype around autonomous agents, and what actually works in production

#248

Post by satyrun »

georgeplusplus wrote: post #104
globular-toast wrote: post #80

Earlier quoted context omitted.

I think in general if everyone is talking about a solution and nobody is talking about problems then it's a sign we're in a bubble. For me the only problem I have is I find typing slow and laborious. I've always said if I could find a way to type less I would take it. That's why I've been using tab completion and refactoring tools etc for years now. So I'm kind of excited about being able to get my thoughts into the…

Maybe you are in a job where it’s not a good use case but there are fields that are handling massive amounts of data or have a huge amount of time waiting for processing data before moving to the next step that I think handing it off to an AI agent to solve then a human puts the pieces together based on its own logic and experiences would work quite nice.

The HN fallacy that is a large % of the posts on AI

"AI is not good for what I do, therefore AI is useless"

Re: The current hype around autonomous agents, and what actually works in production

#249

Post by mritchie712 »

Rexxar wrote: post #154
mritchie712 wrote: post #76

> I've built 12+ production AI agent systems across development, DevOps, and data operations It's hard to make *one* good product (see startup failure rates). You couldn't make 12 (as seemingly a solo dev?) and you're surprised? we've been working on Definite[0] for 2 years with a small team and it only started getting really good in the past 6 months. 0 - data stack + AI agent: https://www.definite.app/

He didn't say he made 12 independent saleable products, he says he built 12 tools that fill a need at his job and are used in production. They are probably quite simple and do a very specific task as the whole article is telling us that we have to keep it simple to have something useable.

that's my point. He's "Betting Against AI Agents" without having taken a serious attempt at building one.

> agents that technically make successful API calls but can't actually accomplish complex workflows because they don't understand what happened.

It takes a long time to get these things right.

Re: The current hype around autonomous agents, and what actually works in production

#250

Post by Too »

Retr0id wrote: post #3

> Each new interaction requires processing ALL previous context I was under the impression that some kind of caching mechanism existed to mitigate this

When inference requires maxing out the memory of a gpu, where are you planning to keep this cache? Unless there is a way to compress the context into a more manageable snapshot, the cloud provider surely won’t keep a gpu idling just for holding a conversation in memory.
Post reply on HN