Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current hype around autonomous agents, and what actually works in production

utkarshkanwat.com

Post reply on HN
Print view View original

141–150 of 268 posts

Re: The current hype around autonomous agents, and what actually works in production

#141

Post by ch4s3 »

psadri wrote: post #133
an0malous wrote: post #128

Earlier quoted context omitted.

> Clearly we have some sort of goal-based self-correction mechanism. Humans can try things, learn, and iterate. LLMs still can't really do the second thing, you can feed back an error message into the prompt but the learning isn't being added to its weights so its knowledge doesn't compound with experience like it does for us. I think there are still a few theoretical breakthroughs needed for LLMs to achieve AGI and…

You could instruct the LLM to formulate a “lesson” based on the error and add this to the tool instructions for future runs.

This isn’t practical at scale. You’ll run into too many novel lessons and burn through too many tokens setting up context.

Re: The current hype around autonomous agents, and what actually works in production

#142

Post by lxgr »

alpha_squared wrote: post #136

One thing I'll add that isn't touched on here is about context windows. While not "infinite", humans have a very large context window for problems they're specialized in solving. Models can often overcome their context window limitations by having larger and more diverse training sets, but that still isn't really a solution to context windows. Yes, I get the context window increases over time and that for many purpos…

Humans don't have this fixed split into "context" and "weights", at least not over non-trivial time spans.

For better or worse, everything we see and do ends up modifying our "weights", which is something current LLMs just architecturally can't do since the weights are read-only.

Re: The current hype around autonomous agents, and what actually works in production

#143

Post by lxgr »

RamblingCTO wrote: post #78

I also build agents/ai automation for a living. Coding agents or anything open-ended is just a stupid idea. It's best to have human validated checkpoints, small search spaces and very specific questions/prompts (does this email contain an invoice? YES/NO). Just because we'd love to have fully intelligent, automatic agents, doesn't mean the tech is here. I don't work on anything that generates content (text, images, c…

Human validation is certainly the most reliable way of introducing checkpoints, but there's others: Running unit tests, doing ad-hoc validations of the entire system etc.

Re: The current hype around autonomous agents, and what actually works in production

#144

Post by afro88 »

> Error rates compound exponentially in multi-step workflows. 95% reliability per step = 36% success over 20 steps. Production needs 99.9%+.

This misses a key feature of agents though. They get feedback from linters, build logs, test runs and even screenshots. And they collect this feedback themselves. This means they can error correct some mistakes along the way.

The math works out differently, depending on how well it can collect automated feedback it is doing what you want.

Re: The current hype around autonomous agents, and what actually works in production

#145

Post by jvanderbot »

swader999 wrote: post #138
jvanderbot wrote: post #111

My AI tool use has been a net positive experience at work. It can take over small tasks when I need a break, clean up or start momentum, and generally provide a good helping hand. But even if it could do my job, the costs pile up really quickly. Claude Code can burn $25/ 1-2 hrs, easily on a large codebase, and that's creeping along at a net positive rate assuming I can keep it on task and provide corrections. If you…

Subscription?

I have one, and upgrades don't have unlimited access as far as I can tell. Correct me if I'm wrong.

This cost scaling will be an issue for this whole AI employee thing, especially because I imagine these providers are heavily discounting.

Re: The current hype around autonomous agents, and what actually works in production

#146

Post by lxgr »

csomar wrote: post #25
KoolKat23 wrote: post #13

Human multi-step workflows tend to have checkpoints where the work is validated before proceeding further, as humans generally aren't 99%+ accurate either. I'd imagine future agents will include training to design these checks into any output, validating against the checks before proceeding further. They may even include some minor risk assessment beforehand, such as "this aspect is crucial and needs to be 99% correc…

Lots of applications have to be redesigned around that. My guess is that micro-services architecture will see a renaissance since it plays well with LLMs.

Somebody will still need to have the entire context, i.e. the full end-to-end use case and corresponding cross-service call stack. That's the biggest disadvantage of microservices, in my experience, especially if service boundaries are aligned with team boundaries.

On the other hand, if LLMs are doing the actual service development, that's something software engineers could be doing :)

Re: The current hype around autonomous agents, and what actually works in production

#147

Post by alpha_squared »

lxgr wrote: post #142
alpha_squared wrote: post #136

One thing I'll add that isn't touched on here is about context windows. While not "infinite", humans have a very large context window for problems they're specialized in solving. Models can often overcome their context window limitations by having larger and more diverse training sets, but that still isn't really a solution to context windows. Yes, I get the context window increases over time and that for many purpos…

Humans don't have this fixed split into "context" and "weights", at least not over non-trivial time spans. For better or worse, everything we see and do ends up modifying our "weights", which is something current LLMs just architecturally can't do since the weights are read-only.

I agree, I'm mostly trying to illustrate how difficult it is to fit our working model of the world into the LLM paradigm. A lot of comments here keep comparing the accuracy of LLMs with humans and I feel that glosses over so much of how different the two are.

Re: The current hype around autonomous agents, and what actually works in production

#148

Post by hemantv »

Llm are great reflections. Issues I have come across too large of context confuse the llm.

Second since llm are non deterministic in nature how do you know if the quality went from 90% to 30% there is no test you can write. What if model provider degrades quality you have no test for it

Re: The current hype around autonomous agents, and what actually works in production

#149

Post by bwfan123 »

actinium226 wrote: post #123

Very nice article. The point about mathematical reliability is interesting. I generally agree with it, but humans aren't 100% reliable, or even 99% reliable, so how do we manage to create things like the Linux kernel or the Mars landers without AI? Clearly we have some sort of goal-based self-correction mechanism. I wonder if there's research into AI on that thread?

Humans build theories of how things work. llms dont. Theories are deterministic and symbolic. Take the turing machine for example as a theory of computation in general, euclidean geometry as a theory for space, and newtonian mechanics as a theory for motion

Even for software applications like the Linux kernel, there would have been a theory in Linus' head - for example of what an operating system is, and how it should work.

A theory gives 100% correct predictions. Although the theory itself may not model the world accurately. Such feedback between the theory, and its application in the world causes iterations to the theory. From newtonian mechanics to relativity etc. From euclidean geometry to geometry of curved spaces etc.

Long story short, the LLM is a long way away from any of this. And to be fair to LLMs, the average human is not creating theories, it takes some genius to create them (newton, turing, etc). The average human is trading memes on social media.

Re: The current hype around autonomous agents, and what actually works in production

#150

Post by anon191928 »

la_fayette wrote: post #98
RamblingCTO wrote: post #78

I also build agents/ai automation for a living. Coding agents or anything open-ended is just a stupid idea. It's best to have human validated checkpoints, small search spaces and very specific questions/prompts (does this email contain an invoice? YES/NO). Just because we'd love to have fully intelligent, automatic agents, doesn't mean the tech is here. I don't work on anything that generates content (text, images, c…

In general I would agree, however the resulting systems of such an approach tend to be "just" expensive workflow systems, which could be done with old tech as well... Where is the real need for anything LLM here?

it would take months with old tech to create a bot that can check multiple websites for specific data or information? so LLM reduces the time a lot? am I wrong?
Post reply on HN