Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The current hype around autonomous agents, and what actually works in production

utkarshkanwat.com

Post reply on HN
Print view View original

111–120 of 268 posts

Re: The current hype around autonomous agents, and what actually works in production

#111

Post by jvanderbot »

My AI tool use has been a net positive experience at work. It can take over small tasks when I need a break, clean up or start momentum, and generally provide a good helping hand. But even if it could do my job, the costs pile up really quickly. Claude Code can burn $25/ 1-2 hrs, easily on a large codebase, and that's creeping along at a net positive rate assuming I can keep it on task and provide corrections. If you automate the corrections we are up to $50/hr or some tradeoff of speed, accuracy, and cost.

Same as it's always been.

For agents, that triangle is not very well quanitfied at the moment which makes all these investigations interesting but still risky.

Re: The current hype around autonomous agents, and what actually works in production

#112

Post by apwell23 »

georgeplusplus wrote: post #104
globular-toast wrote: post #80

Earlier quoted context omitted.

I think in general if everyone is talking about a solution and nobody is talking about problems then it's a sign we're in a bubble. For me the only problem I have is I find typing slow and laborious. I've always said if I could find a way to type less I would take it. That's why I've been using tab completion and refactoring tools etc for years now. So I'm kind of excited about being able to get my thoughts into the…

Maybe you are in a job where it’s not a good use case but there are fields that are handling massive amounts of data or have a huge amount of time waiting for processing data before moving to the next step that I think handing it off to an AI agent to solve then a human puts the pieces together based on its own logic and experiences would work quite nice.

not quite sure what you are proposing here. what exactly is AI agent solving in this example?

I keep hearing vague stuff exactly like your comment at work from management. Its so infuriating.

Re: The current hype around autonomous agents, and what actually works in production

#113

Post by ptero »

corentin88 wrote: post #75
ptero wrote: post #71

Earlier quoted context omitted.

While true, the world doesn't end in 2025. While I would also agree that big financial benefits from agents to companies appear unlikely to arrive this year (and the title specifically mentions 2025) I would bet on agents becoming a disruptive technology in the next 5-10 years. My 2c.

Why this timeline? What’s missing today that would make it possible in 5-10 years?

Just empirical observations. It takes time to propagate technology down to general businesses and business methods up to technology developers. The "propagate down to business methods" is the slower path, as it requires business leaders to become familiar enough with technology to get ideas on how to leverage it.

This is not a new observation -- Clark's note on overestimating short term and underestimating long term impact of technology is one of my favorite patterns. My 2c.

Re: The current hype around autonomous agents, and what actually works in production

#114

Post by johnisgood »

csande17 wrote: post #93
johnisgood wrote: post #88

Earlier quoted context omitted.

Oh, okay, I understand it now, especially with the other comment that said Cursor is one. OK, makes sense. Seems like it "just" reduces friction (quite a lot).

Yeah, it's really just a user experience improvement. In particular, it makes AI look a lot better if it can internally retry a bunch of times until it comes up with valid code or whatever, instead of you having to see each error and prompt it to fix it. (Also, sometimes they can do fancy sampling tricks to force the AI to produce a syntactically valid result the first time. Mostly this is just used for simple JSON s…

Thank you, that is what my initial thought was. I am still doing things the old-fashioned way, thankfully it has worked out for me (and learned a lot in the process), but perhaps this AI agent thing might speed things up a bit. :D Although then I will learn much less.

Re: The current hype around autonomous agents, and what actually works in production

#115

Post by atomon »

Is the main point “let me mathematically prove that it’s impossible to do what I’ve already done 12 times this year?”

Yes, very long workflows with no checks in between will have high error rates. This is true of human workflows too (which also have <100% accuracy at each step). Workflows rarely have this many steps in practice and you can add review points to combat the problem (as evidenced by the author building 12 of these things and not running into this problem)

Re: The current hype around autonomous agents, and what actually works in production

#116

Post by queenkjuul »

apwell23 wrote: post #110
queenkjuul wrote: post #85

Earlier quoted context omitted.

An example: I updated a svelte component at work, and while i could test it in the browser and see it worked fine, the existing unit test suddenly started failing. I spent about an hour trying to figure out why the results logged in the test didn't match the results in the browser. I got frustrated, gave in and asked Claude Code, an AI agent. The tool call loop is something like: it reads my code, then looks up the d…

any examples outside of coding agents ? Even in this example coding agent is short lived . I am curious about continuously running agents that are never done.

No, the fact Claude couldn't remember that JavaScript is zero-indexed for more than 20 minutes has not left me interested in letting it take on bigger tasks

Re: The current hype around autonomous agents, and what actually works in production

#118

Post by ethbr1 »

infecto wrote: post #65
mhog_hn wrote: post #58

Earlier quoted context omitted.

An agent is an LLM + a tool call loop - it is quite a step up in terms of value in my experience

Not a disagreement with you but wanted to further clarify. I do think it’s a step up when done correctly. Thinking of tools like Cursor. Most of my concern and issue comes from the amount of folks I have seen trying to great a system that solves everything. I know in my org people were working on Agents without even a problem they were solving for. They are effectively trying to recreate ChatGPT which to me is a fool…

I’d boil it down thusly:

What do agents provide? Asynchronous work output, decoupled from human time.

That’s super valuable in a lot of use cases! Especially because it’s a prerequisite for parallelizing “AI” use (1 human : many AI).

But the key insight from TFA (which I 100% agree with) is that the tyranny of sub-100% reliability compounded across multiple independent steps is brutal.

Practical agent folks should be engineering risk / reliability, instead of happy path.

And there are patterns and approaches to do that (bounded inputs, pre-classification into workable / not-workable, human in the loop), but many teams aren’t looking at the right problem (risk/reliability) and therefore aren’t architecting to those methods.

And there’s fundamentally no way to compose 2 sequential 99% reliable steps into a 99% reliable system with a risk-naive approach.

Re: The current hype around autonomous agents, and what actually works in production

#119

Post by _fat_santa »

Filligree wrote: post #91
KoolKat23 wrote: post #20

Earlier quoted context omitted.

thats good to hear, theyre on their way there! on a personal note, I'm happy to hear that. I've been apprehensive and haven't tried it, purely due to my fear of the cost.

The standard way to use Claude Code is with a constant-cost subscription; one of their standard website accounts. It’s rate-limited but still generous. You can also use API tokens, yes, but that’s 5-10x more expensive. So I wouldn’t.

> You can also use API tokens, yes, but that’s 5-10x more expensive. So I wouldn’t.

100% agree as someone that uses API tokens. I use it via API tokens only because my work gave me some Anthropic keys and the directive "burn the tokens!" (they want to see us using it and don't give a crap about costs).

Re: The current hype around autonomous agents, and what actually works in production

#120

Post by jampekka »

sarchertech wrote: post #106
Filligree wrote: post #91

Earlier quoted context omitted.

The standard way to use Claude Code is with a constant-cost subscription; one of their standard website accounts. It’s rate-limited but still generous. You can also use API tokens, yes, but that’s 5-10x more expensive. So I wouldn’t.

If API tokens are 10x more expensive doesn’t that imply that the constant-cost subscription is massively subsidized?

Relies on many of the subscribers underusing their quota?
Post reply on HN