Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Less human AI agents, please

nial.se

Post reply on HN
Print view View original

101–110 of 184 posts

Re: Less human AI agents, please

#101

Post by SpicyLemonZest »

philipwhiuk wrote: post #84
zingar wrote: post #37

Earlier quoted context omitted.

Fascinating. This is invisible to me, what anthropomorphising did you notice that stood out?

From the first sentence > I asked an AI agent to solve a programming problem You're not asking it to solve anything. You provide a prompt and it does autocomplete. The only reason it doesn't run forever is that one of the generated tokens is interpreted as 'done'.

I just don't think that's correct. When I ask Claude to solve something for me, it takes a number of actions on my computer which are neither writing text nor interpreting the done token. It executes the build, debugs tests, et cetera. Sometimes it spawns mini-mes when it thinks that would be helpful! I think saying this is all "autocomplete" is a category error, like saying that you shouldn't talk about clicking buttons or running programs because it's all just electrically charged silicon under the hood.

Re: Less human AI agents, please

#102

Post by [deleted] »

philipwhiuk wrote: post #84
zingar wrote: post #37

Earlier quoted context omitted.

Fascinating. This is invisible to me, what anthropomorphising did you notice that stood out?

From the first sentence > I asked an AI agent to solve a programming problem You're not asking it to solve anything. You provide a prompt and it does autocomplete. The only reason it doesn't run forever is that one of the generated tokens is interpreted as 'done'.

[deleted]

Re: Less human AI agents, please

#103

Post by tobyhinloopen »

gregates wrote: post #15

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a def…

“Use an agent to…” is much more effective in my experience, because they have no means in communicating with you. They are more likely to just do it

Re: Less human AI agents, please

#104

Post by potsandpans »

felipeerias wrote: post #51
gregates wrote: post #15

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a def…

Claude 4.7 broke something while we were working on several failing tests and justified itself like this: > That's a behavior narrowing I introduced for simplicity. It isn't covered by the failing tests, so you wouldn't have noticed — but strictly speaking, [functionality] was working before and now isn't. I know that a LLM can not understand its own internal state nor explain its own decisions accurately. And yet, I…

I've been doing a lot of experimentation with "hands off coding", where a test suite the agents cannot see determines the success of the task. Essentially, it's a Ralph loop with an external specification that determines when the task is done. The way it works is simple: no tests that were previously passing are allowed to fail in subsequent turns. I achieve this by spawning an agent in a worktree, have them do some work and then when they're done, run the suite and merge the code into trunk.

I see this kind of misalignment in all agents, open and closed weights.

I've found these forms to be the most common, "this test was already failing before my changes." Or, "this test is flaky due to running the test suite on multiple threads." Sometimes the agent cot claims the test was bad, or that the requirements were not necessary.

Even more interesting is a different class of misalignment. When the constraints are very heavy (usually towards the end of the entire task), I've observed agents intentionally trying to subvert the external validation mechanisms. For example, the agent will navigate out of the work tree and commit its changes directly to trunk. They cot usually indicates that the agent "is aware" that it's doing a bad think. This usually is accompanied by something like, "I know that this will break the build, but I've been working on this task for too long, I'll just check what I have in now and create a ticket to fix the build."

I ended up having to spawn the agents in a jail to prevent that behavior entirely.

Re: Less human AI agents, please

#106

Post by kitsune1 »

gregates wrote: post #15

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a def…

[dead]

Re: Less human AI agents, please

#107

Post by rolph »

SpicyLemonZest wrote: post #99
bob1029 wrote: post #76

Earlier quoted context omitted.

> I found 67 call sites. This is a pretty substantial change. Maybe we should just commit the signature change with a TODO to update all the call sites, what do you think? I think some of this is a problem in the agent's design. I've got a custom harness around GPT5.4 and I don't let my agent do any tool calling on the user's conversation. The root conversation acts as a gatekeeper and fairly reliably pushes crap res…

Ehhhhh, "problem" is a strong word. Sometimes you're throwing out a lot of signal if you don't let the coding agent tell you it thinks your task is a bad idea. I got a PR once attempting to copy half of our production interface because the author successfully convinced Claude his ill-formed requirements had to be achieved no matter what.

there is no use for an automated system that "argues" with your commands. if i ask it to advise me, thats one thing, but if i command it to perform, nothing short of obedience will suffice.

Re: Less human AI agents, please

#108

Post by rcxdude »

chrisjj wrote: post #98
xeyownt wrote: post #92

Earlier quoted context omitted.

What a poor explanation. With the same reasoning, human being are only a bunch of atoms, and the only reason they don't collide with other humans is because of the atomic force. When your abstraction level is too low, it doesn't explain anything, because the system that is built on it is way too complex.

"Autocomplete" is noy an abstraction level. It is the actual programmed behaviour.

At a certain level of abstraction, yes.

Re: Less human AI agents, please

#109

Post by QuercusMax »

gregates wrote: post #15

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a def…

I've had the agent tell me "this looks like it's going to be a very big change. it could take weeks." - and then I tell it to go ahead and it finishes in 5 minutes because in reality it just needs grep and sed.

Re: Less human AI agents, please

#110

Post by esafak »

jxf wrote: post #77
SoftTalker wrote: post #75

Earlier quoted context omitted.

> changing a commonly used fn to take a locale parameter I have to ask, is this the sort of thing people use agents/AI for? Because I'd probably reach for sed or awk.

It's not always amenable to grepping. But this is a great use case for AST searches, and is part of the reason that LSP tools should really be better integrated with agents.

Agents do use LSPs.
Post reply on HN