Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Less human AI agents, please

nial.se

Post reply on HN
Print view View original

41–50 of 184 posts

Re: Less human AI agents, please

#41

Post by chrisjj »

hausrat wrote: post #30

This has very little to do with someone making the LLM too human but rather a core limitation of the transformer architecture itself. Fundamentally, the model has no notion of what is normal and what is exceptional, its only window into reality is its training data and your added prompt. From the perspective of the model your prompt and its token vector is super small compared to the semantic vectors it has generated…

> How should it decide whether your prompt is actually interesting novel exploration of an unknown concept or just complete bogus?

It shouldn't. It should just do what it is told.

Re: Less human AI agents, please

#42

Post by solumunus »

gregates wrote: post #27
grebc wrote: post #24

Earlier quoted context omitted.

If it’s a compiled language, just change the definition and try to compile.

Indeed! You would think it would have some kind of sense that a commit that obviously won't compile is bad! You would think. It would be one thing if it was like, ok, we'll temporarily commit the signature change, do some related thing, then come back and fix all the call sites, and squash before merging. But that is not the proposal. The plan it proposes is literally to make what it has identified as the minimal cha…

On my compiled language projects I have a stop hook that compiles after every iteration. The agent literally cannot stop working until compilation succeeds.

Re: Less human AI agents, please

#43

Post by nialse »

zingar wrote: post #35
fennecbutt wrote: post #31

Earlier quoted context omitted.

Not with tools + supporting (traditional) code. Just as a human would use a task list app or a notepad to keep track of which tasks need to be done so can a model. You can even have a mechanism for it to look at each task with a "clear head" (empty context) with the ability to "remember" previous task execution (via embedding the reasoning/output) in case parts were useful.

The article makes it seem like the author expected this without emptying context in between, which does not yet exist (actually I'm behind on playing with Opus 4.7, the Anthropic claim seems to be that longer sessions are ok now - would be interested to hear results from anyone who has).

That is probably the next step, and in practice it is much of what sub-agents already provide: a kind of tabula rasa. Context is not always an advantage. Sometimes it becomes the problem.

In long editing sessions with multiple iterations, the context can accumulate stale information, and that actively hurts model performance. Compaction is one way to deal with that. It strips out material that should be re-read from disk instead of being carried forward.

A concrete example is iterative file editing with Codex. I rewrite parts of a file so they actually work and match the project’s style. Then Codex changes the code back to the version still sitting in its context. It does not stop to consider that, if an external edit was made, that edit is probably important.

Re: Less human AI agents, please

#44

Post by zingar »

plastic041 wrote: post #18

> There was only one small issue: it was written in the programming language and with the library it had been told not to use. This was not hidden from it. It had been documented clearly, repeatedly, and in detail. What a human thing to do. "Ignoring" instructions is not human thing. It's a bad LLM thing. Or just LLM thing.

The work where I've done well in my life (smashing deadlines, rescuing projects) has so often come because I've been willing to push back on - even explicitly stated - requirements. When clients have tried to replace me with a cheaper alternative (and failed) the main difference I notice is that the cheaper person is used to being told exactly what to do.

Maybe this is more anthropomorphising but I think this pushing back is exactly the result that the LLMs are giving; but we're expecting a bit too much of them in terms of follow-up like: "ok I double checked and I really am being paid to do things the hard way".

Re: Less human AI agents, please

#45

Post by bandrami »

gregates wrote: post #15

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a def…

At the risk of being That Old Guy, this seems like a pretty bad workflow regression from what ctags could do 30 years ago

Re: Less human AI agents, please

#46

Post by anuramat »

gregates wrote: post #15

The version of this I encounter literally every day is: I ask my coding agent to do some tedious, extremely well-specified refactor, such as (to give a concrete real life example) changing a commonly used fn to take a locale parameter, because it will soon need to be locale-aware. I am very clear — we are not actually changing any behavior, just the fn signature. In fact, at all call sites, I want it to specify a def…

whats your setup?

Re: Less human AI agents, please

#47

Post by zingar »

incognito124 wrote: post #3

Your claim, paraphrased, is that AGI is already here and you want ASI

Interesting that what you're talking about as ASI is "as capable of handling explicit requirements as a human, but faster". Which _is_ better than a human, so fair play, but it's striking that this requirement is less about creativity than we would have thought.

Re: Less human AI agents, please

#48

Post by pjc50 »

raincole wrote: post #5

I know anthropomorphizing LLMs has been normalized, but holy shit . I hope the language in this article is intentionally chosen for a dramatic effect.

The thing is .. what else can you do? All the advice on how to get results out of LLMs talks in the same way, as if it's a negotiation or giving a set of instructions to a person.

You can do a mental or physical search and replace all references to the LLM as "it" if you like, but that doesn't change the interaction.

Re: Less human AI agents, please

#50

Post by anuramat »

hausrat wrote: post #30

This has very little to do with someone making the LLM too human but rather a core limitation of the transformer architecture itself. Fundamentally, the model has no notion of what is normal and what is exceptional, its only window into reality is its training data and your added prompt. From the perspective of the model your prompt and its token vector is super small compared to the semantic vectors it has generated…

wdym by "prompt and vector is small"? small as in "less tokens"? that should be a positive thing for any kind of estimation

in any case, how is this specific to transformers?

Post reply on HN