Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How does misalignment scale with model intelligence and task complexity?

alignment.anthropic.com

Post reply on HN
Print view View original

61–70 of 84 posts

Re: How does misalignment scale with model intelligence and task complexity?

#61

Post by cobblestone32 »

smy20011 wrote: post #9

I think It's not because AI working on "misaligned" goals. The user never specify the goal clearly enough for AI system to work. However, I think producing detailed enough specification requires same or even larger amount of work than writing code. We write rough specification and clarify these during the process of coding. I think there are minimal effort required to produce these specification, AI will not help you…

> The user never specify the goal clearly enough for AI system to work.

This is sort of a fundamental problem with all AI. If you tell a robot assistant to "make a cup of tea", how's it supposed to know that that implies "don't break the priceless vase in the kitchen" and "don't step on the cat's tail", et cetera. You're never going to align it well enough with "human values" to be safe. Even just defining in human-understandable terms what those values are is a deep existential question of philosophy, let alone specifying it for a machine that's capable of acting in the world independently.

Re: How does misalignment scale with model intelligence and task complexity?

#62

Post by Earw0rm »

esafak wrote: post #23
xanderlewis wrote: post #8

Earlier quoted context omitted.

What do 'domain valleys' and 'tunneling' mean in this context?

A hallmark of intelligence is the ability to find connections between the seemingly disparate.

That's also a hallmark of some mental/psychological illnesses (paranoid schizophrenia family) and use of certain drugs, particularly hallucinogens.

The hallmark of intelligence in this scenario is not just being able to make the connections, but being able to pick the right ones.

Re: How does misalignment scale with model intelligence and task complexity?

#63

Post by Earw0rm »

gylterud wrote: post #53
TonyStr wrote: post #47

Earlier quoted context omitted.

Does this make conspiracy theorists highly intelligent?

No, but they emulate intelligence by making up connections between seemingly disparate things, where there are none.

They make connections but lack the critical thinking skills to weed out the bad/wrong ones.

Which is why, just occasionally, they're right, but mostly by accident.

Re: How does misalignment scale with model intelligence and task complexity?

#64

Post by xmcqdpt2 »

sandos wrote: post #59
kernc wrote: post #22

Earlier quoted context omitted.

Other actionable insights are: - Merge amendments up into the initial prompt. - Evaluate prompts multiple times (ensemble).

Sometimes when I was stressed, I have used several models to verify each others´ work. They usually find problems, too! This is very useful for things that take time to verify, we have CI stuff that takes 2-3 hours to run and I hate when those fails because of a syntax error.

Syntax errors should be caught by type checking / compiling/ linting. That should not take 2-3 hours!

Re: How does misalignment scale with model intelligence and task complexity?

#65

Post by Soerensen »

The bias-variance framing here maps well to what I've observed building AI-assisted workflows.

In practice, systematic misalignment (bias) is relatively easy to fix - you identify the pattern and add it to your prompt/context. "Always use our internal auth library" works reliably once specified.

Variance-dominated failures are a different beast. The same prompt, same context, same model can produce wildly different quality outputs on complex tasks. I've seen this most acutely when asking models to maintain consistency across multi-file changes.

The paper's finding that "larger models + harder problems = more variance" explains something I couldn't quite articulate before: why Sonnet sometimes outperforms Opus on specific workflows. The "smarter" model attempts more sophisticated solutions, but the solution space it's exploring has more local minima where it can get stuck.

One practical takeaway: decomposing complex tasks into smaller, well-specified subtasks doesn't just help with context limits - it fundamentally changes the bias/variance profile of each inference call. You're trading one high-variance call for multiple lower-variance calls, which tends to be more predictable even if it requires more orchestration overhead.

Re: How does misalignment scale with model intelligence and task complexity?

#66

Post by hnaccount_rng »

gf000 wrote: post #51
jaggederest wrote: post #25

Earlier quoted context omitted.

I believe that the issue right now is that we're using languages designed for human creation in an AI context. I think we probably want languages that are optimized for AI written but human read code, so the surface texture is a lot different. My particular hypothesis on this is something that feels a little bit like python and ruby, but has an absolutely insane overkill type system to help guide the AI. I also threw…

I don't know, LLMs strive on human text, so I would wager that a language designed for humans would quite closely match an ideal one for LLMs. Probably the only difference is that LLMs are not "lazy", they better tolerate boilerplate, and lower complexity structures likely fit them better. (E.g. they can't really one-shot understand some imported custom operator that is not very common in its training data) Also, the…

But that is leas fundamental then you make it sound. LLMs work well with human language because that’s all they are trained on. So what else _could_ an ideal language possible look like?

On the other hand: the usefulness of LLMs will always be gated by their interface to the human world. So even if their internal communication might be superseded at some point. Their contact surface can only evolve if their partners/subjects/masters can interface

Re: How does misalignment scale with model intelligence and task complexity?

#67

Post by StilesCrisis »

nayroclade wrote: post #13

The models they tested are already way behind the current state-of-the-art. Would be interesting to see if their results hold up when repeated with the latest frontier models.

I think we have all seen the latest models turn into a hot mess.

Re: How does misalignment scale with model intelligence and task complexity?

#68

Post by xmcqdpt2 »

smy20011 wrote: post #9

I think It's not because AI working on "misaligned" goals. The user never specify the goal clearly enough for AI system to work. However, I think producing detailed enough specification requires same or even larger amount of work than writing code. We write rough specification and clarify these during the process of coding. I think there are minimal effort required to produce these specification, AI will not help you…

As of today though, that doesn't work. Even straightforward tasks that are perfectly spec-ed can't be reliably done with agents, at least in my experience.

I recently used Claude for a refactor. I had an exact list of call sites, with positions etc. The model had to add .foo to a bunch of builders that were either at that position or slightly before (the code position was for .result() or whatever.) I gave it the file and the instruction, and it mostly did it, but it also took the opportunity to "fix" similar builders near those I specified.

That is after iterating a few times on the prompt (first time it didn't want to do it because it was too much work, second time it tried to do it via regex, etc.)

Re: How does misalignment scale with model intelligence and task complexity?

#69

Post by booleandilemma »

reverius42 wrote: post #58
booleandilemma wrote: post #49

Earlier quoted context omitted.

> the ability to traverse between domain valleys in the cognitive manifold. Couldn't you have just said "know about a lot of different fields"? Was your comment sarcastic or do you actually talk like that?

I think they mean both "know about a lot of different fields" and also "be able to connect them together to draw inferences", the latter perhaps being tricky?

Maybe? They should speak more clearly regardless, so we don't have to speculate over it. The way you worded it is much more understandable.

Re: How does misalignment scale with model intelligence and task complexity?

#70

Post by esafak »

w10-1 wrote: post #54
esafak wrote: post #23

Earlier quoted context omitted.

A hallmark of intelligence is the ability to find connections between the seemingly disparate.

Actually, a hallmark could be to prune illusory connections, right? That would decrease complexity rather than amplifying it.

Yes, that also happens, for example when someone first said natural disasters are not triggered by offending gods. It is all about making explanations as simple as possible but no simpler.
Post reply on HN