Live data from Hacker News

How does misalignment scale with model intelligence and task complexity?

alignment.anthropic.com

81–84 of 84 posts

Re: How does misalignment scale with model intelligence and task complexity?

#81
post #76

The comments so far seem focused on taking a cheap shot, but as somebody working on using AI to help people with hard, long-term tasks, it's a valuable piece of writing. - It's short and to the point - It's actionable in the short term (make sure the tasks per session aren't too difficult) and useful for researchers in the long term - It's informative on how these models work, informed by some of the best in the busi…

There’s not a useful argument here. The article is using current AI to extrapolate future AI failure modes. If future AI models solve the ‘incoherence’ problem, that leaves bias as a primary source of failure (according to the author these are the only two possible failure modes apparently).

That doesn't seem like a useful argument either.

If future AI only manages to solve the variance problem, then it will have problems related to bias.

If future AI only manages to solve the bias problem, then it will have problems related to variance.

If problem X is solved, then the system that solved it won't have problem X. That's not very informative without some idea of how likely it is that X can or will be solved, and current AI is a better prior than "something will happen".

Re: How does misalignment scale with model intelligence and task complexity?

#82
post #76

Earlier quoted context omitted.

There’s not a useful argument here. The article is using current AI to extrapolate future AI failure modes. If future AI models solve the ‘incoherence’ problem, that leaves bias as a primary source of failure (according to the author these are the only two possible failure modes apparently).

That doesn't seem like a useful argument either. If future AI only manages to solve the variance problem, then it will have problems related to bias. If future AI only manages to solve the bias problem, then it will have problems related to variance. If problem X is solved, then the system that solved it won't have problem X. That's not very informative without some idea of how likely it is that X can or will be solv…

> That's not very informative without some idea of how likely it is that X can or will be solved

Exactly, the authors argument would be much better qualified by addressing this assumption.

> current AI is a better prior than "something will happen".

“Current AI” is not a prior, its a static observation.

Re: How does misalignment scale with model intelligence and task complexity?

#83
post #77

Earlier quoted context omitted.

Maybe? They should speak more clearly regardless, so we don't have to speculate over it. The way you worded it is much more understandable.

There wasn't much room to speculate really, but requires some knowledge of understanding problem spaces, topology, and things like minima and maxima.

"inaccessible" rather than "ambiguous" -- but to the uninitiated they are hard to tell apart.
Post reply on HN