The findings are based on older models and assuming recent models behave similarly, what kind of prompt style one should use then to improve the outcome to avoid the increase in variance especially when you ask a model to solve really complex problems?
How does misalignment scale with model intelligence and task complexity?
41–50 of 84 posts
Re: How does misalignment scale with model intelligence and task complexity?
#42[flagged]
Re: How does misalignment scale with model intelligence and task complexity?
#43I feel vindicated when I say that the superintelligence control problem is a total farce, we won't get to superintelligence, it's tantamount to a religious belief. The real problem is the billionaire control problem. The human-race-on-earth control problem.
Re: How does misalignment scale with model intelligence and task complexity?
#44I feel vindicated when I say that the superintelligence control problem is a total farce, we won't get to superintelligence, it's tantamount to a religious belief. The real problem is the billionaire control problem. The human-race-on-earth control problem.
I don’t believe the article makes any claims on the infeasibility of a future ASI. It just explores likely failure modes.
It is fine to be worried about both alignment risks and economic inequality. The world is complex, there are many problems all at once, we don’t have to promote one at the cost of the other.
Re: How does misalignment scale with model intelligence and task complexity?
#45I feel vindicated when I say that the superintelligence control problem is a total farce, we won't get to superintelligence, it's tantamount to a religious belief. The real problem is the billionaire control problem. The human-race-on-earth control problem.
Yeah article aside, looking back on all the AGI stuff from the last year or so really puts our current moment in protective.
This whole paradigm of AI research is cool and all but it's ultimately a simple machine that probabilistically forms text. It's really good at making stuff that sounds smart but like looking at an AI picture, it falls apart the harder you look at it. It's good at producing stuff that looks like code and often kinda works but based on the other comments in this thread I don't think people really grasp how these models work.
Re: How does misalignment scale with model intelligence and task complexity?
#46> Making models larger improves overall accuracy but doesn't reliably reduce incoherence on hard problems. Coherence requires 2 opposing forces to hold coherence in one dimension and at least 3 of them in higher dimensions of quality. My team wrote up a paper titled "If You Want Coherence, Orchestrate a Team of Rivals"[1] because we kept finding that upping the reasoning threshold resulted in less coherence - more ex…
> Coherence requires 2 opposing forces This seems very basic to any kind of information processing beyond straight shot predictable transforms. Expansion and reduction of possibilities, branches, scope, etc. Biological and artificial neural networks converging into multiple signals, that are reduced by competition between them. Scientific theorizing, followed by experimental testing. Evolutionary genetic recombinatio…
> This seems very basic
Yes, this is not some sort of hard-fought wisdom.
It should be common sense, but I still see a lot of experiments which measure the sound of one hand clapping.
In some sense, it is a product of laziness to automate human supervision with more agents, but on the other hand I can't argue with the results.
If you don't really want the experiments and data from the academic paper, we have a white paper which is completely obvious to anyone who's read High Output Management, Mythical Man Month and Philosophy of Software Design recently.
Nothing in there is new, except the field it is applied to has no humans left.
Re: How does misalignment scale with model intelligence and task complexity?
#47Re: How does misalignment scale with model intelligence and task complexity?
#48"model failures become increasingly dominated by incoherence rather than systematic misalignment."
This should not be surprising.
Systematic misalignment, i.e., bias, is still coherent and rational, if it is to be systematic. This would require that AI reason, but AI does not reason (let alone think), it does not do inference.
Re: How does misalignment scale with model intelligence and task complexity?
#49This is a good line: "It found that smarter entities are subjectively judged to behave less coherently" I think this is twofold: 1. Advanced intelligence requires the ability to traverse between domain valleys in the cognitive manifold. Be it via temperature or some fancy tunneling technique, it's going to be higher error (less coherent) in the valleys of the manifold than naive gradient following to the local minima…
> the ability to traverse between domain valleys in the cognitive manifold.
Couldn't you have just said "know about a lot of different fields"? Was your comment sarcastic or do you actually talk like that?
Re: How does misalignment scale with model intelligence and task complexity?
#50When humans dream, we are disconnected from the world around us. Without the grounding that comes from being connected to our bodies, anything can happen in a dream. It is no surprise that models need grounding too, lest their outputs be no more useful than dreams. It’s us engineers who give arms and legs to models, so they can navigate the world and succeed at their tasks.
Also since dreams are built from the combinations of experiences that brain already knows so we cannot die in a dream as our brain does not know how to replicate what it would feel like after being dead. Basically LLMs also cannot produce truly novel ideas.