Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weak-to-Strong Generalization

openai.com

Post reply on HN
Print view View original

91–100 of 203 posts

Re: Weak-to-Strong Generalization

#91

Post by wavemode »

panarky wrote: post #86
wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. Think about the process of a child learning a language. The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of…

> The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of others around them.

I would argue that that learning is always contextualized by visual and spatial information about the real world (which is what our language is meant to describe). And (equally importantly) the child gets real-world feedback on their decisions - some decisions achieve their desired goal and some don't. Some statements cause certain responses from people, some actions have certain consequences.

> Moreover, the analogy to weather prediction or stock market analysis is somewhat misleading. Yes, these models benefit from input data (like air currents for weather, company fundamentals and CEO statements to the media for stocks). But these systems are fundamentally different from intelligence.

> Intelligence, whether artificial or human, is about the ability to learn, adapt, and generate novel responses in a broad range of scenarios, not just about predicting specific outcomes based on specific inputs.

"Intelligence" is kind of a nebulous term. If all it means is the ability to learn, adapt and generate novel responses, then sure, I think we could call almost any neural network intelligent.

But I would argue that we usually do have some expectation that an intelligent system can produce "specific outcomes based on specific inputs". We want to be able to train a worker and have them follow that training so they do their job correctly.

Re: Weak-to-Strong Generalization

#92

Post by wavemode »

famouswaffles wrote: post #87
wavemode wrote: post #84

Earlier quoted context omitted.

> Being "casually predictive" does not mean you have provided all the variables of your prediction in the data. Not sure where I claimed this. > Protein creation is not just _poof_ new proteins. Not sure where I claimed this either. > There are steps and interactions and you don't need to train on all of that. I agree with this statement as well. Have you read what I wrote? Proteins in chains can indeed be used to pr…

Well first you say, >I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. Now you say >I'm using the definition provided verbatim in the linked article: "We believe superintelligence—AI vastly smarter than humans—could be developed within the next ten years."…

Calm down, buddy. Read what I wrote a bit more charitably rather than trying to score points.

Obviously, a prerequisite to becoming more intelligent than a human, is to become equally intelligent as a human. I don't believe LLM's will ever be equal in intelligence to humans, ergo I also don't believe they will become superior in intelligence to human (which is how the linked article defines "AGI").

Re: Weak-to-Strong Generalization

#93

Post by cma »

refulgentis wrote: post #63
JZL003 wrote: post #33

This reminds me of a thing cory doctorow talks about how tech companies control the narrative to focus on fun sexy problems while they have fundamental problems which expose the lie. For example uber/self driving cars always talking about the trolley problem, as if the current (or near future) problem is that self-driving cars are so good they have to choose which one. Not the current very difficult problem of gettin…

After some meditation, I don't find this line of inquiry to bear fruit: I don't recall any entity, nor the entities named (Uber / self-driving cars) talking about the trolley problem - that's a well-known thought experiment in philosophy, but not something covered as a stark binary choice in self-driving cars planner systems. I also don't recall traffic cones being a very difficult problem beyond Cruise + cones on wi…

> I also don't recall traffic cones being a very difficult problem beyond Cruise + cones on windshield in SF. I have no love for Cruise. But its straightforward to pause if there's a large object on the windshield.

Waymo had a big mistake with cones where a lane was blocked off for construction and they thought it was the inverse and started driving down the blocked off side:

https://www.youtube.com/watch?v=B4O9QfUE5uI

full video: https://www.youtube.com/watch?v=zdKCQKBvH-A&t=12m24s

In the statement before that timestamp they say some of it was caused by remote operator error. But it seems to have enough problems with cones to need an operator in the first place, and you can see the planner is wrong.

I think Cruise would be using a human or at least summoning one to oversee in any situation with unexpected cones, based on what they have said about how often they use remote assistance too.

Re: Weak-to-Strong Generalization

#94

Post by nuancebydefault »

logicchains wrote: post #10

>We believe superintelligence—AI vastly smarter than humans—could be developed within the next ten years. However, we still do not know how to reliably steer and control superhuman AI systems Their entire premise is contradictory. An AI incapable of critical thinking cannot be smarter than a human, by definition, as critical thinking is a key component of intelligence. And an AI that is at least as capable of critica…

> critical thinking

I believe _critical_ thinking is just some abstract term invented by humans to describe something not understood by humans. In other words, I'm not convinced _critical_ thinking exists, just like I am not convinced ghosts exists.

Hence, to me relating intelligence with critical thinking does not give any additional insights.

Re: Weak-to-Strong Generalization

#95

Post by oersted »

akprasad wrote: post #14

This method assumes that the weaker model is aligned. I'm curious how the paper addresses that point. > "But what does this second turtle stand on?" persisted James patiently. > To this, the little old lady crowed triumphantly, > "It's no use, Mr. James—it's turtles all the way down."

The weaker model is just a stand-in for human supervisors.

They are experimenting on GPT-2 supervising GPT-4 as an analogue to humans supervising the superhuman AGI.

Re: Weak-to-Strong Generalization

#96

Post by orbital-decay »

wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

> I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence.

This is irrelevant because OpenAI's definition of AGI [1] doesn't imply similarity or equivalence to humans at all:

>artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work

I.e. the stated goal of this company is to put humans out of work and become the censors and gatekeepers, not to produce something human-like.

>You can't model and predict the stock market just by training on the outputs of stock trading decisions (the high today, the low yesterday). You have to train on the inputs (company fundamentals, earnings, market sentiments in the news, etc.)

Most of your intelligence is not actually yours. It's social in nature, obtained by distillation of generations' worth of experience, simplified and passed to you through the stored knowledge. Which is, coincidentally, what the models are being trained on.

[1] https://openai.com/charter

Re: Weak-to-Strong Generalization

#97

Post by ladberg »

wavemode wrote: post #85
naasking wrote: post #78

Earlier quoted context omitted.

> You can't model and predict the weather just by training on the outputs of the weather system Then how did we develop predictive systems just by observing those outputs?

We didn't.

https://deepmind.google/discover/blog/graphcast-ai-model-for...

Re: Weak-to-Strong Generalization

#98

Post by famouswaffles »

wavemode wrote: post #92
famouswaffles wrote: post #87

Earlier quoted context omitted.

Well first you say, >I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. Now you say >I'm using the definition provided verbatim in the linked article: "We believe superintelligence—AI vastly smarter than humans—could be developed within the next ten years."…

Calm down, buddy. Read what I wrote a bit more charitably rather than trying to score points. Obviously, a prerequisite to becoming more intelligent than a human, is to become equally intelligent as a human. I don't believe LLM's will ever be equal in intelligence to humans, ergo I also don't believe they will become superior in intelligence to human (which is how the linked article defines "AGI").

>Calm down, buddy

I'm not agitated.

>Obviously, a prerequisite to becoming more intelligent than a human, is to become equally intelligent as a human. I don't believe LLM's will ever be equal in intelligence to humans, ergo I also don't believe they will become superior in intelligence to human (which is how the linked article defines "AGI").

You have still not answered my question. saying "equivalent to human intelligence" is easy. The real question is equivalent how ? What are you saying it needs to do ? or is this just a vague "i'll know it when i see it" assertion ?

If it's so easy to say "GPT-4 is not agi" then it should be very easy to say what it can't do that disqualifies it

what is it that you're looking for it to do to be called one ?

Re: Weak-to-Strong Generalization

#99

Post by orbital-decay »

naasking wrote: post #78
wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

> You can't model and predict the weather just by training on the outputs of the weather system Then how did we develop predictive systems just by observing those outputs?

This kind of prediction has obvious limitations - for example it cannot reverse the behavior of chaotic systems

Re: Weak-to-Strong Generalization

#100

Post by novaRom »

JZL003 wrote: post #33

This reminds me of a thing cory doctorow talks about how tech companies control the narrative to focus on fun sexy problems while they have fundamental problems which expose the lie. For example uber/self driving cars always talking about the trolley problem, as if the current (or near future) problem is that self-driving cars are so good they have to choose which one. Not the current very difficult problem of gettin…

OpenAI do probably realize they will not win long term vs Open Source (see AI Alliance). Their way of centralized cloud models is simply too risky and not sustainable. What we see instead is more liberation, open source, cooperation, down-scaling, local models. Just look how many more tools and models is available today than even a year ago. And where is OpenAI? Still the same chatGPT, still the same DALL-E, nothing new.
Post reply on HN