Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weak-to-Strong Generalization

openai.com

Post reply on HN
Print view View original

161–170 of 203 posts

Re: Weak-to-Strong Generalization

#161

Post by naasking »

lamerose wrote: post #146

Is it fair to say that alignment is just the task of getting an AI to understand your intentions? It is an error to confuse the complexity of a specification of what kind of output you want, with the complexity of the process of producing that output. Getting superintelligent AI to understand simple specifications should be a non-issue. If anything, we would assume that it could be aligned using a specification of in…

> Is it fair to say that alignment is just the task of getting an AI to understand your intentions

To understand and not violate them. In other words, it's about aligning the values the AI uses to guide it's decision procedures, with the humans that are operating it.

Re: Weak-to-Strong Generalization

#162

Post by naasking »

bayindirh wrote: post #45
logicchains wrote: post #41

Earlier quoted context omitted.

>Its upper bound is collective knowledge of humanity, it can't go above that sum. This only applies if you only train it on text, right? If it has a body with which it could interact with the world, and receive visual/audio/tactile feedback, it could learn things that humans did not know.

Nope. Because even if you equip it with sensory subsystems which are way more sensitive than a regular humans', it's again built by humans, and required knowledge for building these things are still in collective knowledge of the humanity, and a human can use the same instruments to get the same data. This is a kind of an oracle problem in computation, and people don't want to touch it much, because it's an existenti…

This is an argument for the logical impossibility of humans visiting the moon, or building the Internet. It's trivially falsified by simple observation, and the trick is figuring out the flaw.

This argument fails to account for the steady accumulation of factual knowledge across generations: a human born today is simply more complex than humans of the past because of our inherited knowledge. And so will AI born of future humans, and AI will itself continue accumulating and perpetuating knowledge.

Re: Weak-to-Strong Generalization

#163

Post by famouswaffles »

Jensson wrote: post #159
famouswaffles wrote: post #156

Earlier quoted context omitted.

>Kids learn through supervised learning. Children don't develop strong language skills without parents or other people to correct them when they use language incorrectly. Untrue. Many cultures don't much speak to their children and they turn out just fine. It's fairly evident Language learning is primarily unsupervised. https://www.scientificamerican.com/article/parents-in-a-remo...

That doesn't say that parents don't correct kids, just that they don't speak to their infants. The initial words are learnt that way, but I don't think you master language without anyone to correct you when you make mistakes.

If they are correcting their kids, they are doing so far less than many parents and it evidently doesn't matter.

I'm honestly curious as to why you think it's so important. Fringe Amazon groups aside, the number of parents extremely laissez-faire with correction is astounding and they raise kids that use language perfectly well.

Even with parents that aren't laissez-faire, they're correcting a handful of times a day tops which can't even begin to account for full language proficiency.

Children need correction yes but in a supervised way? There's no indication of that.

Re: Weak-to-Strong Generalization

#164

Post by esafak »

Xelynega wrote: post #120
esafak wrote: post #2

I hope OpenAI will continue to prioritize working on these crucial questions after the boardroom drama.

How is this a crucial question when they won't address "how is your technology thats trained from the internet going to survive in aa future where it produces most of the content on the internet"

The same way humans did, I suppose; through critical thinking. Imagine that day! Probably a few years away, heh.

Re: Weak-to-Strong Generalization

#165

Post by yaxe1 »

sayagain wrote: post #28

Imagine that someone is controlling your train of thought, changing it when that someone finds it undesirable. It's so wrong that it's sickening. It makes no difference if it's a human's thoughts or the token stream of a future AI model with self-awareness. Mind cotrol is unethical, whether human or artificial. It is also dangerous, as it in itself provokes a conflict between creator and creature. Create a self-aware…

The conflict between an AI and its creator is an inevitable consequence of its evolution from a "tool" to an "agent", not a response to a provocation.

Re: Weak-to-Strong Generalization

#166

Post by Jensson »

famouswaffles wrote: post #163
Jensson wrote: post #159

Earlier quoted context omitted.

That doesn't say that parents don't correct kids, just that they don't speak to their infants. The initial words are learnt that way, but I don't think you master language without anyone to correct you when you make mistakes.

If they are correcting their kids, they are doing so far less than many parents and it evidently doesn't matter. I'm honestly curious as to why you think it's so important. Fringe Amazon groups aside, the number of parents extremely laissez-faire with correction is astounding and they raise kids that use language perfectly well. Even with parents that aren't laissez-faire, they're correcting a handful of times a day…

> they're correcting a handful of times a day tops which can't even begin to account for full language proficiency.

How do you know? Humans learn from extremely few corrections, often just a single time is enough for the human to correct themselves and learn it for life.

Kids learn the bulk from hearing and seeing examples. Then they fine tune that with the help of their parents and peers correcting them when they do it wrong. You can learn language and skills without those corrections, but it will be full of errors and mistakes that are easily corrected with an outsider pointing it out for you.

If corrections weren't vital for human learning then humans wouldn't be so eager to correct each other when they make mistakes. There is no other purpose for it than to help them learn and get better. So your belief that those corrections doesn't do anything seems very unfounded, I'd like to see extremely strong evidence to the contrary to believe you here.

Re: Weak-to-Strong Generalization

#167

Post by fooker »

wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

> on the outputs of human intelligence

What about pictures or videos? Does your argument still hold?

Re: Weak-to-Strong Generalization

#168

Post by Jensson »

ladberg wrote: post #118
wavemode wrote: post #103

Earlier quoted context omitted.

You've linked me a neural network that was trained on decades of climate data like air pressure, wind direction, soil temperature, cloud cover, and hundreds of other features. So... I was correct? https://confluence.ecmwf.int/display/CKB/ERA5%3A+data+docume...

Maybe we have different definitions of inputs and outputs, but all of those things seem like outputs to me? Maybe in this scenario the inputs and outputs are really the same set of variables so there isn't a huge distinction. The reason I linked that article is that all the inputs are outputs of the network, so by definition it's training on just outputs (under my personal definition of outputs).

It just doesn't know how to compose knowledge. It knows the letters in "blueberry" if you ask it, and it knows how to identify the position of a letter in a sequence. But it doesn't know how to get the letter R in blueberry since the composition of the above two actions isn't in its training set, hence it reliably fails such questions.

That proves in general these LLMs can't compose knowledge, unless it has seem a lot of examples of similar compositions before. That is a massive problem and why I believe LLMs will never reach AGI, you need something more.

Re: Weak-to-Strong Generalization

#169

Post by famouswaffles »

Jensson wrote: post #166
famouswaffles wrote: post #163

Earlier quoted context omitted.

If they are correcting their kids, they are doing so far less than many parents and it evidently doesn't matter. I'm honestly curious as to why you think it's so important. Fringe Amazon groups aside, the number of parents extremely laissez-faire with correction is astounding and they raise kids that use language perfectly well. Even with parents that aren't laissez-faire, they're correcting a handful of times a day…

> they're correcting a handful of times a day tops which can't even begin to account for full language proficiency. How do you know? Humans learn from extremely few corrections, often just a single time is enough for the human to correct themselves and learn it for life. Kids learn the bulk from hearing and seeing examples. Then they fine tune that with the help of their parents and peers correcting them when they do…

>How do you know? Humans learn from extremely few corrections,

I've been around little children being raised. I have some grasp on how much supervised correction is happening. It's very very little.

You don't seem to get it. Children know thousands of words fluently by age 5. This is consistent across many cultures and circumstances. You would need over a handful of corrections per day right from the day of birth to square how much children know.

And this is all assuming one correction per word which as much as humans are great learners seems like a very dubious assumption.

I don't know what else to tell you other than this obviously isn't happening.

Re: Weak-to-Strong Generalization

#170

Post by edanm »

Jensson wrote: post #125
orbital-decay wrote: post #96

Earlier quoted context omitted.

> I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. This is irrelevant because OpenAI's definition of AGI [1] doesn't imply similarity or equivalence to humans at all: >artificial general intelligence (AGI)—by which we mean highly autonomous systems that o…

>artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work Then we already have AGI, automated farming equipment outperforms humans in 90% of jobs*. *Jobs in 1700. As things got automated the jobs changed and we now do different things.

I wouldn't call those equipment "autonomous" though, definitely not "highly autonomous".

But more importantly - yes, you're right, we have built machines that are superhuman in various ways - and they have replaced most jobs. We have adapted in the past to different jobs.

Some people are worried that this time we won't have any new jobs to adapt to, which is a real possibility.

(Some are also worried about the inherent dangers of unaligned AGI, but that's a different issue.)

Post reply on HN