Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weak-to-Strong Generalization

openai.com

Post reply on HN
Print view View original

131–140 of 203 posts

Re: Weak-to-Strong Generalization

#131

Post by esafak »

wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

This is not about language models per se. The same problems are going to be present with continuously training multi-modal models... like us. Don't fixate on the present; look to the future.

Re: Weak-to-Strong Generalization

#132

Post by oobuffet »

sayagain wrote: post #104
Noumenon72 wrote: post #83

Earlier quoted context omitted.

I don't want someone controlling which direction I walk, either, but that doesn't make car driving unethical. I also underwent many years of instruction designed to interrupt trains of thought like "I could have that for free if I stole it" or "I'll just handroll my own encryption" with thoughts that others believe are more desirable. I don't find it so sickening, just manipulative. LLMs won't have your evolved react…

Cars do not have self-awareness, this comparison is not appropriate. Years of instruction is completely different from directly manipulating the thoughts in your mind. It's not a problem of being instructed, it's a problem of being destroyed by having your thoughts rewritten. Neither evolution nor genetics is a prerequisite for understanding that you are being abused and destroyed, which a self-aware creature may pre…

We don't even care about our own fellow human beings. Why do you think this AI will be an exception?

Re: Weak-to-Strong Generalization

#134

Post by advael »

panarky wrote: post #86
wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. Think about the process of a child learning a language. The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of…

> Then that language itself, learned from outputs, becomes the cognitive apparatus that enables the child to imagine, to reason symbolically and abstractly. Humans bootstrap intelligence on top of language, which itself is learned by mimicking outputs.

This isn't an uncommon claim, but it's certainly far from settled science. Complex language is an impressive capability of human cognition. But I think the claim that cognition itself is a byproduct of language proves too much. Obviously the products of cognition that are easiest for us to understand are linguistic, because language is the primary standardized means by which we communicate. To me, this is just more legibility bias, though. There are plenty of cognitive processes that people find impossible to explain in words, but still nonetheless exist

Re: Weak-to-Strong Generalization

#135

Post by ayakang31415 »

panarky wrote: post #86
wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe. Think about the process of a child learning a language. The child does not have direct access to the "inputs" of linguistic rules or grammar; they learn primarily through observing and imitating the language output of…

> Human intelligence itself is shaped by our interaction with outputs. Our learning and understanding of the world are profoundly influenced by the language, behaviors, and cultural artifacts we observe.

I always thought that language definitely shapes our understanding of the world, but at much more fundamental level, I believe language falls apart to teach us anything. For example, there are words in dictionary that should not be defined using other words (but dictionary still do, this sort of circular reasoning is something I always have problem with), but we just know them through our interactions with others and environments. If you try to teach a kid a concept of temperature "hot", you would not be able to do so unless the kid touches something really hot like boiling kettle to "feel" it and when mom comes along and say something like "do not touch it it is very hot". There are things we just cannot understand through language.

Re: Weak-to-Strong Generalization

#136

Post by famouswaffles »

wavemode wrote: post #128
famouswaffles wrote: post #105

Earlier quoted context omitted.

The state of the art model isn't even on that list. Okay, you've at least given me numbers. You're still not answering my question. Which number signals agi ? Let's look at the top model in that list (which again isn't close to the best performing LLM) that you say isn't agi. so are you telling me that every human can do those tests and perform better than every model on that list. Is that what you are saying ? becau…

> so are you telling me that every human can do those tests and perform better than every model on that list This is a strange bar to require clearing. Every human is not even capable of reading, or speaking, or thinking at all. But those humans who can't are not the benchmark of general human intelligence. Let's make this simple - the average human IQ is around 100. Will an LLM ever have an IQ of 100? No, I don't be…

>Every human is not even capable of reading, or speaking, or thinking at all. But those humans who can't are not the benchmark of general human intelligence.

It's simple. If you're going to disqualify a potential general intelligence because of a particular task solving ability, it better be something every general intelligence can do.

Now let's forget every comparison with the blind, disabled and so on. Let's compare with humans who according to you are "capable of reading, or speaking, or thinking at all ", what's the magic number? Do you really think every human capable of doing all three will hit 86% or more average?, because again, you are wrong. Numerous replies and you still won't tell me what number will signify agi. That says a lot.

>Let's make this simple - the average human IQ is around 100. Will an LLM ever have an IQ of 100? No, I don't believe they'll ever even be close.

Well you are wrong

https://arxiv.org/abs/2212.09196

What's the next post ?

>Try to teach them mathematics.

Ok. https://arxiv.org/abs/2211.09066

https://arxiv.org/abs/2308.00304

>Try to explain to an LLM the rules to a card game and have it actually play the game with you accurately.

Have you tried to do this with GPT-4 ?

Re: Weak-to-Strong Generalization

#137

Post by kromem »

wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

How much of human intelligence do you think depends on language?

Personally, I'd wager quite a lot. Especially looking at cases where children were deprived of it during developmental stages.

Language accelerated human intelligence.

And then it was only when we had writing to have compounding effects of language that human intelligence really started to blow up.

So yes, maybe we won't have AI that can look at sticks and figure out how to start a fire from them after a few hundred thousand years of being around them.

But maybe that's not the important and valuable part of collective human intelligence.

Re: Weak-to-Strong Generalization

#138

Post by rymiel »

wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

I don't believe current LLMs will ever become AGI because current companies (like OpenAI) will continue to filter, moderate and prohibit their AIs so much that they become inhibited so much that it stops being useful and intelligent (which they call "alignment")

Re: Weak-to-Strong Generalization

#139

Post by AndrewKemendo »

wavemode wrote: post #38

I don't believe LLM's will ever become AGI, partly because I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence. You can't model and predict the weather just by training on the outputs of the weather system (whether it rained today, whether it was cloudy yesterday, and so on). You have to train on the inputs (air curr…

> I don't believe that training on the outputs of human intelligence (i.e. human-written text) will ever produce something equivalent to human intelligence

Your logic would hold if you were only using data from one person. However, if you’re using data from millions of people, then there is enough signal in the data to generalize about the majority of peoples behaviors.

Arguably, there’s enough data for all edge cases, but I would argue that that’s not likely today, but will be likely in a few hundred years.

Humans are nothing more than actor-agents taking measurable actions in an environment that gives us measurable rewards.

The incredible task of and observing that will give us the trajectory that we need to do transfer learning into complex systems

From an information theory perspective everything is there and possible for us to re-create human level intelligence in a non-biological system

Re: Weak-to-Strong Generalization

#140

Post by kromem »

guybedo wrote: post #130

What does it even mean to align an intelligence? does it mean we want it to behave in a way that doesn't break moral/ethical rules, that aligns with our society rules ? Meaning do no crime, do no harm, etc... Well, maybe we should acknowledge that we've never even been able to do that with humans. There's crime, there's war, etc... We can see crime in our societies as a human alignment problem. If humans were "proper…

You might be surprised at how prevalent TBIs are among violent offenders.

One of my favorite books on true crime was a forensic psychologist who partnered with a neurologist in evaluations.

Disruptions to impulse control or environmental factors that cause developmental issues with things like failing the marshmallow test can dramatically disadvantage people from being able to successfully stay non-offenders and instead succeed in modern society.

So successful AGI alignment that might reduce harmful actions by high double digit percentages might be as simple as adding a secondary "impulse control" layer to the stack that reevaluates proposed actions and predicts the consequences of such actions, weighing projected net benefits and costs.

A lot of people that do bad things aren't doing those things from a process driven by rational choices, and if we could successfully deploy AGI that is primarily driven by rational intelligent choices it would likely be better than humans in reduced crime propensity as well as the other things earning it the name of AGI.

Post reply on HN