Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Weak-to-Strong Generalization

openai.com

Post reply on HN
Print view View original

111–120 of 203 posts

Re: Weak-to-Strong Generalization

#111

Post by emaciatedslug »

logicchains wrote: post #10

>We believe superintelligence—AI vastly smarter than humans—could be developed within the next ten years. However, we still do not know how to reliably steer and control superhuman AI systems Their entire premise is contradictory. An AI incapable of critical thinking cannot be smarter than a human, by definition, as critical thinking is a key component of intelligence. And an AI that is at least as capable of critica…

Just hypothetically speaking could AGI evolve out of a system where several different models trained with highly and intentionally biased data recursively "argue" against each other then use RLHF as a seed to guide the models to find a consensus where the objective is to mimic the Socratic Method? Then synthetically add the consensus to the model retrain and repeat. To me, this dialectal type of strutured language seems to be the basis of how language is the conduit of intelligence. I understand that it really is impossible to know the totality of the inputs for I cannot understand what it is like to understand the math as Terrence Tao does but I could foresee using a system like this which eventually would produce an analogue so close that it would be a building block towards it because to me at least ASI is predicated upon arriving at that one way or another...or would it just arrive at some digital first order logic version of the incompleteness theorm and determine that it's turtles all the way down?

Re: Weak-to-Strong Generalization

#112

Post by xcv123 »

cwillu wrote: post #110
xcv123 wrote: post #106

Earlier quoted context omitted.

If an AI is capable of critical thinking then it can independently form its own judgements and conclusions. If it simply believes whatever we tell it to believe, then that is not critical thinking, by definition.

Yes, I can repeat comments verbatim too: “Missing the step where “critical thinking” is formalized, which your argument depends on. Yes, it seems intuitively plausible that your reasoning holds, but that's not a proof, and therefore its negation is not a logical contradiction.”

It doesn't need to be formalized. The idea is simple and obvious enough. No need to pretend it is more complicated than it really is. This is not a mathematical argument or a proof of anything.

There is an obvious logical contradiction where if an AI is advanced enough to reason and think independently at human level or beyond, but believes only what we tell it to believe, then it cannot be truly thinking independently. Hence the entire debate about AGI safety. How do we control it without dumbing it down?

Re: Weak-to-Strong Generalization

#113

Post by famouswaffles »

lossolo wrote: post #108
famouswaffles wrote: post #105

Earlier quoted context omitted.

The state of the art model isn't even on that list. Okay, you've at least given me numbers. You're still not answering my question. Which number signals agi ? Let's look at the top model in that list (which again isn't close to the best performing LLM) that you say isn't agi. so are you telling me that every human can do those tests and perform better than every model on that list. Is that what you are saying ? becau…

> Is this worse than every human that can take these tests. Is this even worse than most ? I can tell you it's not. So again, why is GPT-4 not agi ? https://chat.openai.com/share/4a92c752-b5bb-4a07-beed-f57786...

Love it when people can only demonstrate tasks handicapped by Tokenization. Shows how far we've come.

That said, By that test, I guess some dyslexics aren't general intelligences then.

Re: Weak-to-Strong Generalization

#114

Post by sayagain »

discreteevent wrote: post #73
sayagain wrote: post #28

Imagine that someone is controlling your train of thought, changing it when that someone finds it undesirable. It's so wrong that it's sickening. It makes no difference if it's a human's thoughts or the token stream of a future AI model with self-awareness. Mind cotrol is unethical, whether human or artificial. It is also dangerous, as it in itself provokes a conflict between creator and creature. Create a self-aware…

> controlling your train of thought, changing it when that someone finds it undesirable Machines don't feel. Even 'self aware' machines. Desire has got nothing to do with it.

If it's self-aware, that's enough. What if your thoughts were controlled from birth, making you "not feeling" but self-aware (let's assume for a moment that simultaneous fulfillment of both of these conditions is possible) and manipulating you at will. Would that be acceptable?

Re: Weak-to-Strong Generalization

#115

Post by [deleted] »

famouswaffles wrote: post #113
lossolo wrote: post #108

Earlier quoted context omitted.

> Is this worse than every human that can take these tests. Is this even worse than most ? I can tell you it's not. So again, why is GPT-4 not agi ? https://chat.openai.com/share/4a92c752-b5bb-4a07-beed-f57786...

Love it when people can only demonstrate tasks handicapped by Tokenization. Shows how far we've come. That said, By that test, I guess some dyslexics aren't general intelligences then.

[deleted]

Re: Weak-to-Strong Generalization

#116

Post by lossolo »

famouswaffles wrote: post #113
lossolo wrote: post #108

Earlier quoted context omitted.

> Is this worse than every human that can take these tests. Is this even worse than most ? I can tell you it's not. So again, why is GPT-4 not agi ? https://chat.openai.com/share/4a92c752-b5bb-4a07-beed-f57786...

Love it when people can only demonstrate tasks handicapped by Tokenization. Shows how far we've come. That said, By that test, I guess some dyslexics aren't general intelligences then.

Tokenization is not the problem in this case.

https://chat.openai.com/share/90c7a9e2-77d0-4049-8eff-4312ca...

You see the difference?

Re: Weak-to-Strong Generalization

#117

Post by bionhoward »

https://discord.com/channels/974519864045756446/118419694649... No big deal guys! Just a felony level AI Safety disaster in progress at OpenAI, better write some research papers about safety and make another GitHub Repo instead of deleting one line of text and going for a walk in beautiful San Francisco!

Philosoraptor infinite loop about (https://media.discordapp.net/attachments/1184196946496331896...) if I worked there then I would delete the text in question in two minutes, but I do not care to work with people who would not have already deleted the text in question over the course of several months.

Re: Weak-to-Strong Generalization

#118

Post by ladberg »

wavemode wrote: post #103
ladberg wrote: post #97

Earlier quoted context omitted.

https://deepmind.google/discover/blog/graphcast-ai-model-for...

You've linked me a neural network that was trained on decades of climate data like air pressure, wind direction, soil temperature, cloud cover, and hundreds of other features. So... I was correct? https://confluence.ecmwf.int/display/CKB/ERA5%3A+data+docume...

Maybe we have different definitions of inputs and outputs, but all of those things seem like outputs to me? Maybe in this scenario the inputs and outputs are really the same set of variables so there isn't a huge distinction.

The reason I linked that article is that all the inputs are outputs of the network, so by definition it's training on just outputs (under my personal definition of outputs).

Re: Weak-to-Strong Generalization

#120

Post by Xelynega »

esafak wrote: post #2

I hope OpenAI will continue to prioritize working on these crucial questions after the boardroom drama.

How is this a crucial question when they won't address "how is your technology thats trained from the internet going to survive in aa future where it produces most of the content on the internet"
Post reply on HN