Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google DeepMind CEO says some form of AGI possible in a few years

wsj.com

Post reply on HN
Print view View original

251–260 of 388 posts

Re: Google DeepMind CEO says some form of AGI possible in a few years

#251

Post by bondarchuk »

riffraff wrote: post #166
bsaul wrote: post #118

Earlier quoted context omitted.

« I don't believe we've seen any evidence of this. » I think chatgpt did show proof of what was considered at least until very recently « intelligence ». aka: understand enough about context and concepts to provide relevant answers to very complex and open questions.

I think this is the culprit: ChatGPT does not understand , as proven by hallucinations, by being wrong on certain questions or puzzle etc.. It just seems to understand. This is useful, and deeply impressive, but it's not the same thing.

You seem to be saying that because it misunderstands certain things, everything else that is evidence of correct understanding also doesn't count as real "understanding". I don't see how this is a coherent position to hold, as we can just as easily apply it to humans who have misunderstood something.

Edit: and I also don't see why it has to be so black-and-white. IMHO there is no problem with saying it understands certain things, and doesn't understand other things. We are talking about general intelligence, not god-like omniscience.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#252

Post by ddalex »

andrepd wrote: post #210
dr_dshiv wrote: post #199

Earlier quoted context omitted.

It’s terrible and math and logic, but ChatGPT is amazing at concepts—that’s why it is so powerful. It doesn’t work programmatically—that’s why it fails at logic. But it can reason inductively very very well. Do you have an example besides logic/math where it doesn’t understand simple concepts?

What do you mean exactly "besides math/logic"? Because logic underpins everything we think. For example it cannot identify musical chords because despite (I presume) ample training material including explanations of how exactly this works, it cannot reasonable represent this as an abstract rigorous rule, as humans do. So I ask what is C E G and it tells me C major correctly, as it presumably appears many times throug…

> logic underpins everything we think.

I hate to break it to you, but humans aren't thinking logically, or exclusively logically. In fact I would say that humans are not using logic most of the time, and we go by intuition most of our lives (intuition is shorthand for experience, pattern matching and extrapolation). There is a reason of why we teach formal logic in certain schools....

Re: Google DeepMind CEO says some form of AGI possible in a few years

#253

Post by AnimalMuppet »

dr_dshiv wrote: post #199
andrepd wrote: post #171

Earlier quoted context omitted.

> ChatGPT does this very well Lol, this is precisely what chatgpt doesn't do well at all! It fails to understand simple concepts, logical inference, simple math, etc.

It’s terrible and math and logic, but ChatGPT is amazing at concepts—that’s why it is so powerful. It doesn’t work programmatically—that’s why it fails at logic. But it can reason inductively very very well. Do you have an example besides logic/math where it doesn’t understand simple concepts?

No, it's amazing at words. Humans use words to encode concepts, but ChatGPT doesn't get the concepts at all - just words and their relationships.

To the extent that humans have encoded the concepts into words, and that text is in the training set, to that degree ChatGPT can work with the words in a way that is at least somewhat true to the concepts encoded in them. But it doesn't actually understand any of the concepts - just words and their relationships.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#254

Post by pflanze »

Xcelerate wrote: post #154
paxys wrote: post #16

I predict that the fight will be more about defining intelligence than inventing it. No one knows what AGI is. There isn't going to be some switch that flips to take us from AI to AGI. These tools we have today will just keep getting incrementally better, and some new ones will pop up, and at some point we'll have to stop and say "yeah, this is good enough to qualify". And everyone will have their own opinion on what…

> I predict that the fight will be more about defining intelligence than inventing it. I think the internet has given a pretty clear definition on what intelligence is: it’s whatever AI can’t do yet.

And when we run out of clues what AI can't do yet, it's AGI?

(You imply that a "can't do yet" will remain forever, which is the open question. If you ask me, AGI is only possible if the tech has ~unlimited agency, which implies control over computer and energy production facilities.)

Re: Google DeepMind CEO says some form of AGI possible in a few years

#255

Post by VHRanger »

ogogmad wrote: post #247
VHRanger wrote: post #221

Earlier quoted context omitted.

You can even do it in a linear regression. Just add enough polynomial terms (x^a) and interaction terms (x1 * x2). The end model looks something like: y = b + x1 + x1^2 + x1^3 + ... + x1 * x2 + (x1 * x2)^2 + ... + x2 + x2^2 + ... By that point you're making a Taylor approximation of the latent function through linear space, which is also a universal approximator. So the commenter above is wrong -- neural networks are…

That would suggest that 1-Hidden-Layer neural nets would work fine, since they are also universal function approximators. But no -- when people talk about "deep learning", the word "deep" refers to having lots of hidden layers. I'm not an expert, but the motivation seems more like this: - Linear regression and SVM sometimes work. But they apply to very few problems. - We can fit those models using gradient descent. A…

RE: 1-layer neural nets, yes, that's the point.

Deep learning has been a series of *engineering* successes stacking over each other rather than theory being applied rigorously.

It's hard to scale training on the "dumb" approximators like a kitchen sink regression, and balancing overfitting becomes a nightmare.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#256

Post by [deleted] »

vidarh wrote: post #175
kranke155 wrote: post #126

Earlier quoted context omitted.

Hallucinations are an indication that something is missing, surely? Does chatGPT need a way to verify reality for itself to become truly intelligent ?

Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too. You'll find people confidently make claims not supported by data or indeed by any of their own data all the time, where when you poke around in their justifications you'll find they have none. The key difference appears a combination of two factors: We appear to be more likely to have learnt which subjects we'…

[deleted]

Re: Google DeepMind CEO says some form of AGI possible in a few years

#257

Post by jebarker »

kranke155 wrote: post #126
earthnail wrote: post #123

Earlier quoted context omitted.

I disagree with you. > Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits No, we haven't pushed linear regression to its limits. If it was only linear regression, it wouldn't work. Neural networks need a non-linearity to model complex things. The beauty is that given an infinite series of nonlinearities, one can model any mathematical function. In practice we…

Hallucinations are an indication that something is missing, surely? Does chatGPT need a way to verify reality for itself to become truly intelligent ?

Hallucinations are a feature, not a bug. GPT pre-training teaches the model to always produce an answer, even when it has little or not relevant training experience, and on average it does well at that. Part of the point of RLHF in ChatGPT was to teach the model not to hallucinate when it doesn't have good supporting experience encoded in it's weights. This helps but is not perfect. However, it seems like there might be a path to much less hallucinations with more RL training data. As others pointed out, humans hallucinate all the time we just have better training for what level of hallucination is appropriate given supporting evidence and context.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#258

Post by Lerc »

bondarchuk wrote: post #241
Lerc wrote: post #226

Earlier quoted context omitted.

I'm ok with believing that all it takes to make intelligence (probably not the only way) is a sufficiently large neural net with the right architecture and weights. I think it is easy to distracted by the specific mechanisms by which these models work, but most of the technological detail is because we want something to happen on systems at the scale of what we can actually build. We simply can't build a human brain…

> The notion "This can't be all thought is" is as old as the idea of AI. Older still: > It must be confessed, moreover, that perception, and that which depends on it, are inexplicable by mechanical causes, that is, by figures and motions, And, supposing that there were a mechanism so constructed as to think, feel and have perception, we might enter it as into a mill. And this granted, we should only find on visiting…

I would argue that that quote contains the idea of AI

Re: Google DeepMind CEO says some form of AGI possible in a few years

#259

Post by ogogmad »

PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

> Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits [emphasis mine]

Where's the data to show that we've hit the limits?

It's obviously not linear regression. You mean gradient descent?

Re: Google DeepMind CEO says some form of AGI possible in a few years

#260

Post by [deleted] »

ml-anon wrote: post #144
kranke155 wrote: post #126

Earlier quoted context omitted.

Hallucinations are an indication that something is missing, surely? Does chatGPT need a way to verify reality for itself to become truly intelligent ?

Calling them "hallucinations" is a shoddy piece of terminology by the community which anthropomorphises them.

[deleted]
Post reply on HN