Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google DeepMind CEO says some form of AGI possible in a few years

wsj.com

Post reply on HN
Print view View original

371–380 of 388 posts

Re: Google DeepMind CEO says some form of AGI possible in a few years

#371

Post by depr »

SanderNL wrote: post #172
depr wrote: post #129

Earlier quoted context omitted.

>I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? People don't need the gigantic amount of input data that ChatGPT needs to learn. However I'm not sure what exactly "this" is you and GP are referring to, and it may be possible to improve existing ideas so that it works with less input data.

You have ingested truly astronomical amounts of data. Your body is covered with sensors of various kinds. Millions of datapoints streaming in day and night. You are not storing it all, but you are finding patterns and correlations in it from the day you were born. This all forms a base where after a decade+ you can learn things fast from books, but that’s comparable to an LLM’s in context learning. It’s fast, but it…

My body may have "ingested" it in some way but filtering starts even at the sense ("sensor") level. So there is no way I have learned from it. I didn't even have to learn how to filter, it is part of my genetic makeup.

And even if I did, most of it would not be of the same category as what LLMs are learning; information about how a fabric feels or the sound of an ambulance doesn't teach me anything about programming or other things GPT can do. So when comparing the inputs, it makes no sense to count all the information that our senses are getting, to all the input for an LLM.

>This all forms a base where after a decade+ you can learn things fast from books, but that’s comparable to an LLM’s in context learning

Not really comparable

Re: Google DeepMind CEO says some form of AGI possible in a few years

#372

Post by ogogmad »

theGnuMe wrote: post #335
ogogmad wrote: post #247

Earlier quoted context omitted.

That would suggest that 1-Hidden-Layer neural nets would work fine, since they are also universal function approximators. But no -- when people talk about "deep learning", the word "deep" refers to having lots of hidden layers. I'm not an expert, but the motivation seems more like this: - Linear regression and SVM sometimes work. But they apply to very few problems. - We can fit those models using gradient descent. A…

1-layer nets are equivalent in theory and universal approximators, they are just hard to train.

Read the first sentence

Re: Google DeepMind CEO says some form of AGI possible in a few years

#373

Post by SilverBirch »

The thing that's interested me about the last 5 or so years is how much its become the case that everyone is now seeking to be a thought leader. It seems like at some point the VC crowd decided that because money was basically free they needed other ways of sourcing investment opportunities so they started trying to become "thought leaders". Which kind of makes sense, if you do it well the branding can be an advantage. But it runs into this problem of toxic positivity. No one wants to hear "Well, this technology, like every technology before it, is promising but unlikely to fully solve 99.99% of the problems it looks promising on today". So instead of having a real conversation about the viability of autonomous vehicles you've got motivated actors and lobotomized supporters making end-of-days style claims about technology, and each time these claims turn out to be totally unfounded we all just move on to the next topic. Can we stop and disect the failing gig-economy model, web3 movement, and elf-driving before we listen to those same people about AI?

That's kind of what I'd like to know, before the CEO of DeepMind opines on AI I'd like to hear his thoughts on what he learned about his last predictions.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#374

Post by SanderNL »

depr wrote: post #371
SanderNL wrote: post #172

Earlier quoted context omitted.

You have ingested truly astronomical amounts of data. Your body is covered with sensors of various kinds. Millions of datapoints streaming in day and night. You are not storing it all, but you are finding patterns and correlations in it from the day you were born. This all forms a base where after a decade+ you can learn things fast from books, but that’s comparable to an LLM’s in context learning. It’s fast, but it…

My body may have "ingested" it in some way but filtering starts even at the sense ("sensor") level. So there is no way I have learned from it. I didn't even have to learn how to filter, it is part of my genetic makeup. And even if I did, most of it would not be of the same category as what LLMs are learning; information about how a fabric feels or the sound of an ambulance doesn't teach me anything about programming…

I think I implied a strong connection between LLMs and how biological intelligence works, but I certainly didn’t intend to.

I think you discount general knowledge acquired non-verbally too easily. The sound of an ambulance and how it moves through space and how it correlates with visual information represents an astronomical amount of information that can be generalized from handsomely. All these data streams have to be connected, they correlate and intertwine in fantastically complex ways.

I think the sound of an ambulance does teach you things that eventually help you “program”. You have witnessed similar (non-)events thousands if not millions of times. Each time it was accompanied with shitloads of sensory data from all modalities, both external and internal.

The base of general patterns you start out from once you are ready for language is staggering.

Again not saying LLMs work like that, because they do not. All I mean to do is put their information requirements in perspective. We ingest a lot more than a bunch of books.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#375

Post by Kbelicius »

marcusverus wrote: post #355
Art9681 wrote: post #313

Earlier quoted context omitted.

It's all moot until we can agree on a definition of intelligence. I don't necessarily think it's relevant to the discussion whether an AI is truly intelligent or not. What matters is whether the technology is good enough to fool an "intelligent" human. It's good enough to do that today. I'd say we would have to train a "dumber" model that can respond with a simple "im not sure" instead of the typical responses we get…

> It's all moot until we can agree on a definition of intelligence. We did agree on a definition intelligence! For 50 years, the Turing test was the unquestioned threshold beyond which machines would be considered "intelligent". The simple fact of the matter is that we've reached that point, but folks remain unimpressed, and so have set about moving the goalposts. I am of the opinion that, when the dust settles, the…

> We did agree on a definition intelligence! For 50 years, the Turing test was the unquestioned threshold beyond which machines would be considered "intelligent".

First of all it is now more then 70 years. Secondly the question of the paper isn't whether a computer is intelligent but whether a computer can win the imitation game, so naturally it doesn't contain any definition of intelligence within it.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#376

Post by riffraff »

vidarh wrote: post #201
riffraff wrote: post #166

Earlier quoted context omitted.

I think this is the culprit: ChatGPT does not understand , as proven by hallucinations, by being wrong on certain questions or puzzle etc.. It just seems to understand. This is useful, and deeply impressive, but it's not the same thing.

Humans are confidently wrong about all kinds of things all the time. We don't call it hallucination other than for very specific, limited subsets. We call each other names over it, call it mistakes, stupidity, or lies. In fact, we structure large parts of society around it (the presence of multiple contradictory world religions means the majority of the worlds population goes through life being confidently wrong). It…

> Humans are confidently wrong about all kinds of things all the time

there is a qualitative difference: humans may be wrong about facts because they think they are true, ChatGPT is wrong because it does not know what anything means. You cannot fix that, because it's just the way LLMs work.

For example, if asked about a URL for something, a human may remember it wrongly, but will in general say "I don't know, let me check", while ChatGPT will just spew something.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#377

Post by riffraff »

bsaul wrote: post #298
riffraff wrote: post #166

Earlier quoted context omitted.

I think this is the culprit: ChatGPT does not understand , as proven by hallucinations, by being wrong on certain questions or puzzle etc.. It just seems to understand. This is useful, and deeply impressive, but it's not the same thing.

"it just seems to understand" : Does stockfish "understand" chess, or does it just "seem to understand" chess ? For all practical purposes and intent, this doesn't make much difference.

yes, for all practical purposes stockfish understands chess. For most practical purposes ChatGPT understands how to string tokens together.

But in AGI, G is the important bit, and neither Stockfish nor ChatGPT have demonstrated general understanding of the world.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#378

Post by bsaul »

riffraff wrote: post #377
bsaul wrote: post #298

Earlier quoted context omitted.

"it just seems to understand" : Does stockfish "understand" chess, or does it just "seem to understand" chess ? For all practical purposes and intent, this doesn't make much difference.

yes, for all practical purposes stockfish understands chess. For most practical purposes ChatGPT understands how to string tokens together. But in AGI, G is the important bit, and neither Stockfish nor ChatGPT have demonstrated general understanding of the world.

i think you're conflating being self-conscious, and being intelligent. In AGI , A is also very important. A universal oracle stuck in a machine, able to correctly predict anything and answer all questions, but still having no "desire" or "will" or consciousness, would be an AGI ( imho )

Re: Google DeepMind CEO says some form of AGI possible in a few years

#379

Post by riffraff »

bsaul wrote: post #378
riffraff wrote: post #377

Earlier quoted context omitted.

yes, for all practical purposes stockfish understands chess. For most practical purposes ChatGPT understands how to string tokens together. But in AGI, G is the important bit, and neither Stockfish nor ChatGPT have demonstrated general understanding of the world.

i think you're conflating being self-conscious, and being intelligent. In AGI , A is also very important. A universal oracle stuck in a machine, able to correctly predict anything and answer all questions, but still having no "desire" or "will" or consciousness, would be an AGI ( imho )

maybe I am, but I would agree with the universal oracle being generally intelligent, I simply don't think our current tools are like that.

E.g. consider this exchange with chatgpt

> hey assistant, can you provide me with a link to learn about LLMs?

> Certainly! LLM stands for Master of Laws, which is a postgraduate degree in law. Here is a link to a website that provides information about LLM programs, including what they entail and how to apply: > https://www.llmstudy.com/what-is-an-llm/what-is-an-llm-.aspx > I hope you find this helpful! Let me know if you have any further questions.

> that is not a real link, can you provide another?

> I apologize for that mistake. Here is the correct link: > https://www.llmstudy.com/what-is-an-llm/what-is-an-llm-.aspx > Please let me know if you have any further questions or if there is anything else I can help you with.

ChatGPT makes up a URL, twice. Does it understand URLs? Does it understand that the URL is wrong? I don't think it does, and I don't think it requires consciousness to do this interaction in a sane way.

Still this does not mean it's useless, it's super useful!

Re: Google DeepMind CEO says some form of AGI possible in a few years

#380

Post by SanderNL »

ZiiS wrote: post #248
SanderNL wrote: post #160

Earlier quoted context omitted.

ZiiS told us all our approaches to this problem are fruitless and will never result in anything resembling AGI. How exactly he knows this is unknown to us, because we are still researching the basic properties of the systems we are currently producing at a staggering rate. Nevertheless it is quite a relief. We can now let this all go and return to frolicking in the meadows again.

My point was not that the current approaches are fruitless; it is that we have no information to decide if they will lead to AGI in a few years or not. Whilst this fits a dictionary definition of "possible" they are trying to make it sound "likely" with no evidence.

Excuse me, I was being snarky. I agree that we are low on information, but I guess we disagree on how low. You imply we have so little information the outcome is basically random, which I think is a bit too extreme.

I don't find it far-fetched to imagine current AI research - not just LLMs - to get us to AGI-likeness in years (5-10). I deliberately use "likeness", because I also think we will never agree on what it means and if we ever reach it.

Digressing here because no one will read it anyway, but if my knowledge of humanity is anything to go by, we will be surrounded by robots and software systems of varying levels of intelligence doing just about anything a normal human could do and we would still be discussing if we ever reach mythical AGI.

Personally I think abstract, pure form AGI is impossible, even in biological systems. We don't scale infinitely and I don't even mean in the physical "too-much-data-to-handle" sense, I mean qualitatively. I think there is a very real ceiling to what kinds of mentation are possible by us and thus what results are achievable. Again, not just quantitatively, but qualitatively.

We have examples of this in the animal realm. Some of them show signs of for all intents and purposes "general" intelligence. But they will never scale to human levels of cognition. Even simple human concepts are out of reach, yet they outperform us on other problems (maze solving, memory, etc). You cannot teach a great ape what you can teach our children. People have tried. They are thus general, because you can teach them just about anything within their "bandwidth", but they are still constrained. I very strongly suspect the same holds for us. It's just that we have no superiors (or even peers) to compare to.

I think future AI systems will doubt we are AGI.

Post reply on HN