Google DeepMind CEO says some form of AGI possible in a few years • Hacker News Board

Re: Google DeepMind CEO says some form of AGI possible in a few years

#181

Post by gwoolhurme » Wed, May 03, 2023, 10:49 AM UTC

elcomet wrote: post #162

gwoolhurme wrote: post #64

Earlier quoted context omitted.

This is my frustration put really well. When people talk about how humans are also just a form of LLM (not that the above comment did exactly that). That might even be true, but simplifying things to that degree doesn't help actually discuss what is going on. The original comment is as far as I know correct... while I don't have a PhD some of my undergraduate work was in control theory and ML, and it works really wel…

What frustrates me is when people say "neural networks cannot show intelligence because they are just a succession of linear layers", and methods that exist since the 70s". I don't understand this argument, and how this has anything to do with intelligence.

I’m not saying they can’t show intelligence. I am saying that the techniques are very old, throwing more hardware seems to have done wonders. It’s been surprising to see how far it’s come. I don’t think that simplifying intelligence in humans and saying we are LLMs helps the conversation. That along with that these techniques being used while they’ve clearly been improved are fairly old. From an engineering standpoint it leaves me wondering if there are not more elegant solutions. That nature actually is fairly cool, it doesn’t help the convo to downplay wetware I think.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#182

Post by maister » Wed, May 03, 2023, 10:50 AM UTC

wg0 wrote: post #44

marricks wrote: post #3

I really wonder what the next AI winter will be like. Edit: Perhaps OpenAI becomes a major tech player and we just see a cooling off of other AI investments as LLM becomes a known in terms of its strengths and weaknesses. Its abilities reach a natural limit which is still generally very useful. Or maybe folks realize the degree of lies/mistruths inherent in its content is actually unmanageable and can’t be improved.…

> Or maybe folks realize the degree of lies/mistruths inherent in its content is actually unmanageable and can’t be improved. After the hype wears off what it will be used for gets greatly curtailed and we see a big contraction. I think it's going to be this the future that's ahead us. There's enormous faith being put in next token predictors as the intelligence breakthrough just because their output coincidentally r…

> I don't understand the "coincidentally" argument.

Nothing is coincidental about those models. They were designed after processes in the brain. They underwent rigorous training to generate a function that probabilistically maps inputs to outputs. Eventually, it exceeded the threshold where most humans consider it to be intelligent. As these models grow larger, they will surpass human intelligence by far. Currently, large language models (LLMs) have fewer weights than human brains, with a difference of a factor in the thousands (based on my superficial research). But what happens when they have an equal or even 100,000 times more weights? These models will be able to model reality in ways humans cannot. Complex concepts like the connection between time and space, which are difficult for humans to grasp, will be easily understood by such models.

> LLMs do not hallucinate sometimes. They hallucinate all the time, it just is a coincident that sometimes these autocompletion of Tokens aligns with the reality. Just by chance, not by craft.

That is such a weird way to think about them. I'd rather say, they always provide the answer that is most probabilistic according to their internal model. Hallucination simply means, that the internal model is not good enough yet and needs to be improved, which it will.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#183

Post by godelski » Wed, May 03, 2023, 10:50 AM UTC

depr wrote: post #129

earthnail wrote: post #123

Earlier quoted context omitted.

I disagree with you. > Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits No, we haven't pushed linear regression to its limits. If it was only linear regression, it wouldn't work. Neural networks need a non-linearity to model complex things. The beauty is that given an infinite series of nonlinearities, one can model any mathematical function. In practice we…

>I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? People don't need the gigantic amount of input data that ChatGPT needs to learn. However I'm not sure what exactly "this" is you and GP are referring to, and it may be possible to improve existing ideas so that it works with less input data.

You have millions of years worth of training data. The model is just pruned, tuned, and specialized. More importantly, you have a lot of control over the tuning process. But it is naive to think that your life is all the training data you have and I'm not sure why this discussion is almost non-existent in threads like these.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#184

Post by ImHereToVote » Wed, May 03, 2023, 10:51 AM UTC

superzamp wrote: post #134

kranke155 wrote: post #126

Earlier quoted context omitted.

Hallucinations are an indication that something is missing, surely? Does chatGPT need a way to verify reality for itself to become truly intelligent ?

Just the other day, my friend asked me if I knew what this bug she took a picture of was. I replied it looked like a house centipede, and it was indeed one. Two things: 1) My confidence was around 80%, without any sense of absolute certainty. 2) I had no idea why I knew that, and where did I originally learned about what a house centipede looks like.

Yeah but your brain is wet. Try to make a GPU wet, and see how general the problem solving becomes. Check mate.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#185

Post by meindnoch » Wed, May 03, 2023, 10:52 AM UTC

matrixdata wrote: post #73

PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

+1. What's new "since the 80s" are faster computation, larger datasets, and a handful of mathematical breakthroughs that enable once-intractable algorithms. It's obvious that human general intelligence operates in frontiers that are plainly outside the scope of computation.

Which mathemathical breakthroughs are we talking about?

Gradient descent is literally highschool calculus-level math.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#186

Post by FredPret » Wed, May 03, 2023, 10:54 AM UTC

goatlover wrote: post #133

FredPret wrote: post #113

Earlier quoted context omitted.

Everything in the universe must by definition be rational. Chemical reactions always happen a certain way, physics always works a certain way, numbers add up the same every time. If something seems irrational, that's because there are rational things layered on top of one another in a way that creates a misapprehension in a partially-informed observer.

> Everything in the universe must by definition be rational. Why? You assume nature must be comprehensible to us because you define it that way? Nature's not under any obligation to us. > If something seems irrational, that's because there are rational things layered on top of one another in a way that creates a misapprehension in a partially-informed observer. Possible irrational things: Collapse of the wave functio…

Rationality != comprehensible to us

We can't look at quantum mechanics and call it irrational. The way the universe works is by definition the right / correct / only way. If logic points one way and reality points another, the logic isn't taking everything into account.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#187

Post by gwoolhurme » Wed, May 03, 2023, 10:56 AM UTC

earthnail wrote: post #124

gwoolhurme wrote: post #64

Earlier quoted context omitted.

This is my frustration put really well. When people talk about how humans are also just a form of LLM (not that the above comment did exactly that). That might even be true, but simplifying things to that degree doesn't help actually discuss what is going on. The original comment is as far as I know correct... while I don't have a PhD some of my undergraduate work was in control theory and ML, and it works really wel…

You can look at very small creatures on earth and think "surely there is more to a human than to this ant". And sure, there is, but also, there isn't. Just as basic life evolves into more complex life, why shouldn't the underlying methods from the NASA in the late 70s evolve into ChatGPT and eventually AGI? Simplifying things to this degree can show how you get from A to B.

I’m not the only one who thinks there is something missing though. That simplification does work to show how to get from A to B, but I don’t think it is analogous when we don’t really understand how we go to human intelligence. None of us were there. All I am saying is that simplifying even mammalian intelligence down to just an LLM doesn’t explain much to me. It might be the case that old 70s math proofs from NASA evolve into a mechanical intelligence maybe? I just personally think that isn’t the case. Humans run on a lot less overall power (though biological necessities like bathrooms and food are real), we also can learn even from less text than a machine and infer from experience. That might not convince you, or you might have your own ideas about what intelligence is.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#188

Post by a13o » Wed, May 03, 2023, 10:59 AM UTC

23B1 wrote: post #152

ml-anon wrote: post #144

Earlier quoted context omitted.

Calling them "hallucinations" is a shoddy piece of terminology by the community which anthropomorphises them.

What would you recommend calling them? Genuinely curious.

When humans do it we call them bullshitters.

But I agree with GP's eager-anthropomorphization complaint. When algorithms produce verifiably wrong output we call them errors.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#189

Post by trabant00 » Wed, May 03, 2023, 11:00 AM UTC

wg0 wrote: post #13

The other few things possible in next few years for past few years are: 1. Fusion 2. Self driving cars. 3. Ubiquitous, generally available and stable 8192 bit quantum computers. 4. High energy density grid scale batteries. Just few more "this maybe the... what could revolutionize..." articles away. What we forget is that it is in these people's business interest (executives making such statements) to state the exagge…

We're not reading news, we're reading press releases. And to be honest I would have hoped for a healthier dose of skepticism from HN. Vaporware and hype are common things in software. Ship it, let me play with it, then we'll talk about it.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#190

Post by etiam » Wed, May 03, 2023, 11:00 AM UTC

GaggiX wrote: post #138

blast wrote: post #100

Earlier quoted context omitted.

"We dont have better algorithms, we just have more data" - Peter Norvig

What about AlphaZero? No prior data.

All of the observable data, implicitly, plus an evaluation criterion. They can operate on that, including starting to mine it in a strategically effective order, because the whole game system except the other player is defined by a small set of rules.