Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google DeepMind CEO says some form of AGI possible in a few years

wsj.com

Post reply on HN
Print view View original

171–180 of 388 posts

Re: Google DeepMind CEO says some form of AGI possible in a few years

#171

Post by andrepd »

dr_dshiv wrote: post #145
misnome wrote: post #132

Earlier quoted context omitted.

> I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? What evidence do we have that this is how intelligence works?

A transformative aspect of human verbal intelligence involves dealing with concepts and their combinations intersection. ChatGPT does this very well. I think we can agree that ChatGPT provides intelligent completions for an astonishing range of human concepts? It seems appropriate to describe what ChatGPT understands and what it doesn’t understand through evals or assessments (in the same way that we can use assessme…

> ChatGPT does this very well

Lol, this is precisely what chatgpt doesn't do well at all! It fails to understand simple concepts, logical inference, simple math, etc.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#172

Post by SanderNL »

depr wrote: post #129
earthnail wrote: post #123

Earlier quoted context omitted.

I disagree with you. > Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits No, we haven't pushed linear regression to its limits. If it was only linear regression, it wouldn't work. Neural networks need a non-linearity to model complex things. The beauty is that given an infinite series of nonlinearities, one can model any mathematical function. In practice we…

>I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? People don't need the gigantic amount of input data that ChatGPT needs to learn. However I'm not sure what exactly "this" is you and GP are referring to, and it may be possible to improve existing ideas so that it works with less input data.

You have ingested truly astronomical amounts of data. Your body is covered with sensors of various kinds. Millions of datapoints streaming in day and night.

You are not storing it all, but you are finding patterns and correlations in it from the day you were born. This all forms a base where after a decade+ you can learn things fast from books, but that’s comparable to an LLM’s in context learning. It’s fast, but it depends on a deep base of prior knowledge.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#174

Post by haneul »

garbagecoder wrote: post #90
liamwire wrote: post #89

Earlier quoted context omitted.

Given that a sufficiently resourced computer ought to be able to run a subatomic-level simulation of an entire human brain, and while acknowledging the usual counterpoint vis a vis C. elegans/OpenWorm but deeming it irrelevant on longer timescales, your take seems quite arrogant. “Outside the scope of computation” is an awfully broad claim.

Prove such a simulation can be created with actual hardware, taking relativity into account.

If you were to try to build such a simulation, given the natural limits of real hardware, wouldn’t some form of relativity be a necessary feature of the simulation to avoid segfaults, rather than something simply to take into account while constructing the hardware?

Re: Google DeepMind CEO says some form of AGI possible in a few years

#175

Post by vidarh »

kranke155 wrote: post #126
earthnail wrote: post #123

Earlier quoted context omitted.

I disagree with you. > Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits No, we haven't pushed linear regression to its limits. If it was only linear regression, it wouldn't work. Neural networks need a non-linearity to model complex things. The beauty is that given an infinite series of nonlinearities, one can model any mathematical function. In practice we…

Hallucinations are an indication that something is missing, surely? Does chatGPT need a way to verify reality for itself to become truly intelligent ?

Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too. You'll find people confidently make claims not supported by data or indeed by any of their own data all the time, where when you poke around in their justifications you'll find they have none.

The key difference appears a combination of two factors: We appear to be more likely to have learnt which subjects we're not very knowledgeable about through extensive feedback, be it through school or conversations where we're told we're wrong, and which also would appear to teach us to be more cautious in general. We also appear to have a (somewhat; far from perfect) better ability to separate memory from thoughts about our knowledge. We certainly can go off on wild tangents and make stuff up about any subject, but we get reinforced from very young that there's a time and a place for making stuff up vs. drawing on memory.

Both goes back simply to extensive (many years of) reinforcement telling us it has negative effects to make stuff up and/or believe things that aren't true, and yet we still do both of those, just not usually as blatantly as current LLMs without being aware.

So I'd expect one missing component is to add a training step that subjects the model to batteries of tests of the limits of their knowledge and incorporating the results of that in the training.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#176

Post by stevenhuang »

PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

It's entirely possible the way our human brains work may very well end up reducible to a sequence of very very complicated matrix operations.

If that's the case, there doesn't need to be anything new; fundamentally, linear regressions may be sufficient.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#177

Post by spuz »

russelldjimmy wrote: post #103
HKH2 wrote: post #82

Earlier quoted context omitted.

Despite our best efforts, we are deeply irrational. Our thinking is based on instinct, not on core principles; it's a top-down approach driven by feelings.

Off topic - a pet peeve of mine is seeing humans termed as “irrational”. Please forgive my rant, as it is not personally targeted at you. We only seem “irrational” when we are talking about a narrow view of “rationality”, i.e., as defined by the cold hard logic of machines. We do not question why we have this definition of rationality. Our “irrationality” simply seems so because we have not bothered to understand the…

It sounds like your main objection is with conflation of "irrational" with "wrong". It's helpful to describe an extreme fear of heights as "irrational" but from an evolutionary point of view it might in fact be correct because it keeps those genes that give the rest of the population a healthy fear of heights in the gene pool. I think these are different concepts and a person can be both irrational and correct in their behaviour.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#178

Post by gwoolhurme »

patrickas wrote: post #149
gwoolhurme wrote: post #66

Earlier quoted context omitted.

That is actually a really interesting find. Some of the verbiage is parallel to the hype we have around AGI right now too "fusion by 1990 instead of 2000..." does that not sound exactly like AGI in X year up from Y year that we read now?

Please see my reply to parent. > "fusion by 1990 instead of 2000..." Those three dots are omitting the most important part of the issue if we spend that much extra money on R&D In the fusion case no one was willing to spend the money, in AGI's case it looks like everyone seems to be willing to spend the money. I personally hope they won't, but that is a crutial point not to be overlooked.

That wasn’t the case at all for fusion. It is that to be viable different scientific breakthroughs in particle physics, and lasers. Especially to the point where the demo recently showed the viability of it. If you read up on it you’ll see that there was a lot of important steps that lead up to the short demo that was achieved. That is my point though, if you read the article even after the ellipses it wasn’t a monetary issue.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#179

Post by krisoft »

PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

> ML hasn't done anything new since the 80s

Lol, let me ask ChatGPT what it thinks about that. :)

> I don't believe we've seen any evidence of this

What kind of evidence would you like to have? Do you want a mathematical proof or what? There is evidence that we are making forward progress in solving problems which were previously in the domain of human cognition. There is evidence that yesterday's "impossible" problem become "possible" while the "hard" problems become "easy" or even "trivial". (Just look at this xkcd[1]. When it was published in 2014 telling whether or not a picture contained a bird was indeed a "5 year and a team of researchers" project. Today it is what, an afternoon? A tutorial you pick up to learn a new ML framework?)

There is also evidence that our solutions are tending toward more generalised ones. Previously you would need a "sentiment detection" network, and a separate "subject disambiguation" network to parse the meaning out of a text. Today you can achieve the same or better with an LLM trained to follow instructions.

Obviously these are not "hard evidence" that this path will lead to AGI. But it is certainly not unreasonable to think that it might.

1: https://xkcd.com/1425/

Re: Google DeepMind CEO says some form of AGI possible in a few years

#180

Post by kypro »

PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

> Personally I think the limits of this technique are far better than I would have thought 10 years ago.

Honestly I've always surprised at the scepticism AI researchers have had of about the limits of training large neural nets with gradient descent. Where I've had my doubts is in the architecture of existing models and still think this is their primary limiting factor (more so than compute and network size).

I think the question that remains now is whether existing models are actually capable of producing a general intelligence that's well rounded and reliable enough to be competitive with human general intelligence. Personally, I think LLMs like GPT-4 are generally intelligent in most ways and should be considered AGI already (at least in a weak sense of the word), but they clearly have notable gaps in their intelligence such as their ability to be consistent, long-term memory, and ability to discern reality from delusion.

I don't think scaling existing models could possibly address these limitations – they seem to be emergent properties of an imperfect architecture. So I suspect we're still a few breakthroughs away from a general intelligence as well rounded as human general intelligence. That said, I suspect existing models (perhaps with a few minor tweaks) are generally intelligent enough that larger models alone are likely still able to replace the majority of human intellectual labour.

I guess what I'm touching on here is the need to more nuanced about what we mean by "AGI" at this point. I think it's quite likely (probable even) that in a few years we'll have an AI that's generally intelligent and capable enough that it can replace a large percentage of existing knowledge work – and also generally intelligent enough to be dangerous. But I suspect despite this it will still have really clear limitations in its abilities when contrasted with human general intelligence.

Post reply on HN