Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google DeepMind CEO says some form of AGI possible in a few years

wsj.com

Post reply on HN
Print view View original

191–200 of 388 posts

Re: Google DeepMind CEO says some form of AGI possible in a few years

#191

Post by vidarh »

saurik wrote: post #151
amitport wrote: post #139

Earlier quoted context omitted.

"People don't need the gigantic amount of input data that ChatGPT needs to learn" Sure they do. Humans rely on tons of audio and video before they can even read (or walk).

But I have learned to program and do complex math having read and analyzed ridiculously less source material on those subjects (and in fact in the case of programming having seen very little code as I mostly only had reference manuals at the time).

We are trained in a way that involves far more back and forth, giving corrections to specific failure modes in our thinking, though. It'll be fascinating to see to what extent interleaving "independent study" (training on large chunks of data) with interaction fed back in as training data could reduce the amount of training needed.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#192

Post by tomhoward »

earthnail wrote: post #124
gwoolhurme wrote: post #64

Earlier quoted context omitted.

This is my frustration put really well. When people talk about how humans are also just a form of LLM (not that the above comment did exactly that). That might even be true, but simplifying things to that degree doesn't help actually discuss what is going on. The original comment is as far as I know correct... while I don't have a PhD some of my undergraduate work was in control theory and ML, and it works really wel…

You can look at very small creatures on earth and think "surely there is more to a human than to this ant". And sure, there is, but also, there isn't. Just as basic life evolves into more complex life, why shouldn't the underlying methods from the NASA in the late 70s evolve into ChatGPT and eventually AGI? Simplifying things to this degree can show how you get from A to B.

We can see how an ant is a more primitive form of the same kind of thing that a human is, and though they're not on the same evolutionary path, we can see how humans evolved from the most primitive forms of life, because we know what the endpoint looks like.

We don't know what 70s AI and ChatGPT will evolve into. That's why everyone keeps debating and prognosticating about it, because nobody actually knows. But whatever people mean by AGI, we don’t know if it will or can evolve from the AI platforms that have come before.

We do know that the thing that evolved into ChatGPT is entirely different from primitive cellular life, so there's no reason to believe that ChatGPT will keep evolving into some kind of humanistic, self-aware intelligence.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#193

Post by ilyt »

FredPret wrote: post #113
HKH2 wrote: post #82

Earlier quoted context omitted.

Despite our best efforts, we are deeply irrational. Our thinking is based on instinct, not on core principles; it's a top-down approach driven by feelings.

Everything in the universe must by definition be rational. Chemical reactions always happen a certain way, physics always works a certain way, numbers add up the same every time. If something seems irrational, that's because there are rational things layered on top of one another in a way that creates a misapprehension in a partially-informed observer.

Quantum effects don't. There are rules but results are within range random.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#194

Post by maister »

vidarh wrote: post #175
kranke155 wrote: post #126

Earlier quoted context omitted.

Hallucinations are an indication that something is missing, surely? Does chatGPT need a way to verify reality for itself to become truly intelligent ?

Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too. You'll find people confidently make claims not supported by data or indeed by any of their own data all the time, where when you poke around in their justifications you'll find they have none. The key difference appears a combination of two factors: We appear to be more likely to have learnt which subjects we'…

> Humans "hallucinate" in the AI sense (it's an awful word that obscures how often we do it) all the time too.

Agreed. I'd like to add another point to the discussion. It seems to me, as if LLMs are held to a higher standard regarding telling the truth than humans are. In my opinion the reason for this is that computers have been traditionally used to solve deterministic tasks and that people are not used to them making wrong claims.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#195

Post by trabant00 »

paxys wrote: post #16

I predict that the fight will be more about defining intelligence than inventing it. No one knows what AGI is. There isn't going to be some switch that flips to take us from AI to AGI. These tools we have today will just keep getting incrementally better, and some new ones will pop up, and at some point we'll have to stop and say "yeah, this is good enough to qualify". And everyone will have their own opinion on what…

I see this argument all the time and it perplexes me. A thing so huge to rival the wheel, the control of fire, etc, and yet people say that we might not be sure when it's here. If it is AGI it will change the world in dramatic ways, you will not spend time arguing that we rolled things on top of logs before we had the wheel. The freaking war chariot is charging your ranks, taking your home and your family in slavery, there is no room for doubt about the impact.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#196

Post by xiphias2 »

kypro wrote: post #180
PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

> Personally I think the limits of this technique are far better than I would have thought 10 years ago. Honestly I've always surprised at the scepticism AI researchers have had of about the limits of training large neural nets with gradient descent. Where I've had my doubts is in the architecture of existing models and still think this is their primary limiting factor (more so than compute and network size). I think…

I agree with you totally.

For me AGI is achieved when AutoGPT is at a point when it's able to improve its own algorithm (specifically improve on the GPT architecture).

Flash attention came out a bit less than a year ago (27 May 2022), which was a great scaling improvement for getting rid of O(n^2) memory requirement in the length of attention.

I guess the next one we need will be one of the solutions that change the FLOPS for computing attention from n^2 to n*log(n) (there was already a paper that achieved it using resizable gauss filters where the size and the base convolution filter is learned separately).

A few of these kind of ,,breakthroughs''/algorithmic improvements are the only ones needed to get to AGI (self improving machine) in my opinion.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#197

Post by vidarh »

goatlover wrote: post #128
earthnail wrote: post #123

Earlier quoted context omitted.

I disagree with you. > Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits No, we haven't pushed linear regression to its limits. If it was only linear regression, it wouldn't work. Neural networks need a non-linearity to model complex things. The beauty is that given an infinite series of nonlinearities, one can model any mathematical function. In practice we…

> I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? Animals have bodies and have to survive in an environment. They're not just sitting there waiting on a prompt to generate text. They do things in the real world. Language is later invention by one particular social animal which serves our needs to communicate, which is different than waiting on prompts.

That is just taking data in and generating data out, though. Language is just data, not something fundamentally different.

There are certainly some missing elements, such as ongoing training and memory, but it's not at all clear that there's any major conceptual differences. There could be, we just don't really know.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#199

Post by dr_dshiv »

andrepd wrote: post #171
dr_dshiv wrote: post #145

Earlier quoted context omitted.

A transformative aspect of human verbal intelligence involves dealing with concepts and their combinations intersection. ChatGPT does this very well. I think we can agree that ChatGPT provides intelligent completions for an astonishing range of human concepts? It seems appropriate to describe what ChatGPT understands and what it doesn’t understand through evals or assessments (in the same way that we can use assessme…

> ChatGPT does this very well Lol, this is precisely what chatgpt doesn't do well at all! It fails to understand simple concepts, logical inference, simple math, etc.

It’s terrible and math and logic, but ChatGPT is amazing at concepts—that’s why it is so powerful.

It doesn’t work programmatically—that’s why it fails at logic. But it can reason inductively very very well. Do you have an example besides logic/math where it doesn’t understand simple concepts?

Re: Google DeepMind CEO says some form of AGI possible in a few years

#200

Post by nologic01 »

PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

Unfortunately this argument, while imho entirely valid and also frequently seen in discussion, is unable to stop the massive train that has been set in motion.

There are various types of retorts: i) the brain is also doing gradient descent; ii) what the brain does does not matter (if you can fake intelligence you have intellegence) iii) the pace is now so fast that has not happened in decades will happen in the next five years etc.

None of them is remotely convincing. I would really start getting worried if indeed some of the AI fanboys had a convincing argument. But we are where we are.

Somehow this whole episode will be just another turn of the screw in adopting algorithms in societal information processing. This in itself is fascinating and dangerous enough.

Post reply on HN