Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google DeepMind CEO says some form of AGI possible in a few years

wsj.com

Post reply on HN
Print view View original

241–250 of 388 posts

Re: Google DeepMind CEO says some form of AGI possible in a few years

#241

Post by bondarchuk »

Lerc wrote: post #226
wesammikhail wrote: post #140

Earlier quoted context omitted.

Andrej Karpathy, the director of AI at tesla, published an AMAZING video about how the current models are built which I think everyone in the tech space should watch. It is intuitive, easy to follow and quite frankly the best video I have seen on the topic. Here is the link: https://www.youtube.com/watch?v=kCc8FmEb1nY&list=FL2tbfd7UpJ... If you conclude, after watching this, that this is all intelligence is - namely…

I'm ok with believing that all it takes to make intelligence (probably not the only way) is a sufficiently large neural net with the right architecture and weights. I think it is easy to distracted by the specific mechanisms by which these models work, but most of the technological detail is because we want something to happen on systems at the scale of what we can actually build. We simply can't build a human brain…

>The notion "This can't be all thought is" is as old as the idea of AI.

Older still:

>It must be confessed, moreover, that perception, and that which depends on it, are inexplicable by mechanical causes, that is, by figures and motions, And, supposing that there were a mechanism so constructed as to think, feel and have perception, we might enter it as into a mill. And this granted, we should only find on visiting it, pieces which push one against another, but never anything by which to explain a perception. This must be sought, therefore, in the simple substance, and not in the composite or in the machine.

- Leibniz, 1714

Re: Google DeepMind CEO says some form of AGI possible in a few years

#242

Post by iainmerrick »

vidarh wrote: post #201
riffraff wrote: post #166

Earlier quoted context omitted.

I think this is the culprit: ChatGPT does not understand , as proven by hallucinations, by being wrong on certain questions or puzzle etc.. It just seems to understand. This is useful, and deeply impressive, but it's not the same thing.

Humans are confidently wrong about all kinds of things all the time. We don't call it hallucination other than for very specific, limited subsets. We call each other names over it, call it mistakes, stupidity, or lies. In fact, we structure large parts of society around it (the presence of multiple contradictory world religions means the majority of the worlds population goes through life being confidently wrong). It…

humans are constantly reinforced from childhood that making confident claims about things we don't actually know well is often going to have negative consequences

If only that were universally true!

Re: Google DeepMind CEO says some form of AGI possible in a few years

#243

Post by dS0rrow »

nologic01 wrote: post #200
PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

Unfortunately this argument, while imho entirely valid and also frequently seen in discussion, is unable to stop the massive train that has been set in motion. There are various types of retorts: i) the brain is also doing gradient descent; ii) what the brain does does not matter (if you can fake intelligence you have intellegence) iii) the pace is now so fast that has not happened in decades will happen in the next…

If not convincing what argument for and against do you think is the strongest?

Re: Google DeepMind CEO says some form of AGI possible in a few years

#244

Post by code_runner »

paxys wrote: post #16

I predict that the fight will be more about defining intelligence than inventing it. No one knows what AGI is. There isn't going to be some switch that flips to take us from AI to AGI. These tools we have today will just keep getting incrementally better, and some new ones will pop up, and at some point we'll have to stop and say "yeah, this is good enough to qualify". And everyone will have their own opinion on what…

No matter how good am AI system gets we can rely on hacker news commenters highlighting hallucinations and saying “it’s only doing a simple task” while totally ignoring whatever the system is actually capable of.

I agree that the goalposts will be moving for a long time (at least for some people)

Re: Google DeepMind CEO says some form of AGI possible in a few years

#245

Post by dr_dshiv »

automatic6131 wrote: post #234
dr_dshiv wrote: post #218

Earlier quoted context omitted.

Sure. Let’s take quantum theory. There are lots of concepts that are based in math but can be reasoned about non-mathematically. The reason that chatGPT can write quantum computer programs to in any domain (despite the lack of existing programs!) is because it can deal with the concepts of quantum computing and the concepts in a domain (eg, predicting housing prices) and align them. Very little of human reasoning is…

Now you are making things up

Want to be specific?

Re: Google DeepMind CEO says some form of AGI possible in a few years

#246

Post by blagie »

Buttons840 wrote: post #34
blagie wrote: post #22

Earlier quoted context omitted.

I don't think that's a limit. If you give me $10M and 5 years, I can tell you a half-dozen ways to train models better than GPT4, primarily by having them do richer tasks than text completion, by having them evaluate themselves in richer ways, by having workflows around them, and by having shared models perform multiple types of tasks (e.g. text, image, controls, etc.). It's not that I'm especially smart; many others…

This reminds me of reinforcement learning and the Bellman equations. Let a neural network based agent act in an environment. Record its rewards. Update the neural network so that the best actions become a little more likely, and the worse actions become a little less likely. Thus, we have demonstrated a training step that improves the agent. Now just run that training step in a loop forever and you'll improve each st…

That sounds like technobabble, which is to say big words like "instability" and "Bellman equations," but with little connecting logic.

If there is meaning behind those words, please write something with multiple paragraphs.

I don't see any reason why there would be a limit, instability, or otherwise, any more or less than there is on biological evolution. I also don't see any reason why there wouldn't be a limit. We just have no idea at this point.

I do think a diversity of tasks is critical. An AGI should be able to not just complete human language but:

- Play a diversity of strategic games like chess or Starcraft

- Perform machine vision like Stable Diffusion

- Predict the output of Python programs (and vice-versa)

- Write (correct) mathematical proofs and arguments

- Control a robotic arm or airplane

... and so on.

We can train all of those independently -- and we could do so in a single network now -- but we're at the very early stages of architecture for how those things would integrate.

At some point, the best solution is a very sophisticated intelligence. How close deep learning in current -- and future -- architectures gets us to super-intelligence is an open question.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#247

Post by ogogmad »

VHRanger wrote: post #221
somewhereoutth wrote: post #164

Earlier quoted context omitted.

You don't need non-linearities, an infinite series of sine functions is enough to model any function. For extraordinary claims ('intelligence'), the burden of proof is on those making the claim, not on others to prove the negative.

You can even do it in a linear regression. Just add enough polynomial terms (x^a) and interaction terms (x1 * x2). The end model looks something like: y = b + x1 + x1^2 + x1^3 + ... + x1 * x2 + (x1 * x2)^2 + ... + x2 + x2^2 + ... By that point you're making a Taylor approximation of the latent function through linear space, which is also a universal approximator. So the commenter above is wrong -- neural networks are…

That would suggest that 1-Hidden-Layer neural nets would work fine, since they are also universal function approximators. But no -- when people talk about "deep learning", the word "deep" refers to having lots of hidden layers.

I'm not an expert, but the motivation seems more like this:

- Linear regression and SVM sometimes work. But they apply to very few problems.

- We can fit those models using gradient descent. Alternatives to gradient descent do exist, but they become less useful as the above models get varied and generalised.

- Empirically, if we compose with some simple non-linearities, we get very good results on otherwise seemingly intractable problems like OCR. See Kernel SVM and Krieging.

- Initially, one might choose this non-linearity from a known list. And then fit the model using specialised optimisation algorithms. But gradient descent still works fine.

- To further improve results, the choice of non-linearity must itself be optimised. Call the non-linearity F. We break F into three parts: F' o L o F'', where L is linear, and F' and F'' are "simpler" non-linearities. We recursively factorise the F' and F'' in a similar way. Eventually, we get a deep feedforward neural network. We cannot use fancy algorithms to fit such a model anymore.

- Somehow, gradient descent, despite being a very generic optimisation algorithm, works much better than expected at successfully fitting the above model. We have derived Deep Learning.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#248

Post by ZiiS »

SanderNL wrote: post #160
ZiiS wrote: post #157

It is _possible_ we will be visited by advanced alien life in a few years. Obviously they could give us AGI or basically any other technology. None of our current approaches have demonstrated they will yield AGI in the next few years.

ZiiS told us all our approaches to this problem are fruitless and will never result in anything resembling AGI. How exactly he knows this is unknown to us, because we are still researching the basic properties of the systems we are currently producing at a staggering rate. Nevertheless it is quite a relief. We can now let this all go and return to frolicking in the meadows again.

My point was not that the current approaches are fruitless; it is that we have no information to decide if they will lead to AGI in a few years or not. Whilst this fits a dictionary definition of "possible" they are trying to make it sound "likely" with no evidence.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#249

Post by dr_dshiv »

chimprich wrote: post #238
dr_dshiv wrote: post #199

Earlier quoted context omitted.

It’s terrible and math and logic, but ChatGPT is amazing at concepts—that’s why it is so powerful. It doesn’t work programmatically—that’s why it fails at logic. But it can reason inductively very very well. Do you have an example besides logic/math where it doesn’t understand simple concepts?

> Do you have an example besides logic/math where it doesn’t understand simple concepts? All the time. It often fails to understand simple concepts. It doesn't really seem to understand anything. For example, try to get it to write some code for a program in a moderately obscure programming language. It's terrible: it will confidently produce stuff, but make errors all over the place. It's unable to understand that i…

Sure, so what are the specific concepts it doesn’t understand?

I don’t think its ability to program in an obscure program is really a great test. That’s a matter of syntax more than semantics, no?

Novel conceptual blends are where it excels. Yes, it needs to understand the concepts involved to blend them —but humans need that too.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#250

Post by bamboozled »

code_runner wrote: post #244
paxys wrote: post #16

I predict that the fight will be more about defining intelligence than inventing it. No one knows what AGI is. There isn't going to be some switch that flips to take us from AI to AGI. These tools we have today will just keep getting incrementally better, and some new ones will pop up, and at some point we'll have to stop and say "yeah, this is good enough to qualify". And everyone will have their own opinion on what…

No matter how good am AI system gets we can rely on hacker news commenters highlighting hallucinations and saying “it’s only doing a simple task” while totally ignoring whatever the system is actually capable of. I agree that the goalposts will be moving for a long time (at least for some people)

Maybe because the "goal posts" aren't well defined?
Post reply on HN