And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants). Such architecture works great for differentiable data, such's images/audios, but the improvement…
Are there ways that an AI practitioner would be able to tell whether a neural network is doing human-style symbolic reasoning?
Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
31–40 of 181 posts
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#32I'm not even convinced that a real AI is possible with conventional computer hardware or anything remotely similar to it. Not even considering software I get the impression there is a fundamental limitation of hardware.
I'm not convinced we've even defined the problem space well enough to solve it. Like what is the concrete measure(something to target) for intelligence? If we develop general intelligence is it going to be human, dog, or fish?
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#33Tangential: This title is weird. As if no one but the top minds in AI didn't know this? This isn't big news to anyone who has done even just a modicum of AI research.
Also, do you believe AGI is currently more a compute/hardware problem, or an algorithmic problem?
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#34And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants). Such architecture works great for differentiable data, such's images/audios, but the improvement…
Hmm, it seems like natural language translation has been getting quite a bit better with statistical techniques, though? I guess it depends what you mean by "only incremental".
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#35Earlier quoted context omitted.
I'm not convinced we've even defined the problem space well enough to solve it. Like what is the concrete measure(something to target) for intelligence? If we develop general intelligence is it going to be human, dog, or fish?
I'm not convinced any of those creatures have general intelligence. I'm similarly unconvinced that we'd recognise general intelligence if we saw it.
The baseline of human capability would definitely still be impressive.
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#36And no one should be surprised by this. The NN advancement of late doesn't help addressing human-style symbolic reasoning at all. All we have is a much more powerful function approximator with a drastic increased capacity (very deep networks with billions of parameters) and scalable training scheme (SGD and its variants). Such architecture works great for differentiable data, such's images/audios, but the improvement…
And - let's be real - a lot of human symbolic reasoning actually happens outside of the brain, on paper or computer screens. We painstakingly learn relatively simple transformations and feedback loops for manipulating this external memory, and then bootstrap it into short-term reaction via lots of practice.
I tend to think that the problems are: a) Tightly defined / domain-specific loss functions. If all I ever do is ask you to identify pictures of bananas, you'll never get around to writing the great american novel. And we don't know how to train the kinds of adaptive or free form loss functions that would get us away from these domain-specific losses.
b) Similarly, I have a soft-spot for the view that a mind is only as good as its set of inputs. We currently mostly build models that are only receptive (image, sound) or generative. Reinforcement learning is getting progress on feedback loops, but I have the sense that there's still a long way to go.
c) I have the feeling that there's still a long way to go in understanding how to deal with time...
d) As great as LSTMs are, there still seems to be some shortcoming in how to incorporate memory into networks. LSTMs seem to give a decent approximation of short-term memory, but still seems far from great. This might be the key to symbolic reasoning, though.
Writing all that down, I gotta say I agree fundamentally with the DeepMind research priorities on reinforcement learning and multi-modal models.
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#37Behind every successful neural network is a human brain. Neural networks are a tool, an advanced tool for sure, but still just a tool. If we are looking for AGI, and assuming the brain is an AGI, then there are still many differences to resolve. For example, back propagation has not been observed in nature. Nor has gradient descent. So the core mechanisms for learning in nature have still to reveal their secrets.
> Behind every successful neural network is a human brain. I've spent a lot of time trying to explain this to people, that there is a confluence between the human brain and the machine, people tend to look at the machine separately, which is a mistake. When I say unequivocally, 'there is no such thing as machine intelligence', I just get blank stares.
Overall, I'd agree that really powerful tools for specific tasks is going to be the majority of "AI" in the coming years.
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#38Tangential: This title is weird. As if no one but the top minds in AI didn't know this? This isn't big news to anyone who has done even just a modicum of AI research.
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#39Not to mention that we don't even know if general intelligence exists. All we know is that mental abilities tend to correlate, but not why they tend to correlate. And if you think about designing machines, in general, the idea of general intelligence is utterly ridiculous. Does a fast car have general speediness? Of course not, it has dozens or hundreds of discrete optimizations that all contribute in some degree to…
I'm not sure you and the OP mean the same thing by "General Intelligence". It seems clear that autonomous systems which can apply their computational machinery to a diverse range of problems, and can, in a diverse range of settings, formulate instrumental goals as part of a plan to attain a final goal, do exist. Because that's what humans are, at least some of the time.
Re: Geoffrey Hinton and Demis Hassabis: AGI is nowhere close to being a reality
#40Earlier quoted context omitted.
Roger Penrose basically suggests what you say in "The Emperor's New Mind". Roughly, it says that the brain (likely, according to him) uses quantum computation, and so we can't make an AI out of a classical computer. The practical flaw with this argument, of course, is that you could instead make an AI that itself uses quantum computation. I asked Roger Penrose about this at a university philosophy meetup over 20 year…
Well, quantum computation is weaker than a nondeterministic Turing machine, so not the same thing I'm saying. Penrose correctly identifies the mind cannot be a deterministic Turing machine, but his invocation of quantum mechanics does not solve the problem he points out. A DTM can simulate an NTM and hence anything inbetween, so the inbetween of quantum computation does not solve anything. The fundamental problem Pen…
What does it explain though? That the human brain has a black box capable of solving certain problems... how exactly?