261–270 of 312 posts
Earlier quoted context omitted.
Does AGI imply human-level intelligence, or would the intelligence of a housefly qualify?
It's a very interesting question. Personally I take mammalian intelligence as the relevant standard we're actually aiming at. So I'd say mouse+. Houseflys, I think, are closer to non-intelligent than intelligent.
Earlier quoted context omitted.
How do you even have reinforcement learning with non-real numbers? The point is to maximize a score. It seems to me, any benefit you'd get from using an alternative number system could be replicated by using an algorithm to convert multiple real number scores into a single value.
Here's an example. Suppose there are two buttons, A and B. If you press A for the nth time, then you get reward n. If you press B for the nth time, then you get reward 0 if n is not a power of 2, or reward omega (the first infinite ordinal number) if n is a power of 2. If the above rewards are shoehorned into real numbers---for example, by replacing omega with 9999 or something---then an RL agent would misunderstand…
I don’t think you want to encode your problem domain in your reward system. It’d be like asking a logic gate to add when you really should be reaching for an FPU. Maybe I’m missing something though?
Earlier quoted context omitted.
You're assuming that intelligence is a computational process, but the sum total of what we know about intelligence says it probably isn't. (Unless you're making a more general reductionist statement that everything in the universe is a computational process - that kind of reductionism is understandable coming from people who work with computers for their job - but this is then a philosophical stance, not scientific,…
> but the sum total of what we know about intelligence says it probably isn't Source? I am not aware of any other known process in the universe that could not be simulated by a Turing machine.
Are the just reformulating the principles of evolution in digital terms, and essentially not providing any new insights at all? Yes, intelligence has been created by evolution. That doesn't imply that any system that is subject to evolutionary forces will lead to the creation of intelligence (and not within a reasonable timeframe, either). The challenge is to create a system that is capable of evolving intelligence.…
I'd say it's even less than that. They seem to be summarizing the ways the problem of teaching an agent to do anything (including be generally intelligent) can be formulated as a problem of maximizing a reward (hence the title). Another way to look at it is, if we had a good enough function (e.g. a universal approximator) it can be made to model any behavior using numerical optimization. Which I think isn't very surp…
So welcome back to the future, and the $trillions the US spent on 20 years of space race and 50 years of cold war. The catchphrase that motivates the next 50 years of government/corporate funding will be...
They've got a Terminator and we don't.
Earlier quoted context omitted.
Here's an example. Suppose there are two buttons, A and B. If you press A for the nth time, then you get reward n. If you press B for the nth time, then you get reward 0 if n is not a power of 2, or reward omega (the first infinite ordinal number) if n is a power of 2. If the above rewards are shoehorned into real numbers---for example, by replacing omega with 9999 or something---then an RL agent would misunderstand…
There are no infinite rewards in biology and yet mathematicians seem to do just fine answering these sorts of questions. I don’t think you want to encode your problem domain in your reward system. It’d be like asking a logic gate to add when you really should be reaching for an FPU. Maybe I’m missing something though?
This is only a problem if you're already assuming we do everything based on our biological reward systems, and in the current context that would be circular reasoning.
Imagine the treasury creates a "superdollar", a product which, if you have one, you can use to create any number of dollars you want, whenever you want, as many times as you want. Obviously a superdollar is more valuable than any finite number of dollars, and humans/mathematicians/AGIs would treat it accordingly, regardless of the finiteness of our biological reward systems.
Earlier quoted context omitted.
I think the title of the paper makes more sense if you consider that ten years ago, someone could have written a paper in a similar spirit with a different take on "what is enough". Back then, it would probably have been titled: "Backpropagation of errors is enough". The last ten years have shown that backpropagation -- while a crucial component -- is not enough. Personally, I would not be shocked to find out in the…
Why are forgetting and sleep relevant? If someone invented a pill that gave you a perfect memory and removed the need to sleep, would you stop being generally intelligent if you took it?
"A sufficiently powerful and general reinforcement learning agent may ultimately give rise to intelligence and its associated abilities. ... We do not offer any theoretical guarantee on the sample efficiency of reinforcement learning agents." OK. This basically says "evolution works". But how fast? Biology took tens of millions of years to boot up. An related question is how much compute power does evolution, viewed…
Great point. Until the promoters of RL can build us a robot that can 1) walk gracefully through a typical home that has stairs and closed doors, 2) cook a meal with pots and pans, and 3) get back up after it falls down -- I suggest we take their claims of impending Singularity with a big grain of salt.
Earlier quoted context omitted.
It's a very interesting question. Personally I take mammalian intelligence as the relevant standard we're actually aiming at. So I'd say mouse+. Houseflys, I think, are closer to non-intelligent than intelligent.
Surely the AGI researchers have a benchmark though, don't they? Somebody else mentioned the Turing Test which is something...
The turing test is also not an AGI test, it's a "good enough" standard for fooling people.
Intelligence fundamentally requires a multitude of environmental capabilities. The turing test considers only a single i/o boundary.
/r/MachineLearning discussion: https://www.reddit.com/r/MachineLearning/comments/nplhy3/r_r... I'm with most of the comments there. This paper is ridiculously hand-wavey.
Many of DeepMind's opinion style papers are like this. Another example of the "handwavy" DeepMind paper: https://arxiv.org/pdf/2102.03406.pdf It's also worth it to note as well that this isn't a homogenous organization, many DeepMind employees have different opinions on issues like this and an individual paper isn't representative of the entire organization.
Please don't consider my critique of this paper as an indictment of DeepMind as a whole!
> Many of DeepMind's opinion style papers are like this.
That's good to know. I have not read many of their opinion papers, and I'll admit I didn't have the context of it being an "opinion" paper.
That said, I don't agree with the opinion. The paper didn't really engage with the concept of AGI in a way that I found satisfying. The conclusion may very well be correct, but this paper wasn't enough to convince me.
Slightly OT: My views were reinforced when I saw the paper was praised by Patricia Churchland. I don't find her take on consciousness a satisfying one, though I find the general direction of her work interesting. See here for another example:
https://www.reddit.com/r/philosophy/comments/nvtgwr/grand_th...
Earlier quoted context omitted.
Intelligence would be produced in any Turing complete automata. But the universe has a frame rate of 10^34 (based on Planck constant.) We don't really have the tech to just run "evolution" of a universe or of even a psuedo biological substrate.
This seems far from clear. Just because a system is capable of turing complete computation does not imply that a generic state of the system will typically eventually produce intelligence or even something which is sophisticated in some sense. As a trivial example, consider a variation of Conway's game of life which, in addition to black and white cells, also has green cells, where any cell next to one or more green…
You're right, that's the actual meaning of action in physics, which is what the Planck constant measures. The amount of change (which is measured in Hz) per joule of energy. But it's a good enough approximation and a good lower bound for the amount of processing power the universe possesses versus our en-silico hardware. We don't have anything near 10^33. Just because we build a system that has the ability to evolve doesn't mean we will ever see it through to the extent that the universe has the capability to.