DeepMind says reinforcement learning is ‘enough’ to reach general AI • Hacker News Board

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#71

Post by otabdeveloper4 » Thu, Jun 10, 2021, 7:27 AM UTC

anyfoo wrote: post #63

otabdeveloper4 wrote: post #52

Earlier quoted context omitted.

You're assuming that intelligence is a computational process, but the sum total of what we know about intelligence says it probably isn't. (Unless you're making a more general reductionist statement that everything in the universe is a computational process - that kind of reductionism is understandable coming from people who work with computers for their job - but this is then a philosophical stance, not scientific,…

> but the sum total of what we know about intelligence says it probably isn't Source? I am not aware of any other known process in the universe that could not be simulated by a Turing machine.

Assuming that the universe can be simulated by a Turing machine is a strong and weird claim that needs to be defended, not the other way around.

We know that Turing machines are very limited things and that the computational processes they carry out also very limited in applicability.

What's the evidence that the universe is more limited that a Turing machine?

Just the fact that we can imagine things that can't be computed by a Turing machine should clue you in that it's probably otherwise.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#72

Post by unishark » Thu, Jun 10, 2021, 7:29 AM UTC

hervature wrote: post #23

Der_Einzige wrote: post #21

Earlier quoted context omitted.

Unsupervised learning is exactly the wrong way to approach chess or other games that MuZero solves. It's also worth noting that traditional alpha-beta pruning + heuristics are basically neck and neck with the very best of neural network based techniques. I'll trust stockfish over a alpha-zero or MuZero for awhile longer if I'm trying to win a computer chess competition ...

Sure, Stockfish just uses millions of years of evolution to build its heuristics and can't be transferred to any other game. The point remains, calling RL a cherry on the cake compared to unsupervised learning when they are completely orthogonal and not mutually exclusive techniques is simplistic and unnecessarily offensive.

So it's bad to be the cake? I assume he means it's the foundation one falls back on when the more specialized categories of methods are not applicable.

You might not like my analogy either. I think of supervised and unsupervised learning as the majority of the genome of ML, while RL is that little Y chromosome sometimes tacked on to address a few high-profile tasks.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#73

Post by anyfoo » Thu, Jun 10, 2021, 7:34 AM UTC

otabdeveloper4 wrote: post #71

anyfoo wrote: post #63

Earlier quoted context omitted.

> but the sum total of what we know about intelligence says it probably isn't Source? I am not aware of any other known process in the universe that could not be simulated by a Turing machine.

Assuming that the universe can be simulated by a Turing machine is a strong and weird claim that needs to be defended, not the other way around. We know that Turing machines are very limited things and that the computational processes they carry out also very limited in applicability. What's the evidence that the universe is more limited that a Turing machine? Just the fact that we can imagine things that can't be co…

> We know that Turing machines are very limited things and that the computational processes they carry out also very limited in applicability.

Again, source? Do you know of anything that is able to perform a computation that a Turing machine cannot?

> Just the fact that we can imagine things that can't be computed by a Turing machine should clue you in that it's probably otherwise.

Like what? Uncomputable numbers like Chaitin's constant? We can "imagine" them by stating their definition, but we cannot compute them. Or do you have something else specific in mind?

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#74

Post by for_i_in_range » Thu, Jun 10, 2021, 7:42 AM UTC

otabdeveloper4 wrote: post #47

harry8 wrote: post #18

Some Bozo who has heard all this many times before is suspicious of claims from places like Deep Mind who have a financial incentive to make them (keep funding) where there aren't working machines to back that claim up. Some Bozo has no credentials, no reputation, no track record of publications and barely supports the claim they're making with anything much. Some Bozo has no financial incentives or otherwise to opin…

I'll bet a sum of real money that Some Bozo is correct. Bozo has the hindsight of history and philosophy going for him, while Deep Mind has a huge financial temptation to sell snake oil.

“Show me the incentives I’ll show you the outcome.”

Google make money, Google Bad.

Deep Mind owned by Google, Deep Mind bad!

The above conclusion stands trite.

Perhaps the inverse is true.

Google and Deep Mind, if correct, could be hurting themselves more than helping themselves.

Why? Creating a future species who’s too smart to click on ads, and too smart to remain subject to its whims, doesn’t sound like it’d be good for quarterly profits...

There’s also the emotional incentive for humans to confirm their own beliefs about humanity being special.

If Google/Deep Mind knows this, yet publishes research anyway in the spirit of truth, why, what they’re doing may be considered heroic.

Two sides of the coin here.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#75

Post by kaba0 » Thu, Jun 10, 2021, 7:42 AM UTC

londons_explore wrote: post #57

Animats wrote: post #48

"A sufficiently powerful and general reinforcement learning agent may ultimately give rise to intelligence and its associated abilities. ... We do not offer any theoretical guarantee on the sample efficiency of reinforcement learning agents." OK. This basically says "evolution works". But how fast? Biology took tens of millions of years to boot up. An related question is how much compute power does evolution, viewed…

In some cases biological 'genetic algorithm' hill climbing can be remarkably ineffective. For example, the classic "design a car that can drive over this terrain" problem, even after a billion generations (~ the same number as life on earth), shows no substantial performance improvement. That makes me suspect something is missing from our biological genetics model.

I think the number of parameters is remarkably (multiple orders of magnitude) different between even the simplest bacteria vs the model used in the car. And then genomics can also do some more advanced techniques like copy a whole gene and start modifying that, etc.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#76

Post by A-Train » Thu, Jun 10, 2021, 7:43 AM UTC

rusk wrote: post #42

harry8 wrote: post #18

Some Bozo who has heard all this many times before is suspicious of claims from places like Deep Mind who have a financial incentive to make them (keep funding) where there aren't working machines to back that claim up. Some Bozo has no credentials, no reputation, no track record of publications and barely supports the claim they're making with anything much. Some Bozo has no financial incentives or otherwise to opin…

Edit: sorry just realised you’re making the same point as me more or less. Putting yourself in third person. I’ll let my comment stand anyhow :) Screwing my face up, looking at this sideways … but it seems as though you’re saying that the Bozos of HN have nothing useful to contribute to this discussion based on … [rereads] … their lack of academic credentials in the area… you could say this about just about any HN po…

The algorithms to train, initialize the networks, new architectures are far more important than the hardware advances. If people knew how to train NNs 50 years ago we would live in a different world.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#77

Post by maaaaattttt » Thu, Jun 10, 2021, 7:44 AM UTC

bildung wrote: post #66

londons_explore wrote: post #58

Earlier quoted context omitted.

Nature has a massive incentive to make good use of energy from light through photosynthesis. Billions of plants compete, and whoever can get most out of the sun will win out. Yet manmade solar cells are more efficient by nearly all measures.

> Yet manmade solar cells are more efficient by nearly all measures. Only because we cheated, though: Houses can't sponantously grow more cells in place when more energy is needed.

On a half-jokingly note, they can, their humans buy them and put them where needed. An alien observer in space would see some houses spontaneously growing solar cells on their roofs.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#78

Post by d110af5ccf » Thu, Jun 10, 2021, 7:46 AM UTC

rich_sasha wrote: post #64

lvncelot wrote: post #56

Earlier quoted context omitted.

Some cynic remarks that during the first AI golden years, claims of imminent success seemed to come from a place of hopeful naïveté of a fledgling science, whereas those same claims nowadays seem to come from a place of cold calculation of a booming business.

Is Deep Mind a “booming business”? They are achieving great things academically, but their business successes are either kept secret or mostly absent. All I know about is the Google data centre cooling scheduling, probably a big saving for Google but hardly an achievement that in its own professes their business success.

Deep Mind is cutting edge ML in general, right? Doesn't Google actively apply the lessons learned all over the place? YouTube content recommendation stands out to me in particular. Translation and automated closed captioning are also obviously ML based. I'd guess that most of the really interesting stuff would be behind the scenes and not immediately visible to end users though.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#79

Post by ackbar03 » Thu, Jun 10, 2021, 7:51 AM UTC

0xB31B1B wrote: post #51

maiodude wrote: post #43

Earlier quoted context omitted.

They just "solved" protein folding late last year. How can you say things are slowing down? Do you honestly expect life-changing discoveries every other week?

Protein folding is a well modeled math problem. The alpha fold solution is extremely good at pruning (aka guessing) folding chain structure possibilities. I am Impressed and this is a difficult problem but this is extremely different from AGI as this is a well scoped easily modelable problem that is basically a chain of 26 inputs types of links of arbitrary length. I am not trying to take away that the protein foldin…

I may be really only speaking for myself here but I have very sincere doubts that anyone who has done any moderately serious work on "AI" and wrangled with the nitty gritty details of it all is really having any huge expectations for AGI. I mean if it comes during my lifetime, hurrah! But personally I'm not gonna sit around waiting for it or depend on it for anything. That being said, is the current AI tech as we have it useless? Of course not. Things like protein folding and alphago are still huge leaps forward in tech, it'd be kind of silly to treat AGI as the only thing worth achieving

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#80

Post by tinco » Thu, Jun 10, 2021, 7:54 AM UTC

0xB31B1B wrote: post #51

maiodude wrote: post #43

Earlier quoted context omitted.

They just "solved" protein folding late last year. How can you say things are slowing down? Do you honestly expect life-changing discoveries every other week?

Protein folding is a well modeled math problem. The alpha fold solution is extremely good at pruning (aka guessing) folding chain structure possibilities. I am Impressed and this is a difficult problem but this is extremely different from AGI as this is a well scoped easily modelable problem that is basically a chain of 26 inputs types of links of arbitrary length. I am not trying to take away that the protein foldin…

It is entirely not necessary for an AGI to be able to drive a car.

Frankly, after seeing AlphaZero and AlphaFold I'm surprised they didn't declare AGI right there and then.

People assume that when AGI happens, computers can suddenly outsmart humans in every way and solve every problem imaginable. The reality is just that it could in theory given enough time and resources.

It is like quantum computing. In theory it can instantly factor and break our nice cryptographic primes. In reality the largest number it factored is 21.