Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind’s new AI with a memory outperforms algorithms 25 times its size

singularityhub.com

Post reply on HN
Print view View original

111–120 of 141 posts

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#111

Post by blovescoffee »

kkjjkgjjgg wrote: post #53

Sounds as if they stored all the correct answers in a database and call it "better". How do they even evaluate these models? Like they already have a billion preprepared correct answers in the database. How do they come up with new questions for the evaluation?

Instead of storing the correct answers in an encoded/embedded form in the weights of the neural net (certain neurons very loosely corresponding to certain "answers") the correct answers are stored elsewhere. That way we can scale down the model to the necessary "thinking" parts and we don't need to use excess neurons for the "memory" part. Kind of handwavey but hopefully that explains the general idea.

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#112

Post by sdenton4 »

regularfry wrote: post #72
muzani wrote: post #70

Earlier quoted context omitted.

I don't know about world changing but it's saved me hundreds of hours. I use it to help read academic papers, put formatting on things like markdown and subtitles, and creative writing. A lot of the things that take it 15 seconds to do take me 2 minute and drain me mentally for about 15 mins. If anything, it's being used in force for social media marketing, where you're trying to say "buy this thing" in different way…

Forgive the ignorance, but how ? What tools are you using on top of GPT3 to do those things?

I recently used GPT-J to create some handouts (fake 1930's newspaper articles) for a roleplaying game. I wrote a headline and byline, then have the model suggest some text. I change the text to include the details I want the players to have + enforce consistency, then reuse the text-so-far as a prompt, and repeat.

I definitely cranked out the newspaper text much faster than I would have on my own, and the model actually made some really nice embellishments and added a couple ideas that I kept in the final text.

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#113

Post by taneq »

katmannthree wrote: post #105
taneq wrote: post #98

Earlier quoted context omitted.

Does the modeled activity track any real world measurements, though? If we're going to claim any real veracity from a simulation then we need to crosscheck it against real measurements, right?

I believe they’ve attempted to measure synapse weights from real worms for training the net, the firings themselves are just basic electrochemistry.

'Attempted' implies 'not succeeded', though? And 'just basic electrochemistry' triggers a massive "is it, though?" response. Sure, we know what theory tells us, I want to know if observations align with theory.

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#114

Post by SubiculumCode »

katmannthree wrote: post #50
saltyfamiliar wrote: post #46

Earlier quoted context omitted.

That proportions in that analogy seem way off. Every single neuron of the human brain could be utilizing quantum mechanical effects at the same time. Not very much at all like a kid's sand castle against the world's tides. I don't think anyone really understand "thinking" and quantum mechanics well enough for the mechanisms to be apparent. I mean, squirrels convert forest detritus into general intelligence. I don't u…

I believe we've been able to model the processes behind neural firings for quite some time now, the issue is more mapping between that micro-scale understanding of how each individual piece in a 10^10 piece puzzle with 10^15 edges affects the macro-scale emergent properties of said puzzle. It's less a lack of understanding of the processes and more a complexity so great that it exceeds the grasp of our best modeling…

Our biophysicalmodesl of neurons are actually woefully simplistic. https://journals.physiology.org/doi/full/10.1152/jn.00360.20...

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#115

Post by neom »

frabcus wrote: post #56
saltyfamiliar wrote: post #19

Earlier quoted context omitted.

Does anyone know if we understand enough about natural, generally intelligent brains to dismiss the idea that they are using quantum phenomenon for computation? Is it unlikely for any reason?

Roger Penrose’s The Emperors New Mind proposed this in the 1990s. He gave circumstantial possibilities, and I don’t think we know more than that now. As an amateur, my instinct is that the mystery of the observer "causing" wave function collapse is the best clue we have either way. I wouldn’t be surprised if we are just weighted neurons, or if we do something quantum too. I doubt the something quantum will be the sam…

Actually I had just done a bunch of researching on Penrose around the time I was having that conversation with Jack, in fact I think I'd watched these two lectures within a few months of that conversation:

Quantum Biology: The Hidden Nature of Nature - https://www.youtube.com/watch?v=ADiql3FG5is

An Introduction to Quantum Biology - with Philip Ball - https://www.youtube.com/watch?v=bLeEsYDlXJk

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#116

Post by neom »

runnerup wrote: post #22
saltyfamiliar wrote: post #19

Earlier quoted context omitted.

Does anyone know if we understand enough about natural, generally intelligent brains to dismiss the idea that they are using quantum phenomenon for computation? Is it unlikely for any reason?

I believe plenty of "quantum phenomenon" are utilized in humans' biochemistry[0]. Whether the brain "calculates" things using a method that is particularly similar to the methods used in today's quantum computers is...unlikely. It probably is "quantum" in other ways though. 0: https://www.the-scientist.com/infographics/infographic--quan...

I had been doing a bunch of research on quantum biology around the time that I had the conversation I mentioned above, that's probably why at the time I wanted to discuss it with him. I think these two lectures are pretty good and I'd watched them around the time I was discussing this stuff with Jack.

Quantum Biology: The Hidden Nature of Nature - https://www.youtube.com/watch?v=ADiql3FG5is

An Introduction to Quantum Biology - with Philip Ball - https://www.youtube.com/watch?v=bLeEsYDlXJk

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#117

Post by blurbleblurble »

saltyfamiliar wrote: post #19
neom wrote: post #12

Earlier quoted context omitted.

I got into a very long debate with an openai person 4/5 years ago about this + adversarial learning + access to a quantum computer (think just straight up world class abacus) was close to the primitives required for more generalized AI. They didn't agree with me, but that's ok! :)

Does anyone know if we understand enough about natural, generally intelligent brains to dismiss the idea that they are using quantum phenomenon for computation? Is it unlikely for any reason?

There's actually a whole field of study that uses tools from quantum information theory to successfully model cognitive and social situations. This doesn't require any assumptions of "quantum physicality". It just makes sense to model probabilistic, reflexive, slippery-to-observe systems with tools that were designed for this. Quantum theory has really great tools and they work well.

I realize this isn't quite the answer you were looking for, but I thought it was worth mentioning.

I personally think this idea that there is some obvious categorical distinction between hard "physical" quantum phenomena and classically probabilistic ones is a fallacy. Quantum theory is in some sense just probability theory with more features.

More and more researchers are catching onto this and I think that's really exciting.

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#118

Post by katmannthree »

taneq wrote: post #113
katmannthree wrote: post #105

Earlier quoted context omitted.

I believe they’ve attempted to measure synapse weights from real worms for training the net, the firings themselves are just basic electrochemistry.

'Attempted' implies 'not succeeded', though? And 'just basic electrochemistry' triggers a massive "is it, though?" response. Sure, we know what theory tells us, I want to know if observations align with theory.

Succeeded within what level of precision? The project is called OpenWorm, they have hydrodynamic models for macroscale cellular behavior and electrochemical simulations for modeling ion channels needed for the neural network. I don't have a reference handy but you should be able to find ample papers discussing accurate electrochemical modeling of synapses. The issue is that the models are complex enough that simulating a couple hundred neurons is expensive. As far as having accurate synaptic data so that they're simulating a specific worm rather than a random one pulled out of the æther, I believe that's still a work in progress.

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#119

Post by PeterisP »

saltyfamiliar wrote: post #19
neom wrote: post #12

Earlier quoted context omitted.

I got into a very long debate with an openai person 4/5 years ago about this + adversarial learning + access to a quantum computer (think just straight up world class abacus) was close to the primitives required for more generalized AI. They didn't agree with me, but that's ok! :)

Does anyone know if we understand enough about natural, generally intelligent brains to dismiss the idea that they are using quantum phenomenon for computation? Is it unlikely for any reason?

The quantum phenomena for calculations require strictly limited interactions as otherwise you just get decoherence. Quantum computations inside of our brain cells are unlikely because it's relatively very hot, having random interactions which would disrupt quantum phenomena.

Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size

#120

Post by PeterisP »

frabcus wrote: post #56
saltyfamiliar wrote: post #19

Earlier quoted context omitted.

Does anyone know if we understand enough about natural, generally intelligent brains to dismiss the idea that they are using quantum phenomenon for computation? Is it unlikely for any reason?

Roger Penrose’s The Emperors New Mind proposed this in the 1990s. He gave circumstantial possibilities, and I don’t think we know more than that now. As an amateur, my instinct is that the mystery of the observer "causing" wave function collapse is the best clue we have either way. I wouldn’t be surprised if we are just weighted neurons, or if we do something quantum too. I doubt the something quantum will be the sam…

It's worth noting that it's not the observer causing anything, but a "technical observation" i.e. any interaction with the macroscopic environment; the analogy of "observer" leads to a misleading implication that there's something special if an agent or conscious entity is doing the observation, which it is not.
Post reply on HN