Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack and Rich Sutton partner to accelerate development of AGI

amii.ca

Post reply on HN
Print view View original

151–160 of 226 posts

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#151

Post by taeric »

FrustratedMonky wrote: post #38
4death4 wrote: post #34

Earlier quoted context omitted.

What makes you think that? What we can LLMs are a specific architecture that literally does predict the next token, one at a time. This isn’t the only way to generate text. Perhaps humans use a method that synthesizes all the text at once. Or some other, unimaginable way.

Not sure about everyone else, but I write one word at a time.

Almost certainly not true. Consider pronouns, as they reference other words, therefore cannot be written in isolation. Or simple rules about when to use "an" or "a".

Shaping your words to sound correct is very common. Both in speech and in writing. Sometimes it is finding how to fit a word you want to use into a sentence. Sometimes it is building a rhyme.

You may feel that you go a word at a time, but that really shows how embedded language is.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#152

Post by FrustratedMonky »

taeric wrote: post #151
FrustratedMonky wrote: post #38

Earlier quoted context omitted.

Not sure about everyone else, but I write one word at a time.

Almost certainly not true. Consider pronouns, as they reference other words, therefore cannot be written in isolation. Or simple rules about when to use "an" or "a". Shaping your words to sound correct is very common. Both in speech and in writing. Sometimes it is finding how to fit a word you want to use into a sentence. Sometimes it is building a rhyme. You may feel that you go a word at a time, but that really sho…

Right now we don't know if LLM's are also doing this, or not.

In their calculating the 'next' word, as part of that 'weighting', are the 'simple rules' for future words, that you are saying humans do but LLM's can't.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#153

Post by belter »

DHaldane wrote: post #134
belter wrote: post #127

Earlier quoted context omitted.

Please watch it again then come back here...

Curious to hear your take. Sutton seems to present AI as worthy successors to humanity. What do you think he is saying?

He is only presenting the arguments of Hans Moravec and arguing for Succession Planning. He even mentions...when we can "no longer contribute" and we "enjoy a comfortable retirement". Almost mention the moral argument that we owe to allow a greater mind to evolve. A trans human might even be the only way humanity can survive. Then he contrasts it with the fearful view of the evolution of AI.

From an abstract on "The Age Of Robots": "...rowing computer power over the next half-century will allow this reptile stage will be surpassed, in stages_ _producing robots that learn like mammals, model their world like primates and eventually reason like humans._ _Depending on your point of view, humanity will then have produced a worthy successor, or transcended inherited_ _limitations and transformed itself into something quite new. No longer limited by the slow pace of human learning and_ _even slower biological evolution, intelligent machinery will conduct its affairs on an ever faster, ever smaller scale, until_ _coarse physical nature has been converted to fine-grained purposeful thought..."

https://en.wikipedia.org/wiki/Hans_Moravec

Now where is arguing for the end of Humanity? To quote from his presentation: "...Why don't we rejoice in their greatness as a symbol and extension of humanity greatness, and work together toward a greater and inclusive civilization"

Is Captain Jean-Luc Picard arguing for the end of humanity, is he less of an human?...Just for advocating for Data? - https://youtu.be/ol2WP0hc0NY

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#154

Post by AndrewKemendo »

JKCalhoun wrote: post #26
mark_l_watson wrote: post #13

Their focus is long running digital intelligences that interact with their environment over long periods of time. I enjoyed reading The Alberta Plan when it was released earlier this year https://arxiv.org/abs/2208.11173

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

This is the long term goal of any Augmented Reality interface IMO

This is why I got into AR initially, because a computing system needs the input persistence of a literal parent or in the case of a self learning agent - something like a baby to perfectly observe in order to be able to create the data environment necessary for learning at the rate humans learn.

You could do it with a collection of sensors, but I think the idealized implementation is basically a perfect recreation of the sense inputs of a person as well as monitoring the person to infer the precise Markov Decision Process.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#155

Post by jedberg »

garblegarble wrote: post #32
JKCalhoun wrote: post #26

Earlier quoted context omitted.

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

>I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM. I read about a paper[1] a while back, it was a rather unpleasant animal study in cats. Using two kittens, one was free to look around, but the other they immobilised in some way, and made it see w…

This isn't a very new concept -- I learned about this in the 90s. Even then the language researchers knew that interaction with the environment was crucial for learning in humans. There were some natural experiments where kids were born without limbs for example and had trouble with early language acquisition because they couldn't experience the environment the same way as most others.

And the AI folks argued even then that the only way to create a language model that truly "understands" would be to put it in a robot that can experience the world in the same way as a human.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#156

Post by gretch »

Qwertious wrote: post #105
tharne wrote: post #59

Earlier quoted context omitted.

You’re presupposing every problem has a nice solution. This is almost certainly false

Lots of problems could be at least severely mitigated if you could throw literally millions of bodies at the problem - if AGI could deliver a $50 chip that could substitute for a human, then could shove each one into a roomba wired to an arm, and you basically have an infinite free workforce.

Yeah except, how do you manufacture them? And distribute them to where the work needs to be? And what type of fuel do they use - we don’t have infinite free energy for an infinite free workforce.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#157

Post by danielbarla »

garblegarble wrote: post #32
JKCalhoun wrote: post #26

Earlier quoted context omitted.

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

>I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM. I read about a paper[1] a while back, it was a rather unpleasant animal study in cats. Using two kittens, one was free to look around, but the other they immobilised in some way, and made it see w…

> They discovered the immobilised kitten's visual processing did not develop normally whereas the mobile kitten's did, suggesting that it's not just the sensory input, but it being feedback to some internal agency within the brain.

> I suspect the same thing would be true for attempts to develop AGI...

On the other hand, I would not be surprised if this did _not_ generalize and translate to LLMs. Biological entities on earth tend to have tons of fairly arbitrary developmental legacy which may or may not translate well to an artificial one. In the experiment, they didn't make billions of clones of the cat with a variety of different parameters and check which one worked, they just observed that yes, this standard v1 cat has some specific hard-wired developmental tendencies. It is not surprising that taking it out of the environment it was evolved for messes things up, but I would not take the extra step that this means that something is essential in general.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#158

Post by romesc »

AJRF wrote: post #106
garblegarble wrote: post #32

Earlier quoted context omitted.

>I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM. I read about a paper[1] a while back, it was a rather unpleasant animal study in cats. Using two kittens, one was free to look around, but the other they immobilised in some way, and made it see w…

Curious to know how you found this paper? I'd like to be able to find interesting papers but I've not found a place of curation unfortunately.

For what it's worth, this was a paper I was exposed to during grad school for robotics/CS. I'm not saying one has to go to school to get good curation of content, but it is certainly one way to be exposed to a well structured foundation of knowledge in your field.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#159

Post by latency-guy2 »

zadler wrote: post #121
isoprophlex wrote: post #25

Earlier quoted context omitted.

"Our moat is first mover advantage on destroying all of humanity"

Yep… I really think the calculus is about integration / immortality, and the staggeringly few humans who might have that opportunity. I hope I am wrong.

Can you explain, you're being extremely cryptic and no one can falsify or tell what they hell you're saying

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#160

Post by ben_w »

xwowsersx wrote: post #52
volent wrote: post #28

Earlier quoted context omitted.

I thought about this too. People say that LLM models are only saying the most common tokens that come after the previous token. And that this makes them incomparable to human intelligence. But Humans are basically long running LLMs that are retrained in real-time. We are the product of our environment.

The claim that "humans are basically long running LLMs" oversimplifies the complexity of human cognition and experience. Humans don't just process information; they experience emotions, desires, and subjective experiences that are deeply intertwined with their cognition. LLMs don't have feelings, motivations, or consciousness. Humans have inner subjective experience, self-awareness, and the ability to reflect on our…

> Humans have inner subjective experience, self-awareness, and the ability to reflect on our own existence. LLMs don't.

We have no test capable of determining whether or not they have those things, not even if we disregarded the limitations of our current technology capabilities and are only asking hypothetically how to differentiate.

We also don't have that for animals, or even other humans — I know I have an experience of being, but no way of telling if someone else who says they do actually does. I have to assume at least some of y'all do or humanity wouldn't have written about it since at least Descartes.

People with aphantasia report being surprised when they realise that other people do have mental images, and that they previously thought such things were invented by the film industry as a metaphor. By analogy, there may well be humans out there without qualia, who just learned to mimic the language of those of us who do, which is after all exactly what LLMs must have done if they don't have inner subjective experience. Philosophers call them P-Zombies.

Post reply on HN