Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack and Rich Sutton partner to accelerate development of AGI

amii.ca

Post reply on HN
Print view View original

91–100 of 226 posts

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#91

Post by beremaki »

JKCalhoun wrote: post #26
mark_l_watson wrote: post #13

Their focus is long running digital intelligences that interact with their environment over long periods of time. I enjoyed reading The Alberta Plan when it was released earlier this year https://arxiv.org/abs/2208.11173

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

> By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) would answer?

It very likely would not compare. Humans are shaped by their subjective experiences not by incoming data and you cannot know what a subject is experiencing solely through the incoming data (except if you have a theory accurately modeling the mind of the subject which is exactly what we are currently missing).

Your LLM won't experience the subject's heartbreaks, joy, grief, shame, hope etc. It will have heard and be able to talk about those, but it will not give accurate answers about what it felt like. Also it won't be able to predict/model accurately how the subject has been changed by those experiences so it could make very wrong assumptions about what the subject could/would do in the future.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#92

Post by beefield »

rvz wrote: post #55
cactusplant7374 wrote: post #48

Earlier quoted context omitted.

It’s AGI. So far everyone’s track record is nonexistent.

No. The parent comment said "AI Research". If you want "AGI" so badly, you certainly need to have an "AI research" background and a track record. Can't achieve your "AGI" without that. Given that Carmack has a close to zero track record in this research, it makes total sense for him to partner with someone that does.

> Can't achieve your "AGI" without that.

Can I ask, what makes you so confident on that? For all I know (which is admittably not much), current AI may be going to a direction completely orthogonal to achieving AGI.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#93

Post by FrustratedMonky »

dartos wrote: post #64
FrustratedMonky wrote: post #38

Earlier quoted context omitted.

Not sure about everyone else, but I write one word at a time.

Yeah but do you write that one word based on the past 1000 or so words that come before it and how each of those words relate to each other with a perfect recollection of each word at the same time? And if you believe you do that subconsciously, then how can you be sure you don’t subconsciously plan a few words ahead?

I don't think I do. To a large degree, GPT does write better.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#94

Post by Michelangelo11 »

sillysaurusx wrote: post #29
throwaway10965 wrote: post #22

Earlier quoted context omitted.

Does that mean you work with them? Can you tell us more about that?

I can’t go into specifics for obvious reasons, but I can tell you they’re a smart, capable crew, with lots of resources, and time is on their side. If I had to place bets on which lab will invent agi, they’d be near the top, precisely because they’re exploring ideas that others don’t seem to be focusing on. It’s the early stages, so it’ll take a few years to get to results. But they’re laser-focused. There are a bunc…

Can you share if the approach is more GOFAI or more neural network-type stuff? Just curious, and I understand if you can't say much, of course.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#95

Post by FrustratedMonky »

the_other wrote: post #50
FrustratedMonky wrote: post #38

Earlier quoted context omitted.

Not sure about everyone else, but I write one word at a time.

You may write one word at a time but the grammar of most languages forces their users to know what they’re going to write several words ahead of the current word.

Ok. Why do you think GPT isn't doing that? or is different?

It might be calculating the next word, because you can only write one word at a time, but you can't say that the current next word isn't influenced by a few words ahead. Don't think the current understanding of what LLM's do internally can rule that out.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#97

Post by JKCalhoun »

xwowsersx wrote: post #52
volent wrote: post #28

Earlier quoted context omitted.

I thought about this too. People say that LLM models are only saying the most common tokens that come after the previous token. And that this makes them incomparable to human intelligence. But Humans are basically long running LLMs that are retrained in real-time. We are the product of our environment.

The claim that "humans are basically long running LLMs" oversimplifies the complexity of human cognition and experience. Humans don't just process information; they experience emotions, desires, and subjective experiences that are deeply intertwined with their cognition. LLMs don't have feelings, motivations, or consciousness. Humans have inner subjective experience, self-awareness, and the ability to reflect on our…

I believe though a significant part of our mind is very much like an LLM. Not all of it of course.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#98

Post by ang_cire »

tharne wrote: post #59
xpe wrote: post #16

Earlier quoted context omitted.

A “solution” that isn’t viable nor palatable isn’t a true solution; it is often merely a hope that people are not people.

You’re presupposing every problem has a nice solution. This is almost certainly false

Any solution is a nice solution depending on your frame of reference. There is no such thing as objectively good or bad without defining the acceptable and unacceptable effects.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#99

Post by Qwertious »

belter wrote: post #87
espadrine wrote: post #82

Earlier quoted context omitted.

Richard Sutton has done fascinating RL research in the past. Lately, though, he has been advocating for ending humanity, which is a bit more chilling[1]: > Rather quickly, [AI] would displace us from existence. […] It behooves us to give them every advantage, and to bow out when we can no longer contribute… [1]: https://youtu.be/NgHFMolXs3U?si=5sriRVFKw61K5yes&t=399

That is not, what he is advocating, at the timestamp you referred to.

>That is ... what he is advocating, at the timestamp you referred to.

There's no such thing as taking a quote out of context, only enhancing it!

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#100

Post by zoogeny »

matusp wrote: post #43

I like Carmack but is it really necessary to have a post about everytime he mentions AI (which is every week for the past year)? So far, his track record in AI research is non-existent.

I agree with the sentiment and I've been watching his tweets on AI to see if Carmack has any relevant input on the field.

But what I think is very interesting is that, from my totally lay-person perspective, the current challenge is scaling out the AI systems we have. We know what systems like GPT-3.5 and GPT-4 require to run and it is now an engineering challenge to scale up those systems (or drastically reduce their costs). And those systems are surprisingly GPU like. Carmack is the kind of low-level bit crunching, memory obsessed kind of guy that might be able to squeeze out the performance from the hardware.

So him teaming up with a high-level first principles math and science research type is a pretty interesting combo. There is no telling if such a partnership will ultimately result in an advancement - but on paper at least it is an interesting approach. All they are missing is a strong hardware guy (maybe Keller will go beyond investing in Carmack's Keen Technologies?).

Post reply on HN