Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack and Rich Sutton partner to accelerate development of AGI

amii.ca

Post reply on HN
Print view View original

141–150 of 226 posts

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#141

Post by TedDoesntTalk »

xpe wrote: post #109
TedDoesntTalk wrote: post #49

Earlier quoted context omitted.

Creator of "Centre for Effective Altruism"? Longtermism? No thanks. I thought that philosophy has been debunked as a scam after the likes of Sam Bankman and others. I mean, living unethically today in order to affect future lives positively is not for me.

> I thought that philosophy has been debunked as a scam after the likes of Sam Bankman and others. How can a philosophy be "debunked as a scam" by a fraudster being in the news? Or are you talking about something else? One can distort and misuse almost any moral philosophy, such as utilitarianism or any other. History is filled with examples. You may disagree with particular assessments, conclusions, or even the basi…

> Or are you talking about something else?

I'm talking about immoral or questionable behavior today to justify a possible future. It's like going back in time to kill Hitler's grandfather to prevent Hitler's birth. You have no idea of unintended consequences, and you are intentionally causing harm now for a future completely out of your control (although you deceive yourself into thinking it is in your control by taking action today).

> Pick an existential risk, such as nuclear proliferation or artificial intelligence. Study the risks and the benefits.

I lived through the Cold War. I don't need this exercise because, the bottom line is, no one knows what existential risk - if any - will happen. The worst of the Cold War never happened, yet many of us thought it would. Likewise, you might choose to focus on AI as your existential risk, go through all the sacrifices and immoral behaviors that longtermism leads you to do in order to mitigate AI, and then a giant meteor collides with Earth. Oops! You chose the wrong existential risk. And meanwhile, you compromised your morals and right behavior in order to mitigate the wrong existential risk. No thanks, not for me.

> My recommendation is simple: read MacAskill's book

No. I might read about 1000 books in my lifetime (85 reading years * 12 per year). You don't get to choose one of them. But I do appreciate your passion and evangalism.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#142

Post by renewiltord »

cactusplant7374 wrote: post #119
loveparade wrote: post #86

Earlier quoted context omitted.

I agree. The guruism on HN about this guy is crazy. Yeah, he made some really cool games 20-30 years ago, but do we really need a front page post every time he tweets? I'd prefer to hear about the company once they actually get something done.

There aren’t enough John Carmack submissions. He has great insight and has lived quite a full life in the industry.

Yeah he's an actual superstar. Very skilled engineer. This trend of dismissing successful people is a bit lame.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#143

Post by TedDoesntTalk »

xpe wrote: post #118
xpe wrote: post #27

Earlier quoted context omitted.

Without taking a particular position on how the probabilities shake out, I suggest critically reading “What We Owe the Future” by William MacAskill and:or other long-term writings that use statistical thinking over a wide range of future scenarios.

I don't get to meet the people who spend time on HN. But I'm curious: who doesn't see the value in _critically_ reading long-term thinking over a wide range of future scenarios? Too busy coding? Reading about something for _your_ career? In service of _your_ family? Trying to get ahead? Yeah, I get it. But tell me: isn't thinking long-term worth something akin to a few hours a month? You don't need to be doom-and-glo…

> isn't thinking long-term worth something akin to a few hours a month?

> Wouldn't it be nice to have some confidence that we're setting up future generations to have the possibility to at least have what we do

This presupposes that you have some control of the long-term. You don't, any more than the flap of a butterfly's wings in Melbourne affects the velocity of the wind in NYC several weeks later.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#145

Post by omneity »

espadrine wrote: post #82
sillysaurusx wrote: post #17

John is nice to work with. Very numbers-focused. If you have a flaw in your logic, he’ll call you on it. And he’s looking for vice-versa —- nothing seems to make him happier than when you point out a flaw that he hadn’t considered. I don’t know Rich, but John has said they work well together, and it’s exciting to see them collaborate. There are some unique ideas at Keen worth pursuing that I haven’t seen elsewhere. L…

Richard Sutton has done fascinating RL research in the past. Lately, though, he has been advocating for ending humanity, which is a bit more chilling[1]: > Rather quickly, [AI] would displace us from existence. […] It behooves us to give them every advantage, and to bow out when we can no longer contribute… [1]: https://youtu.be/NgHFMolXs3U?si=5sriRVFKw61K5yes&t=399

While I don't agree with your conclusion, as it seems Rich Sutton was simply paraphrasing Hans Moravec, I found the video pretty interesting.

Thank you for sharing it.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#146

Post by [deleted] »

sillysaurusx wrote: post #17

John is nice to work with. Very numbers-focused. If you have a flaw in your logic, he’ll call you on it. And he’s looking for vice-versa —- nothing seems to make him happier than when you point out a flaw that he hadn’t considered. I don’t know Rich, but John has said they work well together, and it’s exciting to see them collaborate. There are some unique ideas at Keen worth pursuing that I haven’t seen elsewhere. L…

[deleted]

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#147

Post by MrDresden »

pretendscholar wrote: post #132
zoogeny wrote: post #100

Earlier quoted context omitted.

I agree with the sentiment and I've been watching his tweets on AI to see if Carmack has any relevant input on the field. But what I think is very interesting is that, from my totally lay-person perspective, the current challenge is scaling out the AI systems we have. We know what systems like GPT-3.5 and GPT-4 require to run and it is now an engineering challenge to scale up those systems (or drastically reduce thei…

What exactly are these first principles everyone keeps talking about? Kind of like how everyone keeps talking about updating Bayesian priors without ever actually using bayes rule.

According to Wikipedia[0]:

> In philosophy and science, a first principle is a basic proposition or assumption that cannot be deduced from any other proposition or assumption.

In that sense the OP's comment might read as Carmack partnering up with someone who has a strong foundational understanding of hight level mathematics (if my understanding of the term and OP's post is correct).

[0] https://en.wikipedia.org/wiki/First_principle?wprov=sfla1

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#148

Post by geraldwhen »

JKCalhoun wrote: post #97
xwowsersx wrote: post #52

Earlier quoted context omitted.

The claim that "humans are basically long running LLMs" oversimplifies the complexity of human cognition and experience. Humans don't just process information; they experience emotions, desires, and subjective experiences that are deeply intertwined with their cognition. LLMs don't have feelings, motivations, or consciousness. Humans have inner subjective experience, self-awareness, and the ability to reflect on our…

I believe though a significant part of our mind is very much like an LLM. Not all of it of course.

Yes, the parts of the brain attempting to apply correct grammar to a given idea are likely quite similar.

As in, the part with producing language. I.e large language model.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#149

Post by twodave »

mkl wrote: post #40
otabdeveloper4 wrote: post #23

Earlier quoted context omitted.

Why not? You're using a very bizzare conception of "sentience".

They're using the science fiction definition, which is "sapient" and has been around for years (unfortunately). https://en.wiktionary.org/wiki/sentient

Or maybe the "ramen" (vs "varelse") from the Ender's Game series...

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#150

Post by nlitened »

Gud wrote: post #135

Accelerate AGI So the hardware has finally become powerful enough to create the super mind. Remember this: John Carmack is a hard working games developer. He is not done great philosopher. By chance, some Armageddon level shot has been put in his hands. All I can say, please stop and think for a while Mr Carmack. You are playing with fire. Don’t kill us all, please.

At this point, it’s not about whether we get AGI or not. It’s about who gets AGI first: USA or China
Post reply on HN