Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack and Rich Sutton partner to accelerate development of AGI

amii.ca

Post reply on HN
Print view View original

161–170 of 226 posts

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#161

Post by AJRF »

garblegarble wrote: post #124
AJRF wrote: post #106

Earlier quoted context omitted.

Curious to know how you found this paper? I'd like to be able to find interesting papers but I've not found a place of curation unfortunately.

I'm afraid I don't remember how I found this paper, but it might be from following Dr Abeba Birhane[1], she's an anti-AI academic (which is why I followed her originally, to get reality checks / contrasting viewpoints), but she also posts and boosts a lot of interesting cognitive science (her field). I also follow Olivia Guest[2] for the same reason. Many of the papers go over my head at times of course (despite my i…

thank you for taking the time to answer and to provide links, appreciate it.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#162

Post by tibbydudeza »

There was an Infocom game called AMFV (A Mind Forever Voyaging) where the protagonist wakes up and realizes that he was just a simulation of a human being that has "grown" up into adulthood.

All the formative things that shaped him in life - kindergarten, your first high school romance - a breakup and parent squabbling you experienced.

Always thought that was a way to make an AGI.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#163

Post by tibbydudeza »

waihtis wrote: post #123
cactusplant7374 wrote: post #119

Earlier quoted context omitted.

There aren’t enough John Carmack submissions. He has great insight and has lived quite a full life in the industry.

he has failed to address the most important technological domain of the hacker news crowd - making incremental improvements to frontend web technologies

He would have invented QuakeJS like he did with QuakeC :). React - get the fug out of here :)

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#164

Post by aik »

TedDoesntTalk wrote: post #143
xpe wrote: post #118

Earlier quoted context omitted.

I don't get to meet the people who spend time on HN. But I'm curious: who doesn't see the value in _critically_ reading long-term thinking over a wide range of future scenarios? Too busy coding? Reading about something for _your_ career? In service of _your_ family? Trying to get ahead? Yeah, I get it. But tell me: isn't thinking long-term worth something akin to a few hours a month? You don't need to be doom-and-glo…

> isn't thinking long-term worth something akin to a few hours a month? > Wouldn't it be nice to have some confidence that we're setting up future generations to have the possibility to at least have what we do This presupposes that you have some control of the long-term. You don't, any more than the flap of a butterfly's wings in Melbourne affects the velocity of the wind in NYC several weeks later.

Tell that to the millions of individual humans that have advanced humanity in some way or another for the past tens of thousands of years, through some discovery or invention or inspiring groups of people to do something great.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#166

Post by ben_w »

dartos wrote: post #58
volent wrote: post #28

Earlier quoted context omitted.

I thought about this too. People say that LLM models are only saying the most common tokens that come after the previous token. And that this makes them incomparable to human intelligence. But Humans are basically long running LLMs that are retrained in real-time. We are the product of our environment.

You’re vastly oversimplifying humans. That’s like saying a physics simulation is basically an entire sub universe on your computer. It sounds true, but it’s just not. It’s a gross oversimplification I think Gödel had a proof for how it’s impossible to fully describe a system from within that system. That’s the nail in the coffin for AGI. No matter how much data we give it, no matter how big it is, it’ll never be “hum…

> I think Gödel had a proof for how it’s impossible to fully describe a system from within that system. That’s the nail in the coffin for AGI.

If it was a nail in the coffin for AGI, then humans, who have exactly the same limit, wouldn't count as a general intelligence.

While such a definition would be possible, I don't think it's useful.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#167

Post by taeric »

FrustratedMonky wrote: post #152
taeric wrote: post #151

Earlier quoted context omitted.

Almost certainly not true. Consider pronouns, as they reference other words, therefore cannot be written in isolation. Or simple rules about when to use "an" or "a". Shaping your words to sound correct is very common. Both in speech and in writing. Sometimes it is finding how to fit a word you want to use into a sentence. Sometimes it is building a rhyme. You may feel that you go a word at a time, but that really sho…

Right now we don't know if LLM's are also doing this, or not. In their calculating the 'next' word, as part of that 'weighting', are the 'simple rules' for future words, that you are saying humans do but LLM's can't.

Ah, fair. I was only commenting on the idea that you write one word at a time. Seems fairly unlikely to me. You expand out ideas.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#168

Post by xpe »

TedDoesntTalk wrote: post #141
xpe wrote: post #109

Earlier quoted context omitted.

> I thought that philosophy has been debunked as a scam after the likes of Sam Bankman and others. How can a philosophy be "debunked as a scam" by a fraudster being in the news? Or are you talking about something else? One can distort and misuse almost any moral philosophy, such as utilitarianism or any other. History is filled with examples. You may disagree with particular assessments, conclusions, or even the basi…

> Or are you talking about something else? I'm talking about immoral or questionable behavior today to justify a possible future. It's like going back in time to kill Hitler's grandfather to prevent Hitler's birth. You have no idea of unintended consequences, and you are intentionally causing harm now for a future completely out of your control (although you deceive yourself into thinking it is in your control by tak…

> no one knows what existential risk - if any - will happen.

We live in a world of uncertainty. We can and must make probabilistic assessments. Don't pretend otherwise by using the classic "appeal to ignorance" fallacy.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#169

Post by johnnyworker »

Timon3 wrote: post #104
dartos wrote: post #58

Earlier quoted context omitted.

You’re vastly oversimplifying humans. That’s like saying a physics simulation is basically an entire sub universe on your computer. It sounds true, but it’s just not. It’s a gross oversimplification I think Gödel had a proof for how it’s impossible to fully describe a system from within that system. That’s the nail in the coffin for AGI. No matter how much data we give it, no matter how big it is, it’ll never be “hum…

> I think Gödel had a proof for how it’s impossible to fully describe a system from within that system. That’s the nail in the coffin for AGI. Gödels theorems are about formal axiomatic theories. To apply them to human intelligence, you'd have to prove that human intelligence springs from formal axiomatic theories. I don't think this is possible, which would mean that you can't apply the theorems. > No matter how muc…

> To apply them to human intelligence, you'd have to prove that human intelligence springs from formal axiomatic theories.

It applies to everything we could say or think or measure about intelligence (and anything else). All that probably "spring from" things that don't (rely on our axioms), but they are not accessible to us, so that doesn't really help.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#170

Post by ben_w »

FrustratedMonky wrote: post #38
4death4 wrote: post #34

Earlier quoted context omitted.

What makes you think that? What we can LLMs are a specific architecture that literally does predict the next token, one at a time. This isn’t the only way to generate text. Perhaps humans use a method that synthesizes all the text at once. Or some other, unimaginable way.

Not sure about everyone else, but I write one word at a time.

I seem to have two parts of my inner monologue, one comes up with complete concepts, the other puts them into words. When I started to notice this, I tried skipping the "make words" part to save time, as clearly I had not only had the thought but also was aware that I had already had the thought. This felt wrong in a way I have no words to describe as there's nothing else like that particular feeling of wrongness, though I can analogise it as being almost but not quite entirely unlike annoyance.

Anyway, point is 80% of this comment was already in my head before I started typing; the other 20% was light editing and an H2G2 reference.

Post reply on HN