Live data from Hacker News

John Carmack and Rich Sutton partner to accelerate development of AGI

amii.ca

61–70 of 226 posts

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#61
A complete departure from the topic but I have the same symptomatic awkward pose when standing for a photo as Carmack has in the article :)

But at the same time when you hold your hands back it can look even weirder! Where to put your hands in photos remains a mystery to me.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#62
post #58
post #28

Earlier quoted context omitted.

I thought about this too. People say that LLM models are only saying the most common tokens that come after the previous token. And that this makes them incomparable to human intelligence. But Humans are basically long running LLMs that are retrained in real-time. We are the product of our environment.

You’re vastly oversimplifying humans. That’s like saying a physics simulation is basically an entire sub universe on your computer. It sounds true, but it’s just not. It’s a gross oversimplification I think Gödel had a proof for how it’s impossible to fully describe a system from within that system. That’s the nail in the coffin for AGI. No matter how much data we give it, no matter how big it is, it’ll never be “hum…

> I think Gödel had a proof for how it’s impossible to fully describe a system from within that system.

Gödel's incompleteness theorems

https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#63
post #27

Earlier quoted context omitted.

Without taking a particular position on how the probabilities shake out, I suggest critically reading “What We Owe the Future” by William MacAskill and:or other long-term writings that use statistical thinking over a wide range of future scenarios.

Creator of "Centre for Effective Altruism"? Longtermism? No thanks. I thought that philosophy has been debunked as a scam after the likes of Sam Bankman and others. I mean, living unethically today in order to affect future lives positively is not for me.

> I mean, living unethically today in order to affect future lives positively is not for me.

Though it is an almost-inescapable conclusion of being a utilitarian and significantly smarter than average.

If you can defraud millions and put that money to work saving lives and buying many more QALYs than the defrauded would have otherwise had, you have a moral obligation to do so.

...there are many ways to make utilitarianism fall apart, but that one's my current favorite.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#64
post #34

Earlier quoted context omitted.

What makes you think that? What we can LLMs are a specific architecture that literally does predict the next token, one at a time. This isn’t the only way to generate text. Perhaps humans use a method that synthesizes all the text at once. Or some other, unimaginable way.

Not sure about everyone else, but I write one word at a time.

Yeah but do you write that one word based on the past 1000 or so words that come before it and how each of those words relate to each other with a perfect recollection of each word at the same time?

And if you believe you do that subconsciously, then how can you be sure you don’t subconsciously plan a few words ahead?

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#65
post #60
post #43

I like Carmack but is it really necessary to have a post about everytime he mentions AI (which is every week for the past year)? So far, his track record in AI research is non-existent.

https://en.wikipedia.org/wiki/Richard_S._Sutton

Temporal difference learning was a huge contribution.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#66
post #58
post #28

Earlier quoted context omitted.

I thought about this too. People say that LLM models are only saying the most common tokens that come after the previous token. And that this makes them incomparable to human intelligence. But Humans are basically long running LLMs that are retrained in real-time. We are the product of our environment.

You’re vastly oversimplifying humans. That’s like saying a physics simulation is basically an entire sub universe on your computer. It sounds true, but it’s just not. It’s a gross oversimplification I think Gödel had a proof for how it’s impossible to fully describe a system from within that system. That’s the nail in the coffin for AGI. No matter how much data we give it, no matter how big it is, it’ll never be “hum…

It doesn't have to be perfect. It just has to be convincing. That takes a lot less data and is much more tolerant to simplifications. Just look at the claims that were thrown around GPT's abilities at first.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#67

Earlier quoted context omitted.

How do people apply? They don't seem to have much of a web presence.

I'd accept a massive pay cut to work with such legends :/

I don’t think spending is their primary constraint; they’re probably looking for talent, no matter the cost.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#68
post #12

Earlier quoted context omitted.

Perhaps there is some belief in this that an AGI can save us from our biggest problems? Energy Crisis, Environmental Pollution, Waging Wars. But I agree with you ... it is not really urgent since we know answers for the most of the problems but do not like the solution.

A true AGI would determine the only solution to our crisis is the elimination of the human race...

Are you just basing that idea off of sci fi novels?

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#69

Their focus is long running digital intelligences that interact with their environment over long periods of time. I enjoyed reading The Alberta Plan when it was released earlier this year https://arxiv.org/abs/2208.11173

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

You might enjoy this short story: http://hubski.com/pub/78001

Published in 1990, by the way.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#70
I often wonder if the pursuit of AGI is going to require a greater focus on new hardware architecture.

All the sci-fi seems think this. I have seen Intel chips from the 80s that implemented neuron like behavior on a chip.

We are still waiting on a practical memristor.

Post reply on HN