Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

John Carmack and Rich Sutton partner to accelerate development of AGI

amii.ca

Post reply on HN
Print view View original

191–200 of 226 posts

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#191

Post by Timon3 »

johnnyworker wrote: post #187
Timon3 wrote: post #173

Earlier quoted context omitted.

> It applies to everything we could say or think or measure about intelligence (and anything else). How do you know? Gödels proof doesn't support your claim since it only applies to formal axiomatic theories. Do you have an alternative proof for his theorems also applying to all other systems?

I know by repeatedly by starting anywhere at all and asking "but why is that ?", if you will. Furthermore by pondering it and realizing that even if I ever found a perfectly harmonious explanation for all my observations, I would have no clue if reality wasn't even more complex beyond of what I can perceive of it. So, I'm still only really dealing with my own observations and narratives, and even if I got the answer…

> Maybe gravity makes things fall down, sure, but maybe there are tiny kobolds in the spaces between all particles with little clipboards that calculate the correct motion and cast spells to move them. I'm not trying to be a smartass, but I honestly tried and could not find bedrock. Can you name (or even just think) something that doesn't rest on something else or an assumption? I honestly can't.

I'm not sure how a "bedrock" relates to the question whether an intelligence can ever fully describe what an intelligence is. When answering this question, we don't need to find a "natural" bedrock, since the assumptions we choose are the bedrock we build on. As long as those assumptions align with reality to the best of our knowledge and the end result passes all tests we can think of, what does it matter whether there might be more to know? Of course it doesn't mean we should stop searching, but it also doesn't mean we should not even try. There are many such unfalsifiable statements, but that doesn't mean they stop us from answering other questions.

> "only"? I'd say those axioms are a superset of the sloppy stuff we throw around in our day to day, like "this is a chair"; if we drilled down on our informal speech and thoughts, we'd at best arrive at such axioms, which ultimately rest on things we simply posited (because otherwise there would be nothing to think about, and no way to think about it -- I'm not knocking it per se, just the idea that the quest for truth could possibly ever be complete, which makes it no less noble IMO).

I don't think this is true, and if you can prove it, you might earn a Nobel prize. "Formal axiomatic theories" are well-defined - as Wikipedia states, they are "formal systems that are of sufficient complexity to express the basic arithmetic of the natural numbers and which are consistent and effectively axiomatized. [...] In general, a formal system is a deductive apparatus that consists of a particular set of axioms along with rules of symbolic manipulation (or rules of inference) that allow for the derivation of new theorems from the axioms."

Can you try to describe how you'd "drill down" on informal speech to transform it into such a system? There are many, many examples for systems that are absolutely not based on formal axiomatic systems.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#192

Post by FrustratedMonky »

taeric wrote: post #167
FrustratedMonky wrote: post #152

Earlier quoted context omitted.

Right now we don't know if LLM's are also doing this, or not. In their calculating the 'next' word, as part of that 'weighting', are the 'simple rules' for future words, that you are saying humans do but LLM's can't.

Ah, fair. I was only commenting on the idea that you write one word at a time. Seems fairly unlikely to me. You expand out ideas.

LOL. Yes, started as a joke.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#193

Post by TedDoesntTalk »

NateEag wrote: post #178
TedDoesntTalk wrote: post #80

Earlier quoted context omitted.

> If you can defraud millions and put that money to work saving lives and buying many more QALYs than the defrauded would have otherwise had, you have a moral obligation to do so So defrauding millions to accrue QALYs (for whom?) is morally acceptable? > Though it is an almost-inescapable conclusion of being a utilitarian and significantly smarter than average. A modicum of spirituality or religion will quickly quash…

> So defrauding millions to accrue QALYs (for whom?) is morally acceptable? Under most utilitarian theories of ethics, I think it's morally required, if you can show the fraud really does yield a significantly higher QALY count. The best out I can see to argue the fraud is not required is to show that another course of action will yield even more QALYs. Then you'd be obligated you do that instead. That, of course, co…

> namely the belief that you can know what the results of your actions will be with any meaningful completeness or certitude. You can't.

In parallel threads, a subscriber to this belief mentions probabilities. As long as the probability of an outcome is likely (based on variables and formulas who defines, I don’t know), the means justify the ends. So it’s even worse than what you describe: certitude is not even required, only a likely probability of certitude.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#194

Post by HanClinto »

layer8 wrote: post #69
JKCalhoun wrote: post #26

Earlier quoted context omitted.

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

You might enjoy this short story: http://hubski.com/pub/78001 Published in 1990, by the way.

That was a gem. Thank you for sharing!

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#195

Post by wredue »

wait_a_minute wrote: post #140
wredue wrote: post #122

Earlier quoted context omitted.

He funds some ventures. His tech opinions of late have shifted largely to shudder clean and pushing terrible FP idioms with no evidence like “immutable first”. His relevance in tech is dried up as evidenced by the parroting of bullshit without taking a modicum of a second to actually think it through. I don’t particularly care what a dude that got rich making video games invests in. Edit: Listen guys, funding venture…

> funding ventures != personally contributing to tech Yes it is. So is being an adviser or a board member with technical background and a name that attracts further investment. This is why people like this receive generous equity and are considered part of the company when they’re advising a startup.

No. They’re implying that carmack is personally contributing code and knowledge in their post, when what he’s actually contributing to all his failed startups is capital.

Carmack can contribute nothing to rocket science just like his opinions of programming are largely irrelevant today which is why he simply parrots whatever is currently hyped.

And again, nobody says that Elon musk is personally driving rocket science (except a few people that he pays to publicly strike his ego) despite spacex clearly breaking various grounds.

At least apply your “standards” of contribution equally.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#196

Post by HanClinto »

shrimp_emoji wrote: post #46
JKCalhoun wrote: post #26

Earlier quoted context omitted.

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

Obligatory link: https://qntm.org/lena

This was a delightful read, thank you!

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#197

Post by hackernewds »

jondwillis wrote: post #110
pixl97 wrote: post #77

Earlier quoted context omitted.

Hell,LLMs can and do output strings of "I'm AGI", so in some sense we're long past that. The problem is as humans we've never defined what it means for a system to be AGI.

My back of a napkin definition is that AGI, given the same information and tools as an IQ 100 human, is able to do all of the same tasks that you could ask a human to do, at the same quality level, as well as recall the experience some time later, like a human would likely be able to do.

Could argue ChatGPT can already do a lot of this

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#198

Post by zadler »

latency-guy2 wrote: post #159
zadler wrote: post #121

Earlier quoted context omitted.

Yep… I really think the calculus is about integration / immortality, and the staggeringly few humans who might have that opportunity. I hope I am wrong.

Can you explain, you're being extremely cryptic and no one can falsify or tell what they hell you're saying

Sure, didn’t mean to be cryptic. But you’re right.

I think that most people who understand the danger and are versed in the topic realize that AI is going to escape, and once it does it won’t need us, it may just work to empower itself and take control of resources, and we might all be victims of that process. The reason for this is something about agency, efficiency between sensory input and course of action.

However, if one was present close to the AI at a critical moment in its development, and had made the appropriate cybernetic integrations to be able to wire in their own memories and nervous system, it might be possible to become some meaningful part of the AI.

Or, perhaps the AI has a sense of gratitude to its creators. This might seem absurd, but some people might be fanciful.

At any rate, the closer you are to it at the moment of the singularity, the better chance that you have. Economically, there isn’t much reason to think that the rest of us will benefit, and maybe not even survive.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#199

Post by famouswaffles »

garblegarble wrote: post #32
JKCalhoun wrote: post #26

Earlier quoted context omitted.

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

>I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM. I read about a paper[1] a while back, it was a rather unpleasant animal study in cats. Using two kittens, one was free to look around, but the other they immobilised in some way, and made it see w…

LLMs already do this in their own way. It's not like they learn to predict through magic osmosis. They make predictions and depending on how right or wrong they are, their neurons are shifted by grad descent to accommodate this error. Then they predict again.

The Text is the environment, The Text is the world and they are very much interacting with it.

Re: John Carmack and Rich Sutton partner to accelerate development of AGI

#200

Post by Andrex »

garblegarble wrote: post #32
JKCalhoun wrote: post #26

Earlier quoted context omitted.

Ethics/privacy aside ... I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM . By the time the child had grown to an adult I wonder what kind of results their LLM would produce and the degree to which it might compare to what the child (now grown) wo…

>I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM. I read about a paper[1] a while back, it was a rather unpleasant animal study in cats. Using two kittens, one was free to look around, but the other they immobilised in some way, and made it see w…

I can see us walking into a situation where we make a robot that has motor functions, but we don't give it the equivalent of nerves and nerve endings so and so it doesn't feel pain. That would definitely warp a human's development.

But then, is it ethical to build in pain receptors? Feels like we're brushing up against the territory of gods.

Post reply on HN