Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You are the scariest monster in the woods

jamie.ideasasylum.com

Post reply on HN
Print view View original

161–170 of 405 posts

Re: You are the scariest monster in the woods

#161

Post by itsnowandnever »

boole1854 wrote: post #22
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

the penrose-lucas argument is the best bet: https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument

the basic idea being that either the human mind is NOT a computation at all (and it's instead spooky unexplainable magic of the universe) and thus can't be replicated by a machine OR it's an inconsistent machine with contradictory logic. and this is a deduction based on godel's incompleteness theorems.

but most people that believe AGI is possible would say the human mind is the latter. technically we don't have enough information today to know either way but we know the human mind (including memories) is fallible so while we don't have enough information to prove the mind is an incomplete system, we have enough to believe it is. but that's also kind of a paradox because that "belief" in unproven information is a cornerstone of consciousness.

Re: You are the scariest monster in the woods

#162

Post by nradov »

woeirua wrote: post #44

There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war…

Well it's not everyone. I guess I am in "tech circles" and have zero fear of AGI. Everyone who is (or claims to be) "deathly afraid" is either ignorant or unserious or a grifter. Their arguments are essentially a form of secular religion lacking any firm scientific basis. These are not people worth listening to.

Re: You are the scariest monster in the woods

#163

Post by gpderetta »

andy99 wrote: post #124
bee_rider wrote: post #116

Earlier quoted context omitted.

I think it is bad writing on the part of the author. Or maybe good writing for getting us engaged with the blog post, bad for making an argument though. They include a line that they don’t believe in the possibility of AGI: > I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the…

> humans physically exist so our brains must be possible to build within the existing laws of physics This is an unscientific position to take. We have no idea how our brains work, or how life, consciousness, and intelligence work. It could very well be that’s because our model of the world doesn’t account for these things and they are not in fact possible based on what we know. In fact I think this is likely. So it…

>> humans physically exist so our brains must be possible to build within the existing laws of physics

> This is an unscientific position to take

The universe being constrained observable and understandable, natural laws is pretty much a fundamental axiom of the scientific method.

Re: You are the scariest monster in the woods

#164

Post by zeroonetwothree »

woeirua wrote: post #44

There's this weird disconnect in tech circles, where everyone is deathly afraid of AGI, but totally asleep on the very real possibility of thermonuclear war breaking out in Europe or Asia over the next 10 years. There's already credible evidence that we came perilously close to the use of tactical nuclear weapons in Ukraine which likely would've spiraled out of control. AGI might happen, but the threat of nuclear war…

Everyone thinks their own field is the most important and deserving of attention and funding. Big surprise.

Re: You are the scariest monster in the woods

#165

Post by jncfhnb »

48terry wrote: post #158
jncfhnb wrote: post #40

Earlier quoted context omitted.

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

Wow, who would have thought it was that easy? Wonder why nobody has done this incredibly basic solution to AGI yet.

The framework is easy. The implementation is hard and expensive. The payoff is ambiguous. AGI is not a binary thing that we either have or don’t. General intelligence is a vector.

And people are working on this.

Re: You are the scariest monster in the woods

#166

Post by omnicognate »

nonethewiser wrote: post #136
omnicognate wrote: post #88

Earlier quoted context omitted.

If you genuinely want the strongest statement of it, read The Emperor's New Mind followed by Shadows of the Mind, both by Roger Penrose. These books often get shallowly dismissed in terms that imply he made some elementary error in his reasoning, but that's not the case. The dispute is more about the assumptions on which his argument rests, which go beyond mathematical axioms and include statements about the nature o…

Can you critique my understanding of his argument? 1. Any formal mathematical system (including computers) have true statements that cannot be proven within that system. 2. Humans can see the truth of some such unprovable statements. Which is basically Gödel's Incompleteness Theorem. https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_... Maybe a more ELI5 1. Computers follow set rules 2. Humans can create ru…

That's too brief to capture it, and I'm not going to try to summarise(*). The books are well worth a read regardless of whether you agree with Penrose. (The Emperor's New Mind is a lovely, wide-ranging book on many topics, but Shadows of the Mind is only worth it if you want to go into extreme detail on the AI argument and its counterarguments.)

* I will mention though that "some" should be "all" in 2, but that doesn't make it a correct statement of the argument.

Re: You are the scariest monster in the woods

#167

Post by raldi »

The author comes so close to getting it, with the paragraph about how if you drop a human into an environment, they inevitably take over as the deadliest and most powerful creature.

But the next step is to ask why; in the case of the Gruffalo it was obvious: fangs, claws, strength, size…

In the case of humans, it’s because we’re the most intelligent creature in the forest. And for the first time in our history, we’re about to not be.

Re: You are the scariest monster in the woods

#168

Post by disambiguation »

boole1854 wrote: post #22
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

I suppose intelligence can be partitioned as less than, equal to, or greater than human. Given the initial theory depends on natural evidence, one could argue there's no proof that "greater than human" intelligence is possible - depending on your meaning of AGI.

But then intelligence too is a dubious term. An average mind with infinite time and resources might have eventually discovered general relativity.

Re: You are the scariest monster in the woods

#169

Post by nonethewiser »

jjkaczor wrote: post #120
nonethewiser wrote: post #105

Earlier quoted context omitted.

Where is the "what the thing cares about" part? When I look at that loop my thought is, "OK, the sensory inputs have updated. There are changes. Which ones matter?" The most naive response I could imagine would be like a git diff of sensory inputs. "item 13 in vector A changed from 0.2 to 0.211" etc. Otherwise you have to give it something to care about, or some sophisticated system to develop things to care about. E…

... well, humans are not always known for making correct, logical or sensical decisions when they update their input loops either...

that only makes humans harder to model

Re: You are the scariest monster in the woods

#170

Post by jncfhnb »

zeroonetwothree wrote: post #139
jncfhnb wrote: post #40

Earlier quoted context omitted.

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

This seems to miss a core part of intelligence, which is a model of the world and the actors in it (theory of mind).

That is an emergent property that the system would learn to navigate the world with as a function of sensory inputs and “emotional state”.

Video game bots already achieve this to a limited extent.

Post reply on HN