Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You are the scariest monster in the woods

jamie.ideasasylum.com

Post reply on HN
Print view View original

151–160 of 405 posts

Re: You are the scariest monster in the woods

#151

Post by kayodelycaon »

bee_rider wrote: post #116
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

I think it is bad writing on the part of the author. Or maybe good writing for getting us engaged with the blog post, bad for making an argument though. They include a line that they don’t believe in the possibility of AGI: > I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the…

I don’t think we’ll be able to replicate consciousness until we’re able to make things alive at a biological level.

We can certainly make systems smart enough and people complicit enough to destroy society well before we reach that point.

Re: You are the scariest monster in the woods

#152

Post by walkabout »

amatecha wrote: post #86
boole1854 wrote: post #22

Earlier quoted context omitted.

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

My layman thought about that is that, with consciousness, the medium IS the consciousness -- the actual intelligence is in the tangible material of the "circuitry" of the brain. What we call consciousness is an emergent property of an unbelievably complex organ (that we will probably never fully understand or be able to precisely model). Any models that attempt to replicate those phenomena will be of lower fidelity a…

I think it's also the case that you can't replicate something actually happening, by describing it.

Baseball stats aren't a baseball game. Baseball stats so detailed that they describe the position of every subatomic particle to the Planck scale during every instant of the game to arbitrarily complete resolution still aren't a baseball game. They're, like, a whole bunch of graphite smeared on a whole bunch of paper or whatever. A computer reading that recording and rendering it on a screen... still isn't a baseball game, at all, not even a little. Rendering it on a holodeck? Nope, 0% closer to actually being the thing, though it's representing it in ways we might find more useful or appealing.

We might find a way to create a conscious computer! Or at least an intelligent one! But I just don't see it in LLMs. We've made a very fancy baseball-stats presenter. That's not nothing, but it's not intelligence, and certainly not consciousness. It's not doing those things, at all.

Re: You are the scariest monster in the woods

#153

Post by nonethewiser »

zeroonetwothree wrote: post #150
nonethewiser wrote: post #136

Earlier quoted context omitted.

Can you critique my understanding of his argument? 1. Any formal mathematical system (including computers) have true statements that cannot be proven within that system. 2. Humans can see the truth of some such unprovable statements. Which is basically Gödel's Incompleteness Theorem. https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_... Maybe a more ELI5 1. Computers follow set rules 2. Humans can create ru…

Notably, those true statements can be proven in a higher level mathematical system. So why wouldn’t we say that humans are likewise operating in a certain system ourselves and likewise we have true statements that we can’t prove. We just wouldn’t be aware of them.

>likewise we have true statements that we can’t prove

Yes, and "can't" as in it is absolutely impossible. Not that we simple haven't been able to due to information or tech constraints.

Which is an interesting implication. That there are (or may be) things that are true which cannot be proved. I guess it kinda defies an instinct I have that at least in theory, everything that is true is provable.

Re: You are the scariest monster in the woods

#155

Post by Chance-Device »

omnicognate wrote: post #147
Chance-Device wrote: post #117

Earlier quoted context omitted.

To be honest, the core of Penrose’s idea is pretty stupid. That we can understand mathematics despite incompleteness theorem being a thing, therefore our brains use quantum effects allowing us to understand it. Instead of just saying, you know, we use a heuristic instead and just guess that it’s true. I’m pretty sure a classical system can do that.

I'm sure if you email him explaining how stupid he is he'll send you his Nobel prize. Less flippantly, Penrose has always been extremely clear about which things he's sure of, such as that human intelligence involves processes that algorithms cannot emulate, and which things he puts forward as speculative ideas that might help answer the questions he has raised. His ideas about quantum mechanical processes in the bra…

You realise that this isn’t even a reply so much as a series of insults dressed up in formal language?

Yes, of course you do.

Re: You are the scariest monster in the woods

#156

Post by progbits »

slow_typist wrote: post #149
boole1854 wrote: post #22

Earlier quoted context omitted.

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

In short, by definition, computers are symbol manipulating devices. However complex the rules of symbol manipulation, it is still a symbol manipulating device, and therefore neither intelligent nor sentient. So AGI on computers is not possible.

Computer can simulate human brain on subatomic level (in theory). Do you agree this would be "sentient and intelligent" and not just symbol manipulating?

If yes, everything else is just optimization.

Re: You are the scariest monster in the woods

#157

Post by AlexandrB »

myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

LLMs largely live in the world of pure language and related tokens - something humans invented late in their evolution. Human intelligence comes - at least partially - from more fundamental physical experience. Look at examples of intelligent animals that lack language.

Basically there's something missing with AI. Its conception of the physical world is limited by our ability to describe it - either linguistically or mathematically. I'm not sure what this means for AGI, but I suspect that LLM intelligence is fundamentally not the same as human or animal intelligence at the moment as a result.

Re: You are the scariest monster in the woods

#158

Post by 48terry »

jncfhnb wrote: post #40
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

Wow, who would have thought it was that easy? Wonder why nobody has done this incredibly basic solution to AGI yet.

Re: You are the scariest monster in the woods

#159

Post by jncfhnb »

crdrost wrote: post #144
jncfhnb wrote: post #40

Earlier quoted context omitted.

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

So the current problem with a loop like that is that LLMs in their current form are subject to fixed point theorems, which are these pieces of abstract mathematics that come back when you start to get larger than some subset of your context window and the “big matrix” of the LLM is producing outputs which repeat the inputs. If you have ever had an llm enter one of these loops explicitly, it is infuriating. You can ty…

That’s actually exactly my point. You cannot fake it till you make it by using forever larger context windows. You have to map it back to actual system state. Giant context windows might progressively produce the illusion of working due to unfathomable scale, but it’s a terrible tool for the job.

LLMs are not stateful. A chat log is a truly shitty state tracker. An LLM will never be a good agent (beyond a conceivable illusion of unfathomable scale). A simple agent system that uses an LLM for most of its thinking operations could.

Re: You are the scariest monster in the woods

#160

Post by 0xffff2 »

squidbeak wrote: post #79
kulahan wrote: post #31

Earlier quoted context omitted.

We still haven’t figured out what intelligence even is. Depending on what you care about, the second-smartest animal in the world varies wildly.

This is a bogus argument. There's a lot we don't understand about LLMs, yet we built them.

There's a lot _I_ don't understand about LLMs, but I strongly question whether there is a lot that the best experts in the field don't understand about LLMs.
Post reply on HN