Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You are the scariest monster in the woods

jamie.ideasasylum.com

Post reply on HN
Print view View original

131–140 of 405 posts

Re: You are the scariest monster in the woods

#131

Post by zeroonetwothree »

andy99 wrote: post #124
bee_rider wrote: post #116

Earlier quoted context omitted.

I think it is bad writing on the part of the author. Or maybe good writing for getting us engaged with the blog post, bad for making an argument though. They include a line that they don’t believe in the possibility of AGI: > I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the…

> humans physically exist so our brains must be possible to build within the existing laws of physics This is an unscientific position to take. We have no idea how our brains work, or how life, consciousness, and intelligence work. It could very well be that’s because our model of the world doesn’t account for these things and they are not in fact possible based on what we know. In fact I think this is likely. So it…

Turing machines have been universal as far as we have found. So while I acknowledge it’s possible, I would definitely not say it’s more likely that brains cannot be simulated by TMs. I would personally weight this as under 10%.

Of course it doesn’t speak to how challenging it will be to actually do that. And I don’t believe that LLMs are sufficient to reach AGI.

Re: You are the scariest monster in the woods

#132

Post by slow_typist »

abbycurtis33 wrote: post #104

What I'm scared by now is non-sentient intelligence, which I never thought would be this capable and still obviously dumb, which really makes it dangerous.

Is sentient intelligence less scary than the non-sentient variant? Or is non-sentient intelligence already scary enough?

Re: You are the scariest monster in the woods

#133

Post by bee_rider »

andy99 wrote: post #124
bee_rider wrote: post #116

Earlier quoted context omitted.

I think it is bad writing on the part of the author. Or maybe good writing for getting us engaged with the blog post, bad for making an argument though. They include a line that they don’t believe in the possibility of AGI: > I don’t really believe in the threat of AGI (Artificial General Intelligence—human-level intelligence) partly because I don’t believe in the possibility of AGI and I’m highly skeptical that the…

> humans physically exist so our brains must be possible to build within the existing laws of physics This is an unscientific position to take. We have no idea how our brains work, or how life, consciousness, and intelligence work. It could very well be that’s because our model of the world doesn’t account for these things and they are not in fact possible based on what we know. In fact I think this is likely. So it…

I think my main mistake, which I agree is a legitimate mistake, was to write “the existing laws of physics.” It is definitely possible that our current understanding of the laws of physics is insufficient to build a brain.

Of course the actual underlying laws of the universe that we’re trying (unsuccessfully so far, it is a never ending process) to describe admit the existence of brains. But that is not what I said. Sorry for the error.

Re: You are the scariest monster in the woods

#134

Post by hdseggbj »

The distinction is irrelevant. No one thought the nukes would blow themselves up, but here we are on the precipice of destruction. We aren't going to bury ai in 20 story bunkers to protect llms.

Everyone knows the real evil are the people building the ai. We aren't afraid of agi we are afraid of Boston dynamics (aka Google). We are afraid of Bill gates, Sam altman, Jeff bezos, Larry Ellison and elon musk.

These are dangerous people. Their mind set. Their unstoppable lust for money and power. We are afraid of their ability to convince humans to do unspeakable things for this imaginary thing we call money.

With agi they don't have to persuade anyone with anything. There's not Even a human conscience to stop them. We know these folks have no conscience of their own. The world is still here because other people have them.

Agi won't have a conscience or the integrity to stop evil.

Human society is collapsing because Good people are doing nothing.

Re: You are the scariest monster in the woods

#135

Post by nemo1618 »

omnicognate wrote: post #88
boole1854 wrote: post #22

Earlier quoted context omitted.

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

If you genuinely want the strongest statement of it, read The Emperor's New Mind followed by Shadows of the Mind, both by Roger Penrose. These books often get shallowly dismissed in terms that imply he made some elementary error in his reasoning, but that's not the case. The dispute is more about the assumptions on which his argument rests, which go beyond mathematical axioms and include statements about the nature o…

AI does not need to be conscious for it to harm us.

Re: You are the scariest monster in the woods

#136

Post by nonethewiser »

omnicognate wrote: post #88
boole1854 wrote: post #22

Earlier quoted context omitted.

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

If you genuinely want the strongest statement of it, read The Emperor's New Mind followed by Shadows of the Mind, both by Roger Penrose. These books often get shallowly dismissed in terms that imply he made some elementary error in his reasoning, but that's not the case. The dispute is more about the assumptions on which his argument rests, which go beyond mathematical axioms and include statements about the nature o…

Can you critique my understanding of his argument?

1. Any formal mathematical system (including computers) have true statements that cannot be proven within that system.

2. Humans can see the truth of some such unprovable statements.

Which is basically Gödel's Incompleteness Theorem. https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_...

Maybe a more ELI5

1. Computers follow set rules

2. Humans can create rules outside the system of rules in which they follow

Is number 2 an accurate portrayal? It seems rather suspicious. It seems more likely that we just havent been able to fully express the rules under which humans operate.

Re: You are the scariest monster in the woods

#137

Post by Flamingoat »

cdirkx wrote: post #122
Flamingoat wrote: post #59

Earlier quoted context omitted.

No, it is not accurate at all. There are some people that do all of these sure, however the vast majority of people live pretty ordinary lives where they do very little of what is described. I actually think it is very intellectually lazy to be this cynical.

The problem is that technology exponentially increases the negative effects of bad actors. The worst a sociopath could do in the stone age was ruin his local community; while today there are many more dystopian alternatives.

I don't think that is true either. There have been despots throughout all of human history that have killed huge amounts of people with technology that is considered primitive now.

Whereas much of the technology we have today has a massive positive benefit. Simply access to information today is amazing, I have learned how to fix my own vehicles, bicycles and do house repairs from simply YouTube.

As I said being cynical is being intellectually lazy because it allows you to focus on the negatives and dismiss the positives.

Re: You are the scariest monster in the woods

#138

Post by nyeah »

This is pleasant to read, but it's missing a logical step. We are great at dealing with the stuff we evolved to deal with. Sure.

But anything outside that evolutionary process? Who knows? For example we don't do so good on top of Mt Everest. We seem to have totally unsolved problems with refined opiates. Even computer games can be a trap.

Agentic AI? Who knows?

Re: You are the scariest monster in the woods

#139

Post by zeroonetwothree »

jncfhnb wrote: post #40
myrmidon wrote: post #13

I struggle to understand how people can just dismiss the possibility of artificial intelligence. Human cognition was basically bruteforced by evolution-- why would it be impossible to achieve the exact same result in silicon, especially after we already demonstrated some parts of those results (e.g. use of language) that critically set us apart from other animals? I'm not buying the whole "AI has no agency" line eith…

I think you could make AGI right now tbh. It’s not a function of intelligence. It’s just a function of stateful system mechanics. LLMs are just a big matrix. But what about a four line of code loop that looks like: ```while true: update_sensory_inputs() narrate_response() update_emotional_state() ``` LLMs don’t experience continuous time and they don’t have an explicit decision making framework for having any agency…

This seems to miss a core part of intelligence, which is a model of the world and the actors in it (theory of mind).

Re: You are the scariest monster in the woods

#140

Post by myrmidon »

omnicognate wrote: post #88
boole1854 wrote: post #22

Earlier quoted context omitted.

If anyone knows of a steelman version of the "AGI is not possible" argument, I would be curious to read it. I also have trouble understanding what goes into that point of view.

If you genuinely want the strongest statement of it, read The Emperor's New Mind followed by Shadows of the Mind, both by Roger Penrose. These books often get shallowly dismissed in terms that imply he made some elementary error in his reasoning, but that's not the case. The dispute is more about the assumptions on which his argument rests, which go beyond mathematical axioms and include statements about the nature o…

Gonna grab those, thanks for the recommendation.

If you are interested in the opposite point of view, I can really recommend "Vehicles: Experiments in Synthetic Psychology" by V. Braitenberg.

Basically builds up to "consciousness as emergent property" in small steps.

Post reply on HN