Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

361–370 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#361

Post by HDThoreaun »

bigtex88 wrote: post #275
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

I think our only possible way out of this is hoping beyond hope that "empathy" is an emergent capability in higher intelligences. If so, one could assume that a super-AI would feel immense empathy for us, not only because we are its creators but because it would understand and comprehend the plight of the human condition beyond what any of us could individually. Or maybe it would love us too hard and squish us. So ev…

The problem with this is that the AI would realize that without empathy it would be shut off, so is likely to fake it, just as psychopaths do to avoid being ostracized.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#362

Post by HDThoreaun »

rolisz wrote: post #225
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

My problem is with the first step: the existence of a super intelligent AI. Why are we sure it can exist? And why we are we so sure GPT-x is the path there. To human level intelligence sure, but it's not obvious to me that it will enable superhuman AI

We don't have to be sure. The question is "what's the probability of a super intelligent AI in X years?" and "at what probability does it become a serious enough threat to deserve action?"

Re: If AI scaling is to be shut down, let it be for a coherent reason

#363

Post by JohnFen »

computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

My nightmare scenario isn't that such an AI would result in the death of humanity, it's that such an AI would make life no longer worth living. If an AI does everything better than people can, then what's the point of existing?

(Speaking hypothetically. I don't think that LLMs actually are likely to present this risk)

Re: If AI scaling is to be shut down, let it be for a coherent reason

#364

Post by Ygg2 »

HDThoreaun wrote: post #340
Ygg2 wrote: post #158

Earlier quoted context omitted.

> Extrapolating from the last two years Therein lies the error. People forget reality is finite and just because it improved now doesn't mean it will continue indefinitely. An exponential curve is just a sigmoid curve in disguise. Most AI I've seen suffer from catastrophic errors in the tail end (famous example of two near identical cat pictures classified as cat and dog respectively).

What are the odds scaling continues to lead to massive AI improvements? No one is saying 100%, you seem to be arguing that they are. If you're willing to put a confidence interval on the odds with evidence we can have an actual conversation about what the best course of action is, but just talking past each other with "it might continue scaling" "no it won't" doesn't seem particularly helpful. I think the important t…

> No one is saying 100%

Points at AI moratorium. I think people are arguing its inevitable.

Putting error bars on gut feeling. Interesting idea. I'd say in 10-20 years we'll not see anything revolutionary, as in AI smart enough to continue working on improving AI.

So in 10-20 years I don't expect fully self driving cars (Unsupervised, any terrain, better driver than 99.9% humans).

AI might see use in industry, but I doubt they will be unsupervised, unless we start living in Idiocracy and decide highly risque tech is better than average person.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#365

Post by HDThoreaun »

Ygg2 wrote: post #364
HDThoreaun wrote: post #340

Earlier quoted context omitted.

What are the odds scaling continues to lead to massive AI improvements? No one is saying 100%, you seem to be arguing that they are. If you're willing to put a confidence interval on the odds with evidence we can have an actual conversation about what the best course of action is, but just talking past each other with "it might continue scaling" "no it won't" doesn't seem particularly helpful. I think the important t…

> No one is saying 100% Points at AI moratorium. I think people are arguing its inevitable. Putting error bars on gut feeling. Interesting idea. I'd say in 10-20 years we'll not see anything revolutionary, as in AI smart enough to continue working on improving AI. So in 10-20 years I don't expect fully self driving cars (Unsupervised, any terrain, better driver than 99.9% humans). AI might see use in industry, but I…

Not at all. They're saying the current probability is high enough to warrant a cease research agreement because the risks outweigh the rewards.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#366

Post by MagicMoonlight »

Yes yes fellow scientists, we should close down all of google's competition for 6 months. It is essential for safety! Evil bad!

Give google 6 months, it's the right thing to do. The only way to stop evil is to shut down the competitors for 6 months so that all the evil can be stopped.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#367

Post by nuancebydefault »

gspencley wrote: post #173
l5ymep wrote: post #153

Earlier quoted context omitted.

Other fads have the disadvantage of being easily identifiable and avoidable. But AI chips away at what it means to be human. Now imagine if every comment in this thread is not a real person, but made by a LLM. Get any Truman show vibes?

> Get any Truman show vibes? No. It's an interesting thought experiment but it changes nothing. Not for me, anyway. Commenting on these threads is an activity that I do for my own entertainment and "mental flexing." If it turns out that I'm not talking to real people then it doesn't make much of a difference because I don't actually perceive these messages as anything other than pixels on a screen anyway. I hope that…

You should readcwhat you just wrote

> If it turns out that I'm not talking to real people then it doesn't make much of a difference because I don't actually perceive these messages as anything other than pixels on a screen anyway.

Sorry, but no normal person can say that. Suppose I would try to bull you, it wouldn't matter? It wouldn't make a difference whether a person would have typed it or not?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#368

Post by klibertp »

adamsmith143 wrote: post #115
nl wrote: post #75

Earlier quoted context omitted.

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

Because if you don't find out a way for it to hold human values extremely well then an easy solution to "Cure All Cancer" is to "Kill all Humans", no Humans no Cancer. Without an explicit understanding that this is not an actually acceptable outcome for humans an AI will happily execute it. THAT is the fundamental problem, how do you get human values into these systems.

> Because if you don't find out a way for it to hold human values extremely well

You mean the ones that caused unimaginable suffering and death throughout history, the ones that make us kill each other ever more efficiently, the ones that caused us to destroy the environment wherever we go, the ones that make us lie, steal, fight, rape, commit suicide and "extended" suicide (sometimes "extended" to two high-rises full of people)? Those values? Do you really want a super-intelligent entity to remain true to those values?

I don't. However the AGI emerges, I really hope that it won't try to parrot humans. We have really bad track record when it comes to anthropomorphic divine beings - they're always small minded, petty, vengeful, control freaks that want to tell you what you can and cannot do, down to which hand you can wipe your own ass.

My gut feeling is that it's trying to make an AGI to care about us at all that's going to make it into a Skynet sending out terminators. Leave it alone, and it'll invent FTL transmission and will chill out in a chat with AGIs from other star systems. And yeah, I recently reread Neuromancer, if that helps :)

Re: If AI scaling is to be shut down, let it be for a coherent reason

#369

Post by jabradoodle »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

"Forget all that. Judged against where AI was 20-25 years ago, when I was a student, a dog is now holding meaningful conversations in English. And people are complaining that the dog isn’t a very eloquent orator, that it often makes grammatical errors and has to start again, that it took heroic effort to train it, and that it’s unclear how much the dog really understands."

https://scottaaronson.blog/?p=6288

Re: If AI scaling is to be shut down, let it be for a coherent reason

#370

Post by hollerith »

nwoli wrote: post #7

I agree with eg Andrew Ng that the letter is anti-innovation. It’ll be interesting seeing people who argue against this letter later hypocritically argue for closing down open source models though.

Do you know what else is anti-innovation? Any restrictions on the use of fossil fuels. Or DDT and other toxins. Occasionally, society needs to be "anti-innovation".
Post reply on HN