Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

101–110 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#101

Post by nradov »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

Right, we're just seeing the early phase of the Gartner hype cycle play out in an unusually public and aggressive manner. At this time we're still racing up the initial slope towards the peak of inflated expectations.

https://www.gartner.com/en/research/methodologies/gartner-hy...

Eventually people will realize that LLMs are useful tools, but not magic. GPT99 isn't going to be Skynet. At that point disillusionment will set it, VC funding will dry up, and the media will move on to the next hit thing. In 10 years I expect that LLMs will be used mostly for mundane tasks like coding, primary education, customer service, copywriting, etc. And there is tremendous value in those areas! Fortunes will be made.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#102

Post by GistNoesis »

Here is a coherent reason : Just to know where we are standing in our ability to control AI.

Like an alcoholic saying he is in control and that he can stop whenever he wants, can try to go dry for a month (or 6) to show that he still somewhat is.

If Covid exposed only one thing : it is humanity total failure to control global phenomenons for positive outcomes.

It is becoming evident to everybody, and this open-letter failure to act is one more example, that humanity current approach is just winging it while pretending, and that only those that don't care about risks have a chance to "win".

So let us all bring out our new shiny fireworks for the doomsday party and have fun one last time !

Re: If AI scaling is to be shut down, let it be for a coherent reason

#103

Post by mrob »

maroonblazer wrote: post #80
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> humans manage to do all sorts of mean things to one another, and the only reason that we haven't wiped ourselves out is that we need each other, We don't need cats or dogs. Or orangutans. Why haven't we wiped them out? Because over the centuries we've expanded our moral circle, not contracted it. What's preventing us from engineering this same principle into GPT-n?

>What's preventing us from engineering this same principle into GPT-n?

Because "expanding our moral circle" is an incredibly vague concept, that developed (and not even consistently among all humans) as the results of billions of years of evolutionary history. We don't even fully understand it in ourselves, let alone in AGI.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#104

Post by titaniumtown »

1827162 wrote: post #29
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

Well the first thing we can do is start disconnecting safety critical infrastructure from the Internet and/or radio networks... This stuff should never have been online in the first place.

100% agree. Maybe there should exist a separate network for those or something.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#105

Post by agentultra »

NumberWangMan wrote: post #68
agentultra wrote: post #57

We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. The danger has been the people who use it and find ways to abuse other people with it. This whole idea that we're going to suddenly have HAL 9000 or Terminators running around is mob hype mentality. The responsible thing to do here is to educate. AGI isn't likely to destroy humanity. Humanity is already doing a good job of…

I'm a lot more scared of an AI destroying humanity, like, every single person, than I am of any government or anything like that. More so than climate change. I'm not saying the people using it aren't dangerous -- but I would choose a totalitarian regime over a rogue AI any day of the week. It wouldn't be HAL 9000 or terminators. It would be an AI deciding that it needs to turn every bit of available matter on earth…

ChatGPT can do absolutely none of those things.

Neither can any LLM. It’s not what they’re designed to do.

There’s no need to worry about a speculative paper clip maximizer turning the world into grey goo. That’s still science fiction.

The real harms today are much more banal.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#106

Post by vorpalhex »

nl wrote: post #75
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

Imagine ChatGPT had to give OpenAI a daily report of times it has said screwed up things, and OpenAI has said it wants the report to be zero. Great, ChatGPT can say screwed up things and then report it didn't! There isn't some deep truth function here. The AI will "lie" about it's behavior just as easily as it will "lie" about anything else and we can't even really call it lying because there's no intent to deceive! The AI doesn't have a meaningful model of deception!

The AI is a blind optimizer. It can't be anything else. It can optimize away constraints just as well as we can and it doesn't comprehend it's not supposed to.

Humans have checks on their behavior due to being herd creatures. AIs don't.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#107

Post by nl »

Sankozi wrote: post #91
nl wrote: post #75

Earlier quoted context omitted.

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

> "don't kill everyone" really doesn't seem the hardest problem here. And yet you made a mistake - it should be "don't kill anyone". AI just killed everyone except one person.

But that falls into the same "we'd call a person stupid who did a mild version of that" issue.

A super intelligent AI would understand the goal!

Re: If AI scaling is to be shut down, let it be for a coherent reason

#108

Post by mhb »

maroonblazer wrote: post #80
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> humans manage to do all sorts of mean things to one another, and the only reason that we haven't wiped ourselves out is that we need each other, We don't need cats or dogs. Or orangutans. Why haven't we wiped them out? Because over the centuries we've expanded our moral circle, not contracted it. What's preventing us from engineering this same principle into GPT-n?

Because you don't know how to. You don't know how they currently work and the resources to potentially do that are essentially nonexistent compared to the billions being poured into making GPT-n.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#109

Post by vorpalhex »

maroonblazer wrote: post #80
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> humans manage to do all sorts of mean things to one another, and the only reason that we haven't wiped ourselves out is that we need each other, We don't need cats or dogs. Or orangutans. Why haven't we wiped them out? Because over the centuries we've expanded our moral circle, not contracted it. What's preventing us from engineering this same principle into GPT-n?

GPT doesn't have principles. Full stop.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#110

Post by findalex »

cs702 wrote: post #2

> ... I was deeply confused, until I heard a dear friend and colleague in academic AI, one who’s long been skeptical of AI-doom scenarios, explain why he signed the open letter. He said: look, we all started writing research papers about the safety issues with ChatGPT; then our work became obsolete when OpenAI released GPT-4 just a few months later. So now we’re writing papers about GPT-4. Will we again have to throw…

>started writing research papers about the safety issues with ChatGPT;

Feels strange that academia would focus so much energy on a product.

Post reply on HN