Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

71–80 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#71

Post by agentultra »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

You're not alone. Although I am concerned about the people who wield these latest bits of tech.

All of these service providers jockeying for first-mover advantage in order to close on a monopoly in the space is asking for trouble.

The abilities it gives to scammers to increase the plausibility of their social engineering is going to be problematic. Scams are already quite sophisticated these days. How are we going to keep up?

Those sorts of things. ChatGPT itself is a giant, multi-layered spreadsheet and some code. "It," is not smart, alive, intelligent, or "doing" anything. Speculation of what it could be is muddying the waters as people get stressed out about what all of these charlatans are proselytizing.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#72

Post by bnralt »

computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

Sometime I listen to 90’s AM conspiracy theory radio for fun (Coast to Coast AM). One thing that’s struck me is how much fear there was about the Human Genome Project and designer babies being right around the corner (Gattaca was 1997, for example). Maybe that will come to pass someday. But at the moment, it still seems a long ways off.

A lot of groups have some new technology they’re scared of. Tech folks have latched onto the idea that we’ll create Skynet, or that large scale video surveillance will turn countries into authoritarian dystopian states. Hippy groups are convinced that GMO’s cause cancer or will lead to a biodiversity disaster. Or that nuclear plants are going to lead to meltdowns, environmental destruction, and deaths.

Appropriate safeguards are always important in society. Excessive safeguards can cause harm. Sometimes people have a maximalist view of danger that’s so detached from the current reality that it’s hard to have a rational discussion with them.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#73

Post by bombcar »

tibbon wrote: post #27
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

I don’t think it even requires that AI to be sentient or malicious. The humans already are. Given a tool for carnage and hatred, people will use it. How long did it take from us getting the atomic bomb working to use in production? Less than three months. Will Putin or terrorists hold back from using it in terrible ways if they have it available to them?

Putin already has access to the atomic bomb and hasn’t done much with it lately. So perhaps something can limit actors.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#74

Post by jillesvangurp »

Havoc wrote: post #10

This whole thing is like leaving 20 kids alone in a room with a marshmallow each and telling them don't eat it. ...and then expecting all of them resist. The debate around whether we should tell the kids not to eat it and for what reasons is completely academic. Practically this just isn't happening.

It's a good analogy. Especially considering some of the kids might have "parents" that are not based in the US that are very unlikely to just go and sit on their hands for six months just because some people from the US want them to. It's beyond naive to assume that the rest of the world would agree to do nothing. I don't see that happen.

BTW. I suspect the reasons for this open letter might be a bit disingenuous. A few of the people that signed it represent companies that are effectively competing with OpenAI. Or failing to, rather. They are calling for them to slow down so they can catch up. It's not about saving humanity but about giving themselves a chance to catch up and become part of the problem.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#75

Post by nl »

NumberWangMan wrote: post #58
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone!

I don't understand this and other paperclip maximizer type arguments.

If a person did a minor version of this we'd say they were stupid and had misunderstood the problem.

I don't see why a super-intelligent AI would somehow have this same misunderstanding.

I do get that "alignment" is a difficult problem space but "don't kill everyone" really doesn't seem the hardest problem here.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#76

Post by lukev »

jacquesm wrote: post #64
skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

> I am young, but still lived through several hype phases. My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized ar…

I agree with this 100%, while also disagreeing with the "robot god will kill us all" objections to AI which unfortunately tend to get most of the mindshare.

I think it's important to realize that these are two _completely separate_ concerns. Unfortunately, a lot of the people who get the most air time on this topic are not at all worried about authoritarianism or economic collapse compared to a hypothetical singularity.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#77

Post by realce »

Godel_unicode wrote: post #48
realce wrote: post #17

Earlier quoted context omitted.

Because the NSA isn't going to stop, the CCP isn't going to stop. Anyone who doesn't stop is a threat to my personal freedom, so the only logical reaction to me is to empower yourself as well as possible and keep training.

Huh? How does someone else’s possession of a LLM threaten your personal freedom? How does training your own counteract that? They’re not Pokémon…

It's not someone else's possession - it's someone else's proprietary possession.

A government's proprietary possession of computational power is analogous to it having weapons barred from public ownership, meaning they control a vector of violence than could be used against their citizens without counter-balance.

If someone else has the ability to weaponize information against you, your ability to understand reality is threatened. Without personal LLMs or other AI tools, my ability to analyze things like deepfakes, LLM-written text, or other reality-distortion assets is threatened.

It might sound hyperbolic but we're already hearing people talk about banning GPUs. I'm not trying to fall back into the past.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#78

Post by rhn_mk1 »

lukev wrote: post #36
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

The reasoning pattern of Yud and his ilk is fundamentally theological and eschatological in nature. There isn't any actual understanding of the technology involved. It is fundamentally an ontological argument. Because they can imagine a god-like super intelligent AI, it must be possible. And they're associating that with LLMs because that's the most powerful AI currently available, not based on any actual capabilitie…

> Because they can imagine a god-like super intelligent AI, it must be possible.

It's worse. It may be possible, but we're not equipped to recognize the line as it's crossed. Combined with us making LLMs more and more capable despite not knowing why they work, this extrapolating of LLMs to gods is not insane.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#79

Post by chatmasta »

nickybuzz wrote: post #70
chatmasta wrote: post #31

Earlier quoted context omitted.

I've seen a lot of people worrying about "what will China do with LLMs?" But my question is: how do you know they don't have them already? What if they've been deployed on the internet for the past three years? I guess the same logic can apply to CIA, NSA, DARPA, etc. But you can take solace in the fact that most government organizations are pretty inept. It used to be common understanding that the military/intellige…

I'd also argue the disparity in compensation between private companies and government agencies pushes top engineering talent to the former.

Definitely. I think there is one exception though, which is advanced weapon manufacturers like Lockheed Martin. They're a private company, so they can compete on salary, but they're also a government contractor, building technology that could only be used by military. You won't see Google building stealth fighter jets, for example. So if an advanced physicist wants to work on cutting edge tech, they're more likely to end up at a place like that for their whole career. But even aerospace isn't immune from the private sector, as SpaceX has shown.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#80

Post by maroonblazer »

NumberWangMan wrote: post #58
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> humans manage to do all sorts of mean things to one another, and the only reason that we haven't wiped ourselves out is that we need each other,

We don't need cats or dogs. Or orangutans. Why haven't we wiped them out? Because over the centuries we've expanded our moral circle, not contracted it. What's preventing us from engineering this same principle into GPT-n?

Post reply on HN