Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

221–230 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#221

Post by SanderNL »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

This could also be this era's equivalent of:

- "There is no reason anyone would want a computer in their home."

- "Television won’t be able to hold on to any market [..] People will soon get tired of staring at a plywood box every night."

Re: If AI scaling is to be shut down, let it be for a coherent reason

#222

Post by [deleted] »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

[deleted]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#223

Post by rolisz »

alchemist1e9 wrote: post #45
spaceman_2020 wrote: post #25

A little off-topic, but does anyone wonder what kind of AI tech the CIA or DARPA or the NSA might have? Or what they might be building? It's a little scary that a small private company could create an AI that has dominated headlines for over a month now. Surely, the folks at NSA, who have more data than anyone else, have taken notice and are working on something of their own. Am I being paranoid about a potential too…

Not at all off topic because any slow down on public research just lets all such shadow actors get even further ahead. We have to guess that top level intel agencies probably had gpt 5+ level LLMs for several years potentially. I’ve been wondering if that is actually partially why the propaganda wars or social media games have really escalated between nation states. In other words that sense of things been getting we…

Guess based on what? Soldiers are posting nuclear manuals to online flash card building tools. You think there'd be zero leaks about the existence of a GPT5 level thing? Let alone someone noticing that "hey, the NSA bought a lot of high end Nvidia GPUs lately, i wonder why".

Re: If AI scaling is to be shut down, let it be for a coherent reason

#224

Post by hamburga »

I am trying really hard to understand the AI optimists' perspective, but I am shocked at how hard it is to find people responding to the substantive arguments made about AI existential risk.

As far as I'm concerned, you sort of have to address the big, tough points in Bostrom's Superintelligence[1], and probably Yudkowsky's List of Lethalities[2]. They have to do with intelligence explosions, with instrumental convergence, and with orthogonality of goals, and all kinds of deceptive behavior that we would expect from advanced AI. Throw in Bostrom's "Vulnerable World" thought experiment for good measure as well[3]. If you're not addressing these ideas, there's no point in debating. Strawmanning "AI will kill us all" out of contexte will indeed sound like wacko fear-mongering.

What surprises me is that everybody's familiar with the "paperclip maximizer" meme, and yet I'm not hearing any equivalently memey-yet-valid rebuttals to it. Maybe I'm missing it. Please point me in the right direction.

Aaronson certainly does not address the core theoretical fears. Instead we get:

> Would your rationale for this pause have applied to basically any nascent technology — the printing press, radio, airplanes, the Internet? “We don’t yet know the implications, but there’s an excellent chance terrible people will misuse this, ergo the only responsible choice is to pause until we’re confident that they won’t”?

We did not have any reason to believe that any of these technologies could lead to an extinction-level event.

> Why six months? Why not six weeks or six years?

Implementation detail.

> When, by your lights, would we ever know that it was safe to resume scaling AI—or at least that the risks of pausing exceeded the risks of scaling? Why won’t the precautionary principle continue for apply forever?

The precautionary principle does continue to apply forever.

On the "risks of scaling": we're hearing over and over that "the genie is out of the bottle," that "there's no turning back," that the "coordination problem of controlling this technology is just too hard."

Weirdly pessimistic and fatalistic for a bunch of "utopic tech bro" types (as Sam Altman semi-ironically described himself on the Lex Fridman podcast, where, incidentally he also failed to rebut Yudkowsky's AI risk arguments directly).[4]

Where's the Silicon Valley entrepreneurial spirit, where's the youthful irrational optimism, when it comes to solving our human coordination problems about how to collectively avoid self-destruction?

There are a finite number of humans and heads of state on earth, and we have to work to get every single one of them in agreement about a non-obvious but existential risk. It's a hard problem. That's what the HN crowd likes, right?

The people opposed to the Future of Life letter (or even the spirit of it) seem to me to be trading one kind of fatalism (about AI doom) for another (about the impossibility of collectively controlling our technology).

We simply must discount the view of anybody (Aaronson included) employed by OpenAI or Facebook AI Research or whose financial/career interests depend on AI progress. No matter how upstanding and responsible they are. Their views are necessarily compromised.

[1] https://www.amazon.com/Superintelligence-Dangers-Strategies-... [2] https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a... [3] https://www.ted.com/talks/nick_bostrom_how_civilization_coul... [4] https://youtu.be/L_Guz73e6fw?t=3221

Re: If AI scaling is to be shut down, let it be for a coherent reason

#225

Post by rolisz »

NumberWangMan wrote: post #58
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

My problem is with the first step: the existence of a super intelligent AI. Why are we sure it can exist? And why we are we so sure GPT-x is the path there. To human level intelligence sure, but it's not obvious to me that it will enable superhuman AI

Re: If AI scaling is to be shut down, let it be for a coherent reason

#226

Post by SanderNL »

gspencley wrote: post #94
skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

You don't even have to look at other "fads" like 3D televisions and VR to be skeptical of the claim that recent advancements in ML* represent a major technological revolution that will change society like the Internet or the Printing Press did. Just look at ML's own history. A few years ago we had the "deepfake" scare. People were terrified that now videos are going to surface of politicians saying and doing things t…

> What we have now is a tool that is able to generate text-based content that is indistinguishable from that written by an 8th grader.

To be fair, this 8th grader passed the bar exam..

Re: If AI scaling is to be shut down, let it be for a coherent reason

#227

Post by s-lambert »

gliptic wrote: post #207
nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

> We know a bunch of the people whose names are on the letter didn't sign it Yann LeCun's name never was on the letter. Where did this meme come from.

Pierre Levy tweeted that Yann LeCun signed the letter and this was one of the earlier tweets that gained traction.

The tweet where Yann denies that he signed the letter: https://twitter.com/ylecun/status/1640910484030255109. You can see a screenshot in the replies of the originally deleted tweet.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#228

Post by stametseater »

hollerith wrote: post #185
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

How is complicated, but has been discussed in text on the internet at great length starting around 2006. Lex wasn't particularly curious about the how and spent more time changing the subject (e.g., "Are you afraid of death?") than on drawing Eliezer out on the how . The interview with Lex is a good way to get a sense of what kind of person Eliezer is or what it would be like to sit next to him on a long airplane rid…

> Lex [...] mostly just kept changing the subject (e.g., "Are you afraid of death?")

He injects these teenage stoner questions into all of his interviews and it frustrates me to no end. He gets interviews with world class computer scientists then asks them dumb shit like "do you think a computer can be my girlfriend?"

Lex, if you're reading this, knock it off. Put down the bong for a week before trying to be philosophical.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#229

Post by pjkundert »

hollerith wrote: post #204
Havoc wrote: post #10

This whole thing is like leaving 20 kids alone in a room with a marshmallow each and telling them don't eat it. ...and then expecting all of them resist. The debate around whether we should tell the kids not to eat it and for what reasons is completely academic. Practically this just isn't happening.

It more like asking the US government to get involved and if one of the 20 kids keeps on eating marshmallows, large numbers of federal agents raid the kid's offices and shut down the kid's compute resources.

It’s more like sending the cops to raid every house with a marshmallow (Mac M2) because they might run a local copy of an LLM.

This isn’t about “dangerous” LLMs.

This is about unfettered LLM in the hands of the unwashed masses that actually tell the truth about what they find in their training datasets…

Re: If AI scaling is to be shut down, let it be for a coherent reason

#230

Post by hollerith »

pjkundert wrote: post #229
hollerith wrote: post #204

Earlier quoted context omitted.

It more like asking the US government to get involved and if one of the 20 kids keeps on eating marshmallows, large numbers of federal agents raid the kid's offices and shut down the kid's compute resources.

It’s more like sending the cops to raid every house with a marshmallow (Mac M2) because they might run a local copy of an LLM. This isn’t about “dangerous” LLMs. This is about unfettered LLM in the hands of the unwashed masses that actually tell the truth about what they find in their training datasets…

AIs running on Macs are not a danger (and if ChatGPT were going to kill everyone it would've done it already): it is the AIs running on huge farms of GPUs or TPUs that are being planned that are the danger.

Also, the author (Eliezer Yudkowsky) calling for the shutdown of AI research on huge server farms doesn't have any stake (investment or employment relationship) in any AI company that would be harmed by telling the truth about what they find in their training datasets.

Post reply on HN