Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

301–310 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#301

Post by munificent »

> Were you, until approximately last week, ridiculing GPT as unimpressive, a stochastic parrot, lacking common sense, piffle, a scam, etc. — before turning around and declaring that it could be existentially dangerous? How can you have it both ways? If the problem, in your view, is that GPT-4 is too stupid, then shouldn’t GPT-5 be smarter and therefore safer? Thus, shouldn’t we keep scaling AI as quickly as we can … for safety reasons? If, on the other hand, the problem is that GPT-4 is too smart, then why can’t you bring yourself to say so?

I think the flaw here is equating "smart" with "powerful".

Personally, I think generative AI is scary both when it gets things wrong and when it gets things right. If it was so stupid that it got things wrong all the time and no one cared to use it, then it would be powerless and non-threatening.

But once it crosses a threshold where it's right (or appears to be) often enough for people to find it compelling and use it all the time, then it has become an incredibly powerful force in the hands of millions whose consequences we don't understand. It appears to have crossed that threshold even though it still hilariously gets stuff wrong often.

Making it smarter doesn't walk it back across the threshold, it just makes it even more compelling. Maybe being right more often also makes it safer at an even greater rate, and is thus a net win for safety, but that's entirely unproven.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#302

Post by nullc »

Analemma_ wrote: post #288
sailingparrot wrote: post #192

Earlier quoted context omitted.

> It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded Gary Marcus has been aware that neural nets work for a while now, but he is only in the spotlight for his contrarian take, if he stops having a contrarian take he disappears, because it's not like he is producing any research wort…

This is a depressing pattern that I've seen get repeated over and over: it's easy to become Twitter-famous by just shitting out a steady stream of contrarian "me against the world, I'm the rogue dissident nobody wants to talk about!" takes, and once you start doing this, you have no choice but to keep it up forever (even when the takes are just wrong) because it has become your identity. It has destroyed so many inte…

Adopting a contrarian brand because it brings you attention when you'd otherwise be a nobody is by far not limited to twitter. It's a mainstay of traditional journalism.

If anything twitter disincentivizes the practice because it has commoditized cynicism: It's harder to build a brand by being the well known naysayer of X when someone can source whatever negative view they want by simply typing it into a search box.

Commoditization will be complete once the media starts quoting GPT4 for reliably opposing views.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#303

Post by skybrian »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

Talking about whether AI “works” or “doesn’t work” is a dumb debate. The conversation about AI is confusing with people having a lot of different ideas that are mostly more nuanced than that. I don’t believe Gary Marcus thinks in those terms either.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#304

Post by dTal »

jacquesm wrote: post #64
skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

> I am young, but still lived through several hype phases. My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized ar…

I wish this viewpoint were more common. It's frightening to see how rapidly the "AI safety / alignment" discourse is being co-opted into arguments that AI should follow the whims of large centralized corporations. We have no idea what secret instructions OpenAI or the others are giving their "helpful" assistants. I find the notion that AI will spontaneously become a paperclip maximizer much less immediately terrifying than the clear and present danger of it being co-opted by our existing soulless paperclip maximizers, corporations, to devastating effect.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#305

Post by computerex »

bigtex88 wrote: post #258
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

A mis-aligned super AGI will treat the Earth and everything on it as a playground of atoms and material. Why wouldn't it? What do children do when they see a sandbox? Do they care what happens to any ants or other bugs that might live in it? There does not need to be a "how", as you put it. The logic is "Maybe we should tread carefully when creating an intelligence that is magnitudes beyond our own". The logic is "Ma…

Honestly this fear that people have I think is straight up coming from science fiction. It's not grounded in rational reality. Large language models are just like really smart computer programs.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#306

Post by nonbirithm »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

I can maybe sense some foresight that we didn't have at the time other world-changing technologies were discovered. People reference the printing press often in relation to the arguments against progress, but I haven't seen discussion about the internal combustion engine yet, which hundreds of years later we're now hoping we can replace with EVs because of their unforseen consequences. Books don't have the same impact, especially with paperless formats becoming commonplace. Would we have reacted to the ICE hundreds of years ago the same way some are reacting to AI, that it needs to be stopped or limited to prevent the destruction of $entity, if there had been enough knowledge about how tech can affect e.g. the environment, or the world in general?

There was nothing in place to stop us from taking the tradeoff of transportation efficiency over the at the time unknown negative effects. But should we call the invention of the ICE and other related technologies a mistake that irreparably doomed the planet and our future generations? I have no idea. It's a tough question we were incapable of asking at the time, and might reveal some hard truths about the nature of progress and the price we have to pay for it. At least with the effects known at this point, we can start asking it, and it's a step up from obliviousness.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#307

Post by avidphantasm »

Generative AI differs from things like the printing press or internet in that AI has the potential for agency or agent-like capabilities. Without a human, a printing press does nothing. However, it’s easy to imagine an AI being able to act on its own, potentially intervening in real-world systems. So it’s a straw man argument in my opinion to compare AI to prior technologies lacking agency.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#308

Post by notShabu »

IMO "artificial" intelligence is natural intelligence. Both human brains and silicon brains are formed from stars and are "the universe's way of understanding itself."

AI maps closely to myths of all-knowing all-powerful "Dragons", aspects of nature that destroy and create without regard to human plans. Living with AI will likely be similar to living on a volcano island.

Since human domination over nature has only ever increased, a reversal where humans are subservient to a higher capricious force feels threatening.

The funny thing is... living under the dominion of a "higher force" that creates and destroys yet always does what is "right" b/c it is the Source of Everything (even if it feels unfair and tragic) is what religion deals with.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#309

Post by hamburga »

entropyneur wrote: post #283
bigtex88 wrote: post #258

Earlier quoted context omitted.

A mis-aligned super AGI will treat the Earth and everything on it as a playground of atoms and material. Why wouldn't it? What do children do when they see a sandbox? Do they care what happens to any ants or other bugs that might live in it? There does not need to be a "how", as you put it. The logic is "Maybe we should tread carefully when creating an intelligence that is magnitudes beyond our own". The logic is "Ma…

Most bugs haven't gone extinct though. I doesn't seem obvious that any project the AGI will find worthwhile would necessitate exterminating the humanity.

https://en.wikipedia.org/wiki/Instrumental_convergence

Re: If AI scaling is to be shut down, let it be for a coherent reason

#310

Post by 1attice »

nradov wrote: post #122
GistNoesis wrote: post #102

Here is a coherent reason : Just to know where we are standing in our ability to control AI. Like an alcoholic saying he is in control and that he can stop whenever he wants, can try to go dry for a month (or 6) to show that he still somewhat is. If Covid exposed only one thing : it is humanity total failure to control global phenomenons for positive outcomes. It is becoming evident to everybody, and this open-letter…

There was never any realistic possibility of controlling a highly contagious respiratory virus, so COVID-19 didn't expose anything about humanity's ability to control global phenomenons. What it did expose was the tendency of many people to irrationally overreact to a relatively minor risk, mostly because the risk was novel .

The risk was not minor -- COVID remains a leading cause of death in most countries. https://www.thinkglobalhealth.org/article/just-how-do-deaths...

It is now, thanks to swift and absurdly successful mRNA vaccine research, a minor risk to you.

Post reply on HN