Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

201–210 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#201

Post by WillPostForFood »

chatmasta wrote: post #34
lkbm wrote: post #32

Earlier quoted context omitted.

On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…

Elon Musk is the most ironic signatory of this blog post, considering his decade-long effort to put AI behind the wheel of 100mph+ vehicles. And then he's got the gall to lecture the rest of us on "AI safety" after we finally get a semi-intelligent chatbot? Come on, man.

If we are being fair, even though we might refer to self driving capabilities as "ai" and as self aware supercomputer overlord as "ai" they aren't the same thing and you can hold different opinions on the development of them.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#202

Post by ben_w »

detrites wrote: post #186
tgv wrote: post #129

Earlier quoted context omitted.

> A moratorium only enables the hidden to get there first. That's simply not true. Nobody would have gotten where GPT is today without transformers. That's not a trivial bit of insight anybody could have had. Stopping research funding and publications will prevent rapid evolution.

I mean given the current state. The technology is already sufficiently advanced and in so many peoples hands that "stopping" it now is just an exercise in pushing it underground. Only the opposite can be a useful safeguard. Rapid evolution is well underway. Lone individuals are able to push the envelope of what's possible even with just a new basic interop, maybe in an afternoon. It's much too late to be discussing t…

It's almost impossible to tell.

Yes, plenty of low hanging fruit around; Heck, I can probably literally ask chatGPT to implement for me a few ideas I've got.

OTOH, I've known since secondary school of two distinct ways to make a chemical weapon using only things commonly found in normal kitchens, and absolutely none of the post 9/11 aftershock attacks that got in the news over the next decade did anything remotely so simple, so that example makes me confident that even bad rules passed in haste — as many of them were and remain — can actually help.

(And that's despite my GCSE Chemistry being only grade B).

Re: If AI scaling is to be shut down, let it be for a coherent reason

#203

Post by JohnFen »

judge2020 wrote: post #113
misssocrates wrote: post #100

Earlier quoted context omitted.

What if governments start regulating and locking down advanced computing in the same way they locked down medicine and advanced weapontry?

So far it seems letting private industry iterate on LLMs doesn't directly pose to a risk of ending lives like human trials and nuclear weaponry development do.

I think the fear of LLMs is very overblown. On the other hand, I think that if LLMs actually manage to do what proponents hope it will, some people will die as a result due to economics when they lose their jobs.

That's not unique to LLMs, of course. It's what has happened before every time something has obsoleted a bunch of jobs. There's no reason to think this time would be any different.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#204

Post by hollerith »

Havoc wrote: post #10

This whole thing is like leaving 20 kids alone in a room with a marshmallow each and telling them don't eat it. ...and then expecting all of them resist. The debate around whether we should tell the kids not to eat it and for what reasons is completely academic. Practically this just isn't happening.

It more like asking the US government to get involved and if one of the 20 kids keeps on eating marshmallows, large numbers of federal agents raid the kid's offices and shut down the kid's compute resources.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#205

Post by endtime »

computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

> I ask, how. He is making reductive logical leaps that don't make sense. It's FUD.

His Time article addresses this, as does much of his other writing. It really stems from two key points:

1) The vast majority of possible superintelligences have utility functions that don't include humans. Mind space is large. So by default, we should assume that a superintelligence won't go out of its way to preserve anything we find valuable. And as Eliezer says, we're made of useful atoms.

2) By definition, it can think of things that we can't. So we should have no confidence in our ability to predict its limitations.

It's reasonable to challenge assumptions, but it's not reasonable to say this line of reasoning doesn't exist.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#206

Post by mrob »

agentultra wrote: post #57

We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. The danger has been the people who use it and find ways to abuse other people with it. This whole idea that we're going to suddenly have HAL 9000 or Terminators running around is mob hype mentality. The responsible thing to do here is to educate. AGI isn't likely to destroy humanity. Humanity is already doing a good job of…

Anyone bringing up sci-fi is acting under a fallacy that doesn't have a formal name, but which I'd call "biomorphism", by analogy to anthropomorphism. The human mind is the product of billions of years of evolution, and as such, it's driven by assumptions such as self preservation, sex drive, status seeking, coherent sense of self, that are so fundamental that most of the time we don't even think about them. Sci-fi, even sci-fi written by authors like Peter Watts, who put serious effort into exploring the possible design space of minds, still follows most of these assumptions. A Terminator is ostensibly a machine, but it acts like a caricature of a man.

There's only one genre that writes about truly alien minds (albeit with necessary vagueness), and that's cosmic horror. And unlike sci-fi, which often pretends humans could win, cosmic horror is under no delusion that summoning Elder Gods is ever a good idea.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#207

Post by gliptic »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

> We know a bunch of the people whose names are on the letter didn't sign it

Yann LeCun's name never was on the letter. Where did this meme come from.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#208

Post by JohnFen »

lukev wrote: post #76
jacquesm wrote: post #64

Earlier quoted context omitted.

> I am young, but still lived through several hype phases. My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized ar…

I agree with this 100%, while also disagreeing with the "robot god will kill us all" objections to AI which unfortunately tend to get most of the mindshare. I think it's important to realize that these are two _completely separate_ concerns. Unfortunately, a lot of the people who get the most air time on this topic are not at all worried about authoritarianism or economic collapse compared to a hypothetical singulari…

Personally, I'm worried that if the proponents of LLMs are correct, it will directly lead to authoritarianism and economic collapse.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#209

Post by mattmaroon »

freejak wrote: post #193
hbosch wrote: post #172

Earlier quoted context omitted.

>it seem entirely reasonable that it would go from laughable to godlike in a week. Not if it has a power cord.

It just needs to convince once human to connect it to the grid which wouldn't be a difficult feat for a super-intelligent AI.

Or just distribute itself to computers everywhere.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#210

Post by G_z9 »

nradov wrote: post #56
lkbm wrote: post #32

Earlier quoted context omitted.

On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…

There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…

Yeah, there aren’t experts in something that doesn’t exist. That means we have to make an educated guess. By far the most rational course of action is to halt AI research. And then you say there’s no proof that we are on the path to AGI or that it would harm us. Yeah, and there never could be any proof for either side of the argument. So your dismissal of AI is kind of flaccid without any proof or rational speculation or reasoning. Listen man I’m not a cynical commenter. I believe what I’m saying and I think it’s important. If you really think you’re right then get on the phone with me or video chat so we can actually debate and settle this.
Post reply on HN