Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

311–320 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#311
post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

> It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded Gary Marcus has been aware that neural nets work for a while now, but he is only in the spotlight for his contrarian take, if he stops having a contrarian take he disappears, because it's not like he is producing any research wort…

"Neural networks don't really work" isn't an accurate representation of Marcus' position, and his actual position hasn't been shown to be wrong unless you believe that LLMs and diffusion models display, or are manifestly on the way towards displaying, understanding. That is something many think, and it's not in itself an unreasonable view. However there are also plenty of reasons to think otherwise and many, including me, who haven't conceded the point. It hasn't been settled beyond reasonable debate.

To assume that the person you disagree with can only hold their view out of cynical self-interest, wishful thinking or plain insanity is to assume that you are so obviously right that there can be no valid debate. That is a bad starting position and I'd recommend against it as a matter of principle. Apart from anything else, however convinced you are of your own rightness it's plain rude to assume everyone else is equally convinced, and ad-hominem to ascribe negative motives to those who disagree.

As for Gary Marcus, as far as I've seen he's been consistent in his views and respectful in the way he's argued them. To read about him on HN you'd think he's spent the last few years badmouthing every AI researcher around, but I haven't seen that, just disagreement with people's statements - i.e. healthy debate. I haven't followed him closely though, so if you know of any cases where he's gone beyond that and said the sorts of things about AI researchers that people routinely say about him I'd interested to see them.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#313

Earlier quoted context omitted.

> I'm so offended by the idea of this that I'll personally fund $10k of training runs myself in a jurisdiction where it isn't banned if this ever became law in the US. And this is exactly the risk. These things will just go other places while the US trips over its salty billionaires trying to create political red tape to allow their own businesses to catch up. Surely Elon has no reservations about putting premature s…

To calibrate: Yudkowsky didn't sign because he feels a temporary moratorium doesn't go far enough, and he published a piece in Time [0] with a stance that a shooting war to enforce a ban on AI training would be justified, to prevent the extinction of the human race. I'm not convinced that he's wrong. [0] https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...

The US is not in any position to start a shooting war with anyone that has the ability to push this field forward and come out of it intact. AI training will continue and Yudkowsky will be proven wrong.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#314

> Were you, until approximately last week, ridiculing GPT as unimpressive, a stochastic parrot, lacking common sense, piffle, a scam, etc. — before turning around and declaring that it could be existentially dangerous? How can you have it both ways? If the problem, in your view, is that GPT-4 is too stupid, then shouldn’t GPT-5 be smarter and therefore safer? Thus, shouldn’t we keep scaling AI as quickly as we can ……

I feel like many people who signed the statement did so not because they really agreed with it but because they want a pause on the churn, just as OP had colleagues who admitted as much just for their academic reasons. A lot of people don't really think it's smart but find it dangerous for other reasons, or they have issues with the blatant IP violation that is just being assumed to be okay and "fair use."

Re: If AI scaling is to be shut down, let it be for a coherent reason

#315

Earlier quoted context omitted.

> What we have now is a tool that is able to generate text-based content that is indistinguishable from that written by an 8th grader. To be fair, this 8th grader passed the bar exam..

Well, lawyers pass the bar exam and they're not human either (ba-dum dum!) In all seriousness, I know of a few lawyers who would tell you that's not as impressive as non-lawyers think it is. And the reality is, it did not technically "pass" the bar exam. That's media spin and hype. It doesn't have personhood, it's not a student, it's not being evaluated under the same strict set of conditions. It was an engineering e…

Thanks for this. HN is great to burst my bubble a bit sometimes.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#316

A little off-topic, but does anyone wonder what kind of AI tech the CIA or DARPA or the NSA might have? Or what they might be building? It's a little scary that a small private company could create an AI that has dominated headlines for over a month now. Surely, the folks at NSA, who have more data than anyone else, have taken notice and are working on something of their own. Am I being paranoid about a potential too…

I've seen a lot of people worrying about "what will China do with LLMs?" But my question is: how do you know they don't have them already? What if they've been deployed on the internet for the past three years? I guess the same logic can apply to CIA, NSA, DARPA, etc. But you can take solace in the fact that most government organizations are pretty inept. It used to be common understanding that the military/intellige…

> But you can take solace in the fact that most government organizations are pretty inept.

This is literally the thing that they've been trying to convince you of. Remember Snowden? If not, why not?

So, at least their propaganda wing is working great.

P.S. also, US federal agencies tend to be uniquely stupid, largely due to starve-the-beast politics.

It is not a universal truth. Americans always base their opinions of state institutions in general on their experience with institutions of the American state.

Other countries can & do have actually-effective apparatus, often terrifyingly so.

See, for example, https://www.nytimes.com/2023/03/30/technology/police-surveil...

Re: If AI scaling is to be shut down, let it be for a coherent reason

#317

Earlier quoted context omitted.

A mis-aligned super AGI will treat the Earth and everything on it as a playground of atoms and material. Why wouldn't it? What do children do when they see a sandbox? Do they care what happens to any ants or other bugs that might live in it? There does not need to be a "how", as you put it. The logic is "Maybe we should tread carefully when creating an intelligence that is magnitudes beyond our own". The logic is "Ma…

Honestly this fear that people have I think is straight up coming from science fiction. It's not grounded in rational reality. Large language models are just like really smart computer programs.

There are PhDs who've spent their careers studying AI safety. It's a bit insulting and reductive to cast their work as "coming from science fiction", especially when it sounds like you haven't done much research on the topic.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#318
post #280

Earlier quoted context omitted.

Bullshit. I have used the various LLM tools and have read extensively about them. They have not displayed any emergent capabilities in an AGI sense. Your comment is simply ignorant.

I'm going to just assume you're not being malicious. https://www.microsoft.com/en-us/research/publication/sparks-... https://www.assemblyai.com/blog/emergent-abilities-of-large-... If you'd like you can define "emergent capabilities in an AGI sense".

I am not being malicious. I do not accept those papers as being actual examples of emergent behavior in a true AGI sense. This is just another case of humans imagining that they see patterns in noise. In other words, they haven't rejected the null hypothesis. Junk science. (And just because the science is bad doesn't mean the underlying products aren't useful for solving practical problems and enhancing human productivity.)

The blowhards who are calling for arbitrary restrictions on research are the ones being malicious.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#319

Earlier quoted context omitted.

> I'm so offended by the idea of this that I'll personally fund $10k of training runs myself in a jurisdiction where it isn't banned if this ever became law in the US. And this is exactly the risk. These things will just go other places while the US trips over its salty billionaires trying to create political red tape to allow their own businesses to catch up. Surely Elon has no reservations about putting premature s…

To calibrate: Yudkowsky didn't sign because he feels a temporary moratorium doesn't go far enough, and he published a piece in Time [0] with a stance that a shooting war to enforce a ban on AI training would be justified, to prevent the extinction of the human race. I'm not convinced that he's wrong. [0] https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...

what a weird modern version of religious war

Re: If AI scaling is to be shut down, let it be for a coherent reason

#320

Earlier quoted context omitted.

Honestly this fear that people have I think is straight up coming from science fiction. It's not grounded in rational reality. Large language models are just like really smart computer programs.

There are PhDs who've spent their careers studying AI safety. It's a bit insulting and reductive to cast their work as "coming from science fiction", especially when it sounds like you haven't done much research on the topic.

Why do these arguments not tell us how it will happen?

Show us the steps the AI will take to turn the earth into a playground. Give us a plausible play by play so that we might know what to look for.

Does it gain access to nukes? How does it keep the power on? How does it mine for coal? How does it break into these systems?

How do we not notice an AI taking even one step towards that end?

Has ChatGPT started to fiddle with the power grid yet?

Post reply on HN