If AI scaling is to be shut down, let it be for a coherent reason • Hacker News Board

Re: If AI scaling is to be shut down, let it be for a coherent reason

#261

Post by dwaltrip » Fri, Mar 31, 2023, 4:34 PM UTC

SanderNL wrote: post #226

gspencley wrote: post #94

Earlier quoted context omitted.

You don't even have to look at other "fads" like 3D televisions and VR to be skeptical of the claim that recent advancements in ML* represent a major technological revolution that will change society like the Internet or the Printing Press did. Just look at ML's own history. A few years ago we had the "deepfake" scare. People were terrified that now videos are going to surface of politicians saying and doing things t…

> What we have now is a tool that is able to generate text-based content that is indistinguishable from that written by an 8th grader. To be fair, this 8th grader passed the bar exam..

So many people conveniently disregard facts like this. It's much easier to write it off as "impressive auto-complete", "a writing assistant", "simply regurgitating the training data", etc.

It's an alien intelligence that we barely understand. It has many limitations but also possesses insane capabilities that have not been fully explored.

What happens when you take gpt-5, give it 100x more context / "memory", the ability to think to itself in-between tokens, chain many of them together in such a way that they have more agent-like behavior, along with ten other enhancements we haven't thought of? No one knows...

The biggest limitation of GPT capabilities is our imagination.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#262

Post by ok123456 » Fri, Mar 31, 2023, 4:37 PM UTC

deepsquirrelnet wrote: post #150

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

> I'm so offended by the idea of this that I'll personally fund $10k of training runs myself in a jurisdiction where it isn't banned if this ever became law in the US. And this is exactly the risk. These things will just go other places while the US trips over its salty billionaires trying to create political red tape to allow their own businesses to catch up. Surely Elon has no reservations about putting premature s…

Elon musk doesn't care about this for altruistic reasons or about the future of work. He cares because he owns a social network, and these models can make spam that's impossible to detect, especially given the short size of tweets.

It's naked self-interest.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#263

Post by nradov » Fri, Mar 31, 2023, 4:37 PM UTC

lkbm wrote: post #127

nradov wrote: post #56

Earlier quoted context omitted.

There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…

This is a bit like saying that 1939 Einstein wasn't an expert in nuclear bombs. Sure, it didn't exist, so he wasn't an expert on them, but he was an expert on the thing that led to it and when we said it was possible, sensible people listened. A lot of people working on LLMs say that they believe there is a path to AGI. I'm very skeptical of claims that there's zero evidence in support of their views. I know some of…

That is not a valid analogy. In 1939 there was at least a clear theory of nuclear reactions backed up by extensive experiments. At that point building a weapon was mostly a hard engineering problem. But we have no comprehensive theory of cognition, or even anything that legitimately meets the criteria to be labeled a hypothesis. There is zero evidence to indicate that LLMs are on a path to AGI.

If the people working in this field have some actual hard data then I'll be happy to take a look at it. But if all they have is an opinion then let's go with mine instead.

If you want me to take this issue seriously then show me an AGI roughly on the level of a mouse or whatever. And by AGI I mean something that can reach goals by solving complex, poorly defined problems within limited resource constraints (including time). By that measure we're not even at the insect level.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#264

Post by laichzeit0 » Fri, Mar 31, 2023, 4:37 PM UTC

kami8845 wrote: post #152

nl wrote: post #59

Earlier quoted context omitted.

Of course they would. It's just ridiculous. If people are genuinely concerned about lack of access to the OpenAI models then work at training open ones! OpenAI has a maybe 6 month lead and that's nothing. Plus it's much easier being the follower when you know what is possible. (To be clear, I know at least a few of the projects already working on this. I just want to make it clear that is the intellectually honest ap…

I think their lead might be a bit bigger than that. ChatGPT 3.5 was released 4 months ago and I still haven't seen another LLM come close to it.

A slightly more paranoid me asks whether there’s some magic they’re using that no one is completely aware of. Watching Google fumble around makes me more paranoid that that’s the case.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#265

Post by 1attice » Fri, Mar 31, 2023, 4:40 PM UTC

This essay is literally just Isildur looking at the ring he cut from Sauron and saying "I kind of like it tho".

-> Aaronson admits Yudkowsky's position is consistent -> Main disagreement is that he can imagine other outcomes -> Some soft analogism against other historical technologies (radios, etc) -> "Should I, tho?" segue to the comments.

Yes, Aaronson, you should have signed that letter. You know you should have. Snap out of it.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#266

Post by version_five » Fri, Mar 31, 2023, 4:41 PM UTC

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

All the big named signatories to this were transparently self interested, as well as not real "practitioners" - mostly adjacent folks upset to be out of the limelight. I know it's a blurry line, but people like Andrew Ng and Yann LeCunn who are actually building ML stuff have dismissed this outright. It's almost like a bunch of sore losers got together and took an irrelevant stand. Like Neil Young and Joni Mitchell vs Spotify.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#267

Post by JohnFen » Fri, Mar 31, 2023, 4:42 PM UTC

dwaltrip wrote: post #239

ben_w wrote: post #202

Earlier quoted context omitted.

It's almost impossible to tell. Yes, plenty of low hanging fruit around; Heck, I can probably literally ask chatGPT to implement for me a few ideas I've got. OTOH, I've known since secondary school of two distinct ways to make a chemical weapon using only things commonly found in normal kitchens, and absolutely none of the post 9/11 aftershock attacks that got in the news over the next decade did anything remotely so…

Right, it's amazing to me the extent to which people are throwing their hands in there and saying "There's absolutely NOTHING that can be done!!! We must accept the AGIs however they will manifest"... Clearly, it's a very hard problem with massive uncertainties. But we can take actions that will significantly decrease the risk of utter catastrophe. I don't even think world-ending catastrophe is that likely. But it se…

I suspect that the people who are saying "nothing can be done" are people who want nothing to be done.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#268

Post by dragonwriter » Fri, Mar 31, 2023, 4:42 PM UTC

1attice wrote: post #265

This essay is literally just Isildur looking at the ring he cut from Sauron and saying "I kind of like it tho". -> Aaronson admits Yudkowsky's position is consistent -> Main disagreement is that he can imagine other outcomes -> Some soft analogism against other historical technologies (radios, etc) -> "Should I, tho?" segue to the comments. Yes, Aaronson, you should have signed that letter. You know you should have.…

> Aaronson admits Yudkowsky’s position is consistent

A position can be both consistent and false, because it is based on false premises; its frequently the case when people attempt to derive conclusions about the material universe from pure a priori reasoning from abstract axioms and assumptions without empirical grounding.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#269

Post by bigtex88 » Fri, Mar 31, 2023, 4:42 PM UTC

maroonblazer wrote: post #66

tibbon wrote: post #27

Earlier quoted context omitted.

I don’t think it even requires that AI to be sentient or malicious. The humans already are. Given a tool for carnage and hatred, people will use it. How long did it take from us getting the atomic bomb working to use in production? Less than three months. Will Putin or terrorists hold back from using it in terrible ways if they have it available to them?

>I don’t think it even requires that AI to be sentient or malicious. The humans already are. We are and we aren't. I was struck by this line in the OP: >AI is manifestly different from any other technology humans have ever created, because it could become to us as we are to orangutans; As far as I can tell, we humans treat orangutans quite kindly. I.e., on the whole, we don't go around killing them indiscriminately o…

The problem is that we do not know "How" to engineer those principles. And that's what the entire field of AI alignment is working on. We know what we want the AI to do; the problem is we don't know how to make certain it does that. Because if we only get it 99% right then we're probably all dead in the end.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#270

Post by 1attice » Fri, Mar 31, 2023, 4:43 PM UTC

beauzero wrote: post #196

r3trohack3r wrote: post #136

Earlier quoted context omitted.

This is my view too. The perceived value of LLMs relative to everything that came before is staggering. I wouldn’t expect any regulation or laws to slow it down at this point. You might get the U.S. to crack down on it and stop innovation, GPL or proprietary license holders might win a lawsuit, etc. But I suspect the net effect would be to push research and innovation out of the jurisdiction that made that decision a…

> I’d likely move jurisdictions if the U.S. cracked down on AI progress This is my overriding concern. This is a wave crest that if you don't ride it, it may well crush you or take the ocean with it and leave you in a desert. It is as interesting as fire as a tool for civilization going forward.

What a great metaphor.

Did you recall that your planet is currently in a sixth extinction event, brought about, yes, through the repeated overuse of fire?

Zeus was right. Prometheus belonged on that rock.