Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

131–140 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#131

Post by haberman »

jacquesm wrote: post #64
skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

> I am young, but still lived through several hype phases. My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized ar…

> So far tech was doing what it was told, within certain boundaries. This is the kind of tech that will end up telling you what to do, either directly or through some intermediary

The financial crisis in 2008 was caused by investors who made risky bets based on models they built telling them that mortgage-backed securities were a safe investment. Our understanding of the world is guided by social science that seems to involve an incredible amount of p-hacking using statistics packages that make it easy to crunch through big datasets looking for something publication-worthy. It seems like tech already gives people plenty of tools to make poor decisions that hurt everybody.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#132

Post by dangond »

maroonblazer wrote: post #66
tibbon wrote: post #27

Earlier quoted context omitted.

I don’t think it even requires that AI to be sentient or malicious. The humans already are. Given a tool for carnage and hatred, people will use it. How long did it take from us getting the atomic bomb working to use in production? Less than three months. Will Putin or terrorists hold back from using it in terrible ways if they have it available to them?

>I don’t think it even requires that AI to be sentient or malicious. The humans already are. We are and we aren't. I was struck by this line in the OP: >AI is manifestly different from any other technology humans have ever created, because it could become to us as we are to orangutans; As far as I can tell, we humans treat orangutans quite kindly. I.e., on the whole, we don't go around killing them indiscriminately o…

The whole point is that no one knows how to engineer these principles into a model, and no one has a good plan for doing so either.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#133

Post by rhn_mk1 »

lukev wrote: post #125
rhn_mk1 wrote: post #78

Earlier quoted context omitted.

> Because they can imagine a god-like super intelligent AI, it must be possible. It's worse. It may be possible, but we're not equipped to recognize the line as it's crossed. Combined with us making LLMs more and more capable despite not knowing why they work, this extrapolating of LLMs to gods is not insane.

This is exactly the kind of mysticism I'm talking about. In fact we know precisely how LLMs work. The fact that parts of human linguistic concept-space can be encoded in a high dimensional space of floating point numbers, and that a particular sequence of matrix multiplications can leverage that to perform basic reasoning tasks is surprising and interesting and useful . But we know everything about how how it is trai…

We know how they work, that is true. We don't know why they work, because if we could, then we could extrapolate what happens when you throw more compute at them, and no one would have been surprised about the capabilities of GPT-N+1. Also no one would have been caught with their pants down by seeing people jailbreak their models.

To illustrate it in a different way: on a mechanistic level, we know how animal brains work, as well. Ganglions, calcium channels, the stuff. That doesn't help understand high level phenomena like cognition, which is the part that matters.

If you're right about the LLMs revealing their inner working, that would be indeed a reason to chill out. But I have my doubts, given that LLMs are good at hallucinating. Could you justify why the human readability is actually true, and support that with examples?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#134

Post by dangond »

nl wrote: post #107
Sankozi wrote: post #91

Earlier quoted context omitted.

> "don't kill everyone" really doesn't seem the hardest problem here. And yet you made a mistake - it should be "don't kill anyone". AI just killed everyone except one person.

But that falls into the same "we'd call a person stupid who did a mild version of that" issue. A super intelligent AI would understand the goal!

What stops a super intelligent AI from concluding that we are the ones who misunderstood the goal by letting our morals get in the way of the most obvious solution?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#135

Post by mark_l_watson »

For me, this is complex. My first impression is that many of the signers work on older methods than deep learning and LLMs. Sour grapes.

Of course, real AGI has its dangers, but as Andrew Ng has said, worrying about AGI taking over the world is like worrying about overcrowding of Mars colonies. Both tech fields are far in the future.

The kicker for me though is: we live in an adversarial world, so does it make sense for just a few countries to stop advanced research when most other countries continue at top speed?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#136

Post by r3trohack3r »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

This is my view too. The perceived value of LLMs relative to everything that came before is staggering.

I wouldn’t expect any regulation or laws to slow it down at this point. You might get the U.S. to crack down on it and stop innovation, GPL or proprietary license holders might win a lawsuit, etc.

But I suspect the net effect would be to push research and innovation out of the jurisdiction that made that decision and into another that isn’t willing to kill LLMs in their economy.

Personally, after seeing what LLMs can do first hand, I’d likely move jurisdictions if the U.S. cracked down on AI progress. There is a not-0 chance that putting bread on my table in 10 years requires it.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#137

Post by mrob »

bombcar wrote: post #84
NumberWangMan wrote: post #68

Earlier quoted context omitted.

I'm a lot more scared of an AI destroying humanity, like, every single person, than I am of any government or anything like that. More so than climate change. I'm not saying the people using it aren't dangerous -- but I would choose a totalitarian regime over a rogue AI any day of the week. It wouldn't be HAL 9000 or terminators. It would be an AI deciding that it needs to turn every bit of available matter on earth…

Work out the details on exactly how said end-of-the-world would occur. Note that we already have “AI to end cities” - they’re just sitting, turned off, waiting for the code and button press in silos and submarines throughout the world.

The danger is from something vastly more intelligent than humans, and with a mindset that's incomprehensibly alien. No human is capable of working out the details. That doesn't mean the risk doesn't exist. Failure to understand chemistry does not make ants immune to insecticides. The only thing we can assume about a super-intelligence is that it will be highly capable of achieving its goals. There is no reason to assume those goals will be compatible with human existence.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#138

Post by cableshaft »

lkbm wrote: post #37
dandellion wrote: post #20

Earlier quoted context omitted.

If they're really so worried about it they should start raiding data centres yesterday, writing open letters is such an obvious waste of time that I have a hard time taking it seriously.

Academics, businessmen, and other non-governmental entities are usually better off advocating for the government to enact regulations than engaging in paramilitary action on their own. Do you think Extinction Rebellion is doing more to fight climate change than the people working with governments to enact good climate policy? Do you think PETA is helping animal well-fare more than more than the various organizations…

> Do you think Extinction Rebellion is doing more to fight climate change than the people working with governments to enact good climate policy?

I don't think either have been very effective, at least not recently (getting CFCs banned in the 90s was pretty cool though). And certainly not on the scale that's required at this stage.

> Serious people don't engage in terrorism because the issue is serious. They try to convince the existing power structures to take action.

And the existing power structures dig in their heels, or put on a big show of making changes while really doing almost nothing, or cede some ground for a few years, and then pay to get someone in political power who will reverse all that progress, so that no action ever really gets taken. Fun!

Re: If AI scaling is to be shut down, let it be for a coherent reason

#139

Post by Avicebron »

lukev wrote: post #76
jacquesm wrote: post #64

Earlier quoted context omitted.

> I am young, but still lived through several hype phases. My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized ar…

I agree with this 100%, while also disagreeing with the "robot god will kill us all" objections to AI which unfortunately tend to get most of the mindshare. I think it's important to realize that these are two _completely separate_ concerns. Unfortunately, a lot of the people who get the most air time on this topic are not at all worried about authoritarianism or economic collapse compared to a hypothetical singulari…

[dead]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#140

Post by mellosouls »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

I am maybe not so young and have lived through various hype cycles as well, plus I'm sceptical wrt AGI/sentience via LLMs as well as being a "they should call it ClosedAI" moaner to boot.

So I think I'm pretty hype-averse and a natural scoffer in this instance, but the reality is I've been stunned by the capabilities, and think we are in a transformative cultural moment.

I know "this time it's different" is part of the hype cycle meme, but you know what - this time...

Post reply on HN