Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

121–130 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#121

Post by adamsmith143 »

synaesthesisx wrote: post #51

The arguments for decelerating AI development invoke fear-mongering and cite “existential risk”, which is a fallacy. We’re talking about LLMs, not AGI here (which is quite a ways out, realistically). If anything - we should be accelerating development toward the goal of AGI, as the implications for humanity are profound.

>not AGI here (which is quite a ways out, realistically)

You don't know how far away it is.

>If anything - we should be accelerating development toward the goal of AGI, as the implications for humanity are profound.

Given we don't know how far away it is but current models are matching Human performance on lots of tasks and we don't know any way to ensure their safety it's entirely reasonable to be scared.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#122

Post by nradov »

GistNoesis wrote: post #102

Here is a coherent reason : Just to know where we are standing in our ability to control AI. Like an alcoholic saying he is in control and that he can stop whenever he wants, can try to go dry for a month (or 6) to show that he still somewhat is. If Covid exposed only one thing : it is humanity total failure to control global phenomenons for positive outcomes. It is becoming evident to everybody, and this open-letter…

There was never any realistic possibility of controlling a highly contagious respiratory virus, so COVID-19 didn't expose anything about humanity's ability to control global phenomenons. What it did expose was the tendency of many people to irrationally overreact to a relatively minor risk, mostly because the risk was novel.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#123

Post by A4ET8a8uTh0 »

lkbm wrote: post #32
nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…

Some of us look at patterns and are, understandably, cynical given some of the steps taken ( including those that effectively made OpenAI private the moment its potential payoff became somewhat evident ).

So yeah. There is money on the table and some real stakes that could be lost by the handful of recent owners. Those incumbent and non-incumbent voices are only being amplified now ( as you noted they DID exist before all this ), because it is convenient for the narrative.

They are not being dismissed. They are simply being used.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#124

Post by adamsmith143 »

mrob wrote: post #120
adamsmith143 wrote: post #115

Earlier quoted context omitted.

Because if you don't find out a way for it to hold human values extremely well then an easy solution to "Cure All Cancer" is to "Kill all Humans", no Humans no Cancer. Without an explicit understanding that this is not an actually acceptable outcome for humans an AI will happily execute it. THAT is the fundamental problem, how do you get human values into these systems.

Even "kill all humans" is difficult to define. Is a human dead if you flash-freeze them in liquid helium? It would certainly make it easier to cut out the cancer. And nobody said anything about defrosting them later. And even seemingly healthy humans contain cancerous cells. There's no guarantee their immune system will get all of them.

Fine change the wording to "delete all humans". Same outcome, no humans no cancer.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#125

Post by lukev »

rhn_mk1 wrote: post #78
lukev wrote: post #36

Earlier quoted context omitted.

The reasoning pattern of Yud and his ilk is fundamentally theological and eschatological in nature. There isn't any actual understanding of the technology involved. It is fundamentally an ontological argument. Because they can imagine a god-like super intelligent AI, it must be possible. And they're associating that with LLMs because that's the most powerful AI currently available, not based on any actual capabilitie…

> Because they can imagine a god-like super intelligent AI, it must be possible. It's worse. It may be possible, but we're not equipped to recognize the line as it's crossed. Combined with us making LLMs more and more capable despite not knowing why they work, this extrapolating of LLMs to gods is not insane.

This is exactly the kind of mysticism I'm talking about. In fact we know precisely how LLMs work.

The fact that parts of human linguistic concept-space can be encoded in a high dimensional space of floating point numbers, and that a particular sequence of matrix multiplications can leverage that to perform basic reasoning tasks is surprising and interesting and useful.

But we know everything about how how it is trained and how it is invoked.

In fact, because it's only "state" aside from its parameters is whatever its context window, current LLMs have the interesting property that if you invoke them recursively, all of their "thoughts" are human readable. This is in fact a delightful property for anyone worried about AI safety: our best AIs currently produce a readable transcript of their "mental" processes in English.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#126

Post by AlanYx »

findalex wrote: post #110
cs702 wrote: post #2

> ... I was deeply confused, until I heard a dear friend and colleague in academic AI, one who’s long been skeptical of AI-doom scenarios, explain why he signed the open letter. He said: look, we all started writing research papers about the safety issues with ChatGPT; then our work became obsolete when OpenAI released GPT-4 just a few months later. So now we’re writing papers about GPT-4. Will we again have to throw…

>started writing research papers about the safety issues with ChatGPT; Feels strange that academia would focus so much energy on a product.

Just judging from the volume of papers, it seems to me that there are more academics writing papers on "AI safety" and "AI ethics" than there are academics publishing research papers on actual AI. It's become one of the hottest topics among legal academics, philosophers, ethicists, and a variety of connected disciplines, in addition to its niche among some computer scientists, and the amount of work to get to a paper in these fields is an order of magnitude less than actually publishing technical research.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#127

Post by lkbm »

nradov wrote: post #56
lkbm wrote: post #32

Earlier quoted context omitted.

On Twitter, it was "multi-blillionaires [Elon Musk] want to keep us down!" In the HN open letter thread it was "All the companies falling behind want a chance to catch up." Now it's "Gary Marcus". The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark. A lot of people, incumbent…

There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…

This is a bit like saying that 1939 Einstein wasn't an expert in nuclear bombs. Sure, it didn't exist, so he wasn't an expert on them, but he was an expert on the thing that led to it and when we said it was possible, sensible people listened.

A lot of people working on LLMs say that they believe there is a path to AGI. I'm very skeptical of claims that there's zero evidence in support of their views. I know some of these people and while they might be wrong, they're not stupid, grifters, malicious, or otherwise off-their-rockers.

What would you consider to be evidence that these (or some other technology) could be a path to be a serious physical threat? It's only meaningful for there to be "zero evidence" if there's something that could work as evidence. What is it?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#128

Post by cableshaft »

lukev wrote: post #36
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

The reasoning pattern of Yud and his ilk is fundamentally theological and eschatological in nature. There isn't any actual understanding of the technology involved. It is fundamentally an ontological argument. Because they can imagine a god-like super intelligent AI, it must be possible. And they're associating that with LLMs because that's the most powerful AI currently available, not based on any actual capabilitie…

I suspect that he realized that there's money to be made being the professional luddite of the hot thing du jour that talk shows and schools can give him money for (as so many other people have done for other things), and he's taking this simple claim of 'AGI will murder us all' to get on as many talk shows as possible and make as much money off of it. I doubt he's speaking in public out of any concern for humanity's survival.

It's such an easy claim to make, because you can just say 'yeah, AGI hasn't murdered us all yet, but it will at some point' and keep kicking out that timeline further and further out until your dead and buried and who cares.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#129

Post by tgv »

detrites wrote: post #43
nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

A great many would likely join you, making this entire fiasco a time-wasting distraction, at best, and grave risk at worst. The technologies will continue to be developed, moratorium or not. A moratorium only enables the hidden to get there first. The risks need to be discussed and understood, along with the benefits, and publicly. That's the only sensible way forward. Denying that the technology is here already and…

> A moratorium only enables the hidden to get there first.

That's simply not true. Nobody would have gotten where GPT is today without transformers. That's not a trivial bit of insight anybody could have had. Stopping research funding and publications will prevent rapid evolution.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#130

Post by seanmcdirmid »

misssocrates wrote: post #100
nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

What if governments start regulating and locking down advanced computing in the same way they locked down medicine and advanced weapontry?

They do, which is why China is dumping a bunch of money into ramping up its asic/GPU tech and production.
Post reply on HN