Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

381–390 of 964 posts

Re: Statement on AI Risk

#381

Post by computerphage »

brookst wrote: post #270
patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

> How can it not be obvious You have succinctly and completely summed up the AI risk argument more eloquently than anyone I've seen before. "How can it not be obvious?" Everything else is just intellectual fig leaves for the core argument that intuitively, without evidence, this proposition is obvious. The problem is, lots of "obvious" things have turned out to be very wrong. Sometimes relatively harmlessly, like the…

> borderline offensive to question them

I would be happy to politely discuss any proposition regarding AI Risk. I don't think any claim should go unquestioned.

I can also point you to much longer-form discussions. For example, this post, which has 670 comments, discussing various aspects of the argument: https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...

Re: Statement on AI Risk

#382

Post by mrtranscendence »

patrec wrote: post #221
valine wrote: post #140

Earlier quoted context omitted.

It's not clear at all that we have an avenue to super intelligence. I think the most likely outcome is that we hit a local maximum with our current architectures and end up with helpful assistants similar in capability to George Lucas's C3PO. The scary doomsday scenarios aren't possible without an AI that's capable of both strategic thinking and long term planning. Those two things also happen to be the biggest limit…

> It's not clear at all that we have an avenue to super intelligence AI already beats the average human on pretty much any task people have put time into, often by a very wide margin and we are still seeing exponential progress that even the experts can't really explain, but yes, it is possible this is a local maximum and the curve will become much flatter again. But the absence of any visible fundamental limit on fu…

> we are still seeing exponential progress

I dispute this. What appears to be exponential progress is IMO just a step function that made some jumps as the transformer architecture was employed on larger problems. I am unaware of research that moves beyond this in a way that would plausibly lead to super-intelligence. At the very least I foresee issues with ever-increasing computational requirements that outpace improvements in hardware.

We’ll see similar jumps when other domains begin employing specialized AI models, but it’s not clear to me that these improvements will continue increasing exponentially.

Re: Statement on AI Risk

#383

Post by api »

patrec wrote: post #58
valine wrote: post #17

I have yet to see a solution for “AI safety” that doesn’t involve ceding control of our most powerful models to a small handful of corporations. It’s hard to take these safety concerns seriously when the organizations blowing the whistle are simultaneously positioning themselves to capture the majority of the value.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

Most of the credible threats I see from AI that don't rely on a lot of sci-fi extrapolation involve small groups of humans in control of massively powerful AI using it as a force multiplier to control or attack other groups of humans.

Sam Altman's proposal is to create precisely that situation with himself and a few other large oligarchs being the ones in control of the leading edge of AI. If we really do face runaway intelligence growth and god-like AIs then this is a profound amount of power to place in the hands of just a few people. Even worse it opens the possibility that such developments could happen partly in secret, so the public might not even know how powerful the secret AIs under command of the oligarchs have become.

The analogy with nuclear weapons is profoundly broken in lots of ways. Reasoning from a sloppy analogy is a great way to end up somewhere stupid. AI is a unique technology with a unique set of risks and benefits and a unique profile.

Re: Statement on AI Risk

#384

Post by fauigerzigerk »

duvenaud wrote: post #231
Infernal wrote: post #227

Earlier quoted context omitted.

“Humans are going to be outcompeted by AI” is the concrete bit as best I can tell. Historically humans are not outcompeted by new tools, but humans using old tools are outcompeted by humans using new tools. It’s not “all humans vs the new tool”, as the tool has no agency. If you meant “humans using old tools get outcompeted by humans using AI”, then I agree but I don’t see it any differently than previous efficiency…

I see what you mean - for a while, the best chess was played by humans aided by chess engines. But that era has passed, and now having a human trying to aid the best chess engines just results in worse chess (or the same, if the human does nothing). But whether there a few humans in the loop doesn't change the likely outcomes, if their actions are constrained by competition. What abilities do humans have that AIs wil…

>What abilities do humans have that AIs will never have?

I think the question is what abilities and level of organisation machines would have to acquire in order to outcompete entire human societies in the quest for power.

That's a far higher bar than outcompeting all individual humans at all cognitive tasks.

Re: Statement on AI Risk

#385

Post by stale2002 »

Planktonne wrote: post #315
qt31415926 wrote: post #299

Earlier quoted context omitted.

Her field has also taken the largest hit from the success of LLMs and her research topics and her department are probably no longer prioritized by research grants. Given how many articles she's written that have criticized LLMs it's not surprising she has incentives.

LLMs are in her field; they are one of her research topics and they're definitely getting funding. We absolutely should not be ignoring research that doesn't support popular narratives; dismissing her work because it is critical of LLMs is not reasonable.

It is not that she is critical of LLMs that is the issue.

Instead, it is that she has strong ideological motivations to make certain arguments.

Those motivations being that her research is now worthless, because of LLMs.

I don't believe the alignment doomsayers either, but that is for different reasons than listening to her.

Re: Statement on AI Risk

#386

Post by dns_snek »

ChatGTP wrote: post #322
dns_snek wrote: post #247

Earlier quoted context omitted.

I've never made any statements about, nor do I have any personal beliefs about Max Tegmark in particular.

Well people like him are on the list, so it's a bit strange you're claiming this is mostly about FAANG?

FAANG are the only ones who stand to benefit financially. I'm taking the position that everyone else is simply a "useful idiot" for the likes of Sam Altman, in the most respectful way possible. Nobody is immune from getting wrapped up in hysteria, so I don't care who they are or what they have achieved when their signatures aren't supported by any kind of reasoning, much less sound reasoning.

Re: Statement on AI Risk

#387

Post by computerphage »

escape-big-tech wrote: post #377
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement

Geoffrey Hinton quit google.

Re: Statement on AI Risk

#388

Post by goolulusaurs »

Throughout history there have been hundreds, if not thousands of examples of people and groups who thought the end of the world was imminent. So far, 100% of those people have been wrong. The prior should be that the people who believe in AI doomsday scenarios are wrong also, unless and until there is very strong evidence to the contrary. Vague theoretical arguments are not sufficient, as there are many organizations throughout history who have made similar vague theoretical arguments that the world would end and they were all wrong.

https://en.wikipedia.org/wiki/Category:Apocalyptic_groups

Re: Statement on AI Risk

#389

Post by mitthrowaway2 »

YeGoblynQueenne wrote: post #355
mitthrowaway2 wrote: post #331

Earlier quoted context omitted.

I'm all in favour of fighting climate change, but I'd be more inclined to agree with you if you provide some kind of supporting argument!

Do you mean you're not aware of the arguments in favour of stopping climate change?

No need; like I said, I'm all in favour of fighting climate change. I view it as an existential risk to humanity on the ~200 year timescale, and it should be a high priority. I'm particularly concerned about the impacts on ocean chemistry.

But if you're going to suggest that a Statement on AI risk should mention climate change but not AI risk, because it's a "fantasy", then... well, I'd expect some kind of supporting argument? You can't just declare it and make it true, or point to some other important problem to stir up passions and create a false dichotomy.

Re: Statement on AI Risk

#390

Post by timmytokyo »

sebzim4500 wrote: post #149
Simon321 wrote: post #108

Earlier quoted context omitted.

This fallacy is also known as Pascal's wager: https://en.wikipedia.org/wiki/Pascal%27s_wager But this argument got billions of people to believe in the concept of hell so i expect it to work again to convince people to believe in AI doomsday.

I agree it's a fallacy when the probability is like 10^-10 but in this case I believe that the probability is more like 1%, in which case the argument is sound. I'm not trying to make a pascal's wager argument.

Correction: you perceive it to be a fallacy when others assign high probabilities to things you believe are low probability. Unfortunately, this cuts both ways. Many people believe your 1% estimate is unreasonably high. Are you therefore promoting a fallacy?

Too many ridiculous arguments can be justified on the backs of probability estimates pulled from nether regions.

Post reply on HN