Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

431–440 of 964 posts

Re: Statement on AI Risk

#431

Post by dopamean »

escape-big-tech wrote: post #377
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

Agreed. If the risks were real they would just outright stop working on their AI products. This is nothing more than a PR statement

I agree that nothing about the statement makes me think the risks are real however I disagree that if the risks are real these companies would stop working on their product. I think more realistically they'd shut up about the risk and downplay it a lot. Much like the oil industry did wrt climate change going back to the 70's.

Re: Statement on AI Risk

#432

Post by kypro »

arp242 wrote: post #318
kypro wrote: post #307

Earlier quoted context omitted.

What's the point in dismissing the need for AI safety? Are you guys Russian bots, or do you genuinely see no reason to worry about AI safety? But since I see these kinds of snarky responses often – we obviously do worry about climate change and various other issues. Continued advancements in AI is just one of many issues that face us which humanity should be concerned about. Few concerned about AI would argue it come…

This page talks about "extinction from AI". I'm sorry, but I think that's a complete non-issue for the foreseeable future. I just don't see how that will happen beyond spectacular science fiction scenarios that are just not going to happen. If that makes me a Russian bot then, well, хорошо! The risks from AI will be banal and boring. Spam, blogspam, fake articles, fake pictures, what-have-you. Those things are an iss…

Apologies, the Russian bot comment was more me venting frustration at the prevalence of low-effort response like yours (sorry) to those who try to raise concerns about AI safety.

I do agree with you that extinction from AI isn't likely to be an issue this decade. However, I would note that it's difficult to predict what the rate of change is likely to be once you have scalable general intelligence.

I can't speak for people who signed this, but for me the trends and risks of AI are just as clear as those of climate change. I don't worry that climate change is going to be a major issue this decade (and perhaps not even next), but it's obvious where the trend is going when you project out.

Similarly the "real" risks of AI may not be this decade, but they are coming. And again, I'd stress it's extremely hard to project when that will be since when you have a scalable general intelligence progress is likely to accelerate exponentially.

So that said, where do we disagree here? Are you saying with a high level of certainty that extinction risks from AI are too far in the future to worry about? If so, when do you think extinction risks from AI are likely to be a concern – a couple of decades, more? Do you hold similar views about the present extinction risk of climate change – and if so, why not?

Could I also ask if you believe any resources in the present should be dedicated to the existential risks future AI capabilities could pose to humanity? And if not, when would you like to see resources put into those risks? Is there some level of capability that you're waiting to see before you begin to be concerned?

Re: Statement on AI Risk

#433

Post by verdverm »

holmesworcester wrote: post #378
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by m…

Pretty much anyone can sign it, also notable people like Grimes, not sure why her signature carries weight on this

Re: Statement on AI Risk

#434

Post by holmesworcester »

RandomLensman wrote: post #412
holmesworcester wrote: post #378

Earlier quoted context omitted.

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by m…

Who on the list is an expert on existential risk (and perhaps even beyond academia)?

Signatory Jan Taallin has founded an x-risk organization focused on AI, biotech, nuclear weapons, and climate change:

https://futureoflife.org/

Re: Statement on AI Risk

#435

Post by lxnn »

Veedrac wrote: post #120
Mizza wrote: post #56

Can anybody who really believes this apocalyptic stuff send me in the direction of a convincing _argument_ that this is actually a concern? I'm willing to listen, but I haven't read anything that tries to actually convince the reader of the worry, rather than appealing to their authority as "experts" - ie, the well funded.

The basic argument is trivial: it is plausible that future systems achieve superhuman capability; capable systems necessarily have instrumental goals; instrumental goals tend to converge; human preferences are unlikely to be preserved when other goals are heavily selected for unless intentionally preserved; we don't know how to make AI systems encode any complex preference robustly. Robert Miles' videos are among the…

Technically, I think it's not that instrumental goals tend to converge, but rather that there are instrumental goals which are common to many terminal goals, which are the so-called "convergent instrumental goals".

Some of these goals are ones which we really would rather a misaligned super-intelligent agent not to have. For example:

- self-improvement;

- acquisition of resources;

- acquisition of power;

- avoiding being switched off;

- avoiding having one's terminal goals changed.

Re: Statement on AI Risk

#436

Post by babyshake »

TheOtherHobbes wrote: post #367
valine wrote: post #140

Earlier quoted context omitted.

It's not clear at all that we have an avenue to super intelligence. I think the most likely outcome is that we hit a local maximum with our current architectures and end up with helpful assistants similar in capability to George Lucas's C3PO. The scary doomsday scenarios aren't possible without an AI that's capable of both strategic thinking and long term planning. Those two things also happen to be the biggest limit…

We don't need an avenue to super-intelligence. We just need a system that is better at manipulating human beliefs and behaviour than our existing media, PR, and ad industries. The problem is not science fiction god-mode digital quetta-smart hypercomputing. This is about political, social, and economic influence, and who controls it.

Indeed, an epistemological crisis seems to be the most realistic problem in the next few years.

Re: Statement on AI Risk

#437

Post by staunton »

RandomLensman wrote: post #412
holmesworcester wrote: post #378

Earlier quoted context omitted.

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by m…

Who on the list is an expert on existential risk (and perhaps even beyond academia)?

Who in the world is an expert on existential risk? It's kind of hard to have empirically tested knowledge about that sort of thing.

Re: Statement on AI Risk

#438

Post by s1k3s »

That's it guys they said on a website that mitigating the risk of AI is important. I for one can sleep well at night for the world is saved.

Re: Statement on AI Risk

#439

Post by evrydayhustling »

holmesworcester wrote: post #378
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by m…

> Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted.

The concern is that the most informed names, and those spearheading the publicity around these letters, are the most conflicted.

Also, you can't scan bio lines for the affiliations that impact this kind of statement. I'm not disputing that there are honest reasons for concern, but besides job titles there are sponsorships, friendships, self publicity, and a hundred other reasons for smart, "politically sophisticated" people to look the other way on the fact that this statement will be used as a lobbying tool.

Almost everyone, certainly including myself, can agree that there should be active dialog about AI dangers. The dialog is happening! But by failing to make specifics or suggestions (in order to widen the tentpole and avoid the embarrassment of the last letter), they have produced an artifact of generalized fear, which can and will be used by opportunists of all stripes.

Signatories should consider that they are empowering SOMEBODY, but most will have little say in who that is.

Re: Statement on AI Risk

#440

Post by verdverm »

veerd wrote: post #402
holmesworcester wrote: post #378

Earlier quoted context omitted.

As others have pointed out, there are many on this list (Bruce Schneier, for example) who do not stand to benefit from AI marketing or regulatory capture. Anyone upvoting this comment should take a long look at the names on this letter and realize that many are not conflicted. Many signers of this letter are more politically sophisticated than the average HN commenter, also. So sure, maybe they're getting rolled by m…

Agreed. It's difficult for me to see how the regulatory capture arguments apply to Geoffrey Hinton and Yoshua Bengio (!!). Both of them are criticizing their own life's work and the source of their prestige. That has to be emotionally painful. They aren't doing it for fun. I totally understand not agreeing with AI x-risk concerns on an object level, but I find the casual dismissal bizarre.

Hinton has invested in multiple AI companies: https://www.crunchbase.com/person/geoffrey-hinton
Post reply on HN