Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

541–550 of 964 posts

Re: Statement on AI Risk

#541

Post by hiAndrewQuinn »

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

We could always use a fine-insured bounty system to efficiently route resources that would have gone into increasing AI capabilities into other areas, but that's unfortunately too weird to be part of the Overton window right now. Regulatory capture might be the best we can realistically do.

Re: Statement on AI Risk

#542

Post by chefandy »

fds98324jhk wrote: post #533
chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

They don't want 25k they want jobs in the next presidential administration

> They don't want 25k they want jobs in the next presidential administration

Academics shilling for OpenAI would get them jobs in the next presidential administration?

Re: Statement on AI Risk

#543

Post by randomdata »

mitthrowaway2 wrote: post #532
randomdata wrote: post #494

Earlier quoted context omitted.

> I was trying to illustrate the case that if humans can develop harmful things, like nuclear weapons, then an AI that is as smart as a human can presumably develop similarly harmful things. For the sake of discussion, it was established even before I arrived that those developed things are the danger, not that which creates/uses the things which are dangerous. What is to be gained by ignoring all of that context? >…

I'm specifically worried that an AGI will conceal some instrumental goal of wiping out humans, while posing as helpful. It will helpfully earn a lot of money for a lot of people, by performing services and directing investments, and with its track record, will gain the ability to direct investments for itself. It then plows a billion dollars into constructing a profitable chemicals factory somewhere where rules are l…

> I disagree

Cool, I guess. While I have my opinions too, I'm not about to share them as that would be bad faith participation. Furthermore, it adds nothing to the discussion taking place. What is to be gained by going off on a random tangent that is of interest to nobody? Nothing, that's what.

To bring us back on topic to try and salvage things, it remains that it is established in this thread that the objects of destruction are the danger. AI cannot be the object of destruction, although it may be part of an implementation. Undoubtedly, nuclear missiles already utilize AI and when one talks about the dangers of nuclear missiles they are already including AI as part of that.

Re: Statement on AI Risk

#544

Post by jrockway »

kalkin wrote: post #211
b3nji wrote: post #77

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google note circulating saying "they have no moat", this is their moat. They have to protect their investment, we don't want people running this willy nilly for next to no cost on their own devices, God forbid!

This could be Google's motivation (although note that Google is not actually the market leader right now) but the risk could still be real. Most of the signatories are academics, for one thing, including two who won Turing awards for ML work and another who is the co-author of the standard AI textbook (at least when I was in school). You can be cynical about corporate motives and still worried. I personally am worrie…

> I don't really want my atoms to be ground up to add storage bits

My understanding is that the AI needs iron from our blood to make paperclips. So you don't have to worry about this one.

Re: Statement on AI Risk

#545

Post by kbash9 »

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

The risks are definitely real. Just look at the number of smart individuals speaking out about this.

The argument that anybody can build this in their basement is not accurate at the moment - you need a large cluster of GPUs to be able to come close to state of the art LLMs (e.g. GPT4).

Sam Altman's suggestion of having an IAEA [https://www.iaea.org/] like global regulatory authority seems like the best course of action. Anyone using a GPU cluster above a certain threshold (updated every few months) should be subjected to inspections and get a license to operate from the UN.

Re: Statement on AI Risk

#546

Post by cmilton »

adamsmith143 wrote: post #522
cmilton wrote: post #409

Earlier quoted context omitted.

>that those trends are concerning. Which trends would you be referring to? >it's fairly easy to see how a slightly more advanced LLM could be a destructive force if it was given an unaligned goal by a malicious actor. For example, a slightly more advanced LLM could hack into critical infrastructure killing or injuring many thousands of people. How are you building this progression? Is there any evidence to back up th…

If AI improves by 0.0001% per year on your favorite intelligence metric there will eventually be a point where it surpasses human performance and another point where it surpasses all humans combined on that metric. There is danger in that scenario.

Assuming that timeline, can we agree that we have some time (years?) to hash this out further before succumbing to the ideals of a select few?

Re: Statement on AI Risk

#547

Post by RandomLensman »

staunton wrote: post #437
RandomLensman wrote: post #412

Earlier quoted context omitted.

Who on the list is an expert on existential risk (and perhaps even beyond academia)?

Who in the world is an expert on existential risk? It's kind of hard to have empirically tested knowledge about that sort of thing.

Pretty sure there are people looking into nuclear deterrence, bioterrorism defense, planetary defense etc. (We didn't have a nuclear war or some bioweapon killing everyone, for example, despite warnings).

There are people studying how previous societies got into existential risk situations, too.

We also have a huge amount of socio-economic modelling going into climate change, for example.

So I'd say there should be quite a few around.

Re: Statement on AI Risk

#548

Post by patrec »

AnimalMuppet wrote: post #410
patrec wrote: post #221

Earlier quoted context omitted.

> It's not clear at all that we have an avenue to super intelligence AI already beats the average human on pretty much any task people have put time into, often by a very wide margin and we are still seeing exponential progress that even the experts can't really explain, but yes, it is possible this is a local maximum and the curve will become much flatter again. But the absence of any visible fundamental limit on fu…

Right, and if someone can join the two, that could be something genuinely formidable. But does anyone have a credible path to joining the different flavors to produce a unity that actually works?

Are you willing to make existential bets that no one does and no one will?

Personally, I wouldn't even bet substantial money against it.

Re: Statement on AI Risk

#549

Post by stainablesteel »

i'd like to see AI cause a big societal problem before its regulated

until it does, i call bs. plus, when it actually happens, a legitimate route for regulation will be discovered. as of right now, we have no idea what could go wrong.

Re: Statement on AI Risk

#550

Post by fmap »

chefandy wrote: post #540
fmap wrote: post #523

Earlier quoted context omitted.

This particular statement really doesn't seem like a marketing ploy. It is difficult to disagree with the potential political and societal impacts of large language models as outlined here: https://www.safe.ai/ai-risk These are, for the most part, obvious applications of a technology that exists right now but is not widely available yet . The problem with every discussion around this issue is that there are other sta…

> The problem with every discussion around this issue is that there are other statements on Sure, but we're not talking about those other ones. Dismissing good faith initiatives as marketing ploys because there are bad faith initiatives is functionally no different than just shrugging and walking away. Of course OpenAI et. al. will try to influence the good faith discussions: that's a great reason to champion the one…

I agree completely. I was just speculating on why there is so much discussion about marketing ploys in this comment section.
Post reply on HN