Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

641–650 of 964 posts

Re: Statement on AI Risk

#641

Post by revelio »

slg wrote: post #635
revelio wrote: post #584

Earlier quoted context omitted.

> The risks are definitely real. Just look at the number of smart individuals speaking out about this. In our society smart people are strongly incentivized to invent bizarre risks in order to reap fame and glory. There is no social penalty if those risks never materialize, turn out to be exaggerated or based on fundamental misunderstanding. They just shrug and say, well, better safe than sorry, and everyone lets the…

>In our society smart people are strongly incentivized to invent bizarre risks in order to reap fame and glory. There is no social penalty if those risks never materialize, turn out to be exaggerated or based on fundamental misunderstanding. Are people here not old enough to remember how much Ralph Nader and Al Gore were mocked for their warnings despite generally being right?

Ralph Nader: "Everything will be solar in 30 years" (1978)

Al Gore: "Within a decade, there will be no more snows on Kilimanjaro due to warming temperatures" (An Inconvenient Truth, 2006).

Everything is not solar. Snow is still there. Gore literally made a movie on the back of these false claims. Not only has there been no social penalty for him but you are even citing him as an example of someone who was right.

Here it is again: our society systematically rewards false claims of global doom. It's a winning move, time and again. Even when your claims are falsifiable and proven false, people will ignore it.

Re: Statement on AI Risk

#642

Post by nopinsight »

haldujai wrote: post #632
nopinsight wrote: post #609

Earlier quoted context omitted.

Even if it’s just Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, we’d probably agree the risks are not negligible. There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. Many of whom do not work for those companies. We’re talking about nuclear war level risks here. Even a 1% chance should definitely be a…

> There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. I know the Stanford researchers the most and the “biggest names” in LLMs from HAI and CRFM are absent. It would be useful to have their perspective as well. I’d throw MetaAI in the mix as well. Merely pointing out that healthy skepticism here is not ent…

LLM is already a misnomer. Latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likely capable of more independence.

Even dumb viruses have caused catastrophic harm. Why? It’s capable of rapid self replication in a massive number of existing vessels. You add in some intelligence, vast store of knowledge, huge bandwidth, and some aid by malicious human actors, what could such a group of future autonomous agents do?

More on the risks of “doom”: https://www.lesswrong.com/posts/xWMqsvHapP3nwdSW8/my-views-o...

Re: Statement on AI Risk

#643

Post by chefandy »

AlexandrB wrote: post #594
chefandy wrote: post #580

Earlier quoted context omitted.

Right. It probably needed to be general because there hasn't been enough time to work out sane specific responses, and even if they had, getting buy-in on specifics is a recipe for paralysis by indecision. A credible group of people simply pleading for policy makers, researchers, et. al. to take this seriously will lead to the project approvals, grant money, etc. that will hopefully yield a more sophisticated underst…

Reading the comments here is helping evolve my thinking on the issue for sure. Here's a comment I made in another thread: > As I mentioned in another comment, the listed risks are also notable because they largely omit economic risk. Something that will be especially acutely felt by those being laid off in favor of AI substitutes. I would argue that 30% unemployment is at least as much of a risk to the stability of s…

My own bit of cynicism is that regulating the negative impacts of technology on workforce segments in the US is a non-starter if you approach it from the technology-end of the issue rather than the social safety net end. Most of these automation waves that plunged entire employment categories and large metropolitan areas into oblivion were a net gain for the economy even if it was concentrated at the top. I think the government will temporarily socialize the costs of the corporate profit with stimulus payments, extended unemployment benefits, and any other thing they can do to hold people over until there's a comparatively small risk of triggering real social change. Then they just blame it on the individuals.

Re: Statement on AI Risk

#644

Post by nico »

chefandy wrote: post #487
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

Maybe these people have good intentions and are just being naive

They might not be getting paid, but that doesn’t mean they are not being influenced

AI at this point is pretty much completely open, all the papers, math and science behind it are public

Soon, people will have advanced AI running locally on their phones and watches

So unless they scrub the Internet, start censoring this stuff, and pretty much ban computers, there is absolutely no way to stop AI nor any potentially bad actors from using it

The biggest issues that we should be addressing regarding AI are the potential jobs losses and increased inequality at local and global scale

But of course, the people who usually make these decisions are the ones that benefit the most from inequality, so

Re: Statement on AI Risk

#645

Post by revelio »

cj wrote: post #607
revelio wrote: post #573

Earlier quoted context omitted.

> It is difficult to disagree with the potential political and societal impacts of large language models as outlined here Is it? Unless you mean something mundane like "there will be impact", the list of risks they're proposing are subjective and debatable at best, irritatingly naive at worst. Their list of risks are: 1. Weaponization. Did we forget about Ukraine already? Answer: Weapons are needed. Why is this AI ri…

I appreciate your perspective, but the thing that is missing is the speed at which AI has evolved, seemingly overnight. With crypto, self-driving cars, computers, the internet or just about any other technology, development and distribution happened over decades. With AI, there’s a risk that the pace of change and adoption could be too fast to be able to respond or adapt at a societal level. The rebuttals to each of…

It may seem overnight if you weren't following it, but I've followed AI progress for a long time now. I was reading the Facebook bAbI test paper in 2015:

https://research.facebook.com/downloads/babi/

There's been a lot of progress since then, but it's also nearly 10 years later. Progress isn't actually instant or overnight. It's just that OpenAI spent a ton of money to scale it up then stuck an accessible chat interface on top of tech that was previously being mostly ignored.

Re: Statement on AI Risk

#646

Post by ericb »

lucisferre wrote: post #374
ericb wrote: post #273

Earlier quoted context omitted.

Agreed that it should be spelled out, but... If a superintelligence can be set on any specific task, it could be any task. - Make covid-ebola - Cause world war 3 You may have noticed that chatgpt is sort of goal-less until a human gives it a goal. Assuming nothing other than it can become superintelligent (no one seems to be arguing against that--I argue that it already is ) which is really an upgrade of capability,…

Are you really arguing ChatGPT is already super-intelligent? What is your basis for this conclusion? And many people argue against the idea that GPT is already super intelligent or even can become so at this stage of development and understanding. In fact as far as I can tell it is the consensus right now of experts and it's creators. https://www.calcalistech.com/ctechnews/article/nt9qoqmzz

If super means "surpassing normal human intelligence" then, YES. Take a look at the table in this article. If a human did that, was fluent in every language and coded in every language, we'd say they were superhuman, no?

https://cdn.openai.com/papers/gpt-4.pdf

Re: Statement on AI Risk

#647

Post by hackinthebochs »

patch_cable wrote: post #375
patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

> How can it not be obvious to you It isn't obvious to me. And I've yet to read something that spills out the obvious reasoning. I feel like everything I've read just spells out some contrived scenario, and then when folks push back explaining all the reasons that particular scenario wouldn't come to pass, the counter argument is just "but that's just one example!" without offering anything more convincing. Do you ha…

The history of humanity is replete with examples of the slightly more technologically advanced group decimating their competition. The default position should be that uneven advantage is extremely dangerous to those disadvantaged. This idea that an intelligence significantly greater than our own is benign just doesn't pass the smell test.

From the tech perspective: higher order objectives are insidious. While we may assume a narrow misalignment in received vs intended objective of a higher order nature, this misalignment can result in very divergent first-order behavior. Misalignment in behavior is by its nature destructive of value. The question is how much destruction of value can we expect? The machine may intentionally act in destructive ways as it goes about carrying out its slightly misaligned higher order objective-guided behavior. Of course we will have first-order rules that constrain its behavior. But again, slight misalignment in first-order rule descriptions are avenues for exploitation. If we cannot be sure we have zero exploitable rules, we must assume a superintelligence will find such loopholes and exploit them to maximum effect.

Human history since we started using technology has been a lesson on the outcome of an intelligent entity aimed at realizing an objective. Loopholes are just resources to be exploited. The destruction of the environment and other humans is just the inevitable outcome of slight misalignment of an intelligent optimizer.

If this argument is right, the only thing standing between us and destruction is the AGI having reached its objective before it eats the world. That is, there will always be some value lost in any significant execution of an AGI agent due to misalignment. Can we prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Until we do, x-risk should be the default assumption.

Re: Statement on AI Risk

#649

Post by chefandy »

haldujai wrote: post #612
chefandy wrote: post #542

Earlier quoted context omitted.

> They don't want 25k they want jobs in the next presidential administration Academics shilling for OpenAI would get them jobs in the next presidential administration?

Having their names on something so public is definitely an incentive for prestige and academic promotion. Shilling for OpenAI & co is also not a bad way to get funding support. I’m not accusing any non-affiliated academic listed of doing this but let’s not pretend there aren’t potentially perverse incentives influencing the decisions of academics, with respects to this specific letter and in general. To help dissuade…

Could you be more specific about the conflicts you've uncovered?

Re: Statement on AI Risk

#650

Post by haldujai »

nopinsight wrote: post #642
haldujai wrote: post #632

Earlier quoted context omitted.

> There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. I know the Stanford researchers the most and the “biggest names” in LLMs from HAI and CRFM are absent. It would be useful to have their perspective as well. I’d throw MetaAI in the mix as well. Merely pointing out that healthy skepticism here is not ent…

LLM is already a misnomer. Latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likely capable of more independence. Even dumb viruses have caused catastrophic harm. Why? It’s capable of rapid self replication in a massive number of existing vessels. You add in some intelligence, vast store of knowledge, huge bandwidth, and some aid by ma…

I mean a small group of malicious humans can already bioengineer a deadly virus with CRISPR and open source tech without AI.

This is hardly the first time in history a new technological advancement may be used for nefarious purposes.

It’s a discussion worth having as AI advances but if [insert evil actor] wants to cause harm there are many cheaper and easier ways to do this right now.

To come out and say we need government regulation today does stink at least a little bit of protectionism as practically speaking the “most evil actors” would not adhere to whatever is being proposed, but this would impact the competitive landscape and the corporations yelling the loudest right now have the most to gain, perhaps coincidence but worth questioning.

Post reply on HN