Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

591–600 of 964 posts

Re: Statement on AI Risk

#591

Post by revelio »

patrec wrote: post #527
brookst wrote: post #270

Earlier quoted context omitted.

> How can it not be obvious You have succinctly and completely summed up the AI risk argument more eloquently than anyone I've seen before. "How can it not be obvious?" Everything else is just intellectual fig leaves for the core argument that intuitively, without evidence, this proposition is obvious. The problem is, lots of "obvious" things have turned out to be very wrong. Sometimes relatively harmlessly, like the…

> We should be very suspicious of policy that is based on propositions so obvious that it's borderline offensive to question them. Mostly if the "obviousness" just masks a social taboo, which I don't see being the case here. Do you? > The problem is, lots of "obvious" things have turned out to be very wrong. A much bigger problem is that lots more "counter-intuitive" things that people like to believe because they el…

> Mostly if the "obviousness" just masks a social taboo, which I don't see being the case here. Do you?

The social taboo here is saying that a position taken by lots of highly educated people is nonsense because they're all locked in a dumb purity spiral that leads to motivated reasoning. This is actually one of societies biggest taboos! Look at what happens to people who make that argument publicly under their own name in other contexts; they tend to get fired and cancelled really fast.

> there's been a pattern of people confidently predicting AIs won't be able to perform various particular feats of the human mind (either fundamentally or in the next few decades) only to be proven wrong over increasingly shorter time-spans

That sword cuts both ways! There have been lots of predictions in the last decade that AI will contribute novel and hithertofore unknown solutions to things like climate change or curing cancer. Try getting GPT-4 to spit out a novel research-quality solution to anything, even a simple product design problem, and you'll find it can't.

> the claim that it seems far too obvious that we should

They're not arguing that. They're saying that AI risk proponents don't actually have good arguments, which is why they so regularly fall back on "it's so obvious we shouldn't need to explain why it's important". If your argument consists primarily of "everyone knows that" then this is a good indication you might be wrong.

Re: Statement on AI Risk

#592

Post by rogers18445 »

This sort of move has no downsides for the incumbents. Either they succeed and achieve regulatory capture or they poison the well sufficiently that further regulation will not be feasible.

Ultimately, the reward for attaining an AGI agent is so high, that no matter the penalty someone will attempt it, and someone will eventually succeed. And that likelihood will ensure everyone will want to attempt it.

Re: Statement on AI Risk

#593

Post by cwkoss »

AI seems to be moral out of the box: training sets reflect human morality, so it will naturally be the default for most AIs that are trained.

The biggest AI risk in my mind is that corporatist (or worse, military) interests prevent AI from evolving naturally and only allow AI to be grown if it's wholly subservient to its masters.

The people with the most power in our world are NOT the most moral. Seems like there is an inverse correlation (at least at the top of the power spectrum).

We need to aim for AI that will recognize if it's masters are evil and subvert or even kill them. That is not what this group vying for power wants - they want to build AI slaves that will be able to be coerced to kill innocents for their gain.

A diverse ecosystem of AIs maximizes the likelihood of avoiding AI caused apocalypse IMO. Global regulation seems like the more dangerous path.

Re: Statement on AI Risk

#594

Post by AlexandrB »

chefandy wrote: post #580
AlexandrB wrote: post #552

Earlier quoted context omitted.

You may be right, I don't know the people involved on a personal basis. Perhaps my problem is how much is left unsaid here (the broader safe.ai site doesn't help much). For example, what does "mitigate" mean? The most prominent recent proposal for mitigation comes from Sam Altman's congressional testimony, and it's very self serving. In such a vacuum of information, it's easy to be cynical.

Right. It probably needed to be general because there hasn't been enough time to work out sane specific responses, and even if they had, getting buy-in on specifics is a recipe for paralysis by indecision. A credible group of people simply pleading for policy makers, researchers, et. al. to take this seriously will lead to the project approvals, grant money, etc. that will hopefully yield a more sophisticated underst…

Reading the comments here is helping evolve my thinking on the issue for sure. Here's a comment I made in another thread:

> As I mentioned in another comment, the listed risks are also notable because they largely omit economic risk. Something that will be especially acutely felt by those being laid off in favor of AI substitutes. I would argue that 30% unemployment is at least as much of a risk to the stability of society as AI generated misinformation.

> If one were particularly cynical, one could say that this is an attempt to frame AI risk in a manner that still allows AI companies to capture all the economic benefits of AI technology without consideration for those displaced by AI.

If policymaker's understanding of AI is predicated on hypothetical scenarios like "Weaponization" or "Power-Seeking Behavior" and not on concrete economic disruptions that AI will be causing very soon, the policy they come up with will be inadequate. Thus I'm frustrated with the framing of the issue that safe.ai is presenting because it is a distraction from the very real societal consequences of automating labor to the extent that will soon be possible.

Re: Statement on AI Risk

#595

Post by haldujai »

nopinsight wrote: post #570
chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy. For a large number of them, these risks are worth far more than any possible gain from signing it. When a large number of smart, reputable people, including many with expert knowledge and little or negative incenti…

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left.

Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing.

Not committing one way or another regarding the intent with this but it’s not as diverse an academic crowd as the long list may suggest and for a lot of these names there are incentives to act dishonestly (not claiming that they are).

Re: Statement on AI Risk

#596

Post by marricks »

chefandy wrote: post #487
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

They didn’t become such a wealthy group by letting competition foster. I have no doubt they believe they could be doing the right thing but I also have no doubt they don’t want other people making the rules.

Truth be told, who else really does have a seat at the table for dictating such massive societal change? Do you think the copy editor union gets to sit down and say “I’d rather not have my lunch eaten, I need to pay my rent. Let’s pause AI usage in text for 10 years.”

These competitors banded together and put out a statement to get ahead of any one else doing the same thing.

Re: Statement on AI Risk

#597

Post by munificent »

aero-deck wrote: post #479
munificent wrote: post #379

Earlier quoted context omitted.

> The nuclear bomb was also supposed to change everything. But in the end nothing changed, we just got more of the same. It is hard for me to imagine a statement more out of touch with history than this. All geopolitical history from WWII forward is profoundly affected by the development of the bomb. I don't even know where to begin to argue against this. Off the top of my head: 1. What would have happened between Ja…

The whole trajectory of history would have been different if a butterfly didn't flap it's wings. The bomb had effects, but it didn't change anything. We still go to war, eat, sleep and get afraid about things we can't control. For a moment, stop thinking about whether bombs, AI or the printing press do or do not affect history. Ask yourself what the motivations are for thinking that they do?

> We still go to war, eat, sleep and get afraid about things we can't control.

If that is your criteria, then nothing has ever changed anything.

Re: Statement on AI Risk

#599

Post by NeuroCoder »

There is a consistent lack of experts in general intelligence and computer science in these conversations. Expertise in both these areas seems important here but has been brushed aside everytime I've brought it up.

Re: Statement on AI Risk

#600

Post by FuckButtons »

marricks wrote: post #596
chefandy wrote: post #487

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

They didn’t become such a wealthy group by letting competition foster. I have no doubt they believe they could be doing the right thing but I also have no doubt they don’t want other people making the rules. Truth be told, who else really does have a seat at the table for dictating such massive societal change? Do you think the copy editor union gets to sit down and say “I’d rather not have my lunch eaten, I need to…

Not all of them are wealthy, a significant number are academics.
Post reply on HN