Live data from Hacker News

Statement on AI Risk

safe.ai

531–540 of 964 posts

Re: Statement on AI Risk

#531
post #32

The issue I take with these kind of "AI safety" organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. This is not a real risk today. Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. But of course, "AI will humans extinct" is much sexier and collects clicks. Therefore, the…

>This is not a real risk today.

Many experts believe it is a real risk within the next decade (a “hard takeoff” scenario) That is a short enough timeframe that it’s worth caring about.

Re: Statement on AI Risk

#532

Earlier quoted context omitted.

You said that "AI is an application of math. It is not clear how that can be used directly to harm a body." I was trying to illustrate the case that if humans can develop harmful things, like nuclear weapons, then an AI that is as smart as a human can presumably develop similarly harmful things. If the point you are trying to make is that an AI which secretly creates and deploys nuclear, biological, or chemical weapo…

> I was trying to illustrate the case that if humans can develop harmful things, like nuclear weapons, then an AI that is as smart as a human can presumably develop similarly harmful things. For the sake of discussion, it was established even before I arrived that those developed things are the danger, not that which creates/uses the things which are dangerous. What is to be gained by ignoring all of that context? >…

I'm specifically worried that an AGI will conceal some instrumental goal of wiping out humans, while posing as helpful. It will helpfully earn a lot of money for a lot of people, by performing services and directing investments, and with its track record, will gain the ability to direct investments for itself. It then plows a billion dollars into constructing a profitable chemicals factory somewhere where rules are lax, and nobody looks too closely into what else that factory produces, since the AI engineers have signed off on it. And then once it's amassed a critical stockpile of specific dangerous chemicals, it releases them into the atmosphere and wipes out humanity / agriculture / etc.

Perhaps you would point out that in the above scenario the chemicals (or substitute viruses, or whatever) are the part that causes harm, and the AGI is just an implementation detail. I disagree, because if humanity ends up playing a grand game of chess against an AGI, the specific way in which it checkmates you is not the important thing. The important thing is that it's a game we'll inevitably lose. Worrying about the danger of rooks and bishops is to lose focus on the real reason we lose the game: facing an opponent of overpowering skill, when our defeat is in its interests.

Re: Statement on AI Risk

#533

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

They don't want 25k they want jobs in the next presidential administration

Re: Statement on AI Risk

#534
post #147

Earlier quoted context omitted.

A good way to categorize risk is look at both likelihood and severity of consequences. The most visible issues today (racism, deep fakes, over reliance) are almost certain to occur, but also for the most part have relatively minor consequences (mostly making things that are already happening worse). "Advanced AI will make humans extinct" is much less likely but has catastrophic consequences. Focusing on the catastrop…

This longtermist and Effective Altruism way of thinking is very dangerous. Because using this chain of argumentation, it's "trivial" to say what you're just saying: "So what if there's racism today, it doesn't matter if everybody dies tomorrow. We can't just say that we weigh humanity's extinction with a big number, and then multiply it by all humans that might be born in the future, and use that to say today's REAL…

Today's real issues are not that important compared to human extinction.

Re: Statement on AI Risk

#535

Earlier quoted context omitted.

To use Eliezer's analogy, this is like arguing about which move Stockfish would play to beat you in chess. If we're arguing about whether you can beat Stockfish, I will not be able to tell you the exact moves it will play but I am entirely justified in predicting that you will lose. Obviously we can imagine concrete ways a superintelligence might kill us all (engineer a virus, hack nuclear weapons, misinformation cam…

I understand but agentic/learning general intelligence has not been shown to exist, except ourselves. I’d say this is like worrying about deadly quantum laser weapons that will consume the planet when we are still in the AK47 phase. Edit: it could still be true though. I guess I like some more handholding and pre-chewing before giving governments and large corporations more ropes.

> I’d say this is like worrying about deadly quantum laser weapons that will consume the planet when we are still in the AK47 phase.

Directed energy weapons will almost certainly exist eventually, to some extent they already do.

The reason why it makes more sense to worry about AGI than laser weapons is that when you try to make a laser weapon but fail slightly not much happens: either you miss the target or it doesn't fire.

When you try to make an aligned superintelligence and slightly fail you potentially end up with an unaligned superintelligence, hence the panic.

Re: Statement on AI Risk

#536
post #290
post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

It is possible to believe that AI poses threat, while also thinking that the AI safety organizations currently sprouting up are essentially grifts that will do absolutely nothing to combat the genuine threat. Especially when their primary goal seems to be the creation of well-funded sinecures for a group of like-minded, ideologically aligned individuals who want to limit AI control to a small group of wealthy technol…

I agree.

But as you can see yourself, there are countless people even here, in a technical forum, who claim that AI poses no plausible threat whatsoever. I fail to see how one can reasonably believe that.

Re: Statement on AI Risk

#537
The only risk with AI is that it will be abused by the wealthy and the powerful, especially autocrats, who will no longer need labor, only natural resources. Hence the solution is to promote worldwide democracy and public ownership of natural resources instead of diverting attention to technology.

In this particular case, one cannot miss the irony of the wealthy and the powerful offering us protection if only we entrust them with full control of AI.

Re: Statement on AI Risk

#538
post #510

Earlier quoted context omitted.

This is not how entropy works. The problem with talking about logical physical systems, is you have to understand the gradient against entropy. There are a trillion more ways to kill you then there are to keep you alive. There is only the tiniest sliver of states in which remain human and don't turn to chemical soup or physics. Any AI capable of it's power bill would be able to tell you that today, and that answer is…

Sure, but clever mumbo jumbo won't distract from the principle point. If AI can create a virus that kills all humans, another AI can create a virus that kills that virus. The virus has trillions more ways to be killed than to keep it alive, right?

No, the virus is far harder to kill than a human. You have to crate a virus killer that also does not also kill the human host. That is astronomically harder than making a virus that kills.

Re: Statement on AI Risk

#539

Earlier quoted context omitted.

Consider two different scenarios: 1) Throughout history many people have predicted the world would soon end, and the world did not in fact end. 2) Throughout history no one predicted the world would soon end, and the world did not in fact end. The fact that the real world is aligned with scenario 1 is more an indication that there exists a pervasive human cognitive bias to think that the world is going to end, which…

That argument is still invalid because in scenario 2 we would not be having this discussion. No conclusions can be drawn from such past discourse about the likelihood of definite and complete extinction. Not that, I hope, anyone expected a strong argument to be had there. It seems reasonably certain to me that humanity will go extinct one way or another eventually. That is also not a good argument in this situation.

It depends on what you mean by "this discussion", but I don't think that follows.

If for example, we were in scenario 2 and it was still the case that a large number of people thought AI doomsday was a serious risk, then that would be a much stronger argument for taking the idea of AI doomsday seriously. If on the other hand we are in scenario 1, where there is a long history of people falling prey to apocalypticism, then that means any new doomsday claims are also more likely to be a result of apocalypticism.

I agree that is is likely that humans will go extinct eventually, but I am talking specifically about AI doomsday in this discussion.

Re: Statement on AI Risk

#540
post #523

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

This particular statement really doesn't seem like a marketing ploy. It is difficult to disagree with the potential political and societal impacts of large language models as outlined here: https://www.safe.ai/ai-risk These are, for the most part, obvious applications of a technology that exists right now but is not widely available yet . The problem with every discussion around this issue is that there are other sta…

> The problem with every discussion around this issue is that there are other statements on

Sure, but we're not talking about those other ones. Dismissing good faith initiatives as marketing ploys because there are bad faith initiatives is functionally no different than just shrugging and walking away.

Of course OpenAI et. al. will try to influence the good faith discussions: that's a great reason to champion the ones with a bunch of good faith actors who stand a chance of holding the industry and policy makers to task. Waiting around for some group of experts that has enough clout to do something, but by policy excludes the industry itself and starry-eyed shithead "journalists" trying to ride the wave of the next big thing will yield nothing. This is a great example of perfect being the enemy of good.

Post reply on HN