Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

721–730 of 964 posts

Re: Statement on AI Risk

#721

Post by patch_cable »

patrec wrote: post #682
patch_cable wrote: post #375

Earlier quoted context omitted.

> How can it not be obvious to you It isn't obvious to me. And I've yet to read something that spills out the obvious reasoning. I feel like everything I've read just spells out some contrived scenario, and then when folks push back explaining all the reasons that particular scenario wouldn't come to pass, the counter argument is just "but that's just one example!" without offering anything more convincing. Do you ha…

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…

I think between point 3 and 4 there is a leap to talking about “danger”. Perhaps the disagreement is about what one calls “danger”. I had perhaps mistakenly assumed we were talking about an extinction risk. I’ll grant you concerns about scaling up things like surveillance but there is a leap to being an existential risk that I’m still not following.

Re: Statement on AI Risk

#722

Post by haldujai »

chefandy wrote: post #580
AlexandrB wrote: post #552

Earlier quoted context omitted.

You may be right, I don't know the people involved on a personal basis. Perhaps my problem is how much is left unsaid here (the broader safe.ai site doesn't help much). For example, what does "mitigate" mean? The most prominent recent proposal for mitigation comes from Sam Altman's congressional testimony, and it's very self serving. In such a vacuum of information, it's easy to be cynical.

Right. It probably needed to be general because there hasn't been enough time to work out sane specific responses, and even if they had, getting buy-in on specifics is a recipe for paralysis by indecision. A credible group of people simply pleading for policy makers, researchers, et. al. to take this seriously will lead to the project approvals, grant money, etc. that will hopefully yield a more sophisticated underst…

> will lead to the project approvals, grant money, etc.

In other words, a potential conflict of interest for someone seeking tenure?

Re: Statement on AI Risk

#723

Post by chefandy »

haldujai wrote: post #595
nopinsight wrote: post #570

Earlier quoted context omitted.

It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy. For a large number of them, these risks are worth far more than any possible gain from signing it. When a large number of smart, reputable people, including many with expert knowledge and little or negative incenti…

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left. Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing. Not committing one way or another regarding the intent with this but it’s not as diverse an aca…

I just took that list and separated everyone that had any commercial tie listed, regardless of the company. 35 did and 63 did not.

> "Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left."

You're putting a lot of effort into painting this list in a bad light without any specific criticism or evidence of malfeasance. Frankly, it sounds like FUD to me.

Re: Statement on AI Risk

#724

Post by waterhouse »

revelio wrote: post #627
waterhouse wrote: post #143

Earlier quoted context omitted.

The most obvious paths to severe catastrophe begin with "AI gets to the level of a reasonably competent security engineer in general, and gets good enough to find a security exploit in OpenSSL or some similarly widely used library". Then the AI, or someone using it, takes over hundreds of millions of computers attached to the internet. Then it can run millions of instances of itself to brute-force look for exploits i…

That's not an obvious path at all: - Such exploits happen already and don't lead to extinction or really much more than annoyance for IT staff. - Most of the computers attached to the internet can't run even basic LLMs, let alone hypothetical super-intelligent AIs. - Very few cars (none?) let remote hackers kill people by controlling their acceleration. The available interfaces don't allow for that. Most people aren'…

Addressing your points in order:

- Human hackers who run a botnet of infected computers are not able to run many instances of themselves on those computers, so they're not able to parlay one exploit into many exploits.

- You might notice I said it would take over hundreds of millions of computers, but only run millions of instances of itself. If 1% of internet-attached computers have a decent GPU, that seems feasible.

- If it has found exploits in the software, it seems irrelevant what the interfaces "allow", unless there's some hardware interlock that can't be overridden—but they can drive on the highway, so surely they are able to accelerate at least to 65 mph; seems unlikely that there's a cap. If you mean that it's difficult to work with the software to intelligently make it drive in ways it's designed not to—well, that's why I specified that it would use the software the way it's designed to be used to get onto a main road, and then override it and blindly max out the acceleration; the first part requires minimal understanding of the system, and the second part requires finding a low-level API and using it in an extremely simple way. I suspect a good human programmer with access to the codebase could figure out how to do this within a week; and machines think faster than we do.

There was an incident back in 2015 (!) where, according to the description, "Two hackers have developed a tool that can hijack a Jeep over the internet." In the video they were able to mess with the car's controls and turn off the engine, making the driver unable to accelerate anymore on the highway. They also mention they could mess with steering and disable the brakes. It doesn't specify whether they could have made the car accelerate. https://www.youtube.com/watch?v=MK0SrxBC1xs

Re: Statement on AI Risk

#726

Post by pixl97 »

revelio wrote: post #688
pixl97 wrote: post #659

Earlier quoted context omitted.

Do you remember the days the computing power that's in your pocket took up an entire floor of a building? Because I do. If that is somehow the only barrier between humanity and annihilation, then things don't bode well for us.

Sure. Despite all that progress, computers still have an off switch and power efficiency still matters. It actually matters more now than in the past.

What you're arguing is "what is the minimum viable power envelope for a super intelligence". Currently that answer is "quite a lot". But for the sake of cutting out a lot of argument lets say you have a cellphone sized device that runs on battery power for 24 hours that can support a general intelligence. Lets say, again for arguments sake, there are millions of devices like this distributed in the population.

Do you mind telling me how exactly you turn that off?

Now we're lucky in the sense we don't have that today. AI still requires data centers inputting massive amounts of power and huge cooling bills. Maybe we'll forever require AI to take stupid large amounts of power. But at the same time, a cray super computer required stupid amounts of power and space, and your cellphone has leaps and bounds more computing power than that.

Re: Statement on AI Risk

#727

Post by jiggawatts »

b3nji wrote: post #77

Nonsense, the industry giants are just trying to scare the law makers to license the technology. Effectively, cutting out everyone else. Remember the Google note circulating saying "they have no moat", this is their moat. They have to protect their investment, we don't want people running this willy nilly for next to no cost on their own devices, God forbid!

Imagine if the weights for GPT 4 leaked. It just has to happen one time and then once the torrent magnet link is circulated widely it’s all over… for OpenAI.

This is what they’re terrified of. They’ve invested near a billion dollars and need billions in revenue to enrich their shareholders.

But if the data leaks? They can’t stop random companies or moneyed individuals running the models on their own kit.

My prediction is that there will be copyright enforcement mandated by law in all GPUs. If you upload weights from the big AI companies then the driver will block it and phone home. Or report you to the authorities for violations of corporate profits… err… “AI Safety”.

I guarantee something like this will happen within months because the clock is ticking.

It takes just one employee to deliberately or accidentally leak the weights…

Re: Statement on AI Risk

#728

Post by cocacola1 »

HDThoreaun wrote: post #534
shafyy wrote: post #147

Earlier quoted context omitted.

This longtermist and Effective Altruism way of thinking is very dangerous. Because using this chain of argumentation, it's "trivial" to say what you're just saying: "So what if there's racism today, it doesn't matter if everybody dies tomorrow. We can't just say that we weigh humanity's extinction with a big number, and then multiply it by all humans that might be born in the future, and use that to say today's REAL…

Today's real issues are not that important compared to human extinction.

I’m not sure if extinction is a problem either. No one’s left to care about the issues, then.

Re: Statement on AI Risk

#729

Post by chefandy »

haldujai wrote: post #704
chefandy wrote: post #698

Earlier quoted context omitted.

I'm not really interested in doing a research project on the signatories to investigate your claim, and talking about things like this without specifics seems dubiously useful, so I don't really think there's anything more to discuss.

Huh, you don’t have to do any research. Go to: https://www.safe.ai/statement-on-ai-risk#signatories and uncheck notable figures. Several of the names at the top list a corporate affiliation. If you want me to pick specific ones with obvious conflicts (chosen at a glance): Geoffrey Hinton, Ilya Sutskever, Ian Goodfellow, Shane Legg, Samuel Bowman and Roger Grosse are representative examples based on self-disclosed aff…

Oh so you're saying the ones there with conflicts listed. That's only like 1/3 of the list.

Re: Statement on AI Risk

#730

Post by patch_cable »

patrec wrote: post #682
patch_cable wrote: post #375

Earlier quoted context omitted.

> How can it not be obvious to you It isn't obvious to me. And I've yet to read something that spills out the obvious reasoning. I feel like everything I've read just spells out some contrived scenario, and then when folks push back explaining all the reasons that particular scenario wouldn't come to pass, the counter argument is just "but that's just one example!" without offering anything more convincing. Do you ha…

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…

Reading the lesswrong link, the parts I get hung up on are that it appears in these doomsday scenarios humans lose all agency. Like, no one is wondering why this computer is placing a bunch of orders to DNA factories?

Maybe I’m overly optimistic about the resilience of humans but these scenarios still don’t sound plausible to me in the real world.

Post reply on HN