Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Joint Statement on AI Safety and Openness

open.mozilla.org

Post reply on HN
Print view View original

131–140 of 166 posts

Re: Joint Statement on AI Safety and Openness

#131

Post by jay_kyburz »

hackinthebochs wrote: post #91
jay_kyburz wrote: post #55

Earlier quoted context omitted.

Furthermore, there is little or no defense against a full scale nuclear attack, but a benevolent AI should be sufficient defense against a hostile AI. I think the true fear is that in an AI age, humans are not "useful" and the market and economy will look very different. With AI growing our food, clothing us, building us houses, and entertaining us, humans don't really have anything to do all day.

>but a benevolent AI should be sufficient defense against a hostile AI. What on earth could this possibly mean in practice? Two elephants fighting is not at all good for the mice below.

Better than one Elephant who hates mice.

Re: Joint Statement on AI Safety and Openness

#132

Post by hyeonwho22 »

tsurba wrote: post #72
hyeonwho22 wrote: post #29

Earlier quoted context omitted.

Why not? It has already been shown that AI can be (mis)used to identify good candidates for chemical weapons. [1] Next in the pipeline is obviously some religious nut (who would not otherwise have the capability) using it to design a virus which doesn't set off alarms at the gene synthesis / custom construct companies, and then learning to transfect it. More banally, state actors can already use open source models to…

Essentially you are advocating against information being more efficiently available. Come on. It’s true we are fucked if bioweapons become easy to make, but that is not a question of ”AI”.

The only thing keeping bioweapons from being easy to make is information becoming easily available.

> Essentially you are advocating against information being more efficiently available.

Yes. Some kinds of information should be kept obscure, even if it is theoretically possible for an intelligent individual with access to the world's scientific literature to rediscover them. The really obvious case for this is in regards to the proliferation of WMDs.

For nuclear weapons information is not the barrier to manufacture: we can regulate and track uranium, and enrichment is thought to require industrial scale processes. But the precursors for biological weapons are unregulated and widely available, so we need to gatekeep the relevant skills and knowledge.

I'm sure you will agree with me that if access information on how to make a WMD becomes even a few order of magnitudes as accessible as information on how to steal a Kia or how to steal a catalytic converter, then we will have lost.

My argument is that a truly intelligent AI without safeguards or ethics would make bioweapons accessible to the public, and we would be fucked.

Re: Joint Statement on AI Safety and Openness

#133

Post by shrimpx »

autoexec wrote: post #115
shrimpx wrote: post #108

Earlier quoted context omitted.

That didn't happen with cloud though. I know it's not entirely apples to apples, but you still need a computer the size of a large building to serve Netflix, etc.

Companies will always have an advantage with scale. It's not like you need a super computer though. You can have a single desktop media server in your home that does what netflix does without any problem. A single media server can even serve multiple homes. Netflix cheats. They send non-supercomputer boxes out to ISPs to install locally. If I could convince every ISP to install a bunch of my media servers people coul…

I guess we're speculating that training llama2 will drop by 1000x or something, so anyone can train their own llama2 from scratch for about $2k.

I don't think compute cost has dropped by 1000x since 20 years ago. Maybe by 10 to 50x. And if you add in the demand for higher quality, the cost has probably increased. Like encoding a video for streaming 20 years ago, at that standard, may have cost roughly the same as it does today, or more, when you factor in the increases in resolution and quality.

My prediction is that training the latest model will continue to cost millions to tens of millions for a long time, and these costs may even increase dramatically if significantly more powerful models require proportional increase in training compute.

Unless of course we have some insane algorithmic breakthrough where we find an AI algorithm that blows llama2 out of the water for a small fraction of the compute.

Re: Joint Statement on AI Safety and Openness

#134

Post by hyeonwho22 »

visarga wrote: post #62
hyeonwho22 wrote: post #29

Earlier quoted context omitted.

Why not? It has already been shown that AI can be (mis)used to identify good candidates for chemical weapons. [1] Next in the pipeline is obviously some religious nut (who would not otherwise have the capability) using it to design a virus which doesn't set off alarms at the gene synthesis / custom construct companies, and then learning to transfect it. More banally, state actors can already use open source models to…

> Next in the pipeline is obviously some religious nut (who would not otherwise have the capability) So you're saying that: 1. the religious nut would not find the same information on Google or in books 2. if someone is motivated enough to commit such an act, the ease of use of AI vs. web search would make a difference Has anyone checked how many biology students can prepare dangerous substances with just what they l…

1. The religious nut doesn't have the knowledge or the skill sets right now, but AI might enable them.

2. Accessibility of information makes a huge difference. Prior to 2020 people rarely stole Kias or catalytic converters. When knowledge of how to do this (and for catalytic converters, knowledge of their resale value) became available (i.e. trending on Tiktok), then thefts became frequent. The only barrier which disappeared from 2019 to 2021 was that the information became very easily accessible.

Your last two questions are not counterarguments, since AIs are already outperforming the median biology student, and obviously removing sites from the internet is not feasible. Easier to stop foundation model development than to censor the internet.

> What is to stop someone from training a model on such data anytime they want?

Present proposals are to limit GPU access and compute for training runs. Data centers are kind of like nuclear enrichment facilities in that they are hard to hide, require large numbers of dual-use components that are possible to regulate (centrifuges vs. GPUs), and they have large power requirements which make them show up on aerial imaging.

Re: Joint Statement on AI Safety and Openness

#135

Post by pdonis »

skybrian wrote: post #130
pdonis wrote: post #41

Earlier quoted context omitted.

> Big companies are easier to regulate. But the problem isn't regulating the big companies, or the smaller companies, or underground entities. The problem is state-level adversaries like China who might misuse a technology, whether it's AI or anything else. Such adversaries can't be regulated by laws or executive orders or UN declarations; they have proven that many times in the past. The only way to control them is…

State-level actors are one problem. People defrauding the elderly are a different problem. Ransomware is a third problem. They are all problems. The idea that there's only one important problem is a fallacy.

> The idea that there's only one important problem is a fallacy.

I have made no such claim.

The people advocating for regulating AI are claiming it will solve all the relevant problems--i.e., that it will prevent AI from doing great harm. So pointing out a problem that the regulations will not solve is refuting the claims the advocates of regulation are making. That was my point.

Re: Joint Statement on AI Safety and Openness

#136

Post by pdonis »

smokel wrote: post #71
pdonis wrote: post #41

Earlier quoted context omitted.

> Big companies are easier to regulate. But the problem isn't regulating the big companies, or the smaller companies, or underground entities. The problem is state-level adversaries like China who might misuse a technology, whether it's AI or anything else. Such adversaries can't be regulated by laws or executive orders or UN declarations; they have proven that many times in the past. The only way to control them is…

One could also try to be friendly towards strangers, and not have them be their adversaries in the first place.

> One could also try to be friendly towards strangers, and not have them be their adversaries in the first place.

We have tried this with China, going back to Nixon opening up trade relations in the early 1970s. It hasn't helped.

Re: Joint Statement on AI Safety and Openness

#137

Post by api »

hyeonwho22 wrote: post #134
visarga wrote: post #62

Earlier quoted context omitted.

> Next in the pipeline is obviously some religious nut (who would not otherwise have the capability) So you're saying that: 1. the religious nut would not find the same information on Google or in books 2. if someone is motivated enough to commit such an act, the ease of use of AI vs. web search would make a difference Has anyone checked how many biology students can prepare dangerous substances with just what they l…

1. The religious nut doesn't have the knowledge or the skill sets right now, but AI might enable them. 2. Accessibility of information makes a huge difference. Prior to 2020 people rarely stole Kias or catalytic converters. When knowledge of how to do this (and for catalytic converters, knowledge of their resale value) became available (i.e. trending on Tiktok), then thefts became frequent. The only barrier which dis…

What happens if someone develops a highly effective distributed training algorithm permitting a bunch of people with gaming PCs and fast broadband to train foundation models in a manner akin to Folding@Home?

If that happened open efforts could marshal tens or hundreds of thousands of GPUs.

Right now the barrier is that training requires too much synchronization bandwidth between compute nodes, but I’m not aware of any hard mathematical reason there couldn’t be an algorithm that does not have to sync so much. Even if it were less efficient this could be overcome by the sheer number of nodes you could marshal.

Re: Joint Statement on AI Safety and Openness

#138

Post by pdonis »

ta1243 wrote: post #44
pdonis wrote: post #38

Earlier quoted context omitted.

> without allowing superpowers to escape into the wrong hands The wrong hands will have the same access to whatever "superpowers" AI gives regardless of what regulations are or are not put in place. Regulations can't and won't stop potential bad actors with state-level resources, like China, from using any technology they decide they want to use. So trying to regulate on that basis is a fool's errand. The real questi…

China, Iran, North Korea, Iran, US, Israel, Europe aren't bad actors. The bad actors don't have state level resources.

> China, Iran, North Korea, Iran, US, Israel, Europe aren't bad actors.

Seriously? You don't think China, Iran, and North Korea are bad actors? What planet are you on?

Re: Joint Statement on AI Safety and Openness

#139

Post by hyeonwho22 »

api wrote: post #137
hyeonwho22 wrote: post #134

Earlier quoted context omitted.

1. The religious nut doesn't have the knowledge or the skill sets right now, but AI might enable them. 2. Accessibility of information makes a huge difference. Prior to 2020 people rarely stole Kias or catalytic converters. When knowledge of how to do this (and for catalytic converters, knowledge of their resale value) became available (i.e. trending on Tiktok), then thefts became frequent. The only barrier which dis…

What happens if someone develops a highly effective distributed training algorithm permitting a bunch of people with gaming PCs and fast broadband to train foundation models in a manner akin to Folding@Home? If that happened open efforts could marshal tens or hundreds of thousands of GPUs. Right now the barrier is that training requires too much synchronization bandwidth between compute nodes, but I’m not aware of an…

Is that a serious argument against an AI pause? There are potential scenarios in which regulating AI is challenging, so it isn't worth doing? Why don't we stop regulating nuclear material while we're at it?

In my mind the existential risks make regulation of large training runs worth it. Should distributed training runs become an issue we can figure out a way to inspect them, too.

To respond to the specific htpothetical, if that scenario happens it will presumably be by either a botnet, by a large group of wealthy hobbyists, or by a corporation or a nation state intent on circumventing the pause. Botnets have been dismantled before, and large groups of wealthy hoobyists tend to interested in self preservation (at least more so than individuals). Corporate and state actors defecting on international treaties can be penalized via standard mechanisms.

Re: Joint Statement on AI Safety and Openness

#140

Post by chakalakasp »

mattnewton wrote: post #81
chakalakasp wrote: post #70

Earlier quoted context omitted.

In what other sphere of tech does that apply? "Hackers aren't a problem because we have cybersecurity engineers". And yet somehow entire enterprises and governments are occasionally taken down. What prevents issues in redteam/blueteam is having teams invested in the survivability of the people their organization is working for. That breaks down a bit when all it takes is one biomedical researcher whose wife just left…

I mean, hackers aren’t such a problem that we ban developing new internet apps to only licensed professionals by executive order, so that kinda proves the parent posters point?

The difference is the scale at which a hacker can cause damage. A hacker can ruin a lot of stuff but is unlikely to kill a billion or two people if he succeeds at a hack.

With superintelligent AI you likely have to have every developed use and every end user get it right, air tight, every time, forever.

Post reply on HN