Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Joint Statement on AI Safety and Openness

open.mozilla.org

Post reply on HN
Print view View original

141–150 of 166 posts

Re: Joint Statement on AI Safety and Openness

#141

Post by [deleted] »

hyeonwho22 wrote: post #139
api wrote: post #137

Earlier quoted context omitted.

What happens if someone develops a highly effective distributed training algorithm permitting a bunch of people with gaming PCs and fast broadband to train foundation models in a manner akin to Folding@Home? If that happened open efforts could marshal tens or hundreds of thousands of GPUs. Right now the barrier is that training requires too much synchronization bandwidth between compute nodes, but I’m not aware of an…

Is that a serious argument against an AI pause? There are potential scenarios in which regulating AI is challenging, so it isn't worth doing? Why don't we stop regulating nuclear material while we're at it? In my mind the existential risks make regulation of large training runs worth it. Should distributed training runs become an issue we can figure out a way to inspect them, too. To respond to the specific htpotheti…

[deleted]

Re: Joint Statement on AI Safety and Openness

#142

Post by coryfklein »

drdaeman wrote: post #125
coryfklein wrote: post #118

If you think there is even a 5% chance that super-human level artificial general intelligence could be catastrophic for humanity, then the recipe for how to make AGI is of itself an infohazard far worse than the recipe for methamphetamines/nuclear weapons/etc, and precedent shows that we do in fact lock down access to the latter as a matter of public safety. No, GPT-4 is not AGI and is not going to spell the end of t…

> but neither is pseudoephedrine itself a methamphetemine and yet we regulate its access I don't think it's a correct argument. I believe it's not a secret how metamphetamine is made (though I don't know it myself, but I'm too lazy to research for a sake of argument), and it's known pseudoephedrine is a precursor chemical. Thus, the regulation. No one - I believe - knows how to build an AGI. There is no recipe. It is…

First of all, pseudoephedrine is an analogy. But with AGI there is literally no historical event to compare it to as a perfect analogy, including the ones I gave above.

> it's not a secret how metamphetamine is made

I didn't say it was a secret. But try sharing the recipe to methamphetamine virtually anywhere online and see how long it stays up; we do in fact try to at least make it moderately difficult to find this information. Perfect is the enemy of good here.

> No one - I believe - knows how to build an AGI.

The problem is that if we wait until someone does know how to build an AGI, then the game is up. We do not know if the weights behind GPT-4 will be a critical ingredient. If you make the ingredients common knowledge before you know they were in fact The Ingredients, you've let the cat out of the bag and there's no retrieving it.

Re: Joint Statement on AI Safety and Openness

#143

Post by api »

hyeonwho22 wrote: post #139
api wrote: post #137

Earlier quoted context omitted.

What happens if someone develops a highly effective distributed training algorithm permitting a bunch of people with gaming PCs and fast broadband to train foundation models in a manner akin to Folding@Home? If that happened open efforts could marshal tens or hundreds of thousands of GPUs. Right now the barrier is that training requires too much synchronization bandwidth between compute nodes, but I’m not aware of an…

Is that a serious argument against an AI pause? There are potential scenarios in which regulating AI is challenging, so it isn't worth doing? Why don't we stop regulating nuclear material while we're at it? In my mind the existential risks make regulation of large training runs worth it. Should distributed training runs become an issue we can figure out a way to inspect them, too. To respond to the specific htpotheti…

You are talking about some pretty heavy handed authoritarian stuff to ban math on the basis of hypothetical risks. The nuclear analogy isn’t applicable because we all know that a-bombs really work. There is no proof of any kind of outsized risk from real world AI beyond other types of computing that can be used for negative purposes like encryption or cryptocurrency.

Here’s a legit question: you say pause. Pause until what? What is the go condition? You can never prove an unbounded negative like “AI will never ever become dangerous” so I would think there is no go condition anyone could agree on.

… which means people eventually just ignore the pause when they get tired of it and the hysteria dies out. Why bother then?

Re: Joint Statement on AI Safety and Openness

#144

Post by coryfklein »

akira2501 wrote: post #124
coryfklein wrote: post #118

If you think there is even a 5% chance that super-human level artificial general intelligence could be catastrophic for humanity, then the recipe for how to make AGI is of itself an infohazard far worse than the recipe for methamphetamines/nuclear weapons/etc, and precedent shows that we do in fact lock down access to the latter as a matter of public safety. No, GPT-4 is not AGI and is not going to spell the end of t…

> could be catastrophic for humanity, I do not. How does super human intelligence, on it's own, represent any sort of risk for "humanity?" > then the recipe for how to make AGI is of itself an infohazard [...] nuclear weapons I know how to make nuclear weapons; however, I cannot enrich the fuel enough to actually produce a working version. > and yet we regulate its access Does that actually achieve what it claims to…

> How does super human intelligence, on it's own, represent any sort of risk for "humanity?"

It does not; I was speaking to AGI. But assuming you were referring to AGI as well, you don't have to think very creatively to consider scenarios where it would be harmful.

If you have an agent that can counter your every move with its own superior move, and that agent wants something – anything – differently from what you want, then who wins? Maybe it wants the money in your bank account, maybe it wants to use the atoms in your body to make more graphics cards to reproduce itself.

Think about playing a game of chess against the strongest AI opponent. No matter which move you are considering playing, your opponent has already planned 10 steps ahead and will make a better move than you. Now extrapolate outside the chess board and into the realm where you can use the internet to buy/trade stocks, attack national infrastructure, send custom orders to chemists to follow whatever directions you want, etc.

Re: Joint Statement on AI Safety and Openness

#145

Post by rightbyte »

blacksmith_tb wrote: post #117
trentnix wrote: post #33

Earlier quoted context omitted.

> Leaving AI to a handful of companies is in my opinion the fastest route for inequality and power concentration/centralisation Well yes, that’s precisely why they are lobbying for it.

That does seem awfully convenient, yes. But I am also skeptical that Meta is railing against regulation out of their commitment to protect scrappy little startups doing AI...

I honestly don't see the cynical motive I know is there in what Facebook is doing, releasing Llama. I am down to that they just want to screw with Microsoft for buying "OpenAI", which is fine by me.

Re: Joint Statement on AI Safety and Openness

#146

Post by coryfklein »

jairuhme wrote: post #129
coryfklein wrote: post #118

If you think there is even a 5% chance that super-human level artificial general intelligence could be catastrophic for humanity, then the recipe for how to make AGI is of itself an infohazard far worse than the recipe for methamphetamines/nuclear weapons/etc, and precedent shows that we do in fact lock down access to the latter as a matter of public safety. No, GPT-4 is not AGI and is not going to spell the end of t…

When you say 'catastrophic for humanity', what exactly do you think would happen? And how would regulating models thwart this? Bad actors who are motivated surely will get around US restrictions. I'd imagine when the internet first came around that there was similar sentiment. "People will have access to information on how to create bombs, we need to regulate this so that doesn't happen!"

When playing against a better chess player than me, the whole point is that I cannot predict what move they will make, and that is how the opponent wins. So you don't need to be able to predict what move the AGI will make, only that no matter what move you make it will do better.

But super off-hand idea if I'm trying to be creative. The AGI formulates a chemical that kills humans 90 days after inhalation, hires a chemical lab to synthesize it and forward it to municipalities across the world, and convinces them it's a standard treatment that the WHO has mandated be introduced.

> And how would regulating models thwart this?

I don't think it would. The OP article is about regulation that increases the access to the ingredients for AI, and I'm simply unconvinced that is a recipe for increasing AI safety.

Re: Joint Statement on AI Safety and Openness

#147

Post by coryfklein »

I_am_uncreative wrote: post #121
coryfklein wrote: post #118

If you think there is even a 5% chance that super-human level artificial general intelligence could be catastrophic for humanity, then the recipe for how to make AGI is of itself an infohazard far worse than the recipe for methamphetamines/nuclear weapons/etc, and precedent shows that we do in fact lock down access to the latter as a matter of public safety. No, GPT-4 is not AGI and is not going to spell the end of t…

This is such a stupid take. Even if there was a 5% chance of super-human level AGI destroying the world, we're not going to reduce that risk by restricting general computation. If it is possible to create AGI, it will happen, regardless of what the law stipulates. This is unlike pseudoephedrine or nuclear weapons in that literally anyone with a computer could potentially create it.

Yes, it is in fact going to happen no matter what. So how do we ensure that the very first super-human AGI is even remotely interested in sharing the planet with the human race?

It is not by ensuring that literally anyone with a computer has an equal chance at creating it.

Re: Joint Statement on AI Safety and Openness

#148

Post by chakalakasp »

tsurba wrote: post #82
chakalakasp wrote: post #52

Earlier quoted context omitted.

If you talk to the people on the bleeding edge of AI research instead of nuke-heads (I tend to be kinda deep into both communities), you'll get a better picture that, yeah, a lot of people who work on AI really do think that AI is like nukes in the scale of force multiplication it will be capable of in the near to medium future, and may well vastly exceed nukes in this regard. Even in the "good" scenarios you're look…

I’m pretty sure what communities you are in are not actual research but some hype alarmist bullshit communities, since as a ML researcher absolutely zero of my peers think the things you say.

You have no clue what communities I’m in other than it self affirms your worldview to assume that they must be irrelevant. To be fair, I’m not telling you anything about myself, so you’re not really needing to take my word for it. And I don’t care enough about you to explain in detail.

Though with a tiny bit of Googling you’ll be able to find several Turing Award winners who are saying exactly what I’m saying. In public. Loudly.

Re: Joint Statement on AI Safety and Openness

#149

Post by lelandbatey »

shrimpx wrote: post #108
autoexec wrote: post #89

Earlier quoted context omitted.

> Power is already concentrated in the sense that only the wealthy can train. That situation will change as technology evolves. We'll eventually reach a point where a normal desktop PC can train AI. The wealthy will always be able to do it faster, but the gap will shrink with time. The trick is making sure that laws aren't put in place now that would restrict our ability to do that freely once the technology is there…

That didn't happen with cloud though. I know it's not entirely apples to apples, but you still need a computer the size of a large building to serve Netflix, etc.

Only if you're trying to serve netflix-quality stuff to hundreds of thousands of people. If you're trying to replicate "Netflix the product" (live video streaming with a slick interface) to a small set of individuals, you can do that with a personal computer (see Jellyfin, Plex).

Comparing Netflix (and most highly profitable computer businesses) to the world of producing AI models by training is not going to be fruitful. Netflix takes a lot of effort to operate but you can do on the small scale what Netflix does, quite directly. You can't replicate an AI model like ChatGPT-4 very easily unless you have all the data and huge compute that OpenAI does. Now, once the model has been produced, you can operate that model on the small scale with maybe less amazing results (see llama.cpp, etc) but producing the model is a scale problem like producing high quality steel. You can't escape the need for scale (without some serious technological developments first).

Re: Joint Statement on AI Safety and Openness

#150

Post by chakalakasp »

torginus wrote: post #56
chakalakasp wrote: post #14

Earlier quoted context omitted.

Yes and no. AI is no different than proliferating nuclear weapons or deciding to burn all the fossil fuels -- on an individual competitive level, it makes sense to do this more and more to remain competitive. On a systems-whole level it leads to catastrophe. The whole tragedy of the commons thing, or, more recently described, the Moloch problem. https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditation... Yann LeC…

Why does everybody come up with the 'nuclear weapons' comparison, when there is a much more appropriate one - encryption, specifically public key cryptography? Way back in the 90s, when Phil Zimmerman released PGP, the US government raised hell to keep it from proliferating. Would you rather live in a world where strong encryption for ordinary citizens was illegal?

Because encryption is not an inherently dangerous thing. A superintelligent AI is.

It’s no different than inviting an advanced alien species to visit. Will it go well? Sure hope so, because if they don’t want it to go well it won’t be our planet any more

Post reply on HN