Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Joint Statement on AI Safety and Openness

open.mozilla.org

Post reply on HN
Print view View original

81–90 of 166 posts

Re: Joint Statement on AI Safety and Openness

#81

Post by mattnewton »

chakalakasp wrote: post #70
jay_kyburz wrote: post #55

Earlier quoted context omitted.

Furthermore, there is little or no defense against a full scale nuclear attack, but a benevolent AI should be sufficient defense against a hostile AI. I think the true fear is that in an AI age, humans are not "useful" and the market and economy will look very different. With AI growing our food, clothing us, building us houses, and entertaining us, humans don't really have anything to do all day.

In what other sphere of tech does that apply? "Hackers aren't a problem because we have cybersecurity engineers". And yet somehow entire enterprises and governments are occasionally taken down. What prevents issues in redteam/blueteam is having teams invested in the survivability of the people their organization is working for. That breaks down a bit when all it takes is one biomedical researcher whose wife just left…

I mean, hackers aren’t such a problem that we ban developing new internet apps to only licensed professionals by executive order, so that kinda proves the parent posters point?

Re: Joint Statement on AI Safety and Openness

#82

Post by tsurba »

chakalakasp wrote: post #52
jph00 wrote: post #30

Earlier quoted context omitted.

It is not at all true that "AI is no different than proliferating nuclear weapons". The project manager for the Nuclear Information Project said ( https://www.vox.com/future-perfect/2023/7/3/23779794/artific... ) : """ But what we are seeing too often is a calorie-free media panic where prominent individuals — including scientists and experts we deeply admire — keep showing up in our push alerts because they vaguely…

If you talk to the people on the bleeding edge of AI research instead of nuke-heads (I tend to be kinda deep into both communities), you'll get a better picture that, yeah, a lot of people who work on AI really do think that AI is like nukes in the scale of force multiplication it will be capable of in the near to medium future, and may well vastly exceed nukes in this regard. Even in the "good" scenarios you're look…

I’m pretty sure what communities you are in are not actual research but some hype alarmist bullshit communities, since as a ML researcher absolutely zero of my peers think the things you say.

Re: Joint Statement on AI Safety and Openness

#83

Post by izzydata »

ayakang31415 wrote: post #79

I still don't have any sense of urgency to create and to regulate AI safety protocols. AI that we call is just software with tuned parameters. You input data into it, and it does some computation on chips, and it spits out data. That is all it needs. My sense is that no harm can come from it, but some harm can be inflicted by people who use it.

I don't think there is any possibility of the current "AI" suddenly doing anything damaging on its own or anything at all on its own. The worst that I can see happening which is probably already happening is people using it to assist them with various crimes. Such as social engineering using convincing generated voices and photos or even just fake information.

Re: Joint Statement on AI Safety and Openness

#84

Post by ayakang31415 »

izzydata wrote: post #83
ayakang31415 wrote: post #79

I still don't have any sense of urgency to create and to regulate AI safety protocols. AI that we call is just software with tuned parameters. You input data into it, and it does some computation on chips, and it spits out data. That is all it needs. My sense is that no harm can come from it, but some harm can be inflicted by people who use it.

I don't think there is any possibility of the current "AI" suddenly doing anything damaging on its own or anything at all on its own. The worst that I can see happening which is probably already happening is people using it to assist them with various crimes. Such as social engineering using convincing generated voices and photos or even just fake information.

To deal with such case, we already have existing laws to deal with it. My qualms with AI regulation is "why do we need separate AI safety agencies and laws?"

Re: Joint Statement on AI Safety and Openness

#85

Post by therealpygon »

The fact that we even have to have this discussion is disgusting. It isn’t the open source community or I’ll-equipped developers running singles or handfuls of gpus that are the risk, it is these exact lobbying companies and others like them who run supercomputers comprised of thousands of servers/gpus. They are the risk for exactly the reason they have done all the previous illegal/immoral things they have done and been fined for (or gotten away with).

We all know that these same companies and others are going to ignore any rules anyway, so open source and academics need to just be left alone to innovate.

Regulate by the risk the processing environment presents, not by the technology running on it.

Re: Joint Statement on AI Safety and Openness

#86

Post by flockonus »

shrimpx wrote: post #69
jeanloolz wrote: post #12

Leaving AI to a handful of companies is in my opinion the fastest route for inequality and power concentration/centralisation, and this is in a way an AI disaster we should be worrisome. Power corrupts, great power greatly corrupts, this is not entirely new. From where I stand, it looks like the pandora box has already been opened anyway. The era of hugginface and/or Llama2 models is only going to grow from there.

Power is already concentrated in the sense that only the wealthy can train. We should make kickstarter-type models to fund open source AI training, because it'll likely continue to cost millions to tens of millions for cutting edge models. With Llama2 we're at Meta's mercy, as it cost 20M to train. No guarantee Meta will continue to give us next-gen models. And even if it does, we're stuck with their training biases,…

While i don't disagree, that's missing the forest for the trees - the point is having the freedom to share models and continue to open-source.

I'll argue that between stable diffusion and llama 2, there is nothing highly specific that prevents [very] large amount of people from adopting these models and specializing for them own needs.

The tragedy would be if those went away.

Re: Joint Statement on AI Safety and Openness

#88

Post by nonameiguess »

chakalakasp wrote: post #14
silenced_trope wrote: post #6

Earlier quoted context omitted.

Right, this is existing players (OpenAI, Google, Microsoft, FB, etc) protecting their moat. AI can't be "paused". Sometimes I see the question "should we put a pause on AI development?" It doesn't mean anything. Some countries like China may say they're on board with pausing it but would they actually do so? Or just sign on and allow their companies to get an edge by not enforcing a pause. Same thing with the existin…

Yes and no. AI is no different than proliferating nuclear weapons or deciding to burn all the fossil fuels -- on an individual competitive level, it makes sense to do this more and more to remain competitive. On a systems-whole level it leads to catastrophe. The whole tragedy of the commons thing, or, more recently described, the Moloch problem. https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditation... Yann LeC…

It's quite a bit different. Access to weapons-grade plutonium is inherently scarce. The basic techniques for producing a transformer architecture to emulate human-level text and image generation is out in the open. The only scarce resource right now preventing anyone from reproducing the research themselves from scratch is the data and compute required to do it. But data and compute aren't plutonium. They aren't inherently scarce. Unless we shut down all advances in electronics and communications, period, shutting down AI research only stops it until data and compute is sufficiently abundant that anyone can do what currently only OpenAI and a few pre-existing giants can do.

What does that buy us? An extra decade?

I don't know where this leaves us. If you're in the MIRI camp believing AI has to lead to runaway intelligence explosion to unfathomable godlike abilities, I don't see a lot of hope. If you believe that is inevitable, then as far as I'm concerned, it's truly inevitable. First, because I think formally provable alignment of an arbitrary software system with "human values," however nebulously you might define that, is fundamentally impossible, but even if it were possible, it's also fundamentally impossible to guarantee in perpetuity that all implementations of a system will forever adhere to your formal proof methods. For 50 years, we haven't even been able to get developers to consistently use strnlen. As far as I can tell, if sufficiently advanced AI can take over its light cone and extinguish all value from the universe, or whatever they're up to now on the worry scale, then it will do so.

Re: Joint Statement on AI Safety and Openness

#89

Post by autoexec »

shrimpx wrote: post #69
jeanloolz wrote: post #12

Leaving AI to a handful of companies is in my opinion the fastest route for inequality and power concentration/centralisation, and this is in a way an AI disaster we should be worrisome. Power corrupts, great power greatly corrupts, this is not entirely new. From where I stand, it looks like the pandora box has already been opened anyway. The era of hugginface and/or Llama2 models is only going to grow from there.

Power is already concentrated in the sense that only the wealthy can train. We should make kickstarter-type models to fund open source AI training, because it'll likely continue to cost millions to tens of millions for cutting edge models. With Llama2 we're at Meta's mercy, as it cost 20M to train. No guarantee Meta will continue to give us next-gen models. And even if it does, we're stuck with their training biases,…

> Power is already concentrated in the sense that only the wealthy can train.

That situation will change as technology evolves. We'll eventually reach a point where a normal desktop PC can train AI. The wealthy will always be able to do it faster, but the gap will shrink with time.

The trick is making sure that laws aren't put in place now that would restrict our ability to do that freely once the technology is there, and that massive data sets are compiled and preserved where the public has free access to them.

Re: Joint Statement on AI Safety and Openness

#90

Post by nonameiguess »

nonameiguess wrote: post #88
chakalakasp wrote: post #14

Earlier quoted context omitted.

Yes and no. AI is no different than proliferating nuclear weapons or deciding to burn all the fossil fuels -- on an individual competitive level, it makes sense to do this more and more to remain competitive. On a systems-whole level it leads to catastrophe. The whole tragedy of the commons thing, or, more recently described, the Moloch problem. https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditation... Yann LeC…

It's quite a bit different. Access to weapons-grade plutonium is inherently scarce. The basic techniques for producing a transformer architecture to emulate human-level text and image generation is out in the open. The only scarce resource right now preventing anyone from reproducing the research themselves from scratch is the data and compute required to do it. But data and compute aren't plutonium. They aren't inhe…

I guess I should add, because so few people do, this is what I believe, but it's entirely possible I'm wrong, so by all means, MIRI, keep trying. If you'd asked anyone in the world except three men before 1975 if public key cryptography was possible, they'd have said no, but here we are. Wow me with your math.
Post reply on HN