Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many AI safety orgs have tried to criminalize currently-existing open-source AI

1a3orn.com

Post reply on HN
Print view View original

81–90 of 405 posts

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#81

Post by visarga »

ben_w wrote: post #57
trevyn wrote: post #49

Earlier quoted context omitted.

> nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. I have an alternate proposal: We assume that someone, somewhere will develop AGI without any sort of “alignment”, plan our lives accordingly, and help other humans plan their lives accordingly.

I think that assumption is why Yudkowsky suggested an international binding agreement to not develop a "too smart" AI (the terms AGI and ASI mean different things to different people) wouldn't be worth the paper it was written on unless everyone was prepared to enforce it with air strikes on any sufficiently large computer cluster.

I think not even Sam and Satya agree on the definition of AGI with so much money at stake. Everyone with their own definitions, and hidden interests.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#82

Post by [deleted] »

JoshTriplett wrote: post #48
kolektiv wrote: post #38

Earlier quoted context omitted.

I'm not convinced the onus should be on one side to prove why something isn't an existential risk. We don't start with an assumption that something is world-ending about anything else; we generally need to see a plausibly worked-through example of how the world ends, using technology we can all broadly agree exists/will shortly exist. If we're talking about nuclear weapons, for example, the tech is clear, the pattern…

AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behavi…

[deleted]

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#83

Post by ben_w »

trevyn wrote: post #60
JoshTriplett wrote: post #54

Earlier quoted context omitted.

Ten years ago, even five years ago, I would have said exactly the same thing. I am extremely pro-FOSS. Forget the particulars for just a moment. Forget arguments about the probability of the existential risk, whatever your personal assessment of that risk is. Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on…

> Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity This has nothing to do with should. There are at the very least a handful of people who can, today, unilaterally take risks with the future of humanity without the consent of humanity. I do not see any reason to think that will change in the near future. If these people can…

Hmm.

So, wealth isn't distributed evenly, and computers of any specific capacity are getting cheaper (not Moore's Law any more, IIRC, but still getting cheaper).

If there's a threshold that requires X operations, that currently costs Y dollars, and say only a few thousand individuals (and more corporations) can afford that.

Halve the cost, either by cheaper computers or by algorithmic reduction of the number of operations needed, and you much more than double the number of people who can do it.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#84

Post by lannisterstark »

JoshTriplett wrote: post #54
rokkitmensch wrote: post #45

Earlier quoted context omitted.

I oppose regulating what calculations humans may perform in the strongest possible terms.

Ten years ago, even five years ago, I would have said exactly the same thing. I am extremely pro-FOSS. Forget the particulars for just a moment. Forget arguments about the probability of the existential risk, whatever your personal assessment of that risk is. Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on…

>Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on their unilateral assessment of those risks?

No, we cannot, because that isn't practical. any of the nuclear armed countries can launch a nuclear strike tomorrow (hypothetically - but then again, isn't all "omg ai will kill us all" hypothetical, anyway?) - and they absolutely do not need consent of humanity, much less their own citizenry.

This is honestly, not a great argument.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#85

Post by zarzavat »

mrtksn wrote: post #11
sillysaurusx wrote: post #5

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof. I wish I could be more eloquent on this point, but I’ve mostly just been depressed about this seeming inevitability. Hopefully it won’t be the case. But how could it be otherwise? Hundreds of thousands of people are mad at openai and midjourney for doing exactly what op…

We have infinite data, a microphone and a camera can generate huge amount of it and the public domain literature is wast. Billions of people learn like that everyday.

It’s impossible to learn any technical topic from 70+ year old books. The public domain is small and basically zero if you want to learn anything current. A microphone and camera is fine for learning about daily life, but you cannot get “book smarts” without copyrighted media.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#86

Post by ben_w »

visarga wrote: post #81
ben_w wrote: post #57

Earlier quoted context omitted.

I think that assumption is why Yudkowsky suggested an international binding agreement to not develop a "too smart" AI (the terms AGI and ASI mean different things to different people) wouldn't be worth the paper it was written on unless everyone was prepared to enforce it with air strikes on any sufficiently large computer cluster.

I think not even Sam and Satya agree on the definition of AGI with so much money at stake. Everyone with their own definitions, and hidden interests.

Without knowing them, I can easily believe that. Even without reference to money.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#87

Post by hsuduebc2 »

breadbreadbread wrote: post #25

Ah yes the old "banning things=bad" argument that doesn't offer alternatives to fixing the issues with AI. Just ignore the issues with environmental impact, plagiarism, CP and other non-consensual shit in the data sets, scamming capabilities! All the groups asking for regulation here have **funding** and that means they are evil but we are good for using this tool that is massively subsidized by megacorps that have a…

It's just plainly retarded steering from progress. Only one who would profit from it would be a nations who wouldn't give a flying fuck about your ban. Exactly like with nuclear energy.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#88

Post by visarga »

war321 wrote: post #46
kmeisthax wrote: post #7

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative u…

Pretty sure they do. I follow a few of these safetyist people on twitter and they absolutely argue that companies like OpenAI, Google, Tencent and literally anyone else training a potential AGI should stop training runs and put them under oversight at best and no one should even make an AGI at worst. They just go after open source as well since they're at least aware that open models that anyone can share and use are…

That's a bad call. We would stop openly looking for AI vulnerabilities and create conditions for secret development that would hide away the dangers without being safer. Lots of eyes are better to find the sensitive spots of AI. We need people to hack weaker AIs and help fix them or at least understand the threat profile before they get too strong.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#89

Post by fsflover »

kolektiv wrote: post #62
jagrsw wrote: post #51

Earlier quoted context omitted.

> seems to be a lot of hand-waving between where we are now and "AGI". Modeling an entity that surpasses our intelligence, especially one that interacts with us, is an extraordinarily challenging, if not impossible, task. Concerning the potential for harm, consider the example of Vladimir Putin, who could theoretically cause widespread destruction using nuclear weapons. Although safeguards exist, these could be circu…

Oh, absolutely - such an entity obviously could! Modelling the behaviour of such an entity is very difficult indeed, as you'd need to make all kinds of assumptions without basis. However, you only need to model this behaviour once you've posited the likely existence of such an entity - and that's where (purely subjectively) it feels like there's a gap. Nothing has yet convinced me (and I am absolutely honest about th…

> I am very open to being convinced by evidence - but an "argument from trajectory" seems to be what we have at the moment, and so far, those have stalled at local maxima every single time.

Sounds like the same argument as why flying machines heavier than air deemed impossible at some point.

Post reply on HN