Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many AI safety orgs have tried to criminalize currently-existing open-source AI

1a3orn.com

Post reply on HN
Print view View original

61–70 of 405 posts

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#61

Post by visarga »

JoshTriplett wrote: post #54
rokkitmensch wrote: post #45

Earlier quoted context omitted.

I oppose regulating what calculations humans may perform in the strongest possible terms.

Ten years ago, even five years ago, I would have said exactly the same thing. I am extremely pro-FOSS. Forget the particulars for just a moment. Forget arguments about the probability of the existential risk, whatever your personal assessment of that risk is. Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on…

No, we can't. People have never been able to trust each other so much that they would allow the risk of being marginalised in the name of safety. We don't trust people. Other people are out to get us, or to get ahead. We still think mostly in tribal logic.

If they say "safety" we hear "we want to get an edge by hindering you", or "we want to protect our nice social position by blocking others who would use AI to bootstrap themselves". Or "we want AI to misrepresent your position because we don't like how you think".

We are adversaries that collaborate and compete at the same time. That is why open source AI is the only way ahead, it places the least amount of control on some people by other people.

Even AI safety experts accept that humans misusing AI is a more realistic scenario than AI rebelling against humans. The main problem is that we know how people think and we don't trust them. We are still waging holy wars between us.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#62

Post by kolektiv »

jagrsw wrote: post #51
kolektiv wrote: post #38

Earlier quoted context omitted.

I'm not convinced the onus should be on one side to prove why something isn't an existential risk. We don't start with an assumption that something is world-ending about anything else; we generally need to see a plausibly worked-through example of how the world ends, using technology we can all broadly agree exists/will shortly exist. If we're talking about nuclear weapons, for example, the tech is clear, the pattern…

> seems to be a lot of hand-waving between where we are now and "AGI". Modeling an entity that surpasses our intelligence, especially one that interacts with us, is an extraordinarily challenging, if not impossible, task. Concerning the potential for harm, consider the example of Vladimir Putin, who could theoretically cause widespread destruction using nuclear weapons. Although safeguards exist, these could be circu…

Oh, absolutely - such an entity obviously could! Modelling the behaviour of such an entity is very difficult indeed, as you'd need to make all kinds of assumptions without basis. However, you only need to model this behaviour once you've posited the likely existence of such an entity - and that's where (purely subjectively) it feels like there's a gap.

Nothing has yet convinced me (and I am absolutely honest about the fact that I'm not a deep expert and also not privy to the inner workings of relevant organisations) that it's likely to exist soon. I am very open to being convinced by evidence - but an "argument from trajectory" seems to be what we have at the moment, and so far, those have stalled at local maxima every single time.

We've built some incredibly impressive tools, but so far, nothing that looks or feels like a concept of will (note, not consciousness) yet, to the best of my knowledge.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#64

Post by trevyn »

ben_w wrote: post #57
trevyn wrote: post #49

Earlier quoted context omitted.

> nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. I have an alternate proposal: We assume that someone, somewhere will develop AGI without any sort of “alignment”, plan our lives accordingly, and help other humans plan their lives accordingly.

I think that assumption is why Yudkowsky suggested an international binding agreement to not develop a "too smart" AI (the terms AGI and ASI mean different things to different people) wouldn't be worth the paper it was written on unless everyone was prepared to enforce it with air strikes on any sufficiently large computer cluster.

I think it would help the discussion to understand what the world is like outside of the US and Europe (and… Japan?). There are no rules out here. There is no law. It is a fucking free-for-all. Might makes right. Do there exist GPUs? Shit will get trained.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#65

Post by [deleted] »

visarga wrote: post #50
kmeisthax wrote: post #7

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative u…

> I'm talking about the ability of any AI system to obfuscate plagiarism and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. This should be an explicitly allowed practice, it is following the spirit of copyright to the letter - use the ideas, facts, methods or styles while avoiding to…

[deleted]

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#66

Post by aleph_minus_one »

suslik wrote: post #53
JoshTriplett wrote: post #21

Earlier quoted context omitted.

> AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all . There are risks that apply only to open. There are risks that apply only to closed. But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more…

What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. Should we ban a linear regression while we're at it? It would be great to see some evidence that this risk is real. All I've witnessed so far was scaremongering posts…

> What is the reason to believe that LLMs are an evolutionary step towards AGI at all?

Because this is the marketing pitch of the current wave of venture capital financed AI companies. :-)

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#67

Post by hiAndrewQuinn »

kmeisthax wrote: post #7

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative u…

All future AI research should be banned, a la Bostrom's vulnerable world hypothesis [1]. Every time you pull the trigger in Russian roulette and survive, the next person's chance of getting the bullet is higher.

[1]: https://nickbostrom.com/papers/vulnerable.pdf

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#69

Post by kolektiv »

JoshTriplett wrote: post #48
kolektiv wrote: post #38

Earlier quoted context omitted.

I'm not convinced the onus should be on one side to prove why something isn't an existential risk. We don't start with an assumption that something is world-ending about anything else; we generally need to see a plausibly worked-through example of how the world ends, using technology we can all broadly agree exists/will shortly exist. If we're talking about nuclear weapons, for example, the tech is clear, the pattern…

AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behavi…

I understand your point, I think - and certainly I don't want to go anywhere near name-calling or derision, that doesn't help anyone. But I am reminded of arguments I've had with creationists (I am not comparing you with them, but sometimes the general tone of the debate). It seems like one side is making an extraordinary claim, and then demanding the other side rebut it, and that's not something that seems reasonable to me.

The thing about nuclear weapons is that the theoretical science was clear before the testing - building and testing them was proof by demonstration, but many people agreed with the theory well before that. How they would be used was certainly debated, but there was a clear and well-explained proposal for every step of their creation, which could be tested and falsified if needed. I don't think that's the case here - there seems to be more of a claim for a general acceleration with an inevitable endpoint, and that claim of inevitability feels very short on grounding.

I am more than prepared to admit that I may not be seeing (for various reasons) the evidence that this is near/possible - but I would also claim that nobody is convincingly showing any either.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#70

Post by ben_w »

nerdponx wrote: post #42
kmeisthax wrote: post #7

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative u…

It's way too late to ban any of this. How do you propose to make that work? That would be like banning all "malicious software", it's a preposterous idea when you even begin to think about the practical side of it. And where do you draw the line? Is my XGBoost model "AI", or are we only banning generative AI? Is a Markov chain "generative AI"?

Bans often come after examples, so while I disagree with kmeisthax about… well, everything in that comment… it's almost never too late to pass laws banning GenAI, or to set thresholds at capability levels anywhere, including or excluding Markov chains even.

This is because almost no law is perfectly enforced. My standard example of this is heroin, which nobody defends even if they otherwise think drugs are great, for which the UK has 3 times as many users as its current entire prison population. Despite that failing, the law probably does limit the harm.

Any attempt to enforce a ban on GenAI would by very different, like a cat and mouse game of automatic detection and improved creation (so a GAN even if accidentally), but politicians are absolutely the kind to take credit while kicking the can down the road like that.

Post reply on HN