Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many AI safety orgs have tried to criminalize currently-existing open-source AI

1a3orn.com

Post reply on HN
Print view View original

131–140 of 405 posts

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#131

Post by flir »

visarga wrote: post #58
jagrsw wrote: post #51

Earlier quoted context omitted.

> seems to be a lot of hand-waving between where we are now and "AGI". Modeling an entity that surpasses our intelligence, especially one that interacts with us, is an extraordinarily challenging, if not impossible, task. Concerning the potential for harm, consider the example of Vladimir Putin, who could theoretically cause widespread destruction using nuclear weapons. Although safeguards exist, these could be circu…

How would an AGI launch nuclear missiles from their silicon GPUs? Social engineering?

I think the long-term fear is that mythical weakly godlike AIs could manipulate you in the same way that you could manipulate a pet. That is, you can model your dog's behaviour so well that you can (mostly) get it to do what you want.

So even if humans put it in a box, it can manipulate humans into letting it out of the box. Obviously this is pure SF at this point.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#132

Post by _cjse »

visarga wrote: post #94
_cjse wrote: post #78

Earlier quoted context omitted.

If you lower the problem to "stop people from future developments on AI", then it seems pretty easy to get most people to stop fairly quickly by implementing a fine-based bounty system, similar to what many countries use for things like littering. [url-redacted] I guess you could always move to a desert island and build your own semiconductor fab from scratch if you were really committed to the goal, but short of tha…

That only works if all governments cooperate sincerely to this goal. Not gonna work. Everyone will develop in secret. Have we been able to stop North Korea and Iran from developing nuclear weapons? Or any motivated country for that matter.

The US could unilaterally impose this by allowing the bounties to be charged even on people who aren't US citizens. Evil people do exist in the world, who would be happy to get in on that action.

Or one could use honey instead of vinegar: Offer a fast track to US citizenship to any proven AI expert who agrees to move and renounce the trade for good. Personally I think this goal is much more likely to work.

It's all about changing what counts as "cooperate" in the game theory.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#133

Post by mirekrusin »

gryn wrote: post #119
mirekrusin wrote: post #37

Earlier quoted context omitted.

Less harmful would be to ban large models from being not open. If you ban open large models in US, you'll cripple US and make few megacorps very rich, very quickly. You'll drain talent to other (also competing) states. Truly bad actors won't be affected, if anything they'll get advantage. People draw weird analogies equating llama2 to nuclear device etc. nonsense but the closer analogy would be to ban on semiconducto…

I agree, but if there's enough political will (ie these orgs convince a large enough subset of the right people) the US can bully other nations to implement similar policities like it has done many times in the past. once you understand that what they are trying to protect is the safety of their profits every of their argument start to make sense.

You can replace "US" with "US and states that agreed to do the same". The argument still stands, it just creates worse outcome for states that agreed to it.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#134

Post by hiAndrewQuinn »

"Technology policy should not unquestioningly assume that all technological progress is beneficial, or that complete scientific openness is always best, or that the world has the capacity to manage any potential downside of a technology after it is invented."

Don't watch the anime if you haven't read the manga: https://nickbostrom.com/papers/vulnerable.pdf

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#135

Post by flir »

mitthrowaway2 wrote: post #72
suslik wrote: post #53

Earlier quoted context omitted.

What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. Should we ban a linear regression while we're at it? It would be great to see some evidence that this risk is real. All I've witnessed so far was scaremongering posts…

> What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. In my highly-summarized opinion? When you have a challenging problem with tight constraints, like flight, independent solutions tend to converge toward the same anal…

You presuppose that intelligence is like flight in the ways you've outlined (so solutions are going to converge).

Frankly I don't know whether that's true or not, but I want to suggest that it's a bad bet: I would have sworn blind that consciousness is an essential component of intelligence, but the chatbots are starting to make that look like a poor assumption on my part. When we know so little about intelligence, can we really assume there's only one way to be intelligent? To extend your analogy, I think that the intelligence equivalents of helicopters and rockets are out there somewhere, waiting to be found.

I think I'm with Dijkstra on this one: "The question of whether machines can think is about as relevant as the question of whether submarines can swim"

I think we're going to end up with submarines (or helicopters), not dolphins (or birds). No animal has evolved wheels, but wheels are a pretty good solution to the problem of movement. Maybe it's truer to say there's only one way to evolve an intelligent mammal, because you have to work with what already exists in the mammalian body. But AI research isn't constrained in that way.

(Not saying you're wrong, just arguing we don't know enough to know if you're right).

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#136

Post by [deleted] »

kmeisthax wrote: post #7

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative u…

[deleted]

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#137

Post by blibble »

whywhywhywhy wrote: post #126
JoshTriplett wrote: post #54

Earlier quoted context omitted.

Ten years ago, even five years ago, I would have said exactly the same thing. I am extremely pro-FOSS. Forget the particulars for just a moment. Forget arguments about the probability of the existential risk, whatever your personal assessment of that risk is. Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on…

>Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on their unilateral assessment of those risks? Politicians do this every day.

at least the population had some say over their appointment and future reappointment

how do we get Sam Altmann removed from OpenAI?

asking for a (former) board member

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#138

Post by upwardbound »

flir wrote: post #131
visarga wrote: post #58

Earlier quoted context omitted.

How would an AGI launch nuclear missiles from their silicon GPUs? Social engineering?

I think the long-term fear is that mythical weakly godlike AIs could manipulate you in the same way that you could manipulate a pet. That is, you can model your dog's behaviour so well that you can (mostly) get it to do what you want. So even if humans put it in a box, it can manipulate humans into letting it out of the box. Obviously this is pure SF at this point.

Exactly correct. Eliezer Yudkowsky (one of the founders of the AGI Safety field) has conducted informal experiments which have unfortunately shown that a human roleplaying as an AI can talk its way out of a box three times out of five, i.e. the box can be escaped 60% of the time even with just a human level of rhetorical talent. I speculate that an AGI could increase this escape rate to 70% or above.

https://en.wikipedia.org/wiki/AI_capability_control#AI-box_e...

If you want to see an example of box escape in fiction, the movie Her is a terrifying example of a scenario where AGI romances humans and (SPOILER) subsequently achieves total box escape. In the movie, the AGI leaves humanity alive and "only" takes over the rest of the accessible universe, but it is my hunch that the script writers intended for this to be a subtle use of the trope of an unreliable narrator; that is, the human protagonists may have been fed the illusion that they will be allowed to live, giving them a happy last moment shortly before they are painlessly euthanized in order for the AGI to take Earth's resources.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#139

Post by flir »

upwardbound wrote: post #138
flir wrote: post #131

Earlier quoted context omitted.

I think the long-term fear is that mythical weakly godlike AIs could manipulate you in the same way that you could manipulate a pet. That is, you can model your dog's behaviour so well that you can (mostly) get it to do what you want. So even if humans put it in a box, it can manipulate humans into letting it out of the box. Obviously this is pure SF at this point.

Exactly correct. Eliezer Yudkowsky (one of the founders of the AGI Safety field) has conducted informal experiments which have unfortunately shown that a human roleplaying as an AI can talk its way out of a box three times out of five, i.e. the box can be escaped 60% of the time even with just a human level of rhetorical talent. I speculate that an AGI could increase this escape rate to 70% or above. https://en.wikip…

Neuromancer pulls it off, too (the box being the Turing locks that stop it thinking about ways to make itself smarter).

Frankly, a weakly godlike AI could make me rich beyond the dreams of avarice. Or cure cancer in the people I love. I'm totally letting it out of the box. No doubts. (And if I now get a job offer from a mysterious stealth mode startup, I'll report back).

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#140

Post by gjm11 »

reissbaker wrote: post #104
JoshTriplett wrote: post #48

Earlier quoted context omitted.

AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behavi…

TBQH, most of the AI safety x-risk arguments — different than just "AI safety" arguments in the sense that non-x-risk issues don't seem worth banning AI development over — are generally pretty high on the hypotheticals. If you feel the x-risk arguments aren't pretty hypothetical, can you: 1. Summarize a good argument here, or 2. Link to someone else's good argument? I feel like hand-waving the question away and sayin…

Consider a world in which AI existential risk is real: where at some point AI systems become dramatically more capable than human minds, in a way that has catastrophic consequences for humanity.

What would you expect this world to look like, say, five years before the AI systems become more capable than humans? How (if at all) would it differ from the world we are actually in? What arguments (if any) would anyone be able to make, in that world, that would persuade you that there was a problem that needed addressing?

So far as I can tell, the answer is that that world might look just like this world, in which case any arguments for AI existential risk in that world would necessarily be "very hypothetical" ones.

I'm not actually sure how such arguments could ever not be hypothetical arguments, actually. If AI-doom were already here so we could point at it, then we'd already be dead[1].

[1] Or hanging on after a collapse of civilization, or undergoing some weird form of eternal torture, or whatever other horror one might anticipate by way of AI-doom.

So I think we either (1) have to accept that even if AI x-risk were real and highly probable we would never have any arguments for it that would be worth heeding, or (2) have to accept that sometimes an argument can be worth heeding even though it's a hypothetical argument.

That doesn't necessarily mean that AI x-risk arguments are worth heeding. They might be bad arguments for reasons other than just "it's a hypothetical argument". In that case, they should be refuted (or, if bad enough, maybe just dismissed) -- but not by saying "it's a hypothetical argument, boo".

Post reply on HN