Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many AI safety orgs have tried to criminalize currently-existing open-source AI

1a3orn.com

Post reply on HN
Print view View original

171–180 of 405 posts

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#171

Post by sensanaty »

te_chris wrote: post #71
JoshTriplett wrote: post #48

Earlier quoted context omitted.

AI safety / x-risk folks have in fact made extensive and detailed arguments. Occasionally, folks arguing against them rise to the same standard. But most of the arguments against AI safety look a lot more like name-calling and derision: "nuh-uh, that's sci-fi and unrealistic (mic drop)". That's not a counterargument. > If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behavi…

An extensive HYPOTHETICAL argument, stuffed with assumptions far beyond the capabilities of the technologies they're talking about for their own private ends.

The point of the discussion is to have a look at the possible future ramifications of the technology, so it's only logical to talk about future capabilities and not the current ones. Obviously the current puppet chatbots aren't gonna be doing much ruining (even that's arguable already judging by all the layoffs), but what are future versions of these LLMs/AIs going to be doing to us?

After all, if we only discussed the dangers of nuclear weapons after they've been dropped on cities, well that's too little too late, eh?

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#172

Post by [deleted] »

ben_w wrote: post #110
torginus wrote: post #24

Earlier quoted context omitted.

I wonder how long would it take this to get fixed, if I fed some current best-seller novels into an LLM and instructed it to reword it, renaming the characters and places, and shared the result publicly for free? Although I fear the response would be that powerful AI orgs would put copyright filters in place and lobby for legalization for mandatory AI-DRM in open source AI as well.

What you're describing sounds like a search and replace could already do it. If you mean something more transformative, did Yudkowsky get a licence for HPMOR? Did Pratchett (he might well have) get a licence from Niven to retell Ringworld as Strata? I don't know how true it is, but it's widely repeated that 50 Shades of Grey was originally a fan fiction of the Twilight series. Etc., but note that I'm not saying "no".

[deleted]

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#173

Post by jagrsw »

zer00eyz wrote: post #160
upwardbound wrote: post #138

Earlier quoted context omitted.

Exactly correct. Eliezer Yudkowsky (one of the founders of the AGI Safety field) has conducted informal experiments which have unfortunately shown that a human roleplaying as an AI can talk its way out of a box three times out of five, i.e. the box can be escaped 60% of the time even with just a human level of rhetorical talent. I speculate that an AGI could increase this escape rate to 70% or above. https://en.wikip…

The show "The Walking Dead" always bothered me. Where do they keep finding gas that will still run a car? It wont last forever in tanks, and most gas is just in time delivery (Stations get daily delivery) -- And someone noted on the show that the grass was always mowed. I feel like the AI safety folks are spinning an amazing narrative, the AI is gonna get us like the zombies!!! The retort to the ai getting out of the…

> For the foreseeable future AI is going to be very limited in how much harm it can cause us, because killing us,...

AI's potential for harm might be limited for now in some scenarios (those with warning sings ahead of time), but this might change sooner than we think.

The notion that AGI will be restricted to a single data center and thus susceptible to shutdowns is incorrect. AIs/MLs are, in essence, computer programs + exec environs, which can be replicated, network-transferred, and checkpoint-restored. Please note, that currently available ML/AI systems are directly connected to the outside world, either via its users/APIs/plugins, or by the fact that they're OSS, and can be instantiated by anyone in any computing environment (also those net-connected).

While AGI currently depends on humans for infrastructure maintenance, the future may see it utilizing robots. These robots could range in size (don't need to be movie-like Terminators) and be either autonomously AI-driven or remotely controlled. Their eventual integration into various sectors like manufacturing, transportation, military and domestic tasks implies a vast array for AGI to exploit.

The constraints we associate with AI today might soon be outdated.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#174

Post by neurostimulant »

sillysaurusx wrote: post #5

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof. I wish I could be more eloquent on this point, but I’ve mostly just been depressed about this seeming inevitability. Hopefully it won’t be the case. But how could it be otherwise? Hundreds of thousands of people are mad at openai and midjourney for doing exactly what op…

> There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof.

In the West, maybe. I doubt it'll slow down open source AI development pace in places like China.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#175

Post by [deleted] »

visarga wrote: post #61
JoshTriplett wrote: post #54

Earlier quoted context omitted.

Ten years ago, even five years ago, I would have said exactly the same thing. I am extremely pro-FOSS. Forget the particulars for just a moment. Forget arguments about the probability of the existential risk, whatever your personal assessment of that risk is. Can we agree that people should not be able to unilaterally take existential risks with the future of humanity without the consent of humanity, based solely on…

No, we can't. People have never been able to trust each other so much that they would allow the risk of being marginalised in the name of safety. We don't trust people. Other people are out to get us, or to get ahead. We still think mostly in tribal logic. If they say "safety" we hear "we want to get an edge by hindering you", or "we want to protect our nice social position by blocking others who would use AI to boot…

[deleted]

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#176

Post by simiones »

JoshTriplett wrote: post #21
kmeisthax wrote: post #7

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative u…

> AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all . There are risks that apply only to open. There are risks that apply only to closed. But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more…

> Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks.

The AI safety strawman is "existential risk from magical super-God AI". That is what the unserious "AI safety" grifters or sci-fi enthusiasts are discussing.

The real AI safety risks are the ones that actually exist today: training-set biases extended to decision-making biases, deeply personalized propaganda, plagiarism white-washing, creative works bankrupting, power imbalances from control of working AI tech etc.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#177

Post by bondarchuk »

gjm11 wrote: post #140
reissbaker wrote: post #104

Earlier quoted context omitted.

TBQH, most of the AI safety x-risk arguments — different than just "AI safety" arguments in the sense that non-x-risk issues don't seem worth banning AI development over — are generally pretty high on the hypotheticals. If you feel the x-risk arguments aren't pretty hypothetical, can you: 1. Summarize a good argument here, or 2. Link to someone else's good argument? I feel like hand-waving the question away and sayin…

Consider a world in which AI existential risk is real: where at some point AI systems become dramatically more capable than human minds, in a way that has catastrophic consequences for humanity. What would you expect this world to look like, say, five years before the AI systems become more capable than humans? How (if at all) would it differ from the world we are actually in? What arguments (if any) would anyone be…

I think what is meant is "hypothetical" in the sense of making assumptions about how AI systems would behave under certain circumstances. If an argument relies on a chain of assumptions like that (such as "instrumental convergence" and "reflective stability" to take some Lesswrong classics), it might look superficially like a good argument for taking drastic action, but if the whole argument falls down when any of the assumptions turn out the other way, it can be fairly dismissed as "too hypothetical" until each assumption has strong argumentation behind it.

edit: also I think just in general "show me the arguments" is always a good response to a bare claim that good arguments exist.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#178

Post by Simon_ORourke »

upwardbound wrote: post #145
kmeisthax wrote: post #7

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative u…

ㅤ I work in the AI safety field (my team discovered Prompt Injection) and I absolutely advocate the Dune solution. I even have a tattoo on my arm that I got so that people will ask me about this and I can explain the proposal. The tattoo says "The 1948 Initiative" which has two layers of meaning: (1) It is a proposal for humanity to protect ourselves from being slaughtered by AGI by pre-emptively implementing a total…

Ha! Good one!

Let's just roll back all our healthcare research, logistics, environmental advancements etc. because some boogey man tech might do something bad perhaps at sometime in the future maybe!

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#179

Post by mtlmtlmtlmtl »

ben_w wrote: post #70
nerdponx wrote: post #42

Earlier quoted context omitted.

It's way too late to ban any of this. How do you propose to make that work? That would be like banning all "malicious software", it's a preposterous idea when you even begin to think about the practical side of it. And where do you draw the line? Is my XGBoost model "AI", or are we only banning generative AI? Is a Markov chain "generative AI"?

Bans often come after examples, so while I disagree with kmeisthax about… well, everything in that comment… it's almost never too late to pass laws banning GenAI, or to set thresholds at capability levels anywhere, including or excluding Markov chains even. This is because almost no law is perfectly enforced. My standard example of this is heroin, which nobody defends even if they otherwise think drugs are great, for…

Actually, anyone who knows what they're talking about will tell you the ban makes heroin a much worse problem, not better.

Ban leads to a black market, leads to lousy purity, which leads to fluctuations in purity and switches to potent alternatives, leads to waves of overdose deaths.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#180

Post by [deleted] »

PeterisP wrote: post #170
127361 wrote: post #118

Earlier quoted context omitted.

But if we have 100,000 people with RTX 4090s mining some new AI based cryptocurrency, that happens to train the model in the process, it's going to be a highly effective system. And we can do it anonymously over Tor or I2P.

You really can't, because bandwidth matters as much as compute power - you can only utilize as much power as you can transfer data to/from. The training methods for current LLMs are parallellizable only by very frequently transferring all the data back and forth, and needing a node to gather and merge all the updates very frequently, and redistribute it to every other node so that they can make any progress. And a GP…

[deleted]
Post reply on HN