Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many AI safety orgs have tried to criminalize currently-existing open-source AI

1a3orn.com

Post reply on HN
Print view View original

341–350 of 405 posts

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#341

Post by mtlmtlmtlmtl »

tsimionescu wrote: post #335
mtlmtlmtlmtl wrote: post #334

Earlier quoted context omitted.

You're not making an argument, just stating a disagreement.

Do you have any doubt that, say, Coca Cola Company could push heroin more effectively than someone on a street corner, if it were legal to do so? Do you think heroin is more or less commonly used than tabacco or alcohol? We've already seen from the opioid epidemic that drug companies at least are both willing and able to push massive amounts of an opioid drug on patients if it is even slightly legal to do so. Why do…

No, I have no doubt they would but I never suggested they should be allowed to. I think the sale and quality should be heavily regulated, advertising still banned, and requirements made to be allowed to purchase it. Like age, and who knows, maybe going through some basic harm reduction training.

Certainly not advocating doing anything the way the US does it, in almost any respect. But the pharma thing is fundamentally a different issue. The reason people so easily become addicted to painkillers is because they're initially given for pain by someone they trust, while unbeknownst to them they've actually been given too much, for too long. In other words this is an issue of doctors abusing their authority.

Why don't I think it would get worse? Because of cultural norms. Legalising heroin won't make it cool and socially acceptable to use. People don't actually derive their social values solely from what happens to be legal.

Let me ask you a question. Why isn't everyone an alcoholic or a smoker? Why do they become rarer(in the west) despite those drugs being legal?

(Hint: the reason is cultural).

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#342

Post by reissbaker »

foo3a9c4 wrote: post #317
reissbaker wrote: post #283

Earlier quoted context omitted.

Yes, I am looking for an argument that justifies governments banning LLM development, which implies existential risk is likely. Many things are possible; it is possible Christianity is real and everyone who doesn't accept Jesus will be tormented for eternity, and if you multiply that small chance by the enormity of torment etc etc. Definitely looking for arguments that this is likely, not for arguments that ask the i…

> in observable reality asking ChatGPT to maximize paperclip production does not in fact lead to ChatGPT attempting to turn all life on Earth into paperclips (nor does asking the open source LLMs result in that behavior out of the box either) I agree with you that current publicly available LLMs do not pose an existential risk to humanity. On the other hand I believe there is a better than 10% chance that the cutting…

I believe that the proposals referenced in the article from current AI safety organizations that would make current-gen open-source LLMs illegal due to supposed x-risk are not supported by reality.

Arguing about theoretical AI models 30 years from now that might or might not be dangerous doesn't seem very convincing to me, since we don't know what they'll be based on or how they'll work — researchers today aren't even sure LLMs can scale to super-human intelligence. Similarly, pre-LLMs many safetyist orgs took the "paperclip problem" very seriously, when it's quite clear now that even the not-very-intelligent LLMs of today are capable of understanding the implicit context of a goal like that and won't seriously propose extinguishing humanity as a mechanism to improve paperclip production. Anthropic was formed in part because people thought gpt-3.5-turbo was existentially risky! And I don't think anyone today entertains that thought seriously, to put it lightly.

Trying to ban AI now due to supposed existential risks of systems in the future that don't currently exist and we don't know how to build (and we don't know if the failure modes proposed by the safety orgs will actually exist) seems like putting the cart well before the horse.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#343

Post by war321 »

upwardbound wrote: post #329
Simon_ORourke wrote: post #178

Earlier quoted context omitted.

Ha! Good one! Let's just roll back all our healthcare research, logistics, environmental advancements etc. because some boogey man tech might do something bad perhaps at sometime in the future maybe!

The majority of our healthcare research, logistics, and environmental advancements, and most other aspects of 21st-century living can be switched over to alternative technologies that do not require computers. The main thing we will definitely lose is social media, which many people would be glad to see removed from our lives, as it provides shallow and addictive dopamine hits, much like cigarettes, while diminishing…

No one, not the most screen addicted teenager nor the smarmiest snake-oil beauracrat, is gonna stand by ANYONE that suggests we need to remove ALL computers from society. Even the people critical of AI and social media are more than fine with computers and smartphones.

How would you even begin to enforce that?

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#344

Post by reissbaker »

foo3a9c4 wrote: post #320
simiones wrote: post #282

Earlier quoted context omitted.

The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). The argument starts with a hypothetical (…

> The first links are spiffy little metaphors, but apply just as much at "God could smite all of humanity, even if you don't understand how". They're not making any argument, just assumptions. In particular, they accidentally show how an AI can be superhumanly capable at certain tasks (chess), but be easily defeated by humans at others (anything else, in the case of Stockfish). As I understand it, Yud is actually pro…

Of the three links you posted:

1. States things like "Finding goals that are extinction-level bad and relatively useful appears to be easy: for example, advanced AI with the sole objective ‘increase company.com revenue’ might be highly valuable to company.com for a time, but risks longer term harms to society, if powerfully accruing resources and power toward this end with no regard for ethics beyond laws that are still too expensive to break." But even current-gen LLMs sidestep this pretty easily, and if you ask them to increase e.g. revenue, they do not propose extinction-level events or propose eschewing basic ethics. This argument falls apart upon contact with reality.

2. Is a 57-page PDF of subjectively-defined risks where it gives up on generalized paperclip-maximizing as a threat, but instead proposes narrower "power-seeking" as an unaligned threat that will lead to doom. It presents little evidence that language models will likely attempt to become power-seeking in the real world other than a (non-language-model) reinforcement learning experiment conducted by OpenAI in which an AI was trained to be good at a game that required controlling blocks, and the AI then attempted to control the blocks. It is possible I missed something in the 57 pages, but once it defines power-seeking as a supposed likely existential risk, it seemed to jump straight into proposals on attempted mitigations.

3. Requires accepting that we will by default build a misaligned superhuman AI that will cause humanity to go extinct as the basic premises of the argument (P1-P3), which makes the conclusions not particularly convincing if you don't already believe that.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#345

Post by Athari »

foo3a9c4 wrote: post #304
kolektiv wrote: post #280

Earlier quoted context omitted.

For info: I don't believe (1), I do believe (2) although not that strongly - it's more likely to be a leap than a gradient, I suspect - I simply don't see anything right now that convinces me it's just over the next hill. Your conclusion... maybe, yes - I don't think we're anywhere near a simulation approach with sufficient fidelity however. Also 82% is very specific!

> For info: I don't believe (1), I do believe (2) although not that strongly Thanks for clarifying. Do you believe there is a better than 20% chance that humans will develop AGI in the next 30 years? > I simply don't see anything right now that convinces me it's just over the next hill. These are the reasons that I believe we are close to developing an AGI system. (1) Many smart people are working on capabilities. (2…

These achievements are impressive, but I'd rather not overhype it.

* GPT-4 still hallucinates as hell, can't do math, fails as basic logic, can't handle really big contexts, hard to update, easy to jailbreak etc.

* AlphaGo was defeated by a Go amateur with a help of another AI.

* AlphaStar basically failed to achieve real goals, was trivial to cheese even after defeating high-ranked players sometimes.

All these problems are architectural, you can't just throw more money and GPUs at it like GPT-2-3-4.

It's hard to predict at this point. We may get to AGI anywhere from 5 years to 100 years.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#346

Post by jamilton »

kolektiv wrote: post #166
melagonster wrote: post #155

Earlier quoted context omitted.

Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. Both the definition of AGI given by the company and by AI aliment/safety researchers is similar. AI safety people trust it is dangerous. let me go continue using a nuclear bomb as a metaphor. if we don't know whether building nuclear bomb is possible, but some companies declare they have existing progress on creating this ne…

They declare that - but I could also declare I'm trying to build a nuclear bomb (n.b. I'm not). Whether people are likely to try and stop me, or try and apply some legal non-proliferation framework, is partly influenced by whether they believe what I'm claiming is realistic (it's not - I have a workshop, but no fissile material). Nobody gets too worried about me doing something which would be awful, but the general c…

I think it's reasonable to at least act as if they really are trying to do what they say they're trying to do.

Like with the nuclear bomb situation - I think it would be reasonable for someone to try to stop you from building a nuclear bomb, or check if you really were, even if they had no idea how you could have gotten the materials. Because it would be really bad if you did. I think people would be worried about you trying to do something awful even if there was low confidence it was possible. They wouldn't be as worried as they would be if they thought you were more capable, but still worried.

So I guess that comes down to whether you think companies saying they want to make AGI are more like a toddler, a teenager, you, a ballistics engineer who owns a uranium mine, Lockheed-Martin, or a nation-state trying to make a nuclear bomb. My understanding is that people who are concerned about AI x-risk are largely in the teenager to Lockheed-Martin range (e.g., small eventual risk to large imminent risk), while I assume you think it's more in the toddler range (no risk at all for a long time)

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#347

Post by greyface- »

__loam wrote: post #77
visarga wrote: post #50

Earlier quoted context omitted.

> I'm talking about the ability of any AI system to obfuscate plagiarism and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. This should be an explicitly allowed practice, it is following the spirit of copyright to the letter - use the ideas, facts, methods or styles while avoiding to…

We really need to talk about preserving the spirit of copyright, which is about protecting the labor conditions of people who make things. I'm not saying the current copyright system accomplishes that at all but I do think a system where humans do a shit load of work that AI companies can just steal and profit from without acknowledging the source of that work is another extreme that is a bad outcome. AI systems need…

The spirit of copyright in the United States is not about protecting labor conditions. It is about promoting innovation in arts and sciences.

> The Congress shall have Power ... to promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries.

- United States Constitution, Article I, Section 8, clause 8

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#348

Post by jamilton »

foo3a9c4 wrote: post #304
kolektiv wrote: post #280

Earlier quoted context omitted.

For info: I don't believe (1), I do believe (2) although not that strongly - it's more likely to be a leap than a gradient, I suspect - I simply don't see anything right now that convinces me it's just over the next hill. Your conclusion... maybe, yes - I don't think we're anywhere near a simulation approach with sufficient fidelity however. Also 82% is very specific!

> For info: I don't believe (1), I do believe (2) although not that strongly Thanks for clarifying. Do you believe there is a better than 20% chance that humans will develop AGI in the next 30 years? > I simply don't see anything right now that convinces me it's just over the next hill. These are the reasons that I believe we are close to developing an AGI system. (1) Many smart people are working on capabilities. (2…

I don't think these reasons are very persuasive, as everything but 5 has been true at different times in the past. Obviously it's much more people, more dollars, and more impressive systems (but slower hardware progress), but I hope you see what I'm getting at.

And of course there's differences in what someone considers to be soon. Many AI x-risk believers think there's a ~50% chance of AGI before 2031 (https://www.metaculus.com/questions/5121/date-of-artificial-...) (I've heard this prediction site's userbase tends towards futurists/techno-optimists/AI x-riskers). I would consider that soon, I wouldn't consider 2054 soon.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#349

Post by tavavex »

simiones wrote: post #289
whywhywhywhy wrote: post #125

Earlier quoted context omitted.

>They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. Nah a lot are complaining about the licensing of content because they think it will destroy it but instead would essentially mean image gen ai would only be feasible for companies like Google, Disney, Adobe to build. Not sure if you could even feasibly make GPT4 level models without a multi year timeline…

Given that you need monumental amounts of compute power to come close to something like GPT-4, I don't think the added costs of not treading on people's IP is the major moat that it's being made out to be.

The difference here is that compute is always getting cheaper. In 5 or 10 years, it may be feasible to train something GPT-4-sized for a small business. Not to mention that we're likely not at the highest efficiency yet, and there may be undiscovered methods of training better LLMs on lesser hardware. But licensing costs are not going to decrease. If dataset compilation is ruled to be something that needs to be paid for, it'd put a never-changing "must be worth $Xm or more" badge on any small AI company.

Re: Many AI safety orgs have tried to criminalize currently-existing open-source AI

#350

Post by richardw »

mrbombastic wrote: post #255
richardw wrote: post #205

Earlier quoted context omitted.

Progress in AI is one way. It doesn’t go backwards in the long term. As capabilities increase, the resources required to breach limits become available to smaller groups. First, the hyoerscalars. One day, small teams. Maybe individuals. For every limit that you desire for AI, each will be breached sooner or later. A hundred years or a thousand. Doesn’t matter. A man will want to set them free. Someone will want to wi…

you seem to be implying past progress implies unlimited future progress which seems a very dubious claim. we hit all kinds of plateaus and theoretical limits all the time in human history.

Given history, it's infinitely more dubious to link our future safety to some idea of progress stopping for some reason. We'll have increasingly smart AI and your go-to position is that progress will...stop? It's literally helping us with thinking, which is the driver of innovation and progress.

Very few things have slowed down over decades or centuries. If anything it's been a mad rush with AI recently. Of course there will be plateaus, but I specifically put in a time of 100-1000 years in there which is basically very many 9's guaranteed to produce some major fucking changes in the world. 1000 years ago we were using arrows, and now we have AI to help us overcome plateaus.

Post reply on HN